HK1028696B - Method and apparatus for a copy-once watermark for video recording - Google Patents

Description

Method and apparatus for copy-once watermarking for video recording

Technical Field

The present invention relates to a method and apparatus for implementing a "copy-once" by embedding reliable identification codes, such as Watermark (Watermark) or fingerprint codes, in a video signal and using these identification codes.

Background

Copy protection of motion pictures on video material, such as video tape, is a well known problem. One solution for general analog video tape material copy protection is the well-known Macrovision anti-copy process (Macrovision anti-copy process) which adds a pulse phase to the vertical blanking interval of a video signal (see U.S. Pat. nos. 4,631,603 and 4,695,901 to Ryan, which are incorporated herein by reference). By dividing the image, these added pulses scramble the automatic gain control circuitry of a conventional VCR (video cassette recorder), thereby making any recorded video signal invisible.

Copy protection (copy protection) is a more important problem with digital video technology, and unlike analog recording, one characteristic of digital recording of video signals is their accurate reproduction of the original video signal without degradation. Thus, the use of digital video is more motivating for unauthorized copying of copyrighted video material. This has been a major obstacle to the spread of pre-recorded digital video media and digital video recorders.

One known solution is to use watermarking. A watermark is a digital code embedded in a video signal that can be read by a reader (detector) in a suitable video recorder or player, wherein the watermark itself does not significantly interfere with the video signal (i.e. without significant quality degradation). The watermark is read by detector circuitry in the recording device or player which instructs the recorder or player to perform certain functions, such as disallowing copying of the material or allowing only a single copy of the material based on the watermark. In a suitable recorder or player, special detector circuitry is required to use the watermark.

Watermarks are proprietary signals built into the video signal so that they cannot be removed without substantially damaging the video signal; it is difficult to deliberately or accidentally remove a watermark. Watermarking is a so-called bilateral copy protection scheme requiring a specially adapted ("adapted") recorder/player capable of detecting and responding to the watermark. A normal non-compliant recorder ignores the watermark and thus the watermark does not work. In contrast to the above-described process, this is a "one-sided" macro video copy protection process, but is generally not suitable for protecting a digital signal.

While such watermarks are useful and have been employed to some limited extent in commercial applications, they have significant drawbacks. The most important drawback is in the case of copy-once, which occurs in particular in the cable television industry for pay-per-view broadcasts, such as movies. The cable television industry has created a desire among its users to allow them to use a VCR for their personal recording, such as the recording of a cable-transmitted movie. Although owners of the copyright and cable television industries have apparently accepted that a user engages in a single recording for his individual, the copyright owner (e.g., a movie studio) does not want to allow any second generation recording to be implemented. It is therefore desirable to allow a copy-once video transmission, such as via cable television, while prohibiting any second generation duplication of the first duplication.

It is possible to prohibit such second generation recordings using common watermarking processes. For this case, there are generally two types of watermarking: "prohibited copy" watermarks and "one-time copy" watermarks. Both of which must be read by a suitable recorder that performs the copying, which changes the copy-once watermark to the copy-prohibited watermark when recording. One problem with this is the extra cost of the applicable recording device, since it must be able to detect the two different watermarks and also append (write) the copy-never watermark. In addition to the rather expensive cost of implementing this equipment, such systems are also susceptible to technical failures and are therefore incomplete.

Disclosure of Invention

There is a need in the art of digital video recording for a copy-once method that provides improvements over the prior art in both economy and security. There is also a need for a copy-once method that requires only one watermark, and thus only one watermark detector per applicable recording (or playback) device. Such a method requires no watermarking adder or modifier in the applicable recorder. (where the term "adapted" refers to a device of ordinary skill in the art that has been modified by the addition of specialized circuitry and/or software in accordance with the present invention.) if manufacturers of video players and recorders wish to practice the present invention, they can add suitable circuitry and software to their products, and the supplier of the video material will likewise add the desired signals to their programs (programs).

Attaching such a system should be easy to perform, without keeping any secrets to ensure its security, and operate in both the analog and digital domains. A further requirement is that even if the first (allowed) recording is implemented on an existing regular consumer VCR, either VHS, S-VHS or 8mm signal VCRs (generally VCRs that cannot be expected to meet the digital copy-once protocol), copying of the second generation is still prevented by a compliant recording device.

Thus, a single level (single class) of watermarking according to the invention has two versions: prohibited copy and copy-once, which are identical except for a single bit difference. According to the invention, it is not necessary to attach a watermark to the video signal when video recording is performed, since the copy-once watermark is instead converted to a copy-prohibited watermark by a simple one-bit change. (as used herein, "watermarking" refers to both watermarking and another type of digital "fingerprint.) as described below, the present invention is applicable to both digital and analog video signals (e.g., VHS, S-VHS, or 8 millimeter video signals). The watermark is still viable in the conversion from analog domain to digital domain or digital domain to analog domain, viable compression and viable in the conversion process between television standards, i.e. PAL to NTSC or NTSC to PAL.

According to the invention, a watermark of a digital signal is conventionally embedded in a video image. The subset of watermark bits carries a digital attribute (a number) that is a numerical characteristic of the video signal, such as the average amplitude of the video signal over a video field or frame. The video attributes according to the present invention are preferably certain unique features of the video signal that vary maximally from frame to frame and are not affected by the usual distortions that occur in the associated analog-to-digital transmission channel. This attribute is only used for copy-once situations.

To prepare (encode) the video with that watermark, in addition to conventionally embedding the watermark, a frame (or field) is randomly (or pseudo-randomly) selected, e.g., one frame every 10 seconds of the video signal, and the properties of the frame (or field) of the video signal are digitally computed. A "field flag" is also added to the frame to indicate that the frame is a selected frame. The field mark is a dedicated signal located in an invisible part of the image frame, for example an overscan part.

The encoded video signal is conventionally transmitted via cable television or satellite television to a consumer attempting to record the video signal using his applicable digital video recorder. Prior to recording, the applicable digital video recorder examines the watermark, checks the watermark, detects copy-once bits in the watermark, and extracts the associated attribute values from the watermark. The adapted recording device also detects field marks on a particular mark field, measures properties of the particular field, and compares the measured properties with the extracted property values. If the two attribute values match, the recorder then performs recording. If they do not match, recording (copying) is not initiated.

At the same time, the video recorder strips the field mark from the video signal while performing the recording, so as to prevent any second generation duplication of the formed recording. The absence of field markers prevents any second generation recording of a particular video signal by a suitable video recorder.

Thus, the suitable recorder includes a watermark reader (detector and verifier), an attribute measurement circuit, a field mark remover (stripper) and associated intelligence (software) in a microprocessor, typically resident in such a recorder to perform the necessary computational and logic functions including comparison of the attributes.

The field mark is a signal that cannot be detected and played by ordinary analog VCRs because the field mark is selected to be of a type that can be removed by chrominance filtering circuitry typically present in an analog VCR. Therefore, if the originally transmitted video signal is recorded using a normal analog recorder, the field mark is lost in the analog recording process, thus preventing any applicable digital recorder from implementing second generation duplication.

The field marks are usually inserted in the invisible part of the active video signal, i.e. in the overscan area in the television receiver. The field marks may also be positioned in the vertical or horizontal blanking intervals. The removal of the field mark as described above is preferably a signal of the automatic removal type by a conventional recording apparatus, but this is not a requirement.

While a typical application of the current watermarking process is in the cable television or satellite distribution systems described above, this is not a limitation. Typically, pre-recorded video material, such as pre-recorded on a magnetic tape or DVD (digital versatile disc), will not be affected by this process, since typically such material is not a copy-once but "copy-never". However, the current copy-once watermarking process may be applied to pre-recorded video material.

Drawings

Fig. 1 shows a block diagram of an encoder according to the invention.

Fig. 2 shows a block diagram of a recorder for a watermarking/playback control scheme according to the invention.

FIG. 3 is a table of characteristics in a dual watermark marking system.

FIG. 4 illustrates one frustrating method for current copy-once processing.

Fig. 5 illustrates the frustration device of fig. 4.

Fig. 6A to 6D illustrate the use of one particular videomark according to the invention.

Detailed Description

Basic treatment

According to the invention a watermark is added to (encoded in) a video signal having a payload of e.g. 8 bits (payload is a changeable number of bits). Bits 1 and 2 of the eight payload bits are conventional (in digital video) Copy Generation Management System (CGMS) bits, while bits 3 and 4 are Analog Protection System (APS) bits. The actual analog protection system is for example the well-known Anti-copy (Anti-copy) process for macro video images. These four bits therefore have the usual usage to indicate copy once/copy disabled/no more copies and to switch the analog protection system on or off. (macrovideo or other analog copy protection processes have applicability to digital recording because digital video recorders typically have one analog video signal input capability.) according to the present invention, the bits of the remaining four payloads define the image attributes of the selected frames underlying the video signal, examples of which are described below.

As is common practice, for a "copy-never" program (video signal), bit 1 and bit 2 of the eight payload bits are set to the value (1, 1). Bits 5 through 8 are "don't care". Upon detecting the (1, 1) bit allocation, a suitable recording device refuses to make any recording of material.

For a "copy-once" program, bit 1 and bit 2 are set to the values (1, 0), again as is common practice, bit 5 through bit 8 now display the program's attributes in 4 bits. An "attribute" herein refers to a numerical expression of some characteristic of video material. An attribute preferably changes maximally from frame to frame whenever the image changes. One property preferably does not change with distortion in the video signal, such as noise, non-linearity or frequency response, caused by the analog or digital channel over which the video signal is transmitted.

A suitable video recording device, upon detecting the (1, 0) bit copy-once configuration, calculates the same attribute from an associated image frame and compares it to the value transmitted on payload bits 5 to 8. If there is a match, the copy is allowed to continue. If not, the copying (recording) is terminated. In other words, a copy-once instruction that results in a mismatched attribute will stop the recording process implemented by the applicable recording device. Finally, with the enablement of the first generation recording, the video signal being recorded is modified while it is being recorded so that subsequently applicable recording devices cannot confirm the attribute values conveyed by the watermark, thereby inhibiting further copying.

Detailed description of the preferred embodiments

The following more detailed description of an embodiment of the method takes into account the possibility that the watermark reader circuitry may use, for example, 10 seconds or more to detect and decode without error the watermark in the presence of noise, or the watermark after scaling. The specified time interval is for example only.

Encoding operations

1. Encoding refers to the "front-end" process of protecting the video material. For example, in each 20 second interval of the program, the encoding device randomly selects a video field (or frame) within the first five seconds of achieving a reliable measure of the attribute, even after the video has been affected by distortion in the various analog or digital transmission systems described above. The encoding device makes reliable measurements of the properties of the selected field or frame.

2. After waiting a random selection interval (e.g. 0 to 5 seconds for added security), the encoding apparatus encodes the measured property values of the selected fields into conventional watermarks according to a 4-bit digital code word. (if desired, the watermark is appended to the video signal in 4-bit groups set to some forbidden value, e.g., F (hex), continuously until instructed to convey the property value.)

3. The encoding device marks the field selected in step 1 so that a suitable recording apparatus can reliably locate the field. The following is a method of marking for one example field: approximately the first microsecond of the first horizontal scan line of each field of the video signal is blanked and only a 25IRE amplitude increase is appended to the selected field during that one microsecond interval. A more general marking method will be described below.

Fig. 1 shows an exemplary encoder apparatus at the "front end" of a cable television system. The video program is applied to an input 10 before being cabled to the user. The terminal 10 is typically a multi-wire (bus) connector because of the digital video. As described above, frame selector 12 randomly or pseudo-randomly selects the fields or frames that are particular to be encoded and provides an indication of each selected field or frame on control line 14. In response, field marker circuit 16 marks each frame indicated in the video signal. The control lines 14 are further coupled to a property measurement circuit 18, which property measurement circuit 18 also receives the incoming video signal and calculates the (measured) property of the selected field or frame. The measured digital attribute values are coupled on line (S)26 to watermark modification circuitry 32. Component 32 receives the generic watermark from the conventional watermark generator 28 and modifies the watermark with the 4-bit attribute value. The watermark thus modified is coupled to a frame watermarking circuit 38 which writes the modified watermark to each of the fields or frames provided on the line 22 bearing the field mark. The watermarked video signal is then output at terminal 40 to a distribution network of cable lines (or other lines). It should be understood that some of the components of fig. 1 (and fig. 2) are circuits, while other portions represent circuits or functions performed, for example, in a programmed microprocessor.

Read operation in a suitable video recorder

1. During the recording process, when a field mark of the above-mentioned type in the received video program is detected, the applicable recorder/player measures and stores the properties of the field or frame.

2. The recorder/player compares the measured attribute with the value of the attribute decoded (extracted) from the watermark. The comparison is made as soon as the watermark detector in the recorder/player first has confidence that it has correctly decoded (verified) the watermark.

3. If the two attribute values match, then the recording is allowed to continue. Matching does not require exactly the same values, but may allow some measurement inaccuracies, i.e. with some tolerances. If the two attribute values do not match, or if the periodic field mark lacks a copy-once designated program, recording is inhibited.

4. The recorder deletes the field marks entirely during its recording process. In addition, a "spoof" field mark is added to a field whose properties are significantly different from the field mark carried by the watermark mark, but this additional step may not be required. It should be sufficient to simply delete all field marks. The benefit of deleting the field mark is that the need for modification of the video signal is avoided so that a measurement of a different property during subsequent recording attempts is enforced.

An exemplary suitable recording device is shown in FIG. 2; which complements the encoder of fig. 1.

The input digital video signal with the field marks and the watermark is applied at an input 46 to a field mark detector 48 which detects each field mark and outputs an indication of the marked frame on a control line 50 to a substantially conventional watermark mark detector/verifier 54. The checker 54 checks for the presence of copy-once bits and conventionally checks the watermark. The attribute bits are then taken from the check watermark extracted by extractor 56 and passed to comparator 68 via line 62. Attribute measurement circuit 58 also receives the incoming video signal and, in response to the signal on line 50, measures (calculates) the predetermined attribute of the tagged frame as indicated on control line 50. The measured property value is coupled to a comparator 68 on line 64. If the comparator 68 finds a match, only the conventional recording circuitry 70 is enabled. Coupled to recording circuitry 70 are field mark stripper circuitry 72 and a circuit 74 that changes copy-once bits to an inhibited copy value during recording.

Safety of regulations

For a "thief" (copyright infringer) who would defeat the system, i.e. an infringer who would modify the first generation of legal video so that it can be reproduced by the applicable recorder, he would have to do the following (or an equivalent composition of processes). First, during playback of the first (legitimate) recording, the properties of each field are measured and stored and the watermark is decoded. Subsequently, a field mark is appended to a field whose properties correspond to the properties conveyed by the watermark mark.

Such "thieves" require many seconds of video delay at MPEG-2 (compressed video signal) speed, a watermark reader and an attribute measurement system. (video is typically replicated in MPEG-2 compressed form.) further details are provided below.

Thus, the present process is readily adapted to the goal of preventing "theft" by most people, particularly in view of the minimal gains from misappropriating this portion of the standard overall copy protection system. A copy-once scheme does not require "theft-protection" as does the basic watermarking, since programs marked "copy-once" are generally less valuable in nature than programs that possess "prohibited-copy" copyrights. In any event, a thief who wants several copies will easily (and arguably legally) obtain them simultaneously from a copy-once original video program.

Attribute requirements

Desirable image attributes according to the present invention are:

1. measurable in a consumer type digital video recorder with minimal additional circuitry.

2. The level error should be fairly unaffected for typical video signals: the errors refer to non-linearities, noise, tilt, frequency distortion, quantization and compression artifacts, etc. encountered in various analog and digital transmission systems.

3. So that a thief cannot easily modify a video signal to force a particular attribute value without seriously compromising the viewing value of the program.

The third requirement can be dispensed with because in the context of current copy-once protocols, there is no artifact that might make it substantially more difficult for a thief.

Instance of Properties

The first example: the average d.c. voltage values of the separate regions of the selected image frame are summed in a manner that may result in a large attribute value (largetterbvte value) for a standard video image. The size and location of the image area is selected to maximize immunity to tilt, noise and distortion. The values resulting from the average d.c. amplitude division of the image frames eliminate attribute measurement errors that may be caused by video gain differences in the insertion hardware devices or in the transmission path.

The second example: this example is a specific implementation of the first example described above for an MPEG compressed video signal. Consider an 8 x 8 pixel MPEG-2 macroblock grid placed symmetrically about the center of the image frame. Imagine that each macro-block is colored black or white in the form of a conventional "checkerboard" pattern. This attribute is equal to the total number of d.c. entries for white data blocks minus the total number of d.c. entries for black data blocks divided by the total number of d.c. entries for all data blocks.

Example of a field Mark

First field marker example: the first 1.5 microseconds of the first horizontal scan line of each video field (or frame) is set to 25% of the video gray level. Like the other field markers described herein, the present marker is in the overscan portion of the image frame so as not to be visible on a television receiver. (however, this position is not so limited.) to mark a particular field, a sine wave of, for example, 2.5MHz is added to this gray level with a peak-to-peak value of 50IRE units.

This field mark can be reliably detected even after passing through a poor quality cable television channel. However, it advantageously cannot survive after signal processing and recording circuitry of a VHS or 8mm VCR (video cassette recorder), but it will survive in an S-VHS recording. This is because such field marks are screened out by the chrominance comb filter in VHS or 8mm VCR. Thus, after recording and playback by a VHS or 8mm VCR, the field mark is virtually eliminated, thereby prohibiting second generation copying by a suitable video recorder.

The second example: this example is operable for VHS, S-VHS and 8mm VCR, for example. The first 1.5 microseconds of the first two horizontal scan lines of each video field (or frame) is set to 25% of the video gray level. To mark a particular field, a sine wave of 50IRE unit peak-to-peak subcarrier frequency is added to the gray level so that its phase relative to the sync is the same on both scan lines (not inverted on alternate scan lines as a normal chrominance signal).

This field mark can be reliably detected even after passing through a poor quality cable television channel. However, it will not survive through the chrominance filters of any consumer type VCR. On playback, the comb filter of the VCR will reduce this chromaticity for scan line 1 to 25IRE units and to zero on scan line 2. The field mark detector in a suitable video recorder is arranged to detect only such signals on scan line 2. Thus, after recording and playback by an analog VCR, the field mark is effectively lost, thereby inhibiting further video copying by an applicable digital recorder.

The third example: this field marker is operable for use with, for example, VHS, S-VHS and 8mm VCR. A 24 x 4 pixel block of data at, for example, the bottom left corner of the video field (or frame) is first selected. The chrominance values are then replaced in the 24 x 4 pixel data block with a fixed pattern mark, for example, replacing the 24 x 4 pixel data block with alternating blue and yellow values. This marker must survive MPEG compression to be reliably detected even after passing through a poor quality cable duct. However, it is never viable through the chrominance filters of any consumer type VCR. The field mark detector in a suitable video recorder is arranged to detect only such signals on scan line 2. Thus, after recording and playback by a VHS or 8mm VCR, the field mark is effectively lost, thereby prohibiting second generation copying by a suitable video recorder.

The use of a VCR to eliminate marks is clearly a worthwhile improvement over current copy-once systems from the standpoint of additional protection provided to the copyright owner.

Some of the marking principles may be fully compatible with MPEG compression with minor modifications. It is clear that a desirable improvement over current copy-once systems is provided from the standpoint of providing additional protection to the video material.

Additional embodiments-frame marking and watermarking

1. In another embodiment, a frame (or field) marker is a pseudo-random marker attached to a particular image frame, for example in every 4 second long window, for encoding purposes. The frame marker (as described above) is automatically removed by a VCR. (an example of such a flag is a fixed d.c. color saturation value-the CRed and CBlue values on the first row of the first macroblock). For the marked frame, an attribute value is calculated and saved.

2. The 4 attribute bits of the watermark at a pseudo-random location are modified to equal the attribute values of earlier marked frames, for example, in a subsequent 2 second window.

3. Steps 1 and 2 are repeated throughout the video program during the encoding process.

4. When a copy-once program is subsequently recorded, the frame markers, as described above, are removed.

In addition to the regular, and copy-once only, watermark bits conventionally carried in the coefficients of the low frequency media MPEG DCT, this embodiment appends 4 additional authorization bits, carried by some of the high frequency MPEG DCT coefficients. If a copy-once status is detected from the regular watermark bits, the recorder checks if the 4 additional bits are present; if not, the recorder will not copy. If present, the applicable recorder adds sufficient noise before the copy is made. This will eliminate the 4 additional bits, eliminating the copy grant.

The advantage of this scheme is that the conventional (non-compliant) VCR (without a copy control system) will automatically eliminate the additional authorization bit, rendering the output signal unrecordable by compliant (digital) recorders.

In the present embodiment the watermark comprises two different parts, called WM1 and WM 2. WM1 contains an 8-bit payload that is difficult to eliminate without compromising image quality. The payload bits are distributed over the base/mid/high MPEG DCT coefficient frequencies in the macroblock.

WM2 is, for example, a fixed 4-bit pattern, or a CRC (cyclic redundancy check) value of the payload bits. WM2 is easy to remove because the WM2 bits are only distributed at high frequencies and can be removed with a low-pass or other filter. WM2 survived on 4.2MHz transmission but did not survive on recordings at 2.5MHz VHS. It is easy and inexpensive for WM2 to have most of the energy spread horizontally in the macroblock to eliminate or render it useless. For greater robustness, an optional CRC (4 bits) is used.

The "WM 2 removal" circuitry in the applicable recorder removes WM2 using a low pass filter or other filter on the video data; the necessary circuitry is minimal. The cost of adding WM2 detection circuitry to the WM detection circuitry is minimal because WM2 detection circuitry reuses most of the WM1 detection logic.

Fig. 3 shows the state of the present embodiment. In fig. 3, the content of each watermark field is indicated for WM1 and WM2 along the top row of each column number, with the associated operations in the remainder of the column being performed by an applicable recorder/player. The upper half of fig. 3 is a simpler version of this embodiment; the lower half is a more complex version using CRC. In both versions, copying is only allowed when WM1 is present in copy-once bit groups and WM2 is valid (verified).

Frustration process and apparatus

In addition to the copy-once embodiment described above, one method of defeating the current copy-once process, as described above, is shown in FIG. 4, which diagrammatically describes the defeat process and apparatus. "frustration" means providing a record that it may be copied; this is, of course, a violation of the objectives of the copy-once system of the present invention. The frustration methods and devices proposed therefore will not normally be used commercially by recording device manufacturers, and the disclosure herein is for completeness. (the use of such frustrating methods or devices will typically include copyright violations).

Several conventional components 4 are shown in fig. 4 including a conventional cable line set-top device ("box") 100 connected to a cable television cable 102 and located above a consumer television receiver (not shown). In this case, a suitable recorder 106, including the digital video recorder of the features of fig. 2, is connected to the set-top device 100. A person desiring to defeat the copy-once process couples a defeat "black box" 104 between his cable set-top device 100 and his suitable video recorder 106. Details of the "black box" 104 are provided below; the "black box" 104 houses the defeat circuitry.

In step 1 of fig. 4, a video program comprising copy-once protection of field marks is received and transmitted at the top device 100 via cable 102, to the applicable recorder 106 via "black box" 104. The black box 104 includes circuitry that appends a "marker" (a dedicated field marker) to the first image frame of the (original) field marker that has been detected. (the black box 104 includes its own field mark detector similar to the arrangement of fig. 2.) the mark has specific characteristics as described below. The black box 104 then stores in its internal memory a table of time intervals between detected field markers for the video program. The table provides information about the timing of the original field marks. The video signal with the mark applied to the first field mark frame is then coupled to a suitable recording device 106, of the type shown in fig. 2.

The compliant recording device 106 then causes the allowed copy-once to be recorded to a first tape (or disc) 110. The applicable recorder 106 thus formed removes all of the original field marks, as described above. It cannot however strip the marks (S) appended with the black box 104 because they are marks of each dedicated type field, which cannot be removed by the stripper circuitry in the recording device 110 unlike the original field marks.

In step 2, shown in the lower part of fig. 4, a recorded tape (or disc) 110 is played on a player (or recording device) 114. Of course, the played video signal does not include the original field mark. This video signal is played through the black box 104 ', which black box 104' has been reconfigured to a play mode. (the black box 104 'is programmed to its play mode instead of its recording mode, the black box 104' being the same as shown in the upper half of fig. 4.) the black box 104 'detects the mark (S) appended by the black box 104 in the recording step and uses the frame of the first mark as an indicator (indicator) and then reinserts the new field mark of the type present in the original video material at the time interval indicated by the table previously stored (step 1) in the black box 104'.

Thus, the video signal output from the black box 104' is substantially the same as the original video signal, including all of the re-inserted field marks. This video signal can therefore be recorded by the applicable recorder 106, which will output a second generation video on a second tape or disc 116, again without field marks, but still viewable (but not reproducible). Thus, by the method of FIG. 4, it is possible to provide a second generation video on tape or disk 116, thus defeating the copy-once protection process of the present invention.

The internal structure of the black box 104, 104' is shown in fig. 5. The upper half of fig. 5 shows the black box 104 in a step 1 configuration. The input video signal from the set-top device 100 is applied to an input 120 and then coupled to a frame mark detector 122 which detects the original frame (field) marks. The control line 124 transmits an indication of the occurrence of a detected frame mark to a mark writing circuit 128 which then appends at least the mark (S) to the mark frame of the first field and outputs a video signal of this mark at an output 132 which in fig. 4 is coupled to the applicable recorder 106. Similarly, indications of frames detected on line 124 are transferred to a memory 136 which includes a table for storing the time intervals between detected frame markers.

The lower half of fig. 5 shows the configuration of the black box 104' in step 2. Where the video signal (from player 114) is coupled to black box input 120, which in turn is coupled to a marker detector 140, which detects the markers inserted by marker writing circuit 128. An indication of each such detection mark is provided on control line 142, indicating the particular frame carrying such mark (S). Line 142 controls frame marker write circuit 150 to append frame markers corresponding to those in the original video signal at desired intervals. The time intervals determined by the data previously stored in the memory table 136 are in this case read out on control line 146 to control when the frame mark is applied by the frame mark write circuit 150.

The video signal with the new frame marker inserted by the frame write circuit 150 is then coupled to the black box output 132, which is in turn coupled to (in fig. 4) an input of the applicable recording device 106. This is an illustrative model of a black box and is not limiting.

Given that the knowledge and common knowledge provided herein about the nature of the field marks is available in video technology, the construction of such black boxes is well within the skill of the ordinary art in this field. The black box is essentially a digital device whose function is to add a mark to a particular image frame, sense the original field mark and record a table of the marks. A typical black box is a microprocessor controlled device that includes memory. The actual circuitry in the black box may select various forms of circuitry that perform the functions described above.

Tagging for various video delivery methods

For a copy-once marking system as described above, processes including content preparation, content delivery, and content viewing/recording must be considered. The processing of the various video delivery methods is then analyzed: analog cable and broadcast delivery; delivery of digits through existing installed base stations of digital set-top devices without digital output (conventionally designated P1394); the transfer of digits through existing installed base stations of a digital set-top device in digital output; and digital delivery through the new digital set top device.

For analog cable and broadcast delivery, the content (program) is prepared prior to delivery and the copy-once-to-no-more-copy transformation process is performed in the recording device and may only occur in the video signal domain.

For digital delivery through installed base stations with digital set-top devices without digital output (P1394), the content is preferably prepared prior to programming/MPEG encoding (authoring/MPEG encoding), but may also be prepared in conjunction with programming/MPEG encoding. However, the copy-once marking should be done with live MPEG encoding/decoding. In this case, copy-once to no-more transformation is also performed in the recording device and may only occur in the video domain.

Digital delivery through installed base stations of digital set-top devices with digital outputs is similar, however the copy-once-to-no-copy transformation occurs in the MPEG domain.

Finally, the making of copy-once indicia may also occur in the set-top device itself for digital delivery through a new digital set-top device. Additionally, in the case of P1394 output, copy-once to no-more-copy conversion can also occur in the set-top device, as long as a copy authorization is sent from the set-top device to the decoder.

For analog cable and broadcast delivery, the system includes a fixed copy-once/no-more watermark, and the addition of a frame or field mark appended to randomly selected image frames or sequential image frames to identify the copy-once material. The transformation from copy-once to no-more is performed by eliminating or reducing the mark. In addition, to provide additional security, an attribute of the marked frame or sequence of frames is encoded in the watermark. The watermarking attribute code is randomly delayed to provide greater security.

Examples of attributes include the video characteristics of the display marker frame (S), the position of the marker frame (S) or the length of the marker' S continuation (which will be randomly selected to provide security).

This type of flag can be removed automatically by a normal VCR and even by a non-compliant MPEG encoder. For example, alternate lines of complementary color at a relatively low amplitude (the marker pattern) will be removed by the VHS and MPEG encoders during the field for frame switching. The marker is positioned, for example, at the lower left corner of the video signal (the marker plane is 64 pixels x 4 lines/field) and is not visible on the television receiver (due to overscan). To improve security, the marking process eliminates chroma in the marking area over the entire image frame and attaches the marking pattern to the marking frame. The mark elimination is performed by chroma elimination.

In the case of digital delivery through an installed base station of a set-top device without digital output, the tagging system must be able to survive MPEG compression/decompression and will likely survive VCR and MPEG recording unless the tagging is performed within the encoder itself or the encoding process is limited.

To maximize the effectiveness of copy-once systems, two marking methods are supported with compliant recording devices (compilantrecorders). The "stronger" flag defined above is selected for use in an analog system or for use in a compatible encoder. A "weaker" mark of a mark pattern having a horizontal period of complementary colors of relatively low amplitude is selected for an incompatible encoder.

For digital delivery through an installed base station of a set-top device having a digital output, the digital recorder preferably performs a copy-once to copy-no-more conversion at minimal cost.

One solution is to add and remove an additional marker placed in the bitstream itself. However, the watermark/mark carried in the video data is also necessary to support the video output of the set-top device, and the digitally recorded copy will be marked in the bit stream for no further reproduction and a single copy in the video signal. A synchronization system is then required to be included in each player to ensure that the implementation is no longer copied.

Another more preferred solution is to minimize the additional requirements for label detection/label elimination.

The conventional steps required to read the watermark in an MPEG data stream are:

-demultiplexing the transmission data stream.

-decoding the video data stream into I-frames.

-decoding all slices in the I-frame.

-processing all macro data blocks (of MBs).

-processing the entire luminance data block.

In this case, since only I frames (one kind of MPEG frames) are used in the watermark readout process, the mark detection/removal process should be performed with I frames using only the following steps:

-processing the marked area chroma data blocks for detection.

-buffering the bitstream and replacing the "marked chroma" data block bit sequence in the marked macroblock by a pre-computed "non-chroma" data block bit sequence while ensuring that the video buffer checker size (vbv) is not affected.

Subsequent problems and requirements arise since the label removal is only performed on I-frames and on macroblocks.

For a non-compliant encoder, there must be one continuous marker frame, and it must be long enough to ensure that one I-frame is included. The allocation of one frame or marker frame should be such that there is a sufficient number of I frames among them. If a sequence of tagged frames is used, the properties of the sequence should be inferable from the properties of the I-frame (S). This problem can be better solved by using frame positioning or frame sequence length as an attribute. In the frame location method, one marked frame will carry a particular watermarking code, while an unmarked frame carries another code. Any mismatch between the watermark marking code and the marking state causes no further copying. In the frame sequence length method, the frame sequence length alternates randomly between 3 × D and 5 × D, where D is some number greater than the maximum allowable group of pictures (GOP) size (maxGOP). The presence of a singly labeled I-frame and a sequence of I-frames covering less than the maxgp sum frame in this approach would indicate that no further copying is occurring. So, a sequence of I frames covering more than D and less than the 3D sum frame will match a 5 XD code in the watermark, or a sequence of I frames covering more than the 3D sum frame will match a 3 XD code in the watermark. This latter case is the best approach.

If a sequence of markers is used, none of the markers of the frames preceding the first I frame in the sequence will be removed. The position and sequence length methods described above address this problem. Removing the marker from the I-frame must often remove the marker from other frames in the GOP. This will occur when no motion occurs in the marked area and can be improved by forcing the motion vector in the marked area to zero during encoding. Removing the mark from the I-frame can never cause significant distortion in other frames in the GOP. Such distortion may occur because the marker region may be used to predict any macroblocks in subsequent P and B frames that surround the marker region. This is one of the reasons for using complementary colors in one embodiment. The elimination of the mark should not produce a visible colorimetric change. The elimination of the marks by replacement of a pre-calculated chrominance data block means that the chrominance components in a 54 x 16 region, which generates visible artifacts on some tv sets, are totally deleted or a bit stream is carried in the user data which has to be generated by a compatible encoder.

For a compatible encoder, the flag generation is only performed by the encoder with respect to the I-frame. The motion vectors on the following P and B frames are forced to zero and no intra-block encoding is allowed on these frames. Motion prediction for non-marked data blocks is forced to not use the marked data blocks.

For digital delivery through newly generated digital set-top devices, frame markers appear only in the video output in the same way that ACPs are generated. The video encoder marks (using robust marks) the pre-watermarked content with mark location information conveyed in the bitstream. On the P1394 side, the set-top device implements a P1394 output upon identifying an applicable recorder. This system even allows anyone to specify whether a copy is to be made independently through analog or digital output means. Copyright information and related data (mark positions) are transmitted in the bit stream in the same manner as the ACP information that is currently transmitted.

Three possible systems are disclosed below. All three systems use one common video recorder and one common architecture. The original material is free of artifacts (so that the viewer of the program will not see any effect) and artifacts related to the video copy (e.g., MPEG copy protection).

For a copy-once structure compatible with the encoder, the markers are only for I-frame insertion. For a non-compliant encoder, a sequence of 3 seconds and 5 marks is randomly added to the video signal, and after a random lag time, a "mark length identifier" bit in the watermark is set to 0 for 3 seconds order and 1 for 5 seconds order. The other bits (change bits) are switched each time a new mark sequence is referenced (thus reducing the number of watermarks to 10). One mark uses the horizontal or vertical period of the complementary color at a fairly low amplitude on the 8 bottom lines of the four lower left macroblocks. Bits 11 and 12 are used to distinguish the systems designated therein as 1, 2 and 3.

1. The system 1 has analog-to-digital delivery with compatible encoder/programming. In this system, mark removal is achieved by VHS VCRs and some non-applicable MPEG encoders using vertical periodicity. The marker elimination is performed by eliminating the bottom eight lines of the video signal on a video recorder and the bottom sixteen lines on an MPEG video recorder.

For compatible encoders, the flag is inserted after the field-to-frame conversion, the flag is used only for I-frames, the motion vector is forced to zero for the bottom macroblock, the bottom macroblock is forced unused for motion prediction in other macroblocks and intra coding for the flag region macroblock in P/B frames is disabled.

2. System 2 has a digital delivery without compatible encoder/programming. In this system, a horizontal period is used. The removal of the mark in the MPEG domain is performed by removing the chrominance in the marked macroblocks. Some artifacts in the marked areas of the digitally copied content may be visible on some televisions. The security of this system may not be as robust as MPEG copies because an infringer may exploit the fact that 16 lines of chrominance are missing in the marked area on marked frames and only eight lines are missing on non-marked frames.

3. The system 3 has digital delivery in the new network. This system is not backward compatible with system 1 or 2. It nevertheless provides a high quality, low cost solution to enabling copy-once for new networks or networks that will require the use of new set-top devices.

For video input, when a copy-once watermark marker is detected, the recorder removes the marker by removing the chroma in the bottom 8 lines of a frame in the marker area.

For digital input, when a copy-once (system 1 or 2) is detected, the flag detector indicates whether system 1 or 2 is in use. For the system 1, the mark elimination is performed by replacing the bottom piece with a pre-stored piece (16 lines of the bottom video signal are canceled), and padding is performed so as not to affect vbv. For system 2, mark elimination is performed by replacing the bottom chroma-mark data block with a pre-stored data block (eliminating the chroma signal from the bottom left 4 macroblocks).

Upon detection of a copy-once watermark by system 3, a copy is made as long as the P1394 authentication procedure indicates a copy-once.

Video marker (frame/field marker)

The frame or field marker "described above is a video signal or signal modification inserted into a video data stream which is substantially amenable to MPEG2 compression, but which is essentially rejected by (at least) a standard VHS-type VCR, and which may be subsequently read from the video signal for transmission of information. The marker is preferably convenient for MPEG-2 compression so that it may be inserted into the video signal prior to the MPEG encoding, thereby eliminating the need to modify the MPEG encoder. The information intended for transmission includes the copy protection status of the associated video signal; however, other applications may work equally well and the type of marking is limited to copy protection fields. Other applications are authentication or data transfer.

It is possible to use a chroma-inversion tag (chroma-inversion tag) in which successive selected rows have particular chroma patterns inserted or overlapped so that the modulated chroma signals on successive rows are inverted. The purpose is to cancel the signal in a comb filter in a VCR. However, the chroma vertical sub-sampling (subsampling) required by this format in MPEG-2 coding systems requires vertical anti-aliasing filtering. A pattern consists essentially of high vertical frequency transitions in the chrominance signal spatial domain, and the MPEG-2 antialiasing filter is essentially a vertical low pass filter. (indeed, the structure of the comb filter of the VCR is identical in construction to the MPEG-2 anti-aliasing filter, which typically has a more complex, lower cut-off frequency than the comb filter.) as such, the anti-aliasing filter eliminates the inserted chrominance signal pattern, just as the comb filter does; the insertion signal is therefore undesirably lost in all formats of interest.

According to an embodiment, the baseband chrominance signal bandwidth of an MPEG-2 system is approximately 1.4 MHz. Fig. 6A shows a known color spectrum under a VHS format video signal. Also according to an embodiment, the baseband chrominance signal frequency bandwidth of a standard VHS VCR is approximately 300 kHz. There is thus a region (1.1MHz bandwidth) in the baseband chrominance signal domain extending from approximately 300kHz to approximately 1.4MHz, where one of the signals will pass through an MPEG-2 encoding/decoding path and will then be rejected by a standard (consumer) VHS VCR (NTSC or PAL).

It is desirable that the mark signal to be inserted be present completely and solely in this region so that no component thereof passes through the 300kHz chroma signal path of the VCR. The VCR has chrominance signal channel filtering to ensure that no component of the inserted mark signal is inserted into the luminance signal path of the VCR.

One embodiment of the marker signal (see fig. 6B showing a comparison of the spectrum of the currently inserted marker with the spectrum of the color signal below the chrominance channel) is an 844kHz chrominance signal square wave comprising a repeating horizontal pattern of four consecutive CCIR-6014: 2: 0 samples of the green chrominance signal, followed by four consecutive magenta chrominance signal samples. In the video domain, the chrominance signal reverses its phase approximately every two chrominance signal periods (at 3.58MHz) and can be considered as a double sideband suppressed carrier signal with only two spectral components: one at (3.58MHz-844kHz) and one at (3.58MHz +844 kHz). Specifically, there is no spectral component within 300kHz of 3.58 MHz. Fig. 6C shows a comparison of the frequency spectrum of the inserted mark with the frequency spectrum of the MPEG-2 baseband chrominance channel, and fig. 6D shows the modulated chrominance signal waveform of the inserted mark.

Other labeling embodiments are possible. The pattern need not be a rectangular wave, although it is easy to generate; for example, it may be a sine wave. Similarly, the pattern need not be at 844 kHz; any frequency above approximately 300kHz and below approximately 1.4MHz is feasible. The alternating frequency is 1.125MHz, which includes CCIR601 three green samples and three magenta samples. Moreover, the pattern need not be symmetrical, or even repetitive, with a base energy below approximately 1.4MHz without any pattern or signal of content below approximately 300 kHz; the implementation of this option will be simplified

Embodiments are described.

One way of using this signal to transmit information is to insert several marks of different lengths so that they can be measured in the number of I frames and then insert an accompanying video watermark information that indicates the length of the mark. The system requires two matches before one copy can be made.

To minimize the visibility of one such signal, it is desirable to minimize its various components, both horizontal and vertical, and to have a property that tends to disappear visually. Thus, in another embodiment the indicia comprise small areas of dye, which are interlaced both horizontally and vertically, so as to form a "checkerboard" pattern. The colors in one embodiment are green and magenta because they are complementary colors with very similar luminance values, and the eye tends to spatially average them to a uniform gray color; other colors are possible. The stained area should be as small as the spatial resolution capability of the selected format. In the case of MPEG-2 compression, horizontal and vertical antialiasing filters are a constraint, which precedes chroma subsampling as indicated by the format.

The details of this example vary from implementation to implementation. For a standard MPEG-2 application, using a digital anti-aliasing filter with coefficients [ -29, 0, 88, 0, -29 ]. times.256 for horizontal and vertical sub-sampling, an embodiment of the generated marking signal applies the sequence of a fully sampled (i.e., 13.5MHz) pixel comprising a green pixel, two gray pixels, a purple pixel, and two more gray pixels to the designated anti-aliasing filter. This sequence is repeated in both the horizontal and vertical directions to produce the desired "checkerboard" pattern, and continues (i.e., "repeats") in both directions for the desired distance and benefits use over time.

The respective green and violet pixels constitute the pulse function input to the antialiasing filter and the filter output signal is a sequence of pixels corresponding to the filter coefficients, i.e. a single pixel output to the asserted filter will result in a sequence of pixel outputs having an amplitude pattern of [ -29, 0, 88, 138, 88, 0, -29 ]. The overall output of the filter is the sum of the linearity of each of its response input pixels. In the above embodiment, if magenta pixels were encoded as "-1", gray pixels were encoded as "0", and green pixels were encoded as "+ 1", the steady state response of the filter to the above-described marker sequence would be.

A checkerboard pattern is generated by arranging the input signals such that a given pixel in the scan line of any given mark conforms to the indicated spatial pattern. That is, if the pixel 20 in input row 460 is green, then the pixel 20 in rows 461 and 462 should be gray, the pixel 20 in row 463 should be magenta, and the pixel 20 in rows 464 and 465 should be gray. Pixels 21, 22, 24 and 25 in all marked lines will be grey, while all pixels in the marked part of lines 461, 462, 464 and 465 are also grey. The effect of the anti-aliasing filter is to smear the single colored pixels into the desired sine wave in both the horizontal and vertical directions.

This disclosure is intended to be illustrative, and not restrictive; further modifications to the disclosure will be apparent to those skilled in the art from this disclosure which fall within the scope of the claims of the invention.

Claims (45)

1. A method of copy protection of a digital video signal, comprising the steps of:

placing a mark on a selected field or frame of the video signal;

measuring image attributes of each selected frame; and

the video signal is marked with an identification signal comprising a numerical representation of the measured image property.

2. A method as claimed in claim 1, characterised in that the identification signal is a watermark.

3. The method of claim 1, wherein the image attribute is calculated from the magnitude of a predetermined portion of the selected frame.

4. The method of claim 1, wherein the identification signal includes a bit indicating that copying of the video signal is allowed.

5. The method of claim 1, wherein the marker is located in an overscan portion of the frame.

6. The method of claim 1, wherein the marker is located in a blanking period of the frame.

7. The method of claim 1, wherein the marker is a signal filtered by a conventional analog video recorder.

8. The method of claim 1, wherein the identification signal includes a portion that is a signal filtered by a conventional analog video recorder.

9. The method of claim 1, wherein the marker is a repeating chrominance signal wave.

10. The method of claim 9, wherein the marker is a repeating square wave.

11. The method of claim 1, wherein the marker has a frequency between 300KHz and 1.4 MHz.

12. The method of claim 10, wherein the indicia is a checkerboard pattern comprising two complementary colors.

13. A method of playing a protected video signal having a mark in at least one frame and having an associated identification signal comprising a numerical representation of an image attribute of the frame having the mark, the method comprising the steps of:

detecting the mark to identify a particular frame;

detecting an identification signal associated with the identified frame;

measuring image attributes of the identified frames;

comparing the measured image attribute to a numerical representation of the image attribute for the identified frame; and

recording of the video signal is initiated as long as the comparing step indicates a match result.

14. The method of claim 13, wherein the identification signal is a watermark.

15. The method of claim 13, wherein the image attribute is calculated from an amplitude of a selected portion of the identified frame.

16. The method of claim 13, further comprising the step of: it is determined whether the identification signal includes a copy enable bit and copying is initiated only if the copy enable bit is present.

17. The method of claim 13, further comprising the step of: the copy permission bit is changed during recording of the video signal.

18. The method of claim 13, wherein the marker is located in an overscan portion of the frame.

19. The method of claim 13, wherein the marker is located in a blanking period of the frame.

20. The method of claim 13, wherein the tag is a signal filtered by a conventional analog video recorder.

21. The method of claim 13, wherein the identification signal includes a portion that is filtered out by a conventional analog video recorder.

22. The method of claim 13, wherein the marker is a repeating chrominance signal wave.

23. The method of claim 22, wherein the marker is a repeating square wave.

24. The method of claim 13, wherein the marker has a frequency between 300KHz and 13.4 MHz.

25. The method of claim 22, wherein the indicia is a checkerboard pattern comprising two complementary colors.

26. A digital video recorder adapted for a copy-once protocol for video signals, the recorder comprising:

a video recording circuit;

an input for receiving the video signal;

a mark detector that detects a mark in one frame of the received video signal;

a detector for detecting an identification signal associated with the frame;

an attribute measuring circuit for measuring an image attribute of the frame; and

a comparator that compares the measured image attribute with a numerical representation of the image attribute in the identification signal and initiates operation of the video recording circuit only if there is a match.

27. The apparatus of claim 26, wherein the detector further determines whether a copy enable bit is set in the digital signal watermark, and the comparator only enables copying if the copy enable bit is set.

28. The apparatus of claim 27, further comprising a circuit for changing the copy enable bit during recording of the video signal.

29. An encoding device for protecting a video signal, characterized in that the device comprises:

a terminal for receiving the video signal;

a frame selector that randomly or pseudo-randomly selects a particular field or frame for encoding;

a marking circuit for setting a mark on a selected frame of the received video signal;

a measuring circuit for measuring an image attribute of each of the selected frames; and

a circuit for tagging the video signal with a signal comprising a numerical representation of the measured image attribute for each frame having a tag.

30. The apparatus of claim 29, wherein the marker is located in an overscan portion of the frame.

31. The apparatus of claim 29, wherein the marker is located in a blanking period of the frame.

32. The apparatus of claim 29, wherein the marker is a signal filtered by a conventional analog video recorder.

33. The apparatus of claim 29, wherein the signal includes a portion that is a signal filtered by a conventional analog video recorder.

34. Method for enabling copying of a protected video signal comprising marks in at least one frame and having an associated signal comprising a numerical representation of image properties of the at least one frame, characterized in that the method comprises the steps of:

detecting a frame carrying the marker;

marking one of the detected frames having a predetermined signal;

storing data indicative of a timing of each detected frame;

recording the video signal;

playing back the video signal of the video recorder;

detecting the frame marked in said step of marking one of the detected frames having a predetermined signal while reproducing the video signal; and

inserting a marker matching the marker included in the protected video signal in said marked frame and selected subsequent frames in said step of marking one of the detected frames having a predetermined signal, the inserting being based on the stored data.

35. The method of claim 34, wherein the predetermined signal is different from the signature.

36. The method of claim 34, wherein the predetermined signal is a signal that is not removable by the digital video recorder.

37. The method of claim 34, wherein the correlated signal is a watermark.

38. Apparatus for enabling copying of a protected video signal, the protected video signal including marks in at least one frame and having an associated signal including a numerical representation having an image attribute of the frame, the apparatus comprising:

an input for receiving the protected video signal;

a first detector coupled to the input and detecting the frame marker;

circuitry coupled to the detector for writing a predetermined signal in at least one frame including a mark;

a memory coupled to the first detector for storing data indicative of the timing of each of the frames in which the marker is detected;

a second detector that detects the predetermined signal; and

circuitry for inserting a frame marker into the video signal, the frame marker matching a marker present in the protected video signal, the inserting being based on the stored data.

39. The apparatus of claim 38, wherein the predetermined signal is different from the framing mark.

40. The apparatus of claim 38, wherein the predetermined signal is a signal that is not removable by the digital video recorder.

41. The apparatus of claim 38, wherein the correlated signal is a watermark.

42. A method for processing an original video signal to restrict copying of the original video signal, the original video signal having a watermark in which data relating to copy restriction is conveyed, the method comprising:

inserting a videomark in the original video signal, the videomark having an attribute that corresponds in a predetermined manner to a characteristic of the original video signal, wherein the correspondence indicates that the video signal is the original video signal;

detecting the watermark and verifying that the video signal is the original video signal in response to a request to output the video signal for copying, the verifying comprising verifying a predetermined correspondence with the video mark of the video signal, such that only if the video signal is verified to be the original video signal:

processing the original video signal to produce a copy thereof which is not identical to the original video signal such that the videomark no longer corresponds to the original video signal in said predetermined manner;

wherein subsequent verification of the copied video signal indicates that the video signal is not the original signal, thereby enabling limiting further copying.

43. The method of claim 42, wherein the processing includes a change in videomarks.

44. The method of claim 43, wherein the processing includes elimination of videomarks.

45. The method of claim 43 wherein the processing comprises conversion of the original video signal from digital to analog.