HK1142466B - Video scrambling through relocation of pixel blocks - Google Patents

Description

Video scrambling by resetting pixel blocks

Technical Field

The invention relates to image processing, encryption and decryption of digital visual content, in particular to the field of fuzzy protection of visual content by refusing unauthorized access to the visual content. The invention also relates to encoder/decoder applications in the field of image processing.

Background

The entertainment industry includes movies, cable television, and screenplay, among others. Movies have become a major industry in the entertainment industry. The production of a movie requires a large amount of capital, human energy and a great deal of effort. These efforts, funds, and the availability of too many people to make a movie or screenplay can eventually be a premium if copied or obtained by unauthorized people. Similarly, there are some confidential images or dynamic images, which may be the highest confidentiality concerning the nationality, and strong security is required. Therefore, in order to protect such vital visual information from being illegally obtained, it is critical that a method and system for securing digital visual content be available.

Currently, a number of techniques have been used to prevent copying of visual information. For example, image watermarking technology is a method by which some information is inserted into an original image to achieve, for example, copyright protection or image authentication. There are several requirements for insertion mechanisms, such as invisibility, robustness, high information content and fast and reliable detectability, which often conflict with each other.

However, to date, there has been no conventional technique that prevents unauthorized persons from viewing confidential visual information.

Therefore, there is a strong need for a system and method that can prevent unauthorized persons from viewing important and sensitive visual information.

Encryption techniques for television programming, commonly referred to as "scrambling," are used to control access to charge television services, typically cable or satellite television services. Pay television exists to make profit from users, who sometimes do not pay. The prevention of piracy in cable and satellite networks has become a major factor in the development of pay-tv encryption systems.

Early cable-based pay-tv networks did not employ security measures. This leads to the problem that someone has access to the network but is not bothered with paying for it. Therefore, methods are being developed to block these self-entrants. Early cable television pay-tv systems were based on a number of simple measures. Most often a channel-based filter, which effectively prevents the channel from being received by an unsubscriber. These filters may be added or removed depending on whether subscription is present. As the number of television channels on these cable networks grows, filter-based schemes become increasingly impractical.

Since simple filter schemes are easily bypassed, other techniques, such as adding interfering signals to video and audio, have begun to be used. As technology evolves, addressable set-top boxes are commonly employed, and more sophisticated scrambling techniques, such as digital encryption of audio or video clips and rotations (where a piece of video is cut at a particular point and then two parts are reordered around that point), are also applied to the signal.

Encryption techniques are used to protect the satellite distribution of the cable network. Some systems for cable relay distribution are prohibitively expensive. As the DTH market has expanded, a small number of security systems have begun to be used. Many of these systems (e.g., OAK Orion) are variants of cable scrambling systems that affect the synchronization portion of the video, invert the video signal or add interference frequencies to the video. All such analog scrambling techniques are easily broken.

Typically a Video player, i.e. one of the media players, may be used to play digital Video data from a medium such as an optical disc (e.g. DVD, VCD) and digital Video data from a file of a suitable format such as MPEG, AVI, Real Video and QuickTime. Many of these video players also support simple digital audio playback functions, which make the content easily accessible to unauthorized access. Thus, encryption techniques are used to ensure security in such situations. In recent years, the study of cryptography has relied on the confidentiality of digital content. Cryptography is considered as a branch of both mathematics and computer science, and is closely related to information theory, computer security and engineering.

Therefore, there is a need for a system and method to encrypt digital content to ensure security and to make unauthorized access inaccessible.

Disclosure of Invention

To achieve the above objects and advantages, the present invention encrypts visual contents, i.e., still or moving images, to disable access to the visual contents by any unauthorized person.

Advantageously, the invention results in resetting tiles of the same size of the pixels to hide or encrypt the visual information.

The foregoing objects and advantages have been discussed in the specification to further clarify the invention. The scope of the above invention, however, should not be limited or restricted to the above objects or advantages.

According to the invention, the image is divided into n tiles, each tile having a × b pixels. The n a X b pixel tiles are mapped onto a matrix, i.e. positioned on the X and Y axes. The n tiles are then arranged in a new shape, e.g. rectangular, different from the original shape of the image, but with the number or area of the tiles unchanged. Subsequently, the tiles of the pixels are repositioned into this new shape and a matrix is mapped again for the new tile arrangement. A key is then generated that contains information of the encrypted and previous images and is stored with the still or moving image file.

To decrypt the encrypted visual content or the encrypted image, the player reads the key or the encrypted visual content and decrypts the encrypted visual content or the image by the information provided by this key.

Brief description of the drawings

The detailed description is described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like functions and components.

FIG. 1 depicts a hardware representation of a possible integration of software;

FIG. 2 depicts a rectangular spherical expansion image;

FIG. 3 is a schematic diagram of data flow through a fuzzy codec;

FIG. 4a illustrates the encoding process by a fuzzy encoder;

FIG. 4b illustrates the decoding process by the blur decoder;

FIG. 5a defines a configuration of a group of users;

FIG. 5b defines a configuration of groups of users;

FIGS. 6a to 6e show a legend for a method of key generation;

FIG. 7 shows details of an encryption module;

fig. 8 shows details of the inverse cipher module.

Detailed description of the invention

A method and system for protecting digital visual information is described. The system and method are not intended to be limited to any specific form or arrangement, or any specific embodiment, or any specific use, disclosed herein, since the method and system may be modified in various details or relation without departing from the spirit and scope of the claimed invention as hereinbefore shown and described, with the apparatus or method shown here being for purposes of illustration and disclosure of an operative embodiment only, and not in any way to show all the different forms or variations in which the invention may be carried out or operated.

The encoder portion of the present invention can be used as a stand-alone application or integrated into an existing output tool. However, the decoder part of the invention must be integrated into existing applications, most basically a video or multimedia player delivering and displaying visual content to the end user.

In one embodiment, a rectangular spherical unwrapped image or movie is divided into equal-sized tiles of 16 x 16 pixels and then placed in a new configuration with a ratio of 32: 1 according to the method described herein. The advantages of the method are achieved by resetting the image tiles and in different configurations, for example the position of a 16 × 16 pixel tile can be changed after every 100 16 × 16 pixel tiles.

The image in fig. 2 represents a rectangular spherical expansion image (in pixels) with 360 degrees horizontal information and 180 degrees vertical information. The aspect ratio of this image is 2: 1, that is to say the width is twice the height.

In fig. 3, the source data in 6 is encoded by a blur encoder in 7 which sends out a multimedia file or video stream in 8. The video stream is then sent to a blur decoder in 9, which decodes the information and sends it to a video renderer in 10.

Fig. 4a depicts an encoding process. The source data in 20 may be a video file, a video stream or a still image. The data must be a multimedia file that can be understood at least by an API similar to the DirectShow filter in 25. This source data is described by resolution information (in pixel area), compression type, frame rate (FPS), and color depth. Here, the sound information is not encrypted and is not processed by the blur encoder. The raw sound data 26 is then passed directly from step 25 to step 90 where the encoded file is written. The synchronization information is saved in 25 (same API as 90) and used to write the encoded data in 90.

The source data is loaded into the fuzzy encoder through the frame extraction reader in 30, since the preferred procedure is to compute each frame one by one, thereby preventing data loss. The module reads the incoming data and extracts each frame individually one by one. This process is well known and is handled first by an API from microsoft named DirectShow filter 25. Each frame must be scanned line by line to prevent compression anomalies.

DirectShow divides the processing of multimedia tasks such as video playback into a series of steps called filters. Each filter represents a stage of the data processing process. The filter has a number of input and output pins connecting them together.

The frame extracted in 30 is then parsed by a tile reader in 40 where the entire image expressed in pixels is converted into tiles of 16 x 16 pixels (in order to fit the macroblock size fixed by existing compression codecs for AVI containers and MPEG formats). These tile locations are then saved for scrambling to new locations by the encryption module in 60.

The encryption module in 60 generates a key that provides a new location for each tile extracted in 40. The key may be generated using many well known existing ways, such as Keygen. The key may also be generated manually using empirical methods. The encryption key is required to reformat the encoded data 100 at 140. The key is generated at the encoder device and can be provided to the decoder using all existing data transmission methods. The key may also be embedded in the decoder as an electronic device. Obviously, the key must be delivered to the decoder in a manner independent of the encoded data to maintain a level of security. The key may also be encrypted using a known encryption algorithm, or may be transmitted as a text file, depending on the level of security desired.

The resulting encoded data 100 and the source data 20 have exactly the same number of tiles (macroblocks).

The complete frame is then generated in the macroblock multiplexer (mux) in 50 using the new tile location generated from the source data retrieved from 40 using the key in 60. Once this is done, the frame is written by the frame writer at 70 and stored in the multi-frame buffer at 80.

The function of the multi-frame buffer 80 is to store the required number of frames to compute the final compression algorithm by the DirectShow filter 90. The number of frames stored before compression depends on the compression key frames required by the compression filter. It also indexes the location in the file where each tile location changes with the key. It is also possible to establish the key only at the beginning of the file, but it is also possible to change with a certain frequency, interleaved with the existing frames.

This section completes the encryption processing contained in the encoder section. The encoder portion and the decoder portion are typically not running on the same device. If the encoded data in 100 is stream data rather than a recorded file, the two components may be linked together by known methods.

Fig. 4b shows the decoding process. When the encoded data is transmitted or stored to another device for playback, the data is processed through the DirectShow filter in 110.

The DirectShow filter is then directly connected to the tile reader in 120, which formats the tile information for a macroblock splitter (demultiplexer) in 130, where all the tiles are reorganized using the key imported to the inverse encryption module in 140. As mentioned before, the key can easily be hard-written into the device as ROM data. Here, all tiles are reorganized as the original source data (in the uncompressed case).

The reorganized data is sent to a DirectShow filter in 150 that uses existing functions to generate the decoded data in 160.

The data cannot be saved as a multimedia file using different filters to prevent unauthorized users from obtaining unencrypted data.

The DirectShow filter in 110 separates the audio/video information because the audio does not have to be processed through the blur decoder portion and passed directly to the output portion of the DirectShow filter in 150.

The DirectShow filter in 150 concatenates the two components of the multimedia file and synchronizes them together again. The video portion, referred to herein as decoded data in 160, is sent to a video renderer 170, which video renderer 170 manages image information for the display device. The audio portion is sent to an audio renderer in 180, which manages the audio device.

Fig. 5a shows the case where a single key is provided and used. All users of the end user group a using the player application are allowed to use and display the content encoded by the encoding platform 200. The only limitation is that they can display the content without allowing it to be modified.

Fig. 5b shows a situation where there are multiple user groups.

Each key must be assigned to a group of content for encryption and to a group of end users for display.

Each key may be generated and encrypted on the same encoding platform 200 or may be encoded and used on multiple encoding platforms.

Suppose we have N different groups of users. Each group having a different encryption key. Key a is dedicated to group a, key B is dedicated to group B, and key N is dedicated to group N. There are also no restrictions between different keys, and between different groups.

It should often be the case that group a possesses unique permissions to access media encrypted with key a, group B possesses unique permissions to access media encrypted with key B, and so on.

Fig. 6a to 6e merely provide examples. In this example, we consider that the source data has been extracted and is a progressive rectangular sphere-expanded frame, having a width of 1024 pixels and a height of 512 pixels. The process is integrated into a loop, and each frame must be processed one by one before compression.

Fig. 6a shows a 1024 × 512 pixel rectangular spherical expansion image, i.e. 2048 blocks of 16 × 16 pixels. The frame represents a representative image that can be extracted by the frame extraction reader 30.

Fig. 6b is a graphical representation of the tile extraction function running in the tile reader 40. It can be seen that the image has a length including 64 blocks of 16 × 16 pixels and a width including 32 blocks of 16 × 16 pixels.

Fig. 6c is a matrix formed to represent the positioning of each tile in the X and Y axes. Thus, each tile of 16X 16 pixels is assigned a coordinate of the form (X, Y) by virtue of the matrix.

Fig. 6d shows the new matrix after the encryption process 320 shown in fig. 6e and 7.

In fig. 6e a new shape is created with the same number of tile accommodation, i.e. in this particular embodiment the new shape contains a volume of 2048 tiles, as the original unencrypted image shape. The new shape in which the tile is repositioned is a rectangular shape. However, the area occupied by the tiles remains unchanged. In the embodiment herein, the new pattern is a rectangle with 256 tiles in length and 8 tiles in width. Thus, here the ratio of the number of tiles along the length to the number of tiles along the width is 32: 1. In step 4, the tiles are repositioned into the new shape of the rectangle by using a blurring algorithm to reposition the tiles in a particular order.

Fig. 7 is a detailed view of the encryption module 60 of fig. 4a, where the raw matrix locations 300 receive information about the tiles through the tile reader 40. The information collected here is the number of tiles and their original positions in the matrix. This information is described as a matrix. The matrix variables are then scrambled in the tile position transformer 310 and recombined in a new matrix position 320 to a new matrix. The manner in which these variable positions are scrambled may be managed manually by the end user who manages the encoding process. Or may be imported as a text file conforming to the correct formatting. The new matrix position information is then sent to the key generator 330 and the macroblock multiplexer 50. The key generator 330 generates a key based on the new matrix position information using an existing well-known method. This key source is then stored as a metadata key 65 for later use on the player device.

Fig. 8 is a detailed view of the inverse cipher module 140 of fig. 4 b. The key decoder 400 receives the metadata key 65, where the process recreates the original matrix information relative to the tile position in the frame. The tile location changer 410 receives the tile location and key decoder information delivered from the tile reader 120. This provides the original matrix location to the original matrix location module in 420. This information is then sent to the macroblock splitter in 130 to generate the original frame to be managed by the DirectShow filter in 150. The key information is not limited by the location of the tiles within the frame.

Detailed Description

The above-described codec system (encoder, decoder) combines two independent parts: an encoder to encode the original video file or stream with the key, a decoder to decode the encrypted data using the original key information, delivering the video file restored to the original form to a video renderer for display.

These two separate parts are two separate processes operating on two different computers or CPU systems.

If the key is singular, the key can only provide information to a group of users who have access to the unique information.

The key may have an unlimited time life cycle or may be time limited. For example, if the media is a pay-per-view program, a key may be generated that only allows the end user or group of end users to play the video for a limited time. If the viewing time limit has expired, the end-user is no longer allowed to play the content and the player application may display a warning message.

The size of the source data width and height of a frame must be a multiple of 16 pixels. This is mostly the case for all videos that have been encoded using compression algorithms that satisfy this constraint.

The encoded data and the source data have the same number of pixels, i.e. the same number of tiles. The aspect ratio parameter of the encoded data may be the same as the source data, but is not required. To make the behavior of users who want to access the source data more difficult to achieve, we modify the aspect ratio of the encoded data (video). The pattern shape, the tiles and the number of pixels remain unchanged (rectangular), but the width and length dimensions are different.

The aspect ratio of a two-dimensional shape is the ratio of its longer dimension to its shorter dimension. This may also apply to two characteristic dimensions of a three-dimensional shape, in particular for the longest and shortest "axes", or to symmetrical objects (e.g. sticks) which are described by only two measures (e.g. length and diameter). In this case, the aspect ratio may be a value less than 1 (e.g., consider very short and very long sticks).

The preferred application of the present invention is considered to protect the image content of rectangular spherical unwrapped images or video files and video streams, but is also applicable to all other image or video content. Rectangular spherical expansion forms are well known in the computer graphics arts.

Most importantly, the image data to be processed by the encoder must be in an uncompressed format. Compressed files can also be processed as well, but the end result will not be good enough to meet the needs of different fields of application, including the entertainment industry.

Video compression typically operates on square, contiguous groups of pixels, commonly called macroblocks.

The data may be one or more still images or one or more video files. The final processed file or files output from the encoder can then be saved or streamed directly to the decoder in two different ways:

for raw video compressed using standard video. The AVI container (audio video interleaved) format or the new MPEG-4 format allows all types of compressed files. In this case, the key is processed and transmitted independently in the form of metadata (in case it is not directly embedded in the decoder),

for a metadata file that includes video and data. The use of metadata is to facilitate understanding, using, and managing data. The metadata required for efficient data management varies with the type of data and usage environment. In the context of information systems, data is the content of a computer file, and typical metadata about an individual data item typically contains a field name and length.

Each frame of video must be processed one by one at the bottom and the image must be in a progressive scan format (non-interlaced) containing 24 bits, 32 bits or an indexed color pattern.

The invention is not intended to be limited to any particular form or arrangement, or embodiment, or any particular use, disclosed herein, since the invention may be modified in various details and relation without departing from the spirit and scope of the claimed invention as hereinbefore shown and described, and that the apparatus or method shown herein is intended only to illustrate and disclose operative embodiments, and not to show all the various forms or variations in which the invention may be carried out or operated.

Claims (27)

1. A method of encoding visual information, comprising the steps of:

-segmenting the visual information into a predetermined number of tiles on a matrix;

-determining the position of each tile on the matrix;

-rearranging the tiles on the matrix into a predetermined shape on the matrix using a blurring algorithm; and

-generating a key for decoding the encoded visual information, the key recording a correspondence of the positions of the tiles before and after the rearrangement of the image;

wherein the predetermined shape is different from an original shape of the visual information, and an area of the predetermined shape is equal to an area of the original visual information.

2. The method of claim 1, wherein the visual information is extracted from the multimedia data containing audio information on a frame-by-frame basis by filtering and transmitting the audio without encoding.

3. The method of claim 2, wherein each frame is progressive to prevent compression anomalies.

4. An apparatus for encoding video information, comprising:

-filtering means (25) for processing the input information;

-image processing means, coupled to said filtering means, configured to extract and process information in the visual information frame by frame;

-a tile reader (40) for storing and translating each extracted frame into a predetermined number of tiles; and

-an encryption device (60) coupled to the image processing device to rearrange the tiles into new positions and to generate a symmetric key for decoding the encoded visual information, the symmetric key recording a correspondence of the positions of the tiles before and after the rearrangement of the image;

wherein the encryption device (60) is configured to rearrange the tiles into a predetermined shape, the predetermined shape being different from the original shape of the visual information.

5. The apparatus of claim 4, wherein the image processing device is further configured to:

-generating a complete frame using the generated new position;

-saving and writing a predetermined number of generated frames required for final compression; and

-encrypting the visual information to obtain the encoded information.

6. The device of claim 4, wherein the visual information (20) is a video file or a video stream.

7. The device of claim 4, wherein the visual information (20) is a static image.

8. Device according to claim 4, wherein said visual information (20) is a multimedia file which can be understood at least by an API similar to the filtering means (25) described above.

9. The apparatus of claim 4, wherein the tile format matches a macroblock size fixed by an existing compression codec for AVI containers and MPEG formats or any other video format.

10. The apparatus of claim 6 wherein said encryption means (60) generates a symmetric key (65), said symmetric key (65) providing a new location for each tile extracted by said tile reader (40).

11. A device as claimed in claim 4, wherein said key (65) is encrypted before transmission to the decoder device.

12. A device as claimed in claim 4, wherein said key (65) is embedded in the decoder device in the form of an electronic device.

13. The device of claim 4, wherein said key (65) is stored in ROM data.

14. The apparatus of claim 6, wherein said encoded information (100) has exactly the same number of tiles as said visual information (20).

15. Apparatus according to claim 6, wherein said filtering means (25) removes audio information from the input information.

16. A method of decoding encoded visual information, the method comprising the steps of:

-receiving encoded data comprising encoded visual information divided into a predetermined number of tiles;

-processing said data through a series of filters

-separating audio and video information

-transmitting said audio data directly as output without decoding

-importing a key recording a correspondence of the positions of the tiles before and after the rearrangement of the image;

reorganizing the blocks using the key to reconstruct the original source data for the visual information

-generating decoded data

-concatenating and resynchronizing together the video and audio components of a file

-sending the video part of the data to a video renderer

-sending the audio part of the data to an audio renderer

Wherein the tiles forming the encoded visual information are arranged in a predetermined shape, the predetermined shape being different from the shape of the original visual information and the predetermined shape having an area equal to the area of the original visual information.

17. An apparatus for decoding encoded visual information, the apparatus comprising:

-means for receiving encoded data comprising encoded visual information divided into a predetermined number of tiles;

-a filter (110) for processing the encoded data (100) in a series of steps

-a tile reader (120) for analyzing and formatting tile information in said data (100) into a predetermined tile format

-a macroblock splitter (130) for reorganizing the tile information into a tile organization structure of the original source data using a key that records the correspondence of the tile positions before and after the image rearrangement;

-a reverse encryption module (140) for providing a key for reorganizing said tile information

-a filter (150) for generating decoded data (160);

wherein the macroblock splitter (130) is configured to reorganize tiles forming the encoded visual information from a predetermined shape to a shape of the original visual information, wherein the predetermined shape is different from the shape of the original visual information.

18. The apparatus of claim 17, wherein said decoded data (160) cannot be saved as a multimedia file, thereby preventing unauthorized access to unencrypted data.

19. The apparatus of claim 17, wherein said filter (110) separates audio and video information.

20. The apparatus of claim 17, wherein said decoded data (160) corresponds to a video portion of an incoming multimedia file.

21. The apparatus of claim 20, wherein said video data is sent to a video renderer (170).

22. The apparatus of claim 21, wherein said video renderer (170) manages graphics information for a display device.

23. The apparatus of claim 19, wherein the audio information is sent directly from said filter (110) to an audio renderer (180).

24. The apparatus of claim 23, wherein said audio renderer (180) manages an audio device.

25. The apparatus of claim 19, wherein the audio information is not processed and is passed directly from the first filter (110) to the last filter (160).

26. The apparatus of claim 17, wherein said filter (150) connects and synchronizes audio and video components of the multimedia file.

27. A codec for encoding and decoding visual information, comprising:

-filtering means (25) for processing the input information;

-image processing means, coupled to said filtering means, configured to extract and process information in the visual information frame by frame;

-a tile reader (40) for storing and translating each extracted frame into a tile of predetermined data; and

-an encryption device (60), coupled to said image processing device, to rearrange the tiles into new positions and to generate a symmetric key for decoding the encoded visual information, said symmetric key recording the correspondence of the positions of said tiles before and after the rearrangement of the image; and

-a decoder for decoding the encoded information;

wherein the encryption device is configured to rearrange the tiles into a predetermined shape that is different from an original shape of the visual information.