JP2001268569A - Method and apparatus for encoding residual coefficients of arbitrarily shaped objects - Google Patents

Abstract

A fine-grain scalability method in MPEG-4 is mainly intended for Simple and Core visual profile applications. One of the features of the Core visual profile is the encoding of arbitrary-shaped objects. Currently, the FGS approach only supports efficient encoding of rectangular objects in the enhancement layer. However, current FGS coding techniques are not very efficient for coding arbitrary shaped objects in the enhancement layer. A method for efficiently encoding residual coefficients from arbitrarily shaped objects is described herein. By skipping transparent blocks and using shorter variable length codes to represent the coded block pattern for a macroblock, the present invention improves the coding efficiency of the enhancement layer of arbitrarily shaped objects. This is extremely effective.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video encoding / decoding algorithm, such as MPEG-4, in which content is encoded and transmitted once by a server which adjusts the overall bit rate according to the requirements of a client and a network. It can be used in the fine-grained video scalability method included in.

[0002]

2. Description of the Related Art In recent years, MPEG-4 has been recently developed by MPEG (Moti
on Picture Experts Gro
up) according to the ISO / IEC standard. M
One of the notable features of PEG-4 is the encoding of arbitrarily shaped objects. MPEG-4 has several visual profiles. One of these profiles is the Core visual profile, which supports encoding arbitrary shaped, temporally scalable, rectangular video objects. This is useful for applications that provide relatively simple content interactivity (Internet multimedia applications).

[0003] Version 4 of the MPEG-4 standard, fine-grained video scalability, is primarily intended for video networking applications. This version provides improved video quality at variable bit rates depending on channel capacity. In the FGS method,
MPEG-4 video can be encoded into two different layers, one base layer and one enhancement layer.
The base layer information itself is MPEG-4 video that can be self-decoded to produce a video of reasonable quality. On the other hand, the enhancement layer contains additional information about the base layer video. The combination of these two layers results in better quality video than video with only the base layer.

FIG. 1 shows a simplified video decoding process in MPEG-4. To decode an arbitrary shape, the video bitstream can be demultiplexed to provide three types of encoded bitstream information. FIG.
They are shape, motion and texture bitstreams as shown in FIG. video_object_l
The layer_shape flag 105 is obtained from the header of the bit stream and is used to determine whether or not the shape information exists in the bit stream. If shape information is present, the shape bitstream is decoded in module (or block) 107 to obtain shape information for the VOP. The shape information is then passed to module 109 for texture decoding of the texture bitstream,
And used for motion compensation. For arbitrary shape texture decoding, if the shape information of a block indicates that the current block is not a transparent block, the texture bitstream is variable length decoded, inverse scanned, inverse DC / AC predicted, inverse quantized. And the pixel values of the block are obtained via the respective modules 111, 112, 113, 114 and 115 of the IDCT. The arbitrarily shaped VOP is then reconstructed in module 116.

[0005] Fig. 2 shows a prior art of an encoding process of an FGS enhancement layer for a rectangular object. The quantized DCT coefficients from the base layer are
And the module 202 subtracts the unquantized DCT values to obtain the residual coefficients for the enhancement layer. The VOP residual coefficients are then encoded using bitplane variable length encoding in module 203 to obtain an enhancement layer bitstream.

FIG. 3 shows a prior art of a decoding process of an enhancement layer bit stream. F received by the decoder
The GS enhancement layer bit stream is subjected to bit plane variable length decoding in module 301 to obtain residual coefficients for the enhancement layer. The reconstructed residual coefficients are then reconstructed in module 302 from the reconstructed D
This is added to the CT coefficient to obtain an extended DCT coefficient. The extended DCT coefficients are then converted to the spatial domain using IDCT in module 303 to obtain VOP pixel values.
For macroblocks that are non-intra-coded in the base layer, the motion compensation output from the base layer is added to the output of the IDCT in module 304 to obtain the extended pixel value of the VOP.

[0007]

In the current FGS method, the shape of the VOP in the enhancement layer is rectangular. This means that if the VOP in the base layer is an arbitrarily shaped object, the VOP in the enhancement layer must encode zeros in areas within the rectangular area not occupied by the arbitrarily shaped object. This is very inefficient in terms of coding efficiency, since later when the VOP is reconstructed, the outer area of the arbitrarily shaped object is filled with zeros.

[0008] Thus, the intent of the present disclosure is to solve the coding inefficiencies in the prior art. The problem to be solved is how to avoid coding of residual coefficients existing outside the arbitrary-shaped object in the enhancement layer.

[0009]

SUMMARY OF THE INVENTION In the current MPEG-4 standard, decoding of arbitrarily shaped objects involves the decoding of binary alpha blocks (babs). bab
Is a square block of binary pixels representing whether pixels in a designated block-shaped space area of size 16 × 16 pixels are opaque or transparent.
Thus, if a block is identified by its bab as being transparent, no texture information for this particular block is encoded.

Thus, the problem with the prior art that the encoding of residual coefficients for arbitrary shaped objects is inefficient is to use binary alpha blocks from the base layer to identify blocks that are VOP transparent in the enhancement layer,
This can be solved by avoiding coding of the residual coefficients for these blocks.

In the current state of the art, the enhancement layer VOP
The total number of blocks encoded in
It is based on the total number of blocks that can exist within the rectangular area of the VOP defined by the width of the OP. What is novel in the present invention is that not all blocks present in the VOP of the enhancement layer are coded for arbitrary-shaped objects. The present invention utilizes a binary alpha block from the base layer to determine whether the block is transparent and skips coding of residual coefficients for those blocks in the enhancement layer. The present invention improves the coding efficiency of the enhancement layer for arbitrarily shaped objects.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In the process of decoding an arbitrarily shaped object, a binary alpha block is decoded by a base layer decoder. Table 1 shows seven types of bab types existing in MPEG-4.

[0013]

[Table 1] Table 1: bab types

In the case of the transparent bab type, it means that the texture macroblock coexisting with the bab is a transparent macroblock. In this way, the pixel values of the macroblock need not be encoded at all in the base layer. Similarly, no residual coefficient values for these macroblocks are needed in the enhancement layer, and thus coding of these macroblocks in the enhancement layer should be avoided.

A preferred embodiment of the present invention will be described with reference to FIGS. FIG. 4 shows a block diagram of the encoder of the present invention. The shaded modules 403, 404 are new additions to the existing prior art to form the present invention. The quantized DCT coefficients from the base layer go through an inverse quantization process of a module 401, and a residual coefficient is calculated in a module 402. For an arbitrary shaped object, a binary alpha block of the current VOP is obtained from the base layer. The contents of these binary alpha blocks are then analyzed in module 403 to determine whether the binary alpha blocks are completely transparent, partially opaque, or completely opaque. Based on whether the binary alpha block is opaque or transparent, a determination is made in module 404 as to whether or not to encode the texture block in the enhancement layer. The residual coefficients of those opaque blocks are then
5 using bitplane variable length coding.

FIG. 5 shows a block diagram of the decoder of the present invention. Shaded modules 501, 502, 507
Is a newly added part of the existing prior art to form the present invention. In the decoding process at the enhancement layer for arbitrarily shaped objects, binary alpha blocks are first extracted from the base layer decoder. The contents of these binary alpha blocks are then stored in module 501
To determine whether the corresponding block in the enhancement layer is transparent. Then, based on whether the binary alpha block is transparent or opaque, the module 5 determines whether to decode the block of the enhancement layer or not, based on the module 5.
02. The residual coefficients of the non-transparent blocks are then reconstructed in module 503 and the extended DCT coefficients are calculated in module 504. The extended DCT coefficients are then transformed to the spatial domain in module 505, and if the corresponding macroblock in the base layer is non-intra-coded,
Module 506 adds the motion compensation output from the base layer. Finally, the binary alpha block is used in module 507 to construct an extended arbitrary shape VOP.

A macroblock in the enhancement layer is skipped if the type of bab coexisting with the corresponding macroblock in the base layer is transparent. The reason is that the contents of this macroblock are filled with zeros while the VOP is being reconstructed on the decoder side, so that it is not so important. Therefore, in a macroblock skipped in the enhancement layer, encoding of any information on the contents of the macroblock is avoided.

The contents of the binary alpha block are binary values indicating whether the pixels of the block of size 16 × 16 are opaque / transparent. In the case of a non-transparent bab type, not all pixels of a block having a size of 16 × 16 are necessarily opaque. FIG. 6 shows IntraCAE.
An example of a bab having a type is shown. A binary value of 1 indicates that the pixel is opaque, and a binary value of 0 indicates that the pixel is transparent. As shown in FIG. 6, not all pixels of a block of size 16 × 16 are opaque. The contents of the 8 × 8 block existing in the upper left corner of the 16 × 16 block are all transparent. This means that encoding any information about the contents of a block of size 8 × 8 is inefficient in compression.

In the 4: 2: 0 YUV format, a 16 × 16 block of pixels in a VOP has four 8 × 8 blocks of Y, one 8 × 8 block of U, and one of V Are represented by 8 × 8 blocks. Similarly, a bab of size 16 × 16 can also be divided into four 8 × 8 blocks of binary values.
If the contents of any one of the 8.times.8 blocks in bab are all zero, the corresponding block of luminance components is not encoded in the base layer. Thus, within the enhancement layer, this block of intensity can also be skipped, and its content will not be encoded at all.

As described above, by using the bab information from the base layer, the 8 ×
If the binary values of the eight blocks are all zero, the luminance block included in the macroblock in the enhancement layer is skipped.

In the current FGS syntax, the coded block pattern of a macroblock in the enhancement layer assumes that all four blocks of luminance in the macroblock are not transparent. However, if the object is an arbitrarily-shaped object, there is a macroblock in which all four blocks of luminance in the macroblock are not necessarily transparent. In these cases, different combinations of coded block patterns should be used for these macroblocks to improve coding efficiency.

In the present invention, three additional tables of encoded block patterns are generated for all tables of encoded block patterns present in the current FGS syntax. The three additional tables generated for all existing tables are: if the macroblock has only one non-transparent block, if the macroblock has only two non-transparent blocks, and if the macroblock has only one non-transparent block. This is for the case of having three non-transparent blocks. In the present invention,
4 of the block pattern coded for the enhancement layer
The determination as to whether to use one of the tables is based on the number of non-transparent blocks existing in the macroblock.

Table 2 shows one of the tables of encoded block patterns in the current FGS syntax. The main table of the encoded block pattern is 3
Used for macroblocks of the first bitplane if the number of color components has an equal number of bitplanes.

[0024]

[Table 2] Table 2: Encoded block patterns for four non-transparent blocks

An example of three additional tables generated for the above table is shown below. Tables 3, 4 and 5 are additional tables when a macroblock has three non-transparent blocks, when a macroblock has two non-transparent blocks, and when a macroblock has only one non-transparent block. It is.

[0026]

[Table 3] Table 3: Encoded block patterns for three non-transparent blocks

[0027]

[Table 4] Table 4: Encoded block patterns for two non-transparent blocks

[0028]

[Table 5] Table 5: Encoded block pattern for one non-transparent block

By using different combinations of coded block patterns for macroblocks without four non-transparent blocks, the coded block patterns of these macroblocks are coded. Since fewer bits are required, the coding efficiency of these macroblocks is improved. Table 7, some other examples of shorter variable length codes based on the transparency of the luminance block.
8, 9, 11, 12, 13, 15, 16 are shown. Table 6,
Reference numerals 10 and 14 are existing tables of the current FGS syntax.

[0030]

[Table 6] Table 6: Encoded block patterns for 4 non-transparent blocks

[0031]

[Table 7] Table 7: Encoded block patterns for three non-transparent blocks

[0032]

[Table 8] Table 8: Encoded block patterns for two non-transparent blocks

[0033]

[Table 9] Table 9: Encoded block pattern for one non-transparent block

[0034]

[Table 10] Table 10: Encoded block patterns for 4 non-transparent blocks

[0035]

[Table 11] Table 11: Encoded block patterns for three non-transparent blocks

[0036]

[Table 12] Table 12: Encoded block patterns for two non-transparent blocks

[0037]

[Table 13] Table 13: Encoded block pattern for one non-transparent block

[0038]

[Table 14] Table 14: Encoded block patterns for 4 non-transparent blocks

[0039]

[Table 15] Table 15: Encoded block patterns for three non-transparent blocks

[0040]

[Table 16] Table 16: Encoded block patterns for three non-transparent blocks and two non-transparent blocks

[0041]

The present invention is extremely effective for coding an enhancement layer for an arbitrary-shaped object in the fine-grain scalability method, and therefore, the coding efficiency of the enhancement layer is improved.

[Brief description of the drawings]

FIG. 1 is a simplified block diagram of a base layer decoder

FIG. 2 is a block diagram of a conventional encoder.

FIG. 3 is a block diagram of a prior art decoder;

FIG. 4 is a block diagram of an encoder according to the embodiment of the present invention.

FIG. 5 is a block diagram of a decoder according to the embodiment of the present invention.

FIG. 6 is an explanatory diagram of an example of a block skipped based on the contents of a binary alpha block.

[Explanation of symbols]

101 Demultiplexer, 102 Encoded bit stream (shape), 103 Encoded bit stream (operation), 104 Encoded bit stream (texture), 105 Video object layer shape, 106 107 Shape decoding, 108 Motion decoding, 109 motion compensation, 110 previously reconstructed VOP, 111 variable length decoding, 112 inverse scan, 113 inverse DC & AC prediction, 114 inverse quantization, 115 IDCT, 116 VOP reconstruction, 201 inverse quantization, 202 203 bit plane VLC, 301 bit plane VLD, 302 303 IDCT, 304 401 inverse quantization, 402 403 determine whether block is transparent or opaque, 404 405 bit plane VLC, 501 block is transparent or opaque Determines, 502,503 bitplane VLD, 504 505 IDCT, 506 507 configuration

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Tio Ken Tan Singapore 534415 Singapore, Thai Sen Avenue, Block 1022, 04-3530, Thai Sen Industrial Estate, Panasonic Singapore Research Institute, Inc. F term (reference) 5C059 KK08 MA00 MA47 MB02 MB04 MB14 MC21 MC36 ME01 NN01 NN21 PP04 PP21 SS06 TA50 TB07 TC43 TD09 UA02 UA05 5J064 AA02 BA09 BB03 BC08 BC16 BC25 BD02

Claims (24)

[Claims] 1. A method for encoding a residual coefficient of an arbitrarily shaped object, comprising: extracting a binary alpha block from a video sequence of the arbitrarily shaped object encoded in a base layer encoder; Calculating the residual coefficients of the video sequence from: dividing the residual coefficients into a plurality of blocks; grouping the blocks of the residual coefficients into macroblocks; analyzing the contents of the binary alpha blocks Determining the transparency of the residual block, and encoding the coded block pattern of the macroblock if the macroblock is not a skipped macroblock; Blocks where blocks were skipped Encoding the block of residual coefficients by bit-plane entropy coding if not present, and representing the encoded information in an encoded representation. . 2. A method for decoding an encoded representation of a residual coefficient of an arbitrary shape object, comprising: extracting a binary alpha block from a base layer decoder; and analyzing a content of the binary alpha block. Determining the transparency of the residual block by: decoding the coded block pattern of the macroblock if the macroblock is not a skipped macroblock; and If not, performing inverse bit-plane entropy coding on the coded information to reconstruct the block of residual coefficients; and searching for a block of DCT coefficients reconstructed from the base layer decoder; The block of the reconstructed residual coefficients is Obtaining a block of the extended DCT coefficient by adding the block of the DCT coefficient to the block of the extended DCT coefficient; and transforming the block of the extended DCT coefficient from the frequency domain to the spatial domain by inverse discrete cosine transform to obtain a block of the reconstructed spatial difference value. Obtaining a prediction mode from the base layer decoder; extracting a block of a prediction output from an operation compensation process of the base layer decoder based on the prediction mode; Adding a block to the block of spatial difference values to obtain a block of reconstructed pixel values; embedding (padding) the block of reconstructed pixel values with the binary alpha block to obtain an arbitrary shape Reconstructing the VOP. 3. A method for analyzing the contents of the binary alpha block, comprising: dividing the binary alpha block into four smaller blocks (small blocks) having a size of 8 × 8 pixels; 3. The method according to claim 1, further comprising: determining whether all pixels of the small block are zero; and determining the number of the small blocks including only pixels having a value of zero. 4. the method of. 4. A method for encoding residual coefficients of an arbitrary-shaped object, wherein the residual macroblock has a corresponding binary alpha block in a base layer. the method of. 5. A method for decoding an encoded representation of a residual coefficient of an arbitrarily shaped object, wherein the residual macroblock has a corresponding binary alpha block in a base layer. Item 4. The method according to item 2 or 3. 6. The method of determining a skipped macroblock, wherein the skipped macroblock is defined as a residual macroblock having a zero value for all pixels of the corresponding binary alpha block. The method according to any one of claims 1, 2, 3, 4 or 5, characterized in that: 7. A method for determining a skipped macroblock, wherein the skipped macroblock has a zero value for all pixels in the small block of the corresponding binary alpha block. The method according to claim 1, wherein the method is defined as a luminance block of blocks. 8. A method for encoding a coded (coded) block pattern of the macroblock, comprising: selecting an appropriate table of coded block patterns to be used; 2. The method according to claim 1, comprising: obtaining an encoded block pattern of the following; and expressing the encoded block pattern by a variable length code based on the selected table. . 9. A method for decoding the coded block pattern, comprising: selecting an appropriate table of coded block patterns to be used; and representing the coded block pattern. The method of claim 2, comprising: retrieving a variable length code; and decoding the variable length code based on the selected table to obtain the encoded block pattern. 10. A method for encoding and decoding residual coefficients of an arbitrarily shaped object, wherein different tables of encoded block patterns are selected for residual macroblocks having different numbers of non-skipped blocks. The method according to claim 1, wherein the method is characterized in that: 11. A method for encoding and decoding residual coefficients of an arbitrarily shaped object, wherein the binary alpha block is such that pixels in a specific spatial region having a size of 16 × 16 pixels are transparent or non-transparent. The method according to claim 1, wherein the block is a square block of binary pixels indicating whether it is transparent. 12. A method for encoding and decoding residual coefficients of an arbitrary-shaped object, wherein the pixel in the spatial domain is transparent if the binary value of the binary alpha block is zero, 3. The pixel of claim 1, wherein the pixel in the spatial domain is opaque if the binary value of the value alpha block is one.
The method according to any one of 3, 4, 5, 6, 7 or 11. 13. An apparatus for encoding residual coefficients of an arbitrarily shaped object, comprising: means for extracting a binary alpha block from a video sequence of an arbitrarily shaped object encoded in a base layer encoder; Means for calculating a residual coefficient of the video sequence from a plurality of blocks; means for dividing the residual coefficient into a plurality of blocks; means for grouping the blocks of the residual coefficient into macroblocks; and analyzing the contents of the binary alpha block. Means for determining the transparency of the residual block, and means for coding the coded block pattern of the macroblock if the macroblock is not a skipped macroblock; If the block is not a skipped block, Means for encoding the blocks of said residual coefficients by chromatography emissions entropy coding apparatus characterized in that it comprises a means of expressing the encoded information encoded representation (coding). 14. An apparatus for decoding an encoded representation of a residual coefficient of an arbitrary shape object, comprising: means for extracting a binary alpha block from a base layer decoder; and analyzing the contents of the binary alpha block. Means for determining the transparency of the residual block, and means for decoding a coded block pattern of the macroblock if the macroblock is not a skipped macroblock; and a block in which the block is skipped. Means for performing inverse bit-plane entropy coding on the coded information to reconstruct the block of residual coefficients if not, means for retrieving a block of DCT coefficients reconstructed from the base layer decoder, The block of the reconstructed residual coefficients is divided into blocks of the DCT coefficients. Means for obtaining a block of the extended DCT coefficient by summing the block and a means for converting the block of the extended DCT coefficient from the frequency domain to the spatial domain by inverse discrete cosine transform to obtain a block of reconstructed spatial difference values. Means for determining a prediction mode from the base layer decoder; means for extracting a block of a prediction output from an operation compensation process of the base layer decoder based on the prediction mode; Means for obtaining a reconstructed block of pixel values by adding to the block of spatial difference values; and embedding (padding) the reconstructed block of pixel values with the binary alpha block to reconstruct an arbitrary-shaped VOP. And means for configuring. 15. An apparatus for analyzing the contents of the binary alpha block, comprising: means for dividing the binary alpha block into four smaller blocks (small blocks) having a size of 8 × 8 pixels; 14. The apparatus according to claim 13, further comprising: means for determining whether all pixels of the small block are zero; and means for determining the number of the small blocks including only pixels having a value of zero.
Or the apparatus according to 14. 16. The apparatus for encoding a residual coefficient of an arbitrary-shaped object, wherein the residual macroblock has a corresponding binary alpha block in a base layer. Equipment. 17. An apparatus for decoding an encoded representation of a residual coefficient of an arbitrarily shaped object, wherein the residual macroblock has a corresponding binary alpha block in a base layer. Item 16. The apparatus according to item 14 or 15. 18. The apparatus for determining a skipped macroblock, wherein the skipped macroblock is defined as a residual macroblock having a zero value for all pixels of the corresponding binary alpha block. 13. The method according to claim 13, characterized in that:
The device according to any one of 5, 16, or 17. 19. The apparatus for determining a skipped macroblock, wherein the skipped macroblock has a zero value for all pixels in the small block of the corresponding binary alpha block. The block is defined as a luminance block.
An apparatus according to any one of the preceding claims. 20. An apparatus for encoding a coded (coded) block pattern of said macroblock, comprising: means for selecting an appropriate table of coded block patterns to be used; 14. The apparatus according to claim 13, further comprising: means for obtaining an encoded block pattern, and means for expressing the encoded block pattern by a variable length code based on the selected table. . 21. An apparatus for decoding said coded block pattern, comprising: means for selecting an appropriate table of coded block patterns to be used; and representing said coded block pattern. The apparatus of claim 14, further comprising: means for retrieving a variable length code; and means for decoding the variable length code based on the selected table to obtain the encoded block pattern. . 22. An apparatus for encoding and decoding residual coefficients of an arbitrarily shaped object, wherein different tables of encoded block patterns are selected for residual macroblocks having different numbers of non-skipped blocks. 21. The method according to claim 19, wherein:
Or the apparatus of any one of 21. 23. An apparatus for encoding and decoding residual coefficients of an arbitrary-shaped object, wherein the binary alpha block is such that pixels in a specific spatial region having a size of 16 × 16 pixels are transparent or non-transparent. 19. A square block of binary pixels indicating whether it is transparent or not.
Or the apparatus of any one of claims 19 to 19. 24. An apparatus for encoding and decoding residual coefficients of an arbitrary-shaped object, wherein the pixel in the spatial domain is transparent if the binary value of the binary alpha block is zero; 13. The pixel of claim 13, wherein the pixel in the spatial domain is opaque if the binary value of the value alpha block is one.
24. The device according to any one of 4, 15, 16, 17, 18, 19 or 23.