JP2007266750A - Encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem wherein the number of codes increases when preparing the coding stream of moving pictures for each color depth. <P>SOLUTION: An encoder 200 comprises: a basic layer encoder 100 for generating the encoding data of a basic layer, a first expansion layer encoder 110 for generating the encoding data of a first expansion layer, and a second expansion layer encoder 120 for generating the encoding data of a second expansion layer. Pixel depth is deepened in steps in the order of the basic layer, first expansion layer, and second expansion layer for encoding an input image. In all the bit planes of the input image, a group of bit planes of a different number of bits counted from the most significant bit is taken out for allocating to each layer. The first expansion layer encoder 110 encodes the difference between the input image of the first expansion layer and reconstruction image of the basic one. The second expansion layer encoder 120 encodes the difference between the input image of the second expansion layer and the reconstruction image of the first one. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an encoding method for encoding a moving image with scalability.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. Also, in one stream, images with different image quality (for example, high and low image quality), different resolution (for example, high and low resolution), and different frame rates (for example, high and low frame rates) depending on the code amount H. can be compressed and decompressed. There is a next-generation image compression technique such as SVC (Scalable Video Coding), which is being standardized as an extension of H.264 / AVC.

  SVC, the next-generation image compression technology, encodes moving images with various scalability such as spatial scalability, temporal scalability, and SNR scalability so that moving images can be played at multiple different resolutions, frame rates, and image quality. Turn into. Coding can be performed by arbitrarily combining these scalability, and the scalability function of SVC is very flexible.

Some recent high-quality liquid crystal color televisions can display more than 5 billion colors, and the number of colors that can be displayed on the television is taken into account to increase the number of colors of moving images and encode them with high image quality. It is demanded. In SVC, introduction of the scalability of the number of colors is being studied (for example, see Non-Patent Document 1).
"SVC Requirements Specified by MPEG", JVT-N026, Joint Video Team (JVT) of ISO / IEC MPEG & ITU-T VCEG, 2005

  In the conventional moving image encoding method, the color depth is determined to be a constant value in units of moving image encoded streams. For example, a color depth of 8 bits is sufficient for reproducing video on a standard television, and the number of colors is fixed at 8 bits. In order to meet the need to play moving images with different numbers of colors, it is necessary to prepare separate encoded streams for each color depth and provide them to the user, increasing the overall code amount, communication bandwidth and storage area. There was a problem of squeezing.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an encoding technique for efficiently encoding a moving image with different pixel depths.

  In order to solve the above-described problem, an encoding method according to an aspect of the present invention is configured such that, when a moving image is hierarchically encoded, an upper bit plane group among a plurality of bit planes that provide pixel data is used as a base layer. A bit plane group having a larger number of bits including the upper bit plane group is assigned to an enhancement layer, and the enhancement layer encodes a difference between a picture input to the enhancement layer and a reconstructed picture layer in the base layer. To generate encoded data of the enhancement layer. The pixel data may be any of luminance data, color difference data, and RGB data.

  Here, a picture is a unit of coding, and its concept includes a frame, a field, a VOP (Video Object Plane), and the like.

  According to this aspect, the moving image can be encoded with the scalability of the pixel depth. Moreover, since the difference of the picture from which a pixel depth differs between layers is taken, the information amount of difference data decreases and the code amount of an extended layer can be suppressed.

  The quantization parameter used for the quantization process may be different between the base layer and the enhancement layer, and information regarding the quantization parameter of each layer may be included in the encoded stream of the moving image. In the enhancement layer, the difference between the pre-quantized picture input to the enhancement layer and the dequantized reconstructed picture of the base layer is taken, so even if the quantization parameter differs between the base layer and the enhancement layer, the enhancement layer It does not affect the layer encoding process. Therefore, it is possible to optimize the code amount by changing the quantization parameter independently for each layer.

  In the enhancement layer, encoding may be performed without quantizing a difference between a layer of a picture input to the enhancement layer and a reconstructed picture in the base layer. By omitting the quantization process in the enhancement layer, the speed of the encoding process can be increased, and the circuit scale of the encoding apparatus can be reduced.

  The number of bits allocated to the enhancement layer in the encoded stream of the moving image may be dynamically adjusted, and information regarding the number of bits allocated to the enhancement layer may be included in the encoded stream. As a result, the pixel depth can be increased or decreased as necessary, so that the coding efficiency is improved and the image quality can be adjusted flexibly as necessary. The unit of the region for dynamically changing the pixel depth may be any of a frame, a slice, a macro block, and a region of interest (ROI) region. A region in the image can be selected to change the pixel depth.

  The pixel depth may be changed in a picture other than a picture that is a reference for motion prediction. In the enhancement layer, encoding is performed by taking the difference between the input picture of the enhancement layer and the reconstructed picture of the base layer, so that the encoded data of the enhancement layer has no temporal direction dependency due to motion prediction. Therefore, the pixel depth can be changed by adjusting the number of bits assigned to the enhancement layer in an arbitrary picture without waiting for a picture that is a reference for motion prediction of a moving image.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, moving images having different pixel depths can be efficiently hierarchically encoded.

  FIG. 1 is a configuration diagram of an encoding apparatus 200 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding apparatus 200 according to the present embodiment performs “scalable encoding” in which a moving image is encoded with pixel depth scalability in accordance with SVC (Scalable Video Coding), which is a next-generation image compression technology. I do. Pixel depth scalability refers to scalability related to the number of bits of a pixel value, such as color or luminance, that can be given to one pixel, and is sometimes called color depth scalability in a narrow sense.

  The encoding apparatus 200 may encode a moving image with spatial scalability, temporal scalability, SNR (signal to noise ratio) scalability, and the like in addition to pixel depth scalability.

  In SVC, scalability is realized by hierarchical encoding, and image data having different pixel depths are encoded in layers, and an encoded stream including a plurality of layers having different pixel depths is generated. A moving image that is scalable hierarchically encoded in this way has scalability that an arbitrary pixel depth can be selected and decoded. For example, when only the lower layer encoded data is decoded, a moving image with a small pixel depth, that is, a small number of colors is reproduced, and when decoding including the upper layer encoded data is performed, the pixel depth is deep, that is, the number of colors is large. A moving image is played back.

  The encoding apparatus 200 includes a base layer encoding unit 100 that generates encoded data of the base layer, a first enhancement layer encoding unit 110 that generates encoded data of the first enhancement layer, and a code of the second enhancement layer A second enhancement layer encoding unit 120 that generates encoded data, and encodes an input image by gradually increasing the pixel depth in the order of the base layer, the first enhancement layer, and the second enhancement layer. In the first enhancement layer, the difference from the image data of the base layer is encoded, and in the second enhancement layer, the difference from the image data of the first enhancement layer is encoded.

  In order to vary the pixel depth depending on the layer, a bit plane group corresponding to a different number of bits counted from the most significant bit out of all bit planes of the input image is extracted and assigned to each layer. The number of bits assigned to the base layer is “basic bit number”, the number of bits assigned to the first enhancement layer is “first extension bit number”, and the number of bits assigned to the second enhancement layer is “second extension bit number”. Call. Since the number of bits increases in the order of the number of basic bits, the number of first extension bits, and the number of second extension bits, and the second extension layer is the highest layer, the second extension bit number is equal to the bit length of the pixel.

  The upper bit acquisition unit 150 for the base layer acquires upper bit planes corresponding to the number of basic bits of the input image, and provides them as an input to the base layer encoding unit 100. The upper bit acquisition unit 152 for the first enhancement layer acquires upper bit planes corresponding to the number of first enhancement bits of the input image, and provides it as an input to the first enhancement layer encoding unit 110. All bit planes of the input image are input to the second enhancement layer encoding unit 120.

  As an example, for an input image having a 10-bit pixel depth, upper 8 bits of image data are input to the base layer encoding unit 100, and upper 9 bits of image data are input to the first enhancement layer encoding unit 110, All 10-bit image data is input to the second enhancement layer encoding unit 120.

  The base layer encoding unit 100 performs motion compensation, orthogonal transform, and quantization processes on image data having the number of base bits. The first enhancement layer encoding unit 110 and the second enhancement layer encoding unit 120 respectively perform orthogonal transformation and quantization on image data having a first extension bit number and image data having a second extension bit number, that is, all bit numbers. Each process is performed.

  The first enhancement layer encoding unit 110 encodes the difference between the input image of the first enhancement layer and the reconstructed image of the base layer, and the second enhancement layer encoding unit 120 includes the input image of the second enhancement layer and the first The difference of the reconstructed image of the enhancement layer is encoded.

  The base layer encoding unit 100 generates base layer encoded data by variable-length encoding image data obtained by performing orthogonal transformation and quantization on a difference image obtained by motion prediction, and supplies the encoded data to the stream combining unit 160 To do.

  The base layer encoding unit 100 generates a reconstructed image for motion prediction. The first enhancement layer bit shift section 102 shifts the reconstructed image of the base layer to the left bit, and then gives it to the first enhancement layer encoding section 110. Here, the left bit shift amount by the bit shift unit 102 for the first enhancement layer is the difference between the number of bits of the base layer image and the number of bits of the first enhancement layer image, and the left bit shift by the bit shift unit 102 Thus, 0 is filled in the lower bits of the image data of the base layer, the number of bits of the image data of the base layer and the image data of the first enhancement layer is aligned, and a difference can be taken.

  The first enhancement layer encoding unit 110 obtains a difference between the input image of the first enhancement layer and the reconstructed image of the base layer that has been bit-shifted by the bit shift unit 102, and then orthogonally transforms and quantizes the difference data. Above, variable-length encoding is performed to generate encoded data of the first enhancement layer. The first enhancement layer encoding unit 110 supplies the encoded data of the first enhancement layer to the stream combining unit 160.

  The first enhancement layer encoding unit 110 performs inverse quantization and inverse orthogonal transform to generate a reconstructed image of the first enhancement layer. The second enhancement layer bit shift section 112 shifts the reconstructed image of the first enhancement layer to the second enhancement layer encoding section 120 after shifting the left bit. Here, the left bit shift amount by the bit shift unit 112 for the second enhancement layer is the difference between the number of bits of the first enhancement layer image and the number of bits of the second enhancement layer image. By the bit shift, the number of bits of the image data of the first enhancement layer and the image data of the second enhancement layer are aligned.

  The second enhancement layer encoding unit 120 obtains a difference between the input image of the second enhancement layer and the reconstructed image of the first enhancement layer left-bit shifted by the bit shift unit 112, and then orthogonally transforms the difference data, Then, variable length coding is performed to generate encoded data of the second enhancement layer. The second enhancement layer encoding unit 120 supplies the encoded data of the second enhancement layer to the stream combining unit 160.

  The stream combining unit 160 combines the encoded data of the base layer, the encoded data of the first enhancement layer, and the encoded data of the second enhancement layer, and outputs a moving image encoded stream.

  In FIG. 1, the configuration and operation will be described by taking as an example the case of encoding three layers of the base layer, the first enhancement layer, and the second enhancement layer, but the number of enhancement layers is arbitrary. Also, the number of bits assigned to the enhancement layer is arbitrary, and the number of bits may be increased bit by bit as the process proceeds to the upper layer, and the way of increasing the number of assigned bits may not be constant. For example, for an input image having two enhancement layers and having a 12-bit pixel depth, upper 8 bits of image data are encoded in the base layer, and upper 9 bits of image data are encoded in the first enhancement layer. All 12-bit image data may be encoded by the second enhancement layer.

  Next, the configuration and operation of base layer encoding section 100 will be described in detail.

  The base layer coding unit 100 according to the present embodiment is a moving picture expert group (MPEG-1) standard (MPEG-1) standardized by the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). , MPEG-2 and MPEG-4), H.264 standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264 respectively).

  In the MPEG series standards, I (Intra) frames are used for intra-frame coding, P (Predictive) frames are used for inter-frame predictive coding with a past frame as a reference frame, and past and future frames. A frame that performs bidirectional inter-frame predictive coding using a frame as a reference frame is called a B frame.

  On the other hand, H. In H.264 / AVC, a frame that can be used as a reference frame may be a past two frames as a reference frame or a future two frames as a reference frame regardless of the time. Further, three or more frames can be used as reference frames regardless of the number of frames that can be used as reference frames. Therefore, in MPEG-1 / 2/4, the B frame refers to a Bi-directional prediction frame. Note that in H.264 / AVC, the B frame refers to a bi-predictive prediction frame, since the time of the reference frame does not matter.

  In the embodiment, a frame is used as an example of the encoding unit, but the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The base layer encoding unit 100 receives a moving image input in units of frames, encodes the moving image, and outputs an encoded stream. In the moving image frame input to the base layer encoding unit 100, the upper bits are extracted by the upper bit acquisition unit 150, and a predetermined number of bits counted from the least significant bit is dropped in advance.

  The block generation unit 10a divides the input moving image frame into macro blocks. Macroblocks are formed in order from the upper left to the lower right of the frame. The block generation unit 10a supplies the generated macro block to the differentiator 12a and the motion prediction unit 60a.

  If the frame supplied from the block generation unit 10a is an I frame, the differentiator 12a outputs it to the DCT unit 20a as it is. However, if the frame is a P frame or a B frame, the difference unit 12a and the prediction frame supplied from the motion prediction unit 60a Is calculated and supplied to the DCT unit 20a.

  The motion prediction unit 60a uses a past or future frame stored in the frame buffer 80a as a reference frame, performs motion compensation for each macroblock of the P frame or B frame input from the block generation unit 10a, and performs motion compensation. Generate vectors and prediction frames. The motion prediction unit 60a supplies the generated motion vector to the variable length encoding unit 90a, and supplies the prediction frame to the difference unit 12a and the adder 14a.

  The differentiator 12a obtains a difference between the current encoding target frame output from the block generation unit 10a and the prediction frame output from the motion prediction unit 60a, and outputs the difference to the DCT unit 20a. The DCT unit 20a performs a discrete cosine transform (DCT) on the difference frame given from the differentiator 12a, and gives a DCT coefficient to the quantization unit 30a.

  The quantization unit 30a quantizes the DCT coefficient and provides it to the variable length coding unit 90a. The variable length coding unit 90a performs variable length coding on the quantized DCT coefficient of the difference frame together with the motion vector supplied from the motion prediction unit 60a, and generates an encoded stream. When generating the encoded stream, the variable length encoding unit 90a performs a process of rearranging the encoded frames in time order.

  The quantization unit 30a supplies the quantized DCT coefficient of the frame to the inverse quantization unit 40a. The inverse quantization unit 40a inversely quantizes the given quantized data and gives it to the inverse DCT unit 50a, and the inverse DCT unit 50a performs inverse discrete cosine transform on the given inverse quantized data. Thereby, the encoded frame is restored. The restored frame is input to the adder 14a.

  If the frame supplied from the inverse DCT unit 50a is an I frame, the adder 14a stores it in the frame buffer 80a as it is. If the frame supplied from the inverse DCT unit 50a is a P frame or a B frame, the adder 14a is a difference frame, and thus is supplied from the difference frame supplied from the inverse DCT unit 50a and the motion prediction unit 60a. By adding the predicted frame, the original frame is reconstructed and stored in the frame buffer 80a.

  The reconstructed frame stored in the frame buffer 80a is used as a reference frame for motion compensation by the motion prediction unit 60a.

  In the case of P frame or B frame encoding processing, the motion prediction unit 60a operates as described above. However, in the case of I frame encoding processing, the motion prediction unit 60a does not operate and is not shown here. Intra-frame prediction is performed.

  The motion prediction unit 60a uses the reconstructed frame stored in the frame buffer 80a as a reference frame for the macroblock (referred to as “target macroblock”) of the encoding target frame given from the block generation unit 10a. A region where the difference from the target macroblock is the smallest is identified. As a result, a motion vector indicating the motion from the encoding target frame to the reference frame is determined for each target macroblock.

  The base layer reconstructed frame output from the adder 14 a of the base layer encoding unit 100 is input to the first enhancement layer bit shift unit 102. In order to align the number of bits of the image data of the first enhancement layer, the bit shift unit 102 shifts the data of the reconstructed frame of the base layer to the left bit, and fills the lower bits vacated by the left shift with 0. The bit shift unit 102 provides the first enhancement layer encoding unit 110 with the data of the reconstructed frame of the base layer that has been bit-shifted to the left.

  Next, the configurations and operations of the first enhancement layer encoding unit 110 and the second enhancement layer encoding unit 120 will be described, but the components common to the base layer encoding unit 100 are denoted by the corresponding reference numerals and description thereof is omitted. To do. In the figure, “a” is attached to the code of each component of the base layer encoding unit 100, “b” is attached to the code of each component of the first enhancement layer encoding unit 110, and the second The code of each component of the enhancement layer encoding unit 120 is distinguished by attaching “c”.

  The first enhancement layer encoding unit 110 will be described. The differentiator 16b includes an input frame corresponding to the number of first extension bits provided from the upper bit acquisition unit 152 for the first enhancement layer and a left bit-shifted base layer provided from the bit shift unit 102 for the first enhancement layer. The difference data from the reconstructed frame is obtained and given to the DCT unit 20b. Thereafter, the DCT unit 20b and the quantization unit 30b perform orthogonal transform and quantization on the difference data, and the difference data is input to the variable length coding unit 90b. The variable length encoding unit 90 b performs variable length encoding on the input difference data to generate encoded data of the first enhancement layer, and supplies the encoded data to the stream combining unit 160.

  The quantized difference data output from the quantization unit 30b is input to the inverse quantization unit 40b and inversely quantized, and further input to the inverse DCT unit 50b and subjected to inverse orthogonal transform to restore the original difference data. Is done. The adder 18b reconstructs the input frame of the first enhancement layer by adding the data of the reconstructed frame of the base layer given from the bit shift unit 102 to the restored difference data. That is, the adder 18b performs the reverse calculation of the difference unit 16b.

  The second enhancement layer bit shift unit 112 receives the first enhancement layer reconstructed frame output from the adder 18 b of the first enhancement layer encoding unit 110. The bit shift unit 112 shifts the data of the reconstructed frame of the first enhancement layer to the left bit in order to match the number of bits of the image data of the second enhancement layer, and fills in the lower bits vacated by the left shift. The bit shift unit 112 provides the second enhancement layer encoding unit 120 with the data of the reconstructed frame of the first enhancement layer that has been bit-shifted to the left.

  The second enhancement layer encoding unit 120 will be described. The differentiator 16c obtains difference data between the input frame for the total number of bits and the reconstructed frame of the first enhancement layer shifted from the left enhancement bit provided from the second enhancement layer bit shift unit 112, and sends it to the DCT unit 20c. give. Thereafter, the DCT unit 20c and the quantization unit 30c perform orthogonal transformation and quantization on the difference data, and the difference data is input to the variable length coding unit 90c. The variable length encoding unit 90 c performs variable length encoding on the input difference data to generate encoded data of the second enhancement layer, and supplies the encoded data to the stream combining unit 160.

  In FIG. 1, orthogonal transformation and quantization are performed in the first enhancement layer and the second enhancement layer, but at least one of orthogonal transformation and quantization may be omitted. Since the first enhancement layer and the second enhancement layer encode the difference between the input frame of each layer and the reconstructed frame of the next lower layer, the number of bits of the difference data is reduced. For example, when the base layer is 8 bits, the first enhancement layer is 9 bits, and the second enhancement layer is 10 bits, the difference data between layers has only 1-bit information. Therefore, encoding efficiency is good and the amount of codes can be kept small without performing orthogonal transform or quantization.

  Further, the quantization parameter such as the quantization scale in the quantization process in the first enhancement layer and the second enhancement layer may be a value different from the quantization parameter in the quantization process in the base layer. In this case, information on the quantization parameter of each layer is included in the header of the video stream. By changing the quantization parameter in each layer, the code amount can be controlled independently for each layer.

  Moreover, although variable length encoding was performed in each layer and the encoding data of each layer were produced | generated, you may perform encoding other than variable length encoding.

  In the above description, the pixel depth of the moving image stream is constant, but the pixel depth may be changed dynamically. In that case, the number of bits allocated to the enhancement layer is made variable in the same stream as the pixel depth increases or decreases. For example, 8 bits are allocated to the base layer, and the number of bits allocated to 2 to 4 bits is changed to the enhancement layer. Information on the number of bits allocated to the enhancement layer is included in the header of the video stream. As a result, the number of colors can be increased or decreased in units of frames or in units of frames within a single moving image stream. For example, the number of colors can be increased or decreased according to the scene, or the number of colors can be decreased in an unimportant frame.

  It should be noted that the number of bits allocated to the base layer is preferably not fixed but fixed, so that a moving image can be decoded only by the base layer even in a non-SVC decoding device. In this embodiment, since motion prediction is not performed in the enhancement layer, the encoded data in the enhancement layer has no dependency in the time axis direction. Even if the enhancement layer bit allocation is variable for each frame, the enhancement layer coding is not affected. Therefore, it should be noted that the timing at which the pixel depth can be changed in the encoded stream is not limited to the position of a reference frame such as an I frame, and may be the position of an arbitrary frame.

  According to the encoding apparatus 200 of the present embodiment, a moving image can be encoded hierarchically with pixel depth scalability. Therefore, at the time of decoding, moving images having different pixel depths can be selected and reproduced.

  Also, according to the encoding apparatus 200, in the enhancement layer, the difference between the input frame of the layer and the reconstructed image frame difference of the layer immediately below is taken and encoded. Since there is only a difference in the pixel depth of the image between layers, by taking the difference between layers, the code amount of the encoded data of the enhancement layer becomes extremely small.

  In addition, since motion prediction encoding is not performed in the enhancement layer, the encoded data of the enhancement layer has no dependency in the time direction, and the pixel depth can be changed in an arbitrary frame.

  FIG. 2 is a configuration diagram of the decoding device 500 according to the embodiment. These functional blocks can also be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding device 500 performs “scalable decoding” in which a moving image is decoded with pixel depth scalability in accordance with SVC. The decoding apparatus 500 receives an encoded stream of a moving image that has been hierarchically encoded with pixel depth scalability by the encoding apparatus 200 of FIG. 1, and generates a base layer and an enhancement layer stream from the input encoded stream. Take out and decrypt.

  Corresponding to the encoding device 200 of FIG. 1, the decoding device 500 is configured to decode a moving image in three layers, a base layer, a first enhancement layer, and a second enhancement layer, and decodes each layer. The base layer decoding unit 400, the first enhancement layer decoding unit 410, and the second enhancement layer decoding unit 420 are included, but the number of enhancement layers is arbitrary. Note that the non-SVC compatible decoding device 500 includes only the base layer decoding unit 400, and the decoding device 500 capable of supporting up to the first enhancement layer includes the base layer decoding unit 400 and the first enhancement layer decoding unit 410.

  When decoding apparatus 500 receives an input of an encoded video stream, base layer decoding section 400 decodes base layer encoded data. When the base layer image is reproduced, the image decoded by the base layer decoding unit 400 is output as it is as the final output image.

  When playing back an image of the first enhancement layer, the first enhancement layer decoding unit 410 operates and decodes the encoded data of the first enhancement layer. The first enhancement layer decoding unit 410 bit-shifts the output image of the base layer to match the number of bits of the first enhancement layer, and then adds the decoded data of the first enhancement layer to obtain the final output of the first enhancement layer Generate and output an image.

  When reproducing the second enhancement layer image, the second enhancement layer decoding unit 420 further operates to decode the encoded data of the second enhancement layer. The second enhancement layer decoding unit 420 bit-shifts the output image of the first enhancement layer so as to match the number of bits of the second enhancement layer, and then adds the decoded data of the second enhancement layer to the second enhancement layer. Generate and output the final output image.

  The configuration and operation of the decoding device 500 will be described in detail with reference to FIG. The stream separation unit 430 separates and extracts the encoded data of the base layer, the first enhancement layer, and the second enhancement layer from the input encoded stream, and the encoded data of the base layer is variable by the base layer decoding unit 400. The encoded data of the first enhancement layer is transmitted to the variable length decoding unit 310b of the first enhancement layer decoding unit 410, and the encoded data of the second enhancement layer is converted to the variable length decoding of the second enhancement layer decoding unit 420. To part 310c.

  The base layer decoding unit 400 will be described. The variable length decoding unit 310a performs variable length decoding on the base layer encoded stream, supplies the decoded image data to the inverse quantization unit 320a, and supplies motion vector information to the motion compensation unit 360a.

  The inverse quantization unit 320a inversely quantizes the image data decoded by the variable length decoding unit 310a and supplies the image data to the inverse DCT unit 330a. The image data inversely quantized by the inverse quantization unit 320a is a DCT coefficient. The inverse DCT unit 330a restores the original image data by performing inverse discrete cosine transform (IDCT) on the DCT coefficient inversely quantized by the inverse quantization unit 320a. The image data restored by the inverse DCT unit 330a is supplied to the adder 312a.

  When the image data supplied from the inverse DCT unit 330a is an I frame, the adder 312a outputs the I frame image data as it is, and also generates a reference frame for generating a P frame or a B frame prediction frame. Is stored in the frame buffer 380a.

  When the image data supplied from the inverse DCT unit 330a is a P frame, the adder 312a supplies the difference frame supplied from the inverse DCT unit 330a and the motion compensation unit 360a because the image data is a difference frame. By adding the predicted frames, the original image data is restored and output.

  The motion compensation unit 360a generates a predicted frame of P frame or B frame using the motion vector information supplied from the variable length decoding unit 310a and the reference frame stored in the frame buffer 380a, and supplies the predicted frame to the adder 312a. To do.

  The motion compensation unit 360a acquires the motion vector of the decoding target frame from the variable length decoding unit 310a, specifies the region referred to by the motion vector for the target macroblock of the decoding target frame, and uses the pixel data of the region Thus, a motion-compensated prediction frame is generated and provided to the adder 312a.

  The bit shift unit 402 for the first enhancement layer left-bit shifts the output image of the base layer output from the adder 312a of the base layer decoding unit 400 so as to match the number of bits of the output image of the first enhancement layer, The left-bit shifted base layer output image is provided to adder 314 b of first enhancement layer decoding section 410.

  The first enhancement layer decoding unit 410 will be described. The encoded stream of the first enhancement layer is variable-length decoded by the variable-length decoding unit 310b, dequantized by the inverse quantization unit 320b, and inversely orthogonal transformed by the inverse DCT unit 330b, and then input to the adder 314b. The The image decoded in the first enhancement layer is difference data from the base layer image. The adder 314b adds the difference data decoded by the first enhancement layer to the output image data of the base layer shifted from the bit enhancement unit 402 for the first enhancement layer by the left bit shift, thereby adding the first enhancement layer 314b. Generate and output the final output image of the layer.

  The second enhancement layer bit shift unit 412 shifts the output image of the first enhancement layer output from the adder 314b of the first enhancement layer decoding unit 410 to the left so as to match the number of bits of the output image of the second enhancement layer. The output image of the first enhancement layer that has been bit-shifted and bit-shifted to the left is supplied to the adder 314c of the second enhancement layer decoding unit 420.

  The configuration and operation of the second enhancement layer decoding unit 420 are the same as those of the first enhancement layer decoding unit 410, and the encoded stream of the second enhancement layer is variable length decoded by the variable length decoding unit 310c, and the inverse quantization unit Inverse quantization is performed by 320c, and inverse orthogonal transformation is performed by the inverse DCT unit 330c, which is then input to the adder 314c. The image decoded in the second enhancement layer is difference data from the image of the first enhancement layer. The adder 314c adds the difference data decoded in the second enhancement layer to the output image data of the first enhancement layer shifted from the left enhancement bit provided from the bit shift unit 412 for the second enhancement layer. A final output image of two enhancement layers is generated and output.

  In addition, when orthogonal transformation and quantization are abbreviate | omitted in the 1st enhancement layer encoding part 110 and the 2nd enhancement layer encoding part 120 of FIG. 1, the 1st enhancement layer decoding part 410 and the 2nd enhancement layer decoding part of FIG. In 420, the configuration of inverse orthogonal transform or inverse quantization is unnecessary.

  According to decoding apparatus 500 of the present embodiment, moving images with different pixel depths can be selected by appropriately selecting an enhancement layer in addition to a base layer in an encoded stream of a moving image that has been encoded with pixel depth scalability. Can be decrypted. Thereby, it is possible to reproduce a moving image by selecting an optimal pixel depth according to the number of colors that can be displayed on the display and the image quality requirement level of the user.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a block diagram of the decoding apparatus which concerns on embodiment.

Explanation of symbols

  10a block generation unit, 20a, 20b, 20c DCT unit, 30a, 30b, 30c quantization unit, 40a, 40b inverse quantization unit, 50a, 50b inverse DCT unit, 60a motion prediction unit, 80a frame buffer, 90a, 90b, 90c variable length coding unit, 100 base layer coding unit, 102, 112 bit shift unit, 110 first enhancement layer coding unit, 120 second enhancement layer coding unit, 150, 152 upper bit acquisition unit, 160 stream combination , 200 encoding device, 400 base layer decoding unit, 402, 412 bit shift unit, 410 first enhancement layer decoding unit, 420 second enhancement layer decoding unit, 430 stream separation unit, 500 decoding device.

Claims (4)

  When encoding a moving image hierarchically, among a plurality of bit planes that provide pixel data, an upper bit plane group is a basic layer, and a bit plane group having a larger number of bits including the upper bit plane group is an extension layer. And encoding in the enhancement layer, wherein encoded data of the enhancement layer is generated by encoding a difference between a layer of a picture input to the enhancement layer and a reconstructed picture in the base layer Method.   The quantization parameter used for quantization processing is different between the base layer and the enhancement layer, and information on the quantization parameter of each layer is included in the encoded stream of the moving image. Encoding method.   The encoding according to claim 1, wherein the enhancement layer performs coding without quantizing a difference between a layer of a picture input to the enhancement layer and a reconstructed picture in the base layer. Method.   The number of bits allocated to the enhancement layer in the encoded stream of the moving image is dynamically adjusted, and information on the number of bits allocated to the enhancement layer is included in the encoded stream. 4. The encoding method according to any one of 3.

Images