JP2013038758A - Image encoder, image encoding method, program, image decoder, image decoding method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: In a conventional coding method or a HEVC coding method currently being standardized, a corresponding quantization matrix cannot be defined even when the shape of orthogonal transformation exists other than the conventional square, and the shape of the orthogonal transformation is obtained. The optimal quantization matrix could not be applied.
An input image is divided into a plurality of blocks, the divided blocks are predicted from coded blocks, a prediction error is generated, and a plurality of square or rectangular orthogonal transform processing block shapes are used. Determining a processing block shape to be used, performing the determined orthogonal transform on the generated prediction error, generating a transform coefficient, selecting a quantization matrix based on the determined processing block shape of the orthogonal transform, The transform coefficient is quantized using the selected quantization matrix to generate a quantized coefficient, and the generated quantized coefficient is encoded.
[Selection] Figure 1

Description

  The present invention relates to an image encoding device, an image encoding method and program, an image decoding device, an image decoding method and a program, and more particularly to a quantization method and a quantization matrix encoding method and decoding method.

As a method for compressing and recording a moving image, H.264 is used. H.264 / MPEG-4 AVC (hereinafter referred to as H.264) is known. (Non-Patent Document 1)
H. In H.264, each element of the quantization matrix can be changed to an arbitrary value by encoding scaling_list information. According to Non-Patent Document 1, each element of the quantization matrix can take an arbitrary value by adding delta_scale which is a difference value to the immediately preceding element.

  In recent years, H.C. As a successor to H.264, activities to start international standardization of a more efficient coding method have started. JCT-VC (Joint Collaborative Team on Video Coding) was established between ISO / IEC and ITU-T, and standardized as HEVC (High Efficiency Video Coding) (hereinafter, HEVC).

  In JCT-VC, division with a larger block size than a conventional macroblock (16 × 16 pixels) is being studied as the target screen size increases. This large-sized basic block is called an LCU (Large Coding Unit), and studies are underway with a maximum size of 64 × 64 pixels. (Non-patent document 2) The LCU is further divided into sub-blocks which are units for performing prediction and conversion.

  In order to improve coding efficiency, an intra prediction / orthogonal transform method based on a rectangular sub-block such as a short-range intra prediction is being studied in addition to a conventional intra prediction / orthogonal transform method based on a square sub-block. (Non-Patent Document 3) FIG. 2 shows an example of sub-block division used for short-range intra prediction. 200 in FIG. 2 represents a basic block, and in order to simplify the description, it is assumed that the configuration is 16 × 16 pixels, and each square in the thick frame represents a sub-block. 2 represents an example of a conventional square sub-block division. A basic block of 16 × 16 pixels is divided into square sub-blocks of 8 × 8 pixels or 4 × 4 pixels. On the other hand, 201 and 202 in FIG. 2 represent an example of rectangular sub-block division using short-range intra prediction, and a basic block of 16 × 16 pixels is divided into rectangular sub-blocks of 4 × 16 or 16 × 4 pixels. Has been. In this way, encoding processing is performed using not only a square but also a rectangular sub-block.

ITU-TH. H.264 (03/2010) Advanced video coding for generic audiovisual services JCT-VC contribution JCTVC-E603. doc Internet <http: // phenix. int-evry. fr / jct / doc_end_user / documents / 5_Geneva / wg11 /> JCT-VC contribution JCTVC-E278. doc Internet <http: // phenix. int-evry. fr / jct / doc_end_user / documents / 5_Geneva / wg11 />

  Also in HEVC, H.C. As in the case of H.264, it is considered to introduce a mechanism capable of setting each element of the quantization matrix to an arbitrary value. Further, in HEVC, not only squares but also rectangular sub-block divisions and orthogonal transform shapes corresponding thereto are studied, but the distribution of coefficients resulting from the respective orthogonal transforms differs depending on the orthogonal transform shapes. For this reason, when the shape of the orthogonal transform and the quantization matrix do not match, the quantization is not appropriately performed, and the image quality is greatly deteriorated. Accordingly, the present invention has been made to solve the above-described problems, and an object thereof is to adaptively use a quantization matrix in accordance with the shape of orthogonal transform. Furthermore, it aims at suppressing the code amount when setting a quantization matrix for every shape of orthogonal transformation.

  In order to solve the above-described problems, the image coding apparatus of the present invention has the following configuration. That is, a block dividing unit that divides an input image into a plurality of blocks, a prediction unit that performs prediction from a block that has been encoded with respect to the blocks divided by the block dividing unit, and generates a prediction error; A transform unit that determines a processing block shape to be used from a plurality of processing block shapes of orthogonal transform, performs the determined orthogonal transform on the prediction error generated by the prediction unit, and generates a transform coefficient; and the transform unit Quantization matrix selection means for selecting a quantization matrix based on the processing block shape of the orthogonal transformation determined by the step, and quantizing by quantizing the transform coefficient using the quantization matrix selected by the quantization matrix selection means Quantization means for generating coefficients, and encoding the quantized coefficients generated by the quantization means And a coefficient encoding means that.

  Furthermore, the image decoding apparatus of the present invention has the following configuration. That is, a decoding unit that separates a necessary code from an input bitstream, a determination unit that determines a processing block shape of inverse orthogonal transform from information decoded by the decoding unit, and information decoded by the decoding unit A coefficient decoding means for decoding the quantized coefficients; a quantization matrix decoding means for decoding a quantization matrix used for inverse quantization of the quantized coefficients decoded by the coefficient decoding means; A quantization matrix selecting unit that selects a quantization matrix based on the processed block shape of the inverse orthogonal transform, and a coefficient decoded by the coefficient decoding unit using the quantization matrix selected by the quantization matrix selecting unit An inverse quantization means for inversely quantizing the quantization coefficient to generate a transform coefficient; and Based on the shape of the constant inverse orthogonal transform, and a inverse orthogonal transform means for generating a prediction error of the transform coefficients generated by the inverse quantization unit inverse orthogonal transform to.

  According to the present invention, a quantization matrix can be set for each processing block shape of square or rectangular orthogonal transform, and an optimal quantization process can be performed in each processing block shape.

1 is a block diagram illustrating a configuration of an image encoding device according to a first embodiment. Diagram showing an example of sub-block division FIG. 3 is a block diagram illustrating a configuration of an image decoding device according to a second embodiment. Diagram showing an example of a quantization matrix 7 is a flowchart showing image encoding processing in the image encoding apparatus according to the first embodiment. 7 is a flowchart showing image decoding processing in the image decoding apparatus according to the second embodiment. (A), (b) The figure which shows an example of the structure of a bit stream FIG. 9 is a block diagram illustrating a configuration of an image encoding device according to a third embodiment. FIG. 9 is a block diagram illustrating a configuration of an image decoding device according to a fourth embodiment. 10 is a flowchart showing image encoding processing in the image encoding apparatus according to the third embodiment. 10 is a flowchart showing image decoding processing in the image decoding apparatus according to the fourth embodiment. The block diagram which shows the structural example of the hardware of the computer applicable to the image coding apparatus of this invention, and a decoding apparatus. FIG. 9 is a block diagram illustrating a configuration of an image encoding device according to a sixth embodiment. FIG. 9 is a block diagram illustrating a configuration of an image decoding device according to a seventh embodiment. 10 is a flowchart showing image coding processing in the image coding apparatus according to the sixth embodiment. 10 is a flowchart showing image decoding processing in the image decoding apparatus according to the seventh embodiment. (A) to (c) Example of calculating a rectangular quantization matrix from a square quantization matrix

  Hereinafter, the present invention will be described in detail based on the preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<Embodiment 1>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an image encoding apparatus according to this embodiment.

  In FIG. 1, reference numeral 101 denotes a block dividing unit that divides an input image into a plurality of basic blocks. In the present embodiment, for description, the block dividing unit 101 is described as being divided into basic blocks of 16 × 16 pixels, but is not limited thereto.

  102 divides each basic block divided by the block dividing unit 101 into sub-blocks having the same size as the basic block or smaller than the basic block to generate a sub-block size, and has been encoded in units of sub-blocks A prediction unit that performs prediction from a block. The prediction unit 102 further determines a prediction method, performs prediction and difference value calculation accordingly, and calculates a prediction error. It is assumed that intra prediction is performed for an intra frame in the case of a still image or a moving image, and motion compensation prediction is also performed for a moving image. Intra prediction is generally realized by selecting a method (prediction method) for referring to a reference pixel for calculating a prediction value from data of surrounding pixels for a plurality of reference methods. In the present embodiment, for the sake of explanation, the prediction unit 102 divides the basic block into 16 × 16 pixel, 8 × 8 pixel, 4 × 4 pixel square, or 16 × 4 pixel, 4 × 16 pixel rectangular sub-blocks. However, the present invention is not limited to this.

  Reference numeral 108 denotes a quantization matrix holding unit that generates and temporarily stores a quantization matrix. The method of generating the stored quantization matrix is not particularly limited, but the user may input the quantization matrix or use the one specified in advance as the initial value even if it is calculated from the characteristics of the input image. Of course it is good.

  Reference numeral 103 denotes a conversion unit that performs orthogonal transform on the prediction error of each subblock. In this embodiment, it is assumed that orthogonal transform is performed by performing orthogonal transform in units of sub-block sizes determined by the prediction unit 102. In this embodiment, it is assumed that orthogonal transformation is performed in a processing block shape of 16 × 16 pixels, 8 × 8 pixels, 4 × 4 pixels, or 16 × 4 pixels and 4 × 16 pixels. As for the orthogonal transform, in addition to the discrete cosine transform, a transform such as a discrete sine transform, a Karhunen-Loeve transform, and a Hadamard transform may be used.

  Reference numeral 107 denotes a quantization matrix selection unit that inputs the sub-block size determined by the prediction unit 102 and selects a quantization matrix to be applied from the quantization matrices stored in the quantization matrix holding unit 108.

  Reference numeral 104 denotes a quantization unit that quantizes the orthogonal transform coefficient using the quantization matrix selected by the quantization matrix selection unit 107. A quantization coefficient can be obtained by this quantization.

  Reference numeral 105 denotes a coefficient encoding unit that encodes the quantized coefficient thus obtained to generate quantized coefficient code data. The encoding method is not particularly limited, but a Huffman code or an arithmetic code can be used.

  Reference numeral 109 denotes a quantization matrix encoding unit that encodes the quantization matrix stored in the quantization matrix holding unit 108 and generates quantization matrix code data. The encoding method of the quantization matrix will be described later, but is not particularly limited. A Huffman code, an arithmetic code, or the like can be used for the value of each element of the quantization matrix itself, the difference from the value of the immediately preceding element, or the difference from another quantization matrix.

  106 generates a code related to header information, prediction, and conversion, and integrates the quantization coefficient code data generated by the coefficient encoding unit 105 and the quantization matrix code data generated by the quantization matrix encoding unit 109 It is an encoding part. The code related to prediction and conversion is, for example, a code such as a sub-block size generated by the prediction unit 102 and a prediction method.

  An image encoding operation in the image encoding apparatus will be described below. In the present embodiment, moving image data is input in units of frames, but still image data for one frame may be input. Further, in the present embodiment, only the intra prediction encoding process will be described for ease of explanation, but the present invention is not limited to this and can be applied to the inter prediction encoding process.

  Image data for one frame is input to the block dividing unit 101 and divided into basic block units of 16 × 16 pixels. The divided basic block is input to the prediction unit 102.

  The prediction unit 102 determines the size and prediction method of sub-blocks that are units of prediction, performs prediction on a sub-block basis, and generates a prediction error.

  The transform unit 103 performs orthogonal transform on the prediction error in units of sub-blocks generated by the prediction unit 102 to generate orthogonal transform coefficients. In the present embodiment, the size and shape of the sub-block and the size and shape of the orthogonal transform are the same. The generated orthogonal transform coefficient is input to the quantization unit 104.

  The quantization matrix holding unit 108 holds a quantization matrix used for quantization of the frame, that is, a quantization matrix corresponding to the shape of each sub-block. In this embodiment, there are five types of sub-block sizes: 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, 16 × 4 pixels, and 4 × 16 pixels. Holds a first quantization matrix corresponding to orthogonal transformation of 4 × 4 pixels, a second quantization matrix corresponding to orthogonal transformation of 8 × 8 pixels, and a third quantization matrix corresponding to orthogonal transformation of 16 × 16 pixels. . Furthermore, a fourth quantization matrix corresponding to 16 × 4 pixel orthogonal transformation and a fifth quantization matrix corresponding to 4 × 16 pixel orthogonal transformation are held. In the present embodiment, the above five types of quantization matrices are held, but the held quantization matrices also increase / decrease in accordance with the increase / decrease of the sub-block shape.

  The quantization matrix encoding unit 109 sequentially reads out the quantization matrix from the quantization matrix holding unit 108 according to the sub-block size and encodes it to generate quantization matrix code data. As an encoding method, in this embodiment, each element is encoded as it is, a difference from the immediately preceding element is encoded, and a difference from DPCM or another quantization matrix is used. However, the method is not limited to these. In addition, when the same value continues along with these, the code amount of the quantization matrix can be suppressed by inserting a code for stopping the encoding.

  An example of a specific quantization matrix encoding method will be described with reference to FIG. Reference numeral 400 denotes a quantization matrix. In order to simplify the description, it is assumed that the configuration is for 16 pixels corresponding to a 4 × 4 pixel sub-block, and each square in the thick frame represents an element. Here, the encoding method will be described using the five types of quantization matrices 400 to 404 shown in FIG. 4 as an example.

  400 in FIG. 4 is an example in which each element of the quantization matrix starts from 1 and increases by 1. For example, when the quantization matrix is encoded, the values 1 to 16 are encoded one by one. Further, after encoding the first element 1, the difference from the immediately preceding element, that is, the difference value 1 in this example, may be encoded 15 times. Further, like the DPCM, after encoding the first element 1 and the second element and 1 being the difference between the first elements, the prediction difference 0 may be encoded.

  401 in FIG. 4 is an example in which each element of the quantization matrix starts from 3 and increases by 1. For example, when this quantization matrix is encoded, the values from 3 to 18 are encoded one by one. In addition, after encoding the first element 3, the difference from the immediately preceding element, that is, the difference value 1 in this example, may be encoded 15 times. Furthermore, a difference from each element value of the quantization matrix indicated by 400, that is, a difference value 2 in this example, may be encoded 16 times.

  402 is an example in which each element of the quantization matrix starts from 3 and increases by 1 to 10, and after 10 continues 3 times, it increases to 1 by 16 again. For example, when this quantization matrix is encoded, each element value is encoded one by one as in 400 and 401. Also, a difference from the immediately preceding element or a difference from another quantization matrix element may be encoded. When encoding the difference between the value of each element of 402 and the value of each element of 400, the first 3 to the first 10 are encoded as the difference value 2 and the second 10 is the difference value 1 is encoded. After that, since the difference value is 0 to the end after the third 10, the difference value 0 is encoded seven times. Alternatively, a method of reducing the code amount of the difference value by encoding a code indicating the termination of encoding may be used.

  403 is an example in which each element of the quantization matrix starts from 3 and increases by 1 to 10, and then continues to 10 until the end. For example, when this quantization matrix is encoded, each element value is encoded one by one as in 400 and 401. Also, a difference from the immediately preceding element or a difference from another quantization matrix element may be encoded. When the difference between each element value of 403 and the immediately preceding element is encoded, the difference value 1 is encoded from the first 3 to the first 10. Since the difference value is zero until the end, the difference value 0 is encoded seven times. Alternatively, a method of reducing the code amount of the difference value by encoding a code indicating the termination of encoding may be used.

  404 is an example in which each element of the quantization matrix starts from 6 and gradually increases. For example, when this quantization matrix is encoded, each element value is encoded one by one as in 400 and 401. Also, a difference from the immediately preceding element or a difference from another quantization matrix element may be encoded. Further, when the difference between each element value 404 and the immediately preceding element is encoded, the first element 6 is encoded first. Thereafter, the difference value from the immediately preceding element is encoded. That is, encoding is performed in the order of 7, 0, 7, 0, 0, 8, 0, 0, 0, 4, 0, 0, 5, 0, 5. The quantization matrix encoding method presented above is described for the sake of explanation, and the encoding method is not limited to these.

  The quantization matrix selection unit 107 uses the subblock size determined by the prediction unit 102 to select a quantization matrix used for quantization of the subblock. Specifically, a quantization matrix used for quantization of the sub-block is selected from the quantization matrices stored in the quantization matrix holding unit 108. For example, in the present embodiment, when the sub-block size is 16 × 4 pixels, the fourth quantization matrix is input from the quantization matrix holding unit 108 and output to the quantization unit 104.

  The quantization unit 104 quantizes the orthogonal transform coefficient output from the conversion unit 103 using the quantization matrix selected by the quantization matrix selection unit 107 to generate a quantization coefficient. The generated quantized coefficient is input to the coefficient encoding unit 105.

  The coefficient encoding unit 105 encodes the quantization coefficient generated by the quantization unit 104, generates quantized coefficient code data, and outputs it to the integrated encoding unit 106.

  On the other hand, the quantization matrix encoding unit 109 encodes each quantization matrix stored in the quantization matrix holding unit 108, generates quantization matrix code data, and outputs the data to the integrated encoding unit 106.

  The integrated encoding unit 106 generates codes such as image sequences, frames, pictures, and slice headers. Further, the quantization matrix code data generated by the quantization matrix encoding unit 109 is inserted into any of these headers. The header part code and the quantized coefficient code data generated by the coefficient encoding unit 105 are integrated, the sub block size and prediction information generated by the prediction unit 102 are encoded, and a bit stream is generated and output. .

  FIG. 7A shows an example of the bit stream output in the first embodiment. In this embodiment, the quantization matrix is encoded in the sequence header portion in the bit stream, but the encoding position is not limited to this. Of course, it does not matter even if it takes the structure encoded in the picture header part and other header parts. When the quantization matrix is changed in one sequence, it can be updated by newly encoding the quantization matrix. At this time, all of the quantization matrices may be rewritten, or a part of the quantization matrix can be changed by designating the sub-block size of the quantization matrix to be rewritten. Furthermore, it is of course possible to partially change by specifying the sub-block size of the quantization matrix to be rewritten and the position of the element to be changed.

  FIG. 5 is a flowchart showing an image encoding process in the image encoding apparatus according to the first embodiment. First, in step S501, the quantization matrix holding unit 108 generates one or more quantization matrices. In this embodiment, since there are five types of sub-block sizes, as described above, five types of first to fifth quantization matrices are generated.

  In step S502, the quantization matrix encoding unit 109 encodes each quantization matrix generated by the quantization matrix holding unit 108.

  In step S503, the integrated encoding unit 106 encodes and outputs the header portion of the bitstream. Here, the quantization matrix code data encoded in step S502 is output together with the header.

  In step S504, the block dividing unit 101 divides the input image in frame units into basic block units (16 × 16 pixels).

  In step S505, the prediction unit 102 determines the size of the sub-block. In step S506, the prediction unit 102 performs prediction in units of sub-block sizes generated in step S505.

  In step S507, the image coding apparatus performs determination based on the sub-block size determined in step S506. If the sub-block size is 4 × 4 pixels, the process proceeds to S508; if the sub-block size is 8 × 8 pixels, the process proceeds to S528; if the sub-block size is 16 × 16 pixels, the process proceeds to S548; If it is 16 pixels, the process proceeds to S588.

  In step S508, the transform unit 103 performs orthogonal transform on the prediction error generated in step S505 in units of 4 × 4 pixel sub-blocks to generate orthogonal transform coefficients.

  In step S509, the quantization matrix selection unit 107 selects a first quantization matrix from the quantization matrix generated in step S501.

  In step S528, the transform unit 103 performs orthogonal transform on the prediction error generated in step S505 in units of 8 × 8 pixel sub-blocks to generate orthogonal transform coefficients. In step S529, as in step S509, the quantization matrix selection unit 107 selects the second quantization matrix from the quantization matrix generated in step S501. In step S548, the transform unit 103 performs orthogonal transform on the prediction error generated in step S505 in units of 16 × 16 pixel sub-blocks, and generates orthogonal transform coefficients. In step S549, as in step S509, the quantization matrix selection unit 107 selects a third quantization matrix from the quantization matrix generated in step S501. In step S568, the transform unit 103 performs orthogonal transform on the prediction error generated in step S505 in units of 16 × 4 pixel sub-blocks to generate orthogonal transform coefficients. In step S569, as in step S509, the quantization matrix selection unit 107 selects a fourth quantization matrix from the quantization matrix generated in step S501. Further, in step S588, the transform unit 103 performs orthogonal transform on the prediction error generated in step S505 in units of 4 × 16 pixels, and generates orthogonal transform coefficients. In step S589, as in step S509, the quantization matrix selection unit 107 selects the fifth quantization matrix from the quantization matrix generated in step S501.

  In step S510, the quantization unit 104 inputs the orthogonal transform coefficients generated in steps S508, S528, S548, S568, and S588. Then, quantization is performed using the quantization matrix selected in steps S509, S529, S549, S569, and S589 to generate a quantized coefficient.

  In step S511, the coefficient encoding unit 105 encodes the quantized coefficient generated in step S510 to generate quantized coefficient code data.

  In step S512, the image coding apparatus determines whether or not coding of all the sub-blocks in the basic block has been completed. If finished, the process proceeds to step S513. The process returns to step S505 for the sub-block.

  In step S513, the image encoding apparatus determines whether or not encoding of all the basic blocks has been completed. If it has been completed, the image encoding apparatus stops all operations and ends the processing. The process returns to step S504 for the next basic block.

  With the above configuration and operation, a bitstream that has been quantized using an optimal quantization matrix depending on the shape of the orthogonal transform of the sub-block, particularly by the processing of steps S501, S509, S529, S549, S569, and S589. Can be generated.

  In the present embodiment, a frame using only intra prediction has been described as an example, but it is obvious that a frame that can use inter prediction can also be used.

  In this embodiment, one type of quantization matrix is used for each sub-block size. However, a plurality of quantization matrices may be used and switched depending on the prediction method or the conversion method.

  In addition, even if the sub-block sizes are different, if there is a sub-block size with the same number of pixels in the block, one quantization matrix may be shared. For example, in the present embodiment, there are three types of subblock sizes of 8 × 8 pixels, 16 × 4 pixels, and 4 × 16 pixels with 64 subblock pixels. In this case, an 8 × 8 pixel quantization matrix may be used for the 16 × 4 pixel sub-block and the 4 × 16 pixel sub-block. Similarly, a 16 × 4 pixel quantization matrix may be used for a 4 × 16 pixel sub-block. Even when not shared, a difference from a quantization matrix corresponding to another sub-block may be encoded. For example, in encoding a 16 × 4 pixel quantization matrix, a difference from each element of an 8 × 8 pixel quantization matrix may be encoded.

  Furthermore, regarding the encoding of the quantization matrix corresponding to the sub-block sizes having different numbers of pixels, the encoding may be performed using the quantization corresponding to other sub-block sizes. For example, an 8 × 8 pixel quantization matrix is encoded in units of elements, and a 4 × 4 pixel quantization matrix is an average of 2 × 2 elements in an 8 × 8 pixel quantization matrix or thinned out. The difference may be encoded with the predicted value as the predicted value. In the example of the present embodiment, each element of the 16 × 16 pixel quantization matrix is encoded. Then, the 16 × 4 pixel and 4 × 16 pixel quantization matrices are thinned out in units of 1 × 4 and 4 × 1 elements in the 16 × 16 pixel quantization matrix or average values are taken. Furthermore, it may be used as it is as an element in the quantization matrix, or the difference may be encoded as a predicted value. Conversely, each element of the 4 × 4 pixel quantization matrix is encoded. The 16 × 4 pixel and 4 × 16 pixel quantization matrices are obtained by interpolating each element in the 4 × 4 pixel quantization matrix as an element in the quantization matrix as it is, and using the difference as a predicted value. You may encode. In the encoding of the quantization matrix, the method of encoding each element has been described, but the present invention is not limited to this.

  In the present embodiment, the method of switching the quantization matrix input to the quantization unit 104 according to the sub-block size has been described. However, the present invention is not limited to this. For example, a method may be used in which a quantization unit corresponding to each quantization matrix is prepared and switched based on the output of the quantization matrix selection unit 107.

  Further, in this embodiment, the quantization matrix is selected after performing the orthogonal transformation of the prediction error.However, the present invention is not limited to this, and the orthogonal transformation is performed after the selection of the quantization matrix first. The selection of the quantization matrix and the orthogonal transformation may be performed in parallel.

<Embodiment 2>
FIG. 3 is a block diagram showing the configuration of the image decoding apparatus according to Embodiment 2 of the present invention. In the present embodiment, decoding of the bitstream generated in the first embodiment shown in FIG. 1 will be described.

  In FIG. 3, reference numeral 301 denotes a decoding / separating unit that decodes header information of an input bit stream, separates necessary codes from the bit stream, and outputs them to the subsequent stage. The decoding / separating unit 301 performs the reverse operation of the integrated encoding unit 106 in FIG.

  A quantization matrix decoding unit 308 extracts and decodes quantization matrix code data from the header information of the bitstream.

  Reference numeral 309 denotes a quantization matrix holding unit that temporarily stores the quantization matrix decoded by the quantization matrix decoding unit 308.

  306 decodes prediction / conversion information in units of basic blocks from the code separated by the decoding / separation unit 301, and determines a prediction method and a sub-block size that is a unit of prediction / conversion based on the information. It is a conversion method extraction part.

  Reference numeral 307 denotes a quantization matrix selection unit that selects one of the quantization matrices stored in the quantization matrix holding unit 309 based on the sub-block size determined by the prediction / conversion method extraction unit 306.

  On the other hand, 302 is a coefficient decoding unit that decodes the quantized coefficient code from the code separated by the decoding / separating unit 301 and reproduces the quantized coefficient.

  Reference numeral 303 denotes an inverse quantization unit that performs inverse quantization on the quantization coefficient using the quantization matrix selected by the quantization matrix selection unit 307 and reproduces an orthogonal transform coefficient.

  Reference numeral 304 denotes an inverse transform unit that performs inverse orthogonal transform that is the inverse of the transform unit 103 in FIG. 1 and reproduces a prediction error.

  Reference numeral 305 denotes a prediction reconstruction unit that reproduces image data of a basic block from prediction information including a sub-block size serving as a prediction unit, a prediction error, and decoded image data.

  An image decoding operation in the image decoding apparatus will be described below. In the present embodiment, the moving image bit stream generated in the first embodiment is input in units of frames. However, a configuration may be adopted in which a still image bit stream for one frame is input. Further, in the present embodiment, only the intra prediction decoding process will be described for ease of explanation, but the present invention is not limited to this and can be applied to the inter prediction decoding process.

  In FIG. 3, the input stream data for one frame is input to the decoding / separating unit 301, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated and output. . The quantization matrix code data included in the header information is input to the quantization matrix decoding unit 308, and a quantization matrix corresponding to the sub-block size is reproduced as a quantization matrix used in the subsequent inverse quantization process. In the present embodiment, a first quantization matrix corresponding to a 4 × 4 pixel sub-block, a second quantization matrix corresponding to an 8 × 8 pixel sub-block, and a third quantization matrix corresponding to a 16 × 16 pixel sub-block are provided. Reproduce. Furthermore, five types of fourth quantization matrices corresponding to 16 × 4 pixel sub-blocks and fifth quantization matrices corresponding to 4 × 16 pixel sub-blocks are reproduced. In this embodiment, there are five types of sub-block sizes: 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, 16 × 4 pixels, and 4 × 16 pixels. Therefore, there are five types of quantization matrices to be reproduced, but the quantization matrix to be reproduced is also increased or decreased in accordance with the increase or decrease of the shape of the orthogonal transformation. The reproduced quantization matrix is input to the quantization matrix holding unit 309 and temporarily stored.

  Of the codes separated by the decoding / separation unit 301, the codes related to prediction and conversion are input to the prediction / conversion method extraction unit 306 and decoded to generate information indicating the prediction method and the sub-block size. The sub-block size is output to the quantization matrix selection unit 307 and the coefficient decoding unit 302. The decoded prediction method is output to the prediction reconstruction unit 305.

  The quantization matrix selection unit 307 selects one of the quantization matrices stored in the quantization matrix holding unit 309 according to the input sub-block size information and outputs the selected one to the inverse quantization unit 303. For example, in this embodiment, when the sub-block size is 4 × 16 pixels, the fifth quantization matrix is selected from the quantization matrix holding unit 309 and output to the inverse quantization unit 303.

  Further, among the codes separated by the decoding / separating unit 301, the quantized coefficient code data is input to the coefficient decoding unit 302. Then, the quantization coefficient code data input from the prediction / conversion method extraction unit 306 is decoded in units of sub-block sizes, the quantization coefficient is reproduced, and is output to the inverse quantization unit 303.

  The inverse quantization unit 303 receives the quantization matrix selected by the quantization matrix selection unit 307 and the quantization coefficient reproduced by the coefficient decoding unit 302. Then, inverse quantization is performed using the quantization matrix, and orthogonal transform coefficients are reproduced and output to the inverse transform unit 304.

  The inverse transform unit 304 inputs the reproduced orthogonal transform coefficient, performs inverse orthogonal transform that is the inverse of the transform unit 103 in FIG. 1, reproduces a prediction error, and outputs the prediction error to the prediction reconstruction unit 305.

  The prediction reconstruction unit 305 performs sub-block unit prediction based on the prediction method decoded by the prediction / conversion method extraction unit 306 from the surrounding pixel data that has been decoded into the input prediction error, and generates image data in basic block units. Play and output.

  FIG. 6 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the second embodiment.

  First, in step S601, the decoding / separating unit 301 decodes header information.

  In step S602, the quantization matrix decoding unit 308 decodes the quantization matrix code data included in the header information, and reproduces the quantization matrix used in the subsequent inverse quantization process. The decoded quantization matrix is stored in the quantization matrix holding unit 309. In this embodiment, a 4 × 4 pixel first quantization matrix, an 8 × 8 pixel second quantization matrix, and a 16 × 16 pixel third quantization matrix corresponding to each sub-block size are decoded and reproduced. . Then, five types of quantization matrices of a fourth quantization matrix of 16 × 4 pixels and a fifth quantization matrix of 4 × 16 pixels are decoded and reproduced. However, the decoding order is not limited to this.

  In step S603, the prediction / conversion method extraction unit 306 decodes information related to prediction and conversion in units of basic blocks, and generates a sub-block size that is a unit of the prediction method and prediction / orthogonal transformation from the content.

  In step S604, a determination is made based on the sub-block size generated in step S603. If the sub-block size is 4 × 4 pixels, the process proceeds to S605, if it is 8 × 8 pixels, the process proceeds to S625, if it is 16 × 16 pixels, the process proceeds to S645, and if it is 16 × 4 pixels, the process proceeds to S665. If it is 16 pixels, the process proceeds to S685.

  In step S605, the quantization matrix selection unit 307 selects a quantization matrix to be applied to the subblock based on the subblock size extracted in step S603. Specifically, since the extracted sub-block size is 4 × 4 pixels, the first quantization matrix is selected.

  In step S625, the quantization matrix selection unit 307 selects a quantization matrix to be applied to the subblock based on the subblock size. Specifically, since the extracted sub-block size is 8 × 8 pixels, the second quantization matrix is selected. In step S645, the quantization matrix selection unit 307 selects a quantization matrix to be applied to the subblock based on the subblock size. Specifically, since the extracted sub-block size is 16 × 16 pixels, the third quantization matrix is selected. In step S665, the quantization matrix selection unit 307 selects a quantization matrix to be applied to the subblock based on the subblock size. Specifically, since the extracted sub-block size is 16 × 4 pixels, the fourth quantization matrix is selected. In step S685, the quantization matrix selection unit 307 selects a quantization matrix to be applied to the subblock based on the subblock size. Specifically, since the extracted sub-block size is 4 × 16 pixels, the fifth quantization matrix is selected.

  In step S606, the coefficient decoding unit 302 decodes the quantized coefficient code data in units of subblocks and reproduces the quantized coefficients.

  In step S607, the inverse quantization unit 303 inversely quantizes the quantization coefficient reproduced in step S606 using the quantization matrix selected in steps S605, S625, S645, S665, and S685, and converts the orthogonal transform coefficient. Reproduce.

  In step S608, the determination based on the sub-block size generated in step S603 is performed again. If the sub-block size is 4 × 4 pixels, the process proceeds to step S609, if it is 8 × 8 pixels, the process proceeds to step S629, if it is 16 × 16 pixels, the process proceeds to step S649, and if it is 16 × 4 pixels, the process proceeds to step S669. If it is 4 × 16 pixels, the process proceeds to step S689.

  In step S609, the inverse transformation unit 304 performs inverse orthogonal transformation on the orthogonal transformation coefficient reproduced in step S607 in units of 4 × 4 pixel sub-blocks to reproduce the prediction error. In step S629, the inverse transformation unit 304 performs inverse orthogonal transformation on the orthogonal transformation coefficient reproduced in step S607 in units of 8 × 8 pixel sub-blocks to reproduce the prediction error. In step S649, the inverse transformation unit 304 performs inverse orthogonal transformation on the orthogonal transformation coefficient reproduced in step S607 in units of 16 × 16 pixels, and reproduces a prediction error. In step S669, the inverse transformation unit 304 performs inverse orthogonal transformation on the orthogonal transformation coefficient reproduced in step S607 in units of 16 × 4 pixels, and reproduces a prediction error. In step S689, the inverse transformation unit 304 performs inverse orthogonal transformation on the orthogonal transformation coefficient reproduced in step S607 in units of 4 × 16 pixel sub-blocks to reproduce the prediction error.

  In step S610, the prediction reconstruction unit 305 performs prediction from the decoded surrounding pixel data according to the prediction method reproduced in step S603. It is added to the prediction error reproduced in steps S609, S629, S649, S669, and S689 to reproduce the decoded image of the sub-block.

  In step S611, the image decoding apparatus determines whether or not decoding of all the subblocks in the basic block has been completed. If completed, the process proceeds to step S612. The process returns to step S604 for the block.

  In step S612, the image decoding apparatus determines whether or not the decoding of all the basic blocks has been completed. If the decoding has been completed, the image decoding apparatus stops all the operations and ends the process. Returning to step S603 for the basic block.

  With the above configuration and operation, the bit stream generated by performing the quantization process using the optimal quantization matrix corresponding to the shape of the orthogonal transform represented by the size of the sub-block generated in the first embodiment. Decoding can be performed to obtain a reproduced image.

  Further, as in the first embodiment, the size and shape of the basic block and the size and shape of the sub-block are not limited to this.

  Further, when the quantization matrix code data is included a plurality of times in one sequence of bitstream, the quantization matrix can be updated. The decoding / separation unit 301 detects the quantization matrix code data, and the quantization matrix decoding unit 308 decodes it. The decoded quantization matrix data is replaced with the corresponding quantization matrix in the quantization matrix holding unit 309. At this time, all the quantization matrices may be rewritten, or a part of them can be changed by determining the sub-block size of the rewritten quantization matrix. Furthermore, it is of course possible to partially change the elements in the matrix by determining the sub-block size of the quantization matrix to be rewritten and the position of the element to be changed.

  Moreover, although this embodiment demonstrated the method of switching the quantization matrix input into the inverse quantization part 303 according to subblock size, it is not limited to this. For example, a method of preparing an inverse quantization unit corresponding to each quantization matrix and switching it based on the output of the quantization matrix selection unit 307 may be used. Similarly, a plurality of steps S606 and S607 in FIG. 6 may be prepared, and the selection of the quantization matrix, the coefficient decoding, the inverse quantization, and the inverse transform may be performed in parallel with each sub-block size determined in step S604. .

  In this embodiment, the quantization coefficient is decoded after selecting the quantization matrix. However, the present invention is not limited to this, and the quantization matrix is selected after decoding the quantization coefficient first. Alternatively, the quantization matrix selection and the quantization coefficient decoding may be performed in parallel.

<Embodiment 3>
FIG. 8 is a block diagram showing an image encoding apparatus according to this embodiment. In FIG. 8, the same numbers are assigned to portions that perform the same functions as those in FIG. 1 of the first embodiment, and description thereof is omitted.

  Reference numeral 851 denotes an encoding method generation unit that generates encoding information indicating how to encode each quantization matrix.

  Reference numeral 809 denotes a quantization matrix encoding that encodes the quantization matrix stored in the quantization matrix holding unit 108 based on the encoding information generated by the encoding method generation unit 851 to generate quantization matrix code data. Part.

  As in the integrated encoding unit 106 in FIG. 1, an integrated encoding unit 806 generates a code related to header information, prediction, and conversion. The integrated encoding unit 106 is encoded information from the encoding method generation unit 851. Is different from the above and is encoded.

  An image encoding operation in the image encoding apparatus will be described below.

  The encoding method generation unit 851 first generates encoding information indicating how to encode each quantization matrix. In this embodiment, if the encoding method information is 0, a method of encoding each element of the quantization matrix as it is is used. Also, if 1 is used, a method of encoding a difference from the immediately preceding element is used, and if 2, a method of encoding a difference from an element of a quantization matrix having the same number of elements encoded immediately before is used. To do. Further, if the number is 3, the prediction quantization matrix is generated by thinning out the elements of the quantization matrix having the more number of elements encoded immediately before, and the difference from the prediction matrix is encoded. Also, if it is 4, the prediction quantization matrix is generated by interpolating the elements of the quantization matrix with the smaller number of elements encoded immediately before, and the difference between them is encoded. The encoding method of each element of the quantization matrix is not limited to these, and after encoding the difference between the first element, the second element, and the first element as in DPCM, the prediction difference may be encoded. . Further, it is of course possible to use a method of performing encoding with reference to a decoded quantization matrix designated by the immediately preceding or another code. Further, thinning is used as a prediction method for a quantization matrix having a smaller number of elements from a quantization matrix having a larger number of elements, but an average value, a median value, or the like may be used. Furthermore, an identifier indicating that the difference is not encoded at all and shared with other quantization matrices may be encoded. The combination of the quantization matrix encoding method information and the quantization matrix encoding method is not limited to this. The generation method of the quantization matrix encoding method information is not particularly limited, but may be input by the user, may be used as a fixed value, or may be stored in the quantization matrix holding unit 108. Of course, it may be calculated from the characteristics of the quantization matrix. The generated quantization matrix encoding method information is input to the quantization matrix encoding unit 809 and the integrated encoding unit 806.

  The quantization matrix encoding unit 809 encodes each quantization matrix stored in the quantization matrix holding unit 108 based on the input encoding information, generates quantization matrix code data, and generates an integrated encoding unit. Output to 806.

  The integrated encoding unit 806 encodes the encoded information generated by the encoding method generating unit 851, generates an encoded information code, and outputs the encoded information code after being included in the header information. Furthermore, the sub block size information generated by the prediction unit 102 is also encoded and output. The encoding method is not particularly limited, but a Huffman code or an arithmetic code can be used. FIG. 7B shows an example of a bit stream including an encoded information code. The encoding information code may be included in any of the headers of sequences, pictures, etc., but exists before each quantization matrix code data.

  FIG. 10 is a flowchart showing an image encoding process in the image encoding apparatus according to the third embodiment. In FIG. 10, the same numbers are assigned to portions that perform the same functions as those in FIG.

  In step S1051, the encoding method generation unit 851 determines the encoding information used in the subsequent step S1002.

  In step S1002, the quantization matrix encoding unit 809 encodes the quantization matrix generated in step S501 based on the encoding information determined in step S1051.

  In step S1003, the integrated encoding unit 806 encodes the encoded information, generates an encoded information code, and outputs the encoded information code in the header portion in the same manner as other codes.

  With the above configuration and operation, each quantization matrix is encoded with the optimal encoding method, and a bitstream is generated that has been quantized using the optimal quantization matrix corresponding to the orthogonal transform represented by the sub-block size. can do. Thereby, it is possible to suppress the amount of code generated by encoding the quantization matrix.

  In this embodiment, the case where one encoding method is selected for all the quantization matrices to be encoded has been described as an example, but the present invention is not limited to this. Of course, the encoding method may be selected for each quantization matrix, or the encoding method may be selected for each sub-block size.

<Embodiment 4>
FIG. 9 is a block diagram showing an image decoding apparatus according to this embodiment. In FIG. 9, the same numbers are assigned to portions that perform the same functions as those in FIG. 3 of the second embodiment, and description thereof is omitted.

  Reference numeral 901 denotes a decoding / separating unit that decodes header information of the input bit stream, separates necessary codes from the bit stream, and outputs them to the subsequent stage. 3 is different from the decoding / separating unit 301 in that the encoded information code is separated from the header information of the bitstream and output to the subsequent stage.

  A decoding method decoding unit 951 decodes the encoded information code separated by the decoding / separating unit 901 and reproduces information on the quantization matrix coding method.

  Reference numeral 908 denotes a quantization matrix decoding unit that decodes the quantization matrix code data separated by the decoding / separation unit 901 from the header information of the bitstream based on the information on the encoding method.

  An image decoding operation in the image decoding apparatus will be described below.

  In FIG. 9, the input stream data for one frame is input to the decoding / separating unit 901, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated and output. . The encoding information code included in the header information is input to the decoding method decoding unit 951, and information on the encoding method of the quantization matrix is reproduced. Then, the reproduced information on the encoding method of the quantization matrix is input to the quantization matrix decoding unit 908.

  On the other hand, the quantization matrix code data included in the header information is input to the quantization matrix decoding unit 908. Then, the quantization matrix decoding unit 908 selects the quantization matrix decoding method based on the input quantization matrix encoding method information, decodes the quantization matrix code data, and uses it in the subsequent dequantization processing. The resulting quantization matrix is reproduced. The reproduced quantization matrix is temporarily stored in the quantization matrix holding unit.

  FIG. 11 is a flowchart showing image decoding processing in the image decoding apparatus according to the fourth embodiment. The same numbers are assigned to portions that perform the same functions as those in FIG. 6 of the second embodiment, and descriptions thereof are omitted.

  In step S1101, the decoding / separating unit 901 decodes the header information.

  In step S1151, the decoding method decoding unit 951 decodes the quantization matrix coding information code included in the header information, and reproduces the quantization matrix coding method information.

  In step S1102, the quantization matrix decoding unit 908 selects a quantization matrix decoding method based on the quantization matrix encoding method information reproduced in step S1151. Then, the quantization matrix code data included in the header information is decoded to reproduce the quantization matrix.

  With the above-described configuration and operation, each quantization matrix generated in the third embodiment is encoded by an optimal encoding method, and is a bitstream that is quantized using an optimal quantization matrix depending on the sub-block size. Can be decoded to obtain a reproduced image. As a result, it is possible to decode a bitstream with a reduced code amount generated by encoding adapted to the quantization matrix.

  In the present embodiment, the case where one decoding method is selected for all the quantization matrix code data to be decoded has been described as an example, but the present invention is not limited to this. Of course, the decoding method may be selected in units of quantization matrix, or the encoding method may be selected for each sub-block size.

<Embodiment 5>
FIG. 13 is a block diagram showing an image encoding apparatus according to this embodiment. In FIG. 13, the same numbers are assigned to portions that perform the same functions as those in FIG. 1 of the first embodiment, and descriptions thereof are omitted.

  Reference numeral 1308 denotes a quantization matrix holding unit that generates and temporarily stores a quantization matrix. The method of generating the stored quantization matrix is not particularly limited, but the user may input the quantization matrix or use the one specified in advance as the initial value even if it is calculated from the characteristics of the input image. May be. In the present embodiment, it is assumed that a quantization matrix corresponding to orthogonal transformation of square shapes such as 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels is generated and stored. Furthermore, it is assumed that a quantization matrix corresponding to orthogonal transformation of a rectangular shape such as 16 × 4 pixels and 4 × 16 pixels generated by a quantization matrix reduction unit 1351 described later is further input and stored. The generated and stored quantization matrices are not limited to these. For example, a 32 × 32 pixel square quantization matrix is further generated, and reduced 32 × 8 pixel and 8 × 32 pixel rectangular quantization is performed. A matrix may be further stored.

  Reference numeral 1351 denotes a quantization matrix reduction unit that generates a rectangular quantization matrix by reducing one side of a square quantization matrix. In the present embodiment, a 16 × 16 pixel square quantization matrix generated by the quantization matrix holding unit 1308 is input, reduction processing is performed, and a 4 × 16 pixel and 16 × 4 pixel quantization matrix is generated. Shall. The reduction process will be described with reference to FIG.

  FIG. 17A shows a 16 × 16 pixel quantization matrix modeled using symbols. The symbols ◎, ○, Δ, and × indicate each element of the quantization matrix. In this embodiment, the quantization matrix reduction unit 1351 thins out the elements existing at the positions of ◎ and ○ in the vertical direction from the 16 × 16 pixel quantization matrix of FIG. Assume that the 16 × 4 pixel quantization matrix shown is generated. Further, from the 16 × 16 pixel quantization matrix of FIG. 17A, the elements existing at the positions of ◎ and Δ are thinned out in the horizontal direction, and the 4 × 16 pixel quantization matrix shown in FIG. Shall be generated.

  Reference numeral 1309 denotes a quantization matrix encoding unit that encodes a square quantization matrix in the quantization matrix stored in the quantization matrix holding unit 1308 to generate quantization matrix code data. Like the quantization matrix encoding unit 109 in FIG. 1, the quantization matrix encoding method is not particularly limited. A Huffman code, an arithmetic code, or the like can be used for the value of each element of the quantization matrix itself, or a difference from the immediately preceding element or a difference from another quantization matrix.

  An integration 1306 generates codes related to header information, prediction, and conversion, and integrates the quantization coefficient code data generated by the coefficient encoding unit 105 and the quantization matrix code data generated by the quantization matrix encoding unit 1309. It is an encoding part.

  An image encoding operation in the image encoding apparatus will be described below.

  The quantization matrix holding unit 1308 holds a quantization matrix corresponding to the shape of the square sub-block among the quantization matrices used for quantization of the frame. In this embodiment, there are five types of sub-block sizes: 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, 16 × 4 pixels, and 4 × 16 pixels. Therefore, a first quantization matrix corresponding to an orthogonal transformation of 4 × 4 pixels having a square shape therein, a second quantization matrix corresponding to an orthogonal transformation of 8 × 8 pixels, and an orthogonal transformation of 16 × 16 pixels A third quantization matrix corresponding to is held. In the present embodiment, the above three types of quantization matrices are held, but the held quantization matrices are also increased or decreased in accordance with the increase or decrease in the shape of the sub-block. Since it is necessary to generate a quantization matrix corresponding to the orthogonal transformation of 16 × 4 pixels and 4 × 16 pixels, a 16 × 16 pixel quantization matrix is output to the quantization matrix reduction unit 1351.

  The quantization matrix reduction unit 1351 reduces the input 16 × 16 pixel quantization matrix by the method described above, generates a 16 × 4 pixel and 4 × 16 pixel quantization matrix, and a quantization matrix holding unit 1308. Output to.

  The quantization matrix holding unit 1308 further stores the 16 × 4 pixel and 4 × 16 pixel quantization matrices input from the quantization matrix reduction unit 1351. That is, the fourth quantization matrix corresponding to the 16 × 4 pixel orthogonal transformation and the fifth quantization matrix corresponding to the 4 × 16 pixel orthogonal transformation are held.

  The quantization matrix encoding unit 1309 sequentially reads the quantization matrix corresponding to the square orthogonal transform from the quantization matrix holding unit 1308 according to the sub-block size and encodes it to generate quantization matrix code data. In the present embodiment, a first quantization matrix corresponding to orthogonal transformation of 4 × 4 pixels, a second quantization matrix corresponding to orthogonal transformation of 8 × 8 pixels, and a third quantum corresponding to orthogonal transformation of 16 × 16 pixels. Three types of quantization matrices are encoded. The encoding method is not limited as in the quantization matrix encoding unit 109 in FIG.

  The integrated encoding unit 1306 generates codes such as an image sequence, a frame, a picture, and a slice header as in the integrated encoding unit 106 of FIG. Further, the quantization matrix code data generated by the quantization matrix encoding unit 1309 is inserted into any of these headers. FIG. 7C is an example of a bit stream output in this embodiment. In this embodiment, the quantization matrix is encoded in the sequence header portion in the bit stream, but the encoding position is not limited to this.

  FIG. 15 is a flowchart showing an image encoding process in the image encoding apparatus according to the fifth embodiment. In FIG. 15, the same numbers are assigned to portions that perform the same functions as those in FIG.

  First, in step S1501, the quantization matrix holding unit 1308 generates one or more quantization matrices. In the present embodiment, there are five types of sub-block sizes, and among them, there are three types of square shapes. As described above, three types of quantization matrices from the first to the third are generated.

  In step S1502, the quantization matrix encoding unit 1309 encodes each square quantization matrix generated by the quantization matrix holding unit 1308.

  In step S1551, the quantization matrix reduction unit 1351 reduces the third quantization matrix by the above-described method, and supports the fourth quantization matrix corresponding to the orthogonal transformation of 16 × 4 pixels and the orthogonal transformation of 4 × 16 pixels. A fifth quantization matrix is generated.

  In step S1503, the integrated encoding unit 1306 encodes and outputs the header portion of the bitstream. Here, the quantization matrix code data encoded in step S1502 is output together with the header.

  In this embodiment, step S1551 is executed between step S1502 and step S1503, but the execution order of step S1551 is not limited to this. It suffices if it is after step S1501 and before step S504.

  With the above configuration and operation, it is possible to generate a bit stream in which only a square-shaped quantization matrix is encoded while performing quantization using an optimal quantization matrix corresponding to the orthogonal transform represented by each sub-block size. . Thereby, it is possible to suppress the amount of code generated by encoding the quantization matrix.

  In the present embodiment, the quantization matrix corresponding to the rectangular orthogonal transform generated by the quantization matrix reduction unit 1351 is stored in the quantization matrix holding unit 1308. However, the embodiment is not limited thereto. Not. The quantization matrix holding unit 1308 may store only a square quantization matrix and reduce the square quantization matrix each time a rectangular quantization matrix is used, thereby generating a rectangular quantization matrix. In this way, it is also possible to reduce the amount of memory necessary for holding the quantization matrix in the quantization matrix holding unit 1308.

  In this embodiment, when the reduction process from 16 pixels to 4 pixels is performed, the 4n (where n is an integer) element is extracted and the reduction process is realized by thinning out other elements. The processing method is not limited to this. The reduction process may be realized by extracting the 4n + 1, 4n + 2, or 4n + 3th element and thinning out other elements. Furthermore, the reduction process may be realized by taking the average of the 4n, 4n + 1, 4n + 2, 4n + 3th four elements.

  Further, in the present embodiment, the reduction processing of 1/4 times that generates 16 × 4 pixels and 4 × 16 pixels from 16 × 16 pixels is shown, but the magnification is not limited thereto. The essence of the invention does not change from 16 × 16 pixels to 1/8 times 16 × 2 pixels and 2 × 16 pixels, or from 16 × 16 pixels to 1/2 times 16 × 8 pixels and 8 × 16 pixels. Furthermore, a method of converting 16 × 4 pixels generated after generating 16 × 4 pixels from 16 × 16 pixels into 16 × 2 pixels may be used.

  Furthermore, in the present embodiment, the 16 × 16 pixel quantization matrix is reduced. However, the present invention is not limited to this, and the 32 × 32 pixel, 8 × 8 pixel, or larger quantization matrix is reduced. Thus, a rectangular quantization matrix may be generated.

<Embodiment 6>
FIG. 14 is a block diagram showing a configuration of an image decoding apparatus according to Embodiment 6 of the present invention. In the present embodiment, decoding of the bitstream generated in the fifth embodiment illustrated in FIG. 13 will be described. In FIG. 14, the same numbers are assigned to portions that perform the same functions as those in FIG. 3 of the second embodiment, and description thereof is omitted.

  In FIG. 14, reference numeral 14001 denotes a decoding / separating unit that decodes header information of an input bit stream, separates necessary codes from the bit stream, and outputs them to the subsequent stage. The decoding / separating unit 14001 performs the reverse operation of the integrated encoding unit 1306 in FIG.

  Reference numeral 14008 denotes a quantization matrix decoding unit that extracts and decodes quantization matrix code data from the header information of the bit stream. The bit stream generated in the sixth embodiment is encoded with a quantization matrix corresponding to orthogonal transformation of three types of squares of 4 × 4 pixels, 8 × 8 pixels, and 16 × 16 pixels. Then, these three kinds of quantization matrices are reproduced.

  Reference numeral 14051 denotes a quantization matrix reduction unit that generates a rectangular quantization matrix by reducing one side of a square quantization matrix. The same operation as that of the quantization matrix reduction unit 1351 of FIG. In the present embodiment, a 16 × 16 pixel square quantization matrix reproduced by the quantization matrix decoding unit 14008 is input, reduction processing is performed, and 4 × 16 pixel and 16 × 4 pixel quantization matrices are generated. Shall. The reduction process is the same as that of the quantization matrix reduction unit 1351 in FIG.

  Reference numeral 14009 denotes a quantization matrix holding unit that temporarily stores the square quantization matrix reproduced by the quantization matrix decoding unit 14008 and the rectangular quantization matrix generated by the quantization matrix reduction unit 14051.

  An image decoding operation in the image decoding apparatus will be described below. In the present embodiment, the moving image bitstream generated in the fifth embodiment is input in units of frames. However, a configuration may be adopted in which a still image bitstream for one frame is input. Alternatively, a divided bit stream such as a slice unit or a packet unit may be sequentially input.

  In FIG. 14, the input stream data for one frame is input to the decoding / separating unit 14001, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated and output. .

  The quantization matrix code data included in the header information is input to the quantization matrix decoding unit 14008, and a quantization matrix corresponding to the sub-block size is reproduced as a quantization matrix used in the subsequent inverse quantization process. In the present embodiment, the first quantization matrix corresponding to the 4 × 4 pixel sub-block, the second quantization matrix corresponding to the 8 × 8 pixel sub-block, and the third quantization matrix corresponding to the 16 × 16 pixel sub-block Three types of square quantization matrix are reproduced. In the present embodiment, there are five types of sub-block sizes of 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, 16 × 4 pixels, and 4 × 16 pixels. It is a kind. For this reason, there are three types of quantization matrices to be reproduced, but the quantization matrices to be reproduced also increase / decrease as the shape of orthogonal transformation increases / decreases.

  The reproduced square quantization matrix is input to the quantization matrix reduction unit 14051, and a quantization matrix corresponding to rectangular orthogonal transformation is generated from the input square quantization matrix. In this embodiment, since there are two types of orthogonal transform sizes of 16 × 4 pixels and 4 × 16 pixels, a 16 × 4 pixel quantization matrix is replaced with a 16 × 4 pixel quantization matrix. Reduce to generate. The generated rectangular quantization matrix is input to the quantization matrix holding unit 14009 together with the square quantization matrix reproduced by the quantization matrix decoding unit 14008, and is temporarily stored.

  Of the codes separated by the decoding / separation unit 14001, the codes related to prediction and conversion are input to the prediction / conversion method extraction unit 306 and decoded to generate information indicating the prediction method and the sub-block size. Further, among the codes separated by the decoding / separating unit 14001, the quantized coefficient code data is input to the coefficient decoding unit 302.

  FIG. 16 is a flowchart illustrating an image decoding process in the image decoding apparatus according to the sixth embodiment. The same numbers are assigned to portions that perform the same functions as those in FIG. 6 of the second embodiment, and descriptions thereof are omitted.

  First, in step S1601, the decoding / separating unit 14001 decodes the header information.

  In step S1602, the quantization matrix decoding unit 14008 decodes the quantization matrix code data included in the header information, and the quantization matrix used in the subsequent inverse quantization process is reproduced. In this embodiment, the first quantization matrix of 4 × 4 pixels, the second quantization matrix of 8 × 8 pixels, and the third quantization matrix of 16 × 16 pixels corresponding to each square sub-block size are decoded and reproduced. To do. However, the decoding order is not limited to this.

  In step S1651, the quantization matrix reduction unit 14051 reduces the third quantization matrix by the above-described method, and supports the fourth quantization matrix corresponding to the orthogonal transformation of 16 × 4 pixels and the orthogonal transformation of 4 × 16 pixels. A fifth quantization matrix is generated. The processing in this step is the same as that in step S1551 in FIG.

  With the above configuration and operation, a bit stream generated by encoding the square quantization matrix only while quantizing using the optimum quantization matrix corresponding to the orthogonal transform represented by each sub-block size, generated in the fifth embodiment. Can be decoded to obtain a reproduced video. As a result, it is possible to decode a bitstream with a reduced code amount generated by encoding the quantization matrix.

  In the present embodiment, the quantization matrix corresponding to the rectangular orthogonal transform generated by the quantization matrix reduction unit 14051 is stored in the quantization matrix holding unit 14009. However, the embodiment is not limited thereto. Not. The quantization matrix holding unit 14009 may store only a square quantization matrix and reduce the square quantization matrix every time a rectangular quantization matrix is used, thereby generating a rectangular quantization matrix. In this way, it is possible to reduce the amount of memory required for holding the quantization matrix in the quantization matrix holding unit 14009.

  In this embodiment, when the reduction process from 16 pixels to 4 pixels is performed, the 4n (where n is an integer) element is extracted and the reduction process is realized by thinning out other elements. The processing method is not limited to this. The reduction process may be realized by extracting the 4n + 1, 4n + 2, or 4n + 3th element and thinning out other elements. Furthermore, the reduction process may be realized by taking the average of the 4n, 4n + 1, 4n + 2, 4n + 3th four elements.

  Further, in the present embodiment, the reduction processing of 1/4 times that generates 16 × 4 pixels and 4 × 16 pixels from 16 × 16 pixels is shown, but the magnification is not limited thereto. The essence of the invention does not change from 16 × 16 pixels to 1/8 times 16 × 2 pixels and 2 × 16 pixels, or from 16 × 16 pixels to 1/2 times 16 × 8 pixels and 8 × 16 pixels.

  Furthermore, in the present embodiment, the 16 × 16 pixel quantization matrix is reduced, but the 32 × 32 pixel and 8 × 8 pixel quantization matrix is reduced to generate a rectangular quantization matrix. Also good.

<Embodiment 7>
In the above embodiment, each processing unit shown in FIGS. 1, 3, 8, 9, 13, and 14 is configured by hardware. However, the processing performed in each processing unit shown in FIG. 1, FIG. 3, FIG. 8, FIG. 9, FIG.

  FIG. 12 is a block diagram illustrating a configuration example of computer hardware applicable to the image display device according to each of the embodiments.

  The CPU 1401 controls the entire computer using computer programs and data stored in the RAM 1402 and the ROM 1403, and executes each process described above as performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 1401 functions as each processing unit shown in FIGS. 1, 3, 8, 9, 13, and 14.

  The RAM 1402 has an area for temporarily storing computer programs and data loaded from the external storage device 1406, data acquired from the outside via an I / F (interface) 1409, and the like. Further, the RAM 1402 has a work area used when the CPU 1401 executes various processes. That is, the RAM 1402 can be allocated as, for example, a frame memory or can provide other various areas as appropriate.

  The ROM 1403 stores setting data of the computer, a boot program, and the like. The operation unit 1404 is configured by a keyboard, a mouse, and the like, and can input various instructions to the CPU 1401 when operated by a user of the computer. A display unit 1405 displays a processing result by the CPU 1401. The display unit 1405 is configured by a display device such as a liquid crystal display.

  The external storage device 1406 is a mass information storage device represented by a hard disk drive device. The external storage device 1406 stores an OS (operating system) and computer programs for causing the CPU 1401 to realize the functions of the respective units shown in FIGS. 1, 3, 8, 9, 13, and 14. Yes. Further, each image data as a processing target may be stored in the external storage device 1406.

  Computer programs and data stored in the external storage device 1406 are appropriately loaded into the RAM 1402 under the control of the CPU 1401 and are processed by the CPU 1401. The I / F 1407 can be connected to a network such as a LAN or the Internet, and other devices such as a projection device and a display device. The computer can acquire and send various information via the I / F 1407. Can be. A bus 1408 connects the above-described units.

  The operation having the above-described configuration is controlled by the CPU 1401 as the operation described in the above flowchart.

<Other embodiments>
The object of the present invention can also be achieved by supplying a storage medium storing a computer program code for realizing the above-described functions to the system, and the system reading and executing the computer program code. In this case, the computer program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the computer program code constitutes the present invention. In addition, the operating system (OS) running on the computer performs part or all of the actual processing based on the code instruction of the program, and the above-described functions are realized by the processing. .

  Furthermore, you may implement | achieve with the following forms. That is, the computer program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the above-described functions are realized by the CPU or the like provided in the function expansion card or function expansion unit performing part or all of the actual processing.

  When the present invention is applied to the above storage medium, the computer program code corresponding to the flowchart described above is stored in the storage medium.

Claims (7)

Block dividing means for dividing the input image into a plurality of blocks;
A prediction unit that performs prediction from a block that has been encoded with respect to the block divided by the block division unit, and generates a prediction error;
Transform means for determining a processing block shape to be used from a plurality of square or rectangular orthogonal processing block shapes, performing the determined orthogonal transform on the prediction error generated by the prediction means, and generating transform coefficients; ,
Quantization matrix selection means for selecting a quantization matrix based on the processing block shape of the orthogonal transformation determined by the transformation means;
Quantization means for quantizing a transform coefficient using the quantization matrix selected by the quantization matrix selection means to generate a quantization coefficient;
An image encoding apparatus comprising coefficient encoding means for encoding the quantized coefficient generated by the quantization means. A quantization matrix generating means for generating a quantization matrix corresponding to a plurality of square or rectangular orthogonal transform processing block shapes used when quantizing the transform coefficient;
Quantization matrix encoding means for encoding the quantization matrix generated by the quantization matrix generation means;
2. The apparatus according to claim 1, further comprising: a quantization matrix encoded by the quantization matrix encoding unit; and an integrated encoding unit that integrates the quantized coefficients encoded by the coefficient encoding unit. The image encoding device described. Decoding means for separating necessary codes from the input bitstream;
Determining means for determining a processing block shape of inverse orthogonal transform from the information decoded by the decoding means;
Coefficient decoding means for decoding quantized coefficients from the information decoded by the decoding means;
Quantization matrix decoding means for decoding a quantization matrix used when inversely quantizing the quantized coefficients decoded by the coefficient decoding means;
A quantization matrix selection unit that selects a quantization matrix based on the processing block shape of the inverse orthogonal transform determined by the determination unit;
Using the quantization matrix selected by the quantization matrix selection means, inverse quantization means for dequantizing the quantized coefficients decoded by the coefficient decoding means to generate transform coefficients;
And an inverse orthogonal transform unit that generates a prediction error by performing an inverse orthogonal transform on the transform coefficient generated by the inverse quantization unit based on the shape of the inverse orthogonal transform determined by the determination unit. Image decoding device. An image encoding method in an image encoding device, comprising:
A block dividing step of dividing the input image into a plurality of blocks;
A prediction step of performing prediction from the encoded block on the divided block and generating a prediction error;
A transformation step of determining a processing block shape to be used from a plurality of square or rectangular orthogonal transformation block shapes, performing the determined orthogonal transformation on the generated prediction error, and generating a transformation coefficient;
A quantization matrix selection step of selecting a quantization matrix based on the determined processing block shape of the orthogonal transform;
A quantization step of quantizing the transform coefficient using the selected quantization matrix to generate a quantization coefficient;
An image encoding method comprising: a coefficient encoding step for encoding the generated quantized coefficient. An image decoding method in an image decoding device, comprising:
A decoding step of separating necessary codes from the input bitstream;
A determination step of determining a processing block shape of inverse orthogonal transform from the decoded information;
A coefficient decoding step of decoding quantized coefficients from the decoded information;
A quantization matrix decoding step for decoding a quantization matrix used when dequantizing the decoded quantization coefficient;
A quantization matrix selection step of selecting a quantization matrix based on the determined inverse orthogonal transform processing block shape;
Using the selected quantization matrix, dequantizing the decoded quantized coefficients to generate transform coefficients; and
An image decoding method comprising: an inverse orthogonal transform step of generating a prediction error by performing an inverse orthogonal transform on a generated transform coefficient based on the determined shape of the inverse orthogonal transform.   A program that causes the computer to function as the encoding device according to claim 1 by being read and executed by the computer.   A program that causes the computer to function as the decoding device according to claim 3 by being read and executed by the computer.

Images