JP2012147290A - Image coding apparatus, image coding method, program, image decoding apparatus, image decoding method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To encode intra prediction mode information as side information for identifying a selected mode in an encoding method for performing intra prediction having a plurality of modes, and a bit for that is required.
An encoding control parameter is set, pixel data of a block having a first size is subjected to frequency conversion, a part of the conversion coefficient is removed and replaced with a predetermined value, and conversion is performed. The pixel data of a block having a small second size is frequency-converted, the use of the first conversion step is restricted based on the encoding control parameter, and the first conversion step is performed based on the determined block size Or the output of the second conversion step is controlled and encoded.
[Selection] Figure 1

Description

  The present invention relates to an image encoding device, an image encoding method and program, an image decoding device, an image decoding method and a program, and more particularly to an encoding method for encoding and decoding an image by dividing the image into blocks of a plurality of sizes.

  As a method for compressing and recording a moving image, H.264 is used. H.264 / MPEG-4 AVC (hereinafter referred to as H.264) is known. H. The H.264 encoding method is widely used in one-segment digital terrestrial broadcasting. H. The H.264 encoding method is characterized in that a plurality of intra predictions are prepared using integer conversion in units of 4 × 4 pixels in addition to the conventional encoding method. H. In the H.264 encoding method, a 16 × 16 macroblock is used as a unit for encoding, and the inside thereof is divided into 4 × 4 blocks and used as a unit for conversion. In addition, conversion by 8 × 8 blocks is also adopted in a part of intra coding. Further, H.C. As an improved technique based on the H.264 encoding method, for example, H.264. There is a technique for expanding the block size of H.264 to a block of 64 pixels × 64 pixels.

ISO / IEC14496-10: 2004 Information technology--Coding of audio-visual objects--Part10: Advanced Video Coding ITU-TH. H.264 Advanced video coding for generic audiovisual services

  However, in a large block such as a 64 pixel × 64 pixel block or a 32 pixel × 32 pixel block, the cost of the orthogonal transform itself is high and the number of generated coefficients is very large. take time. On the other hand, as a method of reducing the coefficient and performing encoding and compressing the code amount, a method of forcibly setting a high frequency coefficient to 0 after orthogonal transformation is known. Further, a method of sending only 64 coefficients in a 32 pixel × 32 pixel block is conceivable. By adopting this, the coefficient transmitted in a large block can be reduced, and the compression efficiency can be improved.

  However, since the high frequency coefficient is reduced uniformly, the high frequency is not reproduced and the image quality is greatly deteriorated. H. Since the residual component by motion compensation and intra prediction, which is also employed in H.264, contains more high frequency, the deterioration is further increased. Also, if the quantization step is small, the number of quantization coefficients having a value other than 0 after quantization becomes very large. However, since the high frequency is uniformly set to 0, there is a problem that the image quality does not increase even if the quantization step is lowered. Has occurred.

  In addition, conversion with a large block size involves many input / output of pixels and input / output of conversion coefficients, and the calculation cost is very high.

  In order to solve the above-described problems, the image coding apparatus of the present invention has the following configuration. That is, a setting unit that sets an encoding control parameter, and a first conversion unit that performs frequency conversion on pixel data of a block having a first size and replaces the data with a predetermined value except for a part of the conversion coefficient. A second conversion unit that frequency-converts pixel data of a block having a second size smaller than the first size, and a block that restricts use of the first conversion unit based on the encoding control parameter It comprises size determining means and encoding means for controlling and encoding the output of the first converting means or the output of the second converting means based on the determined block size.

  According to the present invention, it is possible to improve the image quality by restricting replacement with a fixed value of the transform coefficient in a large block. In particular, the block size can be suitably limited by determining the block size in relation to the quantization parameter that greatly contributes to the generation of coefficients. In addition, by limiting the block size, there is an effect that encoded data representing block division can be reduced.

It is a block diagram showing the structure of the image coding apparatus of 1st Embodiment. It is a block diagram showing the structure of the conversion block size determination part of 1st Embodiment. It is a figure showing arrangement | positioning of the value of the coefficient of a conversion block. It is a figure showing an example of arrangement | positioning of a conversion block. It is a figure showing an example of arrangement | positioning of a conversion block. It is a block diagram showing the detail of the orthogonal transformation part of 1st Embodiment. It is a block diagram showing another structure of the image coding apparatus of 1st Embodiment. It is a block diagram showing another structure of the conversion block size determination part of 1st Embodiment. It is a flowchart figure which shows the operation | movement of the image coding process of 1st Embodiment. It is a block diagram showing another structure of the image coding apparatus of 1st Embodiment. It is a flowchart figure which shows the operation | movement of the division | segmentation flag encoding process of 1st Embodiment. It is a block diagram showing the structure of the image coding apparatus of 2nd Embodiment. It is a flowchart figure which shows the operation | movement of the division | segmentation flag encoding process of 2nd Embodiment. It is a block diagram showing another structure of the image coding apparatus of 2nd Embodiment. It is a block diagram showing another structure of the image coding apparatus of 2nd Embodiment. It is a block diagram showing the structure of the image decoding apparatus of 3rd Embodiment. It is a block diagram showing the detail of the inverse orthogonal transformation part of 3rd Embodiment. It is a flowchart figure which shows the operation | movement of the image decoding process of 3rd Embodiment. It is a flowchart figure which shows the operation | movement of the division flag decoding process of 3rd Embodiment. It is a block diagram showing the structure of the image decoding apparatus of 4th Embodiment. It is a flowchart figure which shows the operation | movement of the division | segmentation flag decoding process of 4th Embodiment. It is a block diagram showing another structure of the image decoding apparatus of 4th Embodiment. It is a block diagram which shows the structural example of the hardware of the computer applicable to the image coding apparatus of this invention, and a decoding apparatus.

  Hereinafter, the present invention will be described in detail based on the preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<Embodiment 1>
An embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing a configuration of an image encoding apparatus according to the first embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a terminal for inputting image data. A buffer 2 temporarily stores image data. Reference numeral 3 denotes a basic block dividing unit that divides an image into a plurality of basic blocks. In the present embodiment, for ease of explanation, the basic block size is described as 32 pixels × 32 pixels, but the present invention is not limited to this. Reference numeral 4 denotes a transform block size / division determination unit that divides a basic block and forms a transform block having a block size for performing orthogonal transform. Here, the conversion block is divided into conversion blocks of 32 pixels × 32 pixels, 16 pixels × 16 pixels, 8 pixels × 8 pixels, and 4 pixels × 4 pixels, which are not divided into blocks. However, the division method is not limited to this.

  A prediction unit 8 performs intra prediction or motion compensation prediction in units of divided transform blocks, and calculates a prediction error. An orthogonal transform unit 9 obtains a spatial frequency coefficient by performing orthogonal transform on the prediction error. In the present embodiment, DCT conversion is described as an example of frequency conversion, but the present invention is not limited to this. Of course, conversion such as Hadamard conversion may be used. The orthogonal transformation unit 9 has a function of performing orthogonal transformation corresponding to a plurality of types of transformation block sizes divided by the transformation block division / size determination unit 4. However, for the orthogonal transformation of 32 pixels × 32 pixels and 16 pixels × 16 pixels, only the low frequency 8 pixels × 8 pixels coefficients are calculated, and 0 is output otherwise. FIG. 3 is a diagram showing the arrangement of the coefficient values of the transform block. FIG. 3A is a diagram illustrating coefficients after orthogonal transformation of 32 pixels × 32 pixels. Similarly, FIG. 3B is a diagram illustrating coefficients after orthogonal transformation of 16 pixels × 16 pixels. In any of the figures, the black pixel holds the coefficient obtained by the orthogonal transformation, but other white pixels have a value of zero. The arrangement of the retained coefficients is not limited to this.

  A quantization unit 10 quantizes the obtained coefficient in accordance with a quantization parameter (encoding control parameter) to obtain a quantization coefficient. Reference numeral 11 denotes an entropy encoding unit that encodes various headers, quantization parameters of each block, and the like, and further performs entropy encoding on the quantized coefficients. The entropy encoding method is not particularly limited, but encoding by arithmetic code or Huffman code is performed.

  Reference numeral 5 denotes a terminal that receives an instruction for a user (not shown) to control the image quality, and reference numeral 6 denotes a quantization step setting unit that generates an initial value of a quantization parameter to be set in the quantization unit 10 in accordance with the instruction. Reference numeral 7 denotes a maximum transform block size determination unit that determines the maximum transform block size when the transform block partition unit divides by referring to the initial value of the quantization parameter.

Reference numeral 15 denotes an inverse quantization unit that restores a coefficient from the quantized coefficient generated by the quantization unit 10, and reference numeral 16 denotes an inverse orthogonal that performs inverse transformation of the restored coefficient by the orthogonal transformation unit 9 to restore a prediction error. It is a conversion unit. Reference numeral 17 denotes a prediction error adding unit that adds the prediction error and the prediction result. A frame memory 18 stores the decoded image. Reference numeral 19 denotes a motion compensation unit that performs motion compensation by performing motion detection by comparing the contents of the frame memory 18 and input image data in units of transform blocks. Reference numeral 20 denotes a motion vector encoder that encodes a motion vector as a result of the motion detection to generate a motion vector code. Reference numeral 21 denotes a block division information encoding unit that encodes information in a state where the basic block is divided into conversion blocks by the conversion block division / size determination unit 4 and generates a block division code. Reference numeral 12 denotes a multiplexing unit that integrates and aligns outputs from the entropy encoding unit 11, the motion vector encoding unit 20, and the block division information encoding unit 21, and outputs the result as a bit stream. Reference numeral 13 denotes a terminal for outputting the bit stream to the outside. A rate control unit 14 controls the quantization parameter based on the bitstream code amount.
In the above configuration, the operation of the image encoding process will be described.

  Prior to the processing, a user (not shown) sends an instruction to the quantization parameter setting unit 6 via the terminal 5 regarding the image quality at the time of encoding. The quantization parameter setting unit 6 sets a small value quantization parameter if the user desires high image quality, and sets a large value quantization parameter if the user desires to reduce bits even with low image quality. In the present embodiment, for ease of explanation, it is assumed that a high image quality mode in which the quantization parameter QP is set to a small value QS and a high efficiency mode in which the quantization parameter QP is set to a large value QL can be set. . However, the setting method is not limited to this.

  The quantization parameter set by the quantization parameter setting unit 6 is input to the maximum transform block size determination unit 7 and the rate control unit 14. If the value of the input quantization parameter QP is QL, the maximum transform block size determining unit 7 sets the transform block size to a maximum of 32 pixels × 32 pixels. Further, if the value of the input quantization parameter QP is QS, the maximum block size of the conversion is limited to 8 pixels × 8 pixels.

  Image data is sequentially input from the outside via the terminal 1 and stored in the buffer 2. The basic block dividing unit 3 cuts out a block of 32 pixels × 32 pixels in the order of input and inputs the block to the transform block dividing / size determining unit 4.

  The transform block division / size determination unit 4 calculates a coding cost when dividing by each block size, and selects a combination of divisions with a low cost. FIG. 2 is a block diagram showing an example of a detailed configuration of the transform block division / size determination unit 4. In FIG. 2, reference numeral 31 denotes a terminal for inputting basic block pixel data from the basic block 3. A buffer 32 stores the input pixel data.

  Reference numeral 33 denotes a 32 × 32 conversion unit that performs DCT conversion on the 32 pixels × 32 pixels stored in the buffer 32. Reference numeral 34 denotes a 16 × 16 conversion unit which divides the 32 pixels × 32 pixels stored in the buffer 32 into four parts in the vertical and horizontal directions and performs DCT conversion on each block of 16 pixels × 16 pixels. Reference numeral 35 denotes an 8 × 8 conversion unit that divides the 32 pixels × 32 pixels stored in the buffer 32 vertically and horizontally into four and performs DCT conversion on each 8 pixel × 8 pixel block. Reference numeral 36 denotes a 4 × 4 conversion unit that divides the 32 pixels × 32 pixels stored in the buffer 32 into 8 parts in the vertical and horizontal directions and performs DCT conversion on each 4 pixel × 4 pixel block.

  Reference numerals 37 to 40 denote cost calculation units which input orthogonal transformation results as respective inputs and calculate encoding costs. As a calculation method, there is a method of obtaining a block cost using a Lagrange multiplier. In addition, the cost calculation method is not limited to this, and for example, there is a method of determination using activities such as distribution and edge amount in each conversion block.

  Reference numerals 41 to 44 denote cost buffers corresponding to the corresponding cost calculation units 37 to 40 for accumulating the obtained costs in units of conversion blocks. Reference numeral 45 denotes a transform block division determination unit that reads the cost of each transform block stored in these cost buffers and selects a combination that minimizes the cost in the basic block. 46 is a terminal for outputting information on the selected combination, which is the output of the transform block division determining unit 45, as transform block division information. The prediction unit 8, the orthogonal transform unit 9, the motion compensation unit 19, and the block division in FIG. The information is output to the information encoding unit 21.

  47 is a transform block dividing unit that receives the transform block partition information and cuts out and outputs transform blocks from the buffer 32 according to the transform block partition information, and 48 is a terminal that sequentially outputs the pixel data of the split transform blocks. It is.

  49 is a terminal for inputting the maximum size of the transform block from the maximum transform block size determining unit 7 in FIG. In this embodiment, 32 is input if the value of the quantization parameter is QL, and 8 is input if the value is QS. 50 is a control for controlling whether or not to operate the 32 × 32 conversion unit 33, the cost calculation unit 37, the cost buffer 41, the 16 × 16 conversion unit 34, the cost calculation unit 38, and the cost buffer 42 in accordance with the input from the terminal 49. It is a vessel. In addition to these, the control signal is also input to the transform block division determining unit 45.

The operation of transform block division processing in the above configuration will be described.
The maximum size of the conversion block is input from the terminal 49 to the controller 50. If the maximum size of the conversion block is 32, the controller 50 operates the 32 × 32 conversion unit 33, the cost calculation unit 37, the cost buffer 41, the 16 × 16 conversion unit 34, the cost calculation unit 38, and the cost buffer 42. Set to. Further, if the maximum size of the conversion block is 8, the 32 × 32 conversion unit 33, the cost calculation unit 37, the cost buffer 41, the 16 × 16 conversion unit 34, the cost calculation unit 38, and the cost buffer 42 are set in a paused state. To do. At this time, the power supplied to the hardware is also cut off.

Pixel data (32 pixels × 32 pixels) of the basic block is input from the terminal 31 and stored in the buffer 32.
Assume that the controller 50 operates the 32 × 32 conversion unit 33, the cost calculation unit 37, the cost buffer 41, the 16 × 16 conversion unit 34, the cost calculation unit 38, and the cost buffer 42. At that time, the 32 × 32 conversion unit 33, 16 × 16 conversion unit 34, 8 × 8 conversion unit 35, and 4 × 4 conversion unit 36 read the data in the buffer 32 for each conversion block size. Then, DCT conversion is performed at each size, and the conversion coefficient is input to the cost calculation units 37 to 40 connected to each size.

  The cost calculation units 37 to 40 calculate the encoding cost from the coefficients and store them in the cost buffers connected to each. The cost buffer 41 stores one cost when the conversion block is 32 pixels × 32 pixels. In the cost buffer 42, four costs when the conversion block is 16 pixels × 16 pixels are stored according to the position. In the cost buffer 43, 16 costs when the conversion block is 8 pixels × 8 pixels are stored according to the position. In the cost buffer 44, 64 costs when the conversion block is 4 pixels × 4 pixels are stored according to the position.

  On the other hand, it is assumed that the controller 50 pauses the 32 × 32 conversion unit 33, the cost calculation unit 37, the cost buffer 41, the 16 × 16 conversion unit 34, the cost calculation unit 38, and the cost buffer 42. At that time, the data of the buffer 32 is read by the 8 × 8 conversion unit 35 and the 4 × 4 conversion unit 36 for each conversion block size, DCT conversion is performed at each size, and the conversion coefficient is connected to the cost calculation unit 39 connected to each. And 40.

  The cost calculation units 39 and 40 calculate the encoding cost from the coefficients and store them in the cost buffers connected to each. In the cost buffer 43, 16 costs when the conversion block is 8 pixels × 8 pixels are stored according to the position. In the cost buffer 44, 64 costs when the conversion block is 4 pixels × 4 pixels are stored according to the position.

  If all the costs are input to the cost buffers 41 to 44, the transform block division determination unit 45 compares these costs if the controller 50 operates all of the cost buffers 41 to 44, and the lowest cost is obtained. Select a combination of splits.

  FIG. 4 shows an example of transform block division when the high-efficiency mode is selected by the quantization parameter setting unit 6 in FIG. 1 and the maximum transform block size is 32. The 32 pixel × 32 pixel block is a 16 pixel × 16 pixel conversion block A and D, an 8 pixel × 8 pixel conversion block BA, BB, BC, BD, CB, CD, and a 4 pixel × 4 pixel conversion block. Are divided into CAA, CAB, CAC, CAD, CBA, CBB, CBC, and CBD.

  That is, regarding the state and size of block division, “a block of 32 pixels × 32 pixels is divided into 16 pixel × 16 pixel blocks”, “block A of 16 pixels × 16 pixels is not divided”, and a block of 16 pixels × 16 pixels B is divided into 8 pixel × 8 pixel blocks ”“ 8 pixel × 8 pixel block BA is not divided ”“ 8 pixel × 8 pixel block BB is not divided ”“ 8 pixel × 8 pixel block BC is divided "The block BD of 8 pixels x 8 pixels is not divided" "The block C of 16 pixels x 16 pixels is divided into 8 pixels x 8 pixels block" "The block CA of 8 pixels x 8 pixels is 4 pixels x 4 “Divided into pixel blocks” “Block CB of 8 pixels × 8 pixels is not divided” “Block CC of 8 pixels × 8 pixels is divided into 4 pixels × 4 pixel blocks” “8 strokes” × 8 pixel block CD of block D undivided "" 16 × 16 pixels is represented as divided not ", which is the division information of the transform block.

  FIG. 5 shows an example of transform block division when the high quality mode is selected by the quantization parameter setting unit 6 in FIG. 1 and the maximum transform block size is 8. 8-pixel x 8-pixel conversion block AA, AB, AC, AD, BA, BB, BC, BD, CB, CD, DA, DB, DC, DD and 4-pixel x 4-pixel conversion block CAA, CAB, It is divided into CAC, CAD, CBA, CBB, CBC, and CBD. The 16 pixel × 16 pixel conversion block in FIG. 4 is subdivided into 8 pixel × 8 pixel conversion blocks. At this time, the division information of the conversion block of FIG. 5 is “32 pixel × 32 pixel block is divided into 16 pixel × 16 pixel block” “16 pixel × 16 pixel block A is divided into 8 pixel × 8 pixel block” "8 pixel x 8 pixel block AA is not divided" "8 pixel x 8 pixel block AB is not divided" "8 pixel x 8 pixel block AC is not divided" "8 pixel x 8 pixel block "AD is not divided" "Block B of 16 pixels x 16 pixels is divided into 8 pixels x 8 pixel blocks" "Block BA of 8 pixels x 8 pixels is not divided" "Block BB of 8 pixels x 8 pixels is divided Not "" 8 pixel x 8 pixel block BC is not divided "" 8 pixel x 8 pixel block BD is not divided "" 16 pixel x 16 pixel block C is 8 pixel x 8 pixel block "8 pixel x 8 pixel block CA is divided into 4 pixel x 4 pixel block" "8 pixel x 8 pixel block CB is not divided" "8 pixel x 8 pixel block CC is 4 pixels "Divided into 4 pixel blocks" "block CD of 8 pixels x 8 pixels is not divided" "block D of 16 pixels x 16 pixels is divided into 8 pixels x 8 pixel blocks" "8 pixels x 8 pixels The block DA is not divided, “the block DB of 8 pixels × 8 pixels is not divided”, the block DC of 8 pixels × 8 pixels is not divided, and the block DD of 8 pixels × 8 pixels is not divided.

  When the division information of these transform blocks is output from the transform block division determination unit 45 to the prediction unit 8, the orthogonal transformation unit 9, the motion compensation unit 19, and the block division information encoding unit 21 in FIG. At the same time, it is input to the transform block division unit 47. The transform block division unit 47 sequentially reads transform block data from the buffer 32 based on the transform block division information and outputs the data from the terminal 48 to the prediction unit 8 and the motion compensation unit 19 in FIG.

  Returning to FIG. 1, the transform block division information generated by the transform block size / division determination unit 4 and the pixel data of the transform block are input to the prediction unit 8 and the motion compensation unit 19. The motion compensation unit 19 performs motion prediction on the converted block from the frame memory 18 based on the pixel block data, generates a pixel block to be referred to as a motion vector, and inputs the pixel block to the prediction unit 8. The motion vector is encoded by the motion vector encoding unit 20 and input to the multiplexing unit 12 as motion vector code data.

  The prediction unit 8 performs intra prediction on the input pixel data, compares it with the prediction result of motion compensation input from the motion compensation unit 19, selects the one with a small prediction error, and determines the prediction mode. The difference between the selected prediction result and the input pixel block is taken to generate prediction error data. The generated prediction error data is input to the orthogonal transform unit 9. The prediction result used at that time is input to the prediction error adding unit 17. The prediction mode is output to the multiplexing unit 12.

  The orthogonal transform unit 9 performs orthogonal transform indicated by transform block division information on the prediction error. A detailed block diagram of the orthogonal transform unit 9 is shown in FIG. In FIG. 6, reference numeral 60 denotes a terminal for inputting transformed block division information from the transformed block size / division determination unit 4 in FIG. Reference numeral 61 denotes a terminal for inputting the maximum size of the transform block from the maximum transform block size determination unit 7 in FIG. Similar to the controller 50 in FIG. 2, 62 is a controller that controls whether or not to operate the 32 × 32 conversion unit 65 and the 16 × 16 conversion unit 66 in accordance with the input from the terminal 61.

  63 is a terminal for inputting prediction error data from the prediction unit 8 of FIG. Reference numerals 64 and 69 denote selectors that select an input destination or an output destination based on the transform block size of the transform block division information input from the terminal 60.

  Reference numeral 65 denotes a 32 × 32 conversion unit that performs DCT conversion on an input conversion block of 32 × 32 pixels to obtain 32 × 32 coefficient data. Reference numeral 34 denotes a 16 × 16 conversion unit that performs DCT conversion on the input 16 pixel × 16 pixel conversion block to obtain 16 × 16 coefficient data. Reference numeral 67 denotes an 8 × 8 conversion unit that performs DCT conversion on the 8 pixel × 8 pixel conversion block input to the buffer 32 and obtains 8 × 8 coefficient data. Reference numeral 68 denotes a 4 × 4 conversion unit that performs DCT conversion on a 4 pixel × 4 pixel conversion block input to the buffer 32 and obtains 4 × 4 coefficient data.

  Reference numeral 70 denotes a terminal for outputting coefficient data obtained by each conversion. In the above configuration, the operation of orthogonal transform processing will be described.

  Prior to processing, the maximum size of the conversion block is input from the terminal 61 and input to the controller 62. If the maximum size of the conversion block is 32, the controller 62 sets the 32 × 32 conversion unit 65 and the 16 × 16 conversion unit 66 to operate. If the maximum size of the conversion block is 8, the 32 × 32 conversion unit 65 and the 16 × 16 conversion unit 66 are set in a paused state. At this time, the power supplied to the hardware is also cut off.

  The transform block division information is input from the terminal 60 and input to the selector 64 and the selector 69. If the input conversion block size is 32, the selector 64 sets the output destination to the 32 × 32 conversion unit 65, and the selector 69 sets the input destination to the 32 × 32 conversion unit 65. Similarly, if the input conversion block size is 16, the selector 64 sets the output destination to the 16 × 16 conversion unit 65, and the selector 69 sets the input destination to the 16 × 16 conversion unit 66. If the input conversion block size is 8, the selector 64 sets the output destination to the 8 × 8 conversion unit 65, and the selector 69 sets the input destination to the 8 × 8 conversion unit 67. If the input conversion block size is 4, the selector 64 sets the output destination to the 4 × 4 conversion unit 65, and the selector 69 sets the input destination to the 4 × 4 conversion unit 68.

  The pixel data of the conversion block input from the terminal 63 is input to the conversion unit of the corresponding size by the selector 64. The coefficient data of the conversion block obtained by conversion by each conversion unit is output from the terminal 70 via the selector 69. At this time, the 32 × 32 conversion unit 65 calculates the value of the 8 × 8 portion shown in FIG. 3A, and converts the other white portions to 0. Similarly, the 16 × 16 conversion unit 66 calculates the value of the 8 × 8 portion shown in FIG. 3B, and converts the other white portions to 0.

  Returning to FIG. 1, the coefficient data generated by the orthogonal transform unit 9 is input to the quantization unit 10. The quantization unit 10 performs quantization using the quantization parameter determined by the rate control unit 14, generates quantized coefficient data, and outputs the quantized coefficient data to the entropy encoding unit 11. In addition, the quantization parameter used at this time is also output to the entropy encoder 11. The entropy encoding unit 11 encodes the input quantization parameter, quantization coefficient, and the like. The encoding method is not particularly limited. For example, the difference between the quantization parameter and the quantization parameter used in the quantization of the immediately preceding transform block is Huffman encoded. The quantization coefficient is also H.264. Like the H.264 encoding method, it is realized by making a one-dimensional array and performing variable length encoding.

  On the other hand, the quantized coefficient data obtained by the quantizing unit 10 is input to the inverse quantizing unit 15 and is dequantized using the quantization parameter to restore the coefficient data. The restored coefficient data is input to the inverse orthogonal transform unit 16 to restore the prediction error data. The restored prediction error data is input to the prediction error adding unit 17, and the prediction value of intra prediction generated by the prediction unit 8 or the prediction value obtained by motion compensation is input and added to correspond to the frame memory 18. Stored in the area.

  Further, the division information of the transformation block determined by the transformation block size / division decision unit 4 is encoded by the block division information encoding unit 21. In the following description, it is assumed that the transform block division information is encoded in the order of upper left, upper right, lower left, and lower right. It is assumed that 1 is encoded if it is divided and 0 is allocated if it is not divided.

  First, since the basic block is divided, 1 is set to the basic block division flag indicating that the basic block is divided. If conversion of 32 pixels × 32 pixels is performed without division, the basic block division flag is set to 0. Subsequently, information indicating whether or not the 16-pixel × 16-pixel conversion block A is divided is represented as an A block division flag. That is, this value is 1 if it is divided, and 0 otherwise. When the block A is divided again and divided into blocks of 8 pixels × 8 pixels, a block division flag indicating whether or not the block A is divided again into blocks of 4 pixels × 4 pixels is set.

Regarding the division of the block of FIG.
1 A block of 32 pixels × 32 pixels is divided into 16 pixel × 16 pixel blocks 0 A block A of 16 pixels × 16 pixels is not divided 1 A block B of 16 pixels × 16 pixels is divided into 8 pixels × 8 pixel blocks 0 The block BA of 8 pixels × 8 pixels is not divided 0 The block BB of 8 pixels × 8 pixels is not divided 0 The block BC of 8 pixels × 8 pixels is not divided 0 The block BD of 8 pixels × 8 pixels is not divided 1 16 pixel × 16 pixel block C is divided into 8 pixel × 8 pixel block 1 8 pixel × 8 pixel block CA is divided into 4 pixel × 4 pixel block 0 8 pixel × 8 pixel block CB is divided No 1 8 pixel × 8 pixel block CC is divided into 4 pixel × 4 pixel block 0 8 pixel × 8 pixel block CD is not divided 0 16 pixel × 1 The 6-pixel block D is not divided. Similarly, for the block division of FIG. 5, the following division block division flags are generated.
1 Block of 32 pixels x 32 pixels is divided into 16 pixels x 16 pixel blocks 1 Block A of 16 pixels x 16 pixels is divided into blocks of 8 pixels x 8 pixels 0 Block AA of 8 pixels x 8 pixels is divided 0 No 8-pixel x 8-pixel block AB is divided 0 8-pixel x 8-pixel block AC is not divided 0 8-pixel x 8-pixel block AD is not divided 1 16-pixel x 16-pixel block B is 8 pixels x Divided into 8 pixel blocks 0 Block BA of 8 pixels x 8 pixels is not divided 0 Block BB of 8 pixels x 8 pixels is not divided 0 Block BC of 8 pixels x 8 pixels is not divided 0 8 pixels x 8 pixels Block BD is not divided 1 16 pixel × 16 pixel block C is divided into 8 pixel × 8 pixel block 1 8 pixel × 8 pixel block CA is 4 The block CB of 8 pixels × 8 pixels is not divided 1 The block CC of 8 pixels × 8 pixels is divided into 4 pixels × 4 pixels blocks 0 The block CD of 8 pixels × 8 pixels Is not divided 1 Block D of 16 pixels x 16 pixels is divided into 8 pixels x 8 pixels block 0 Block DA of 8 pixels x 8 pixels is not divided 0 Block DB of 8 pixels x 8 pixels is not divided 0 8 pixels The block DC of × 8 pixels is not divided. 0 The block DD of 8 pixels × 8 pixels is not divided.

  The order of the codes is not limited to this. For example, if a division flag of a block of 32 pixels × 32 pixels is first divided into 16 pixels × 16 pixel blocks, then a 16 pixel × 16 pixel block is subsequently performed. It may be continued with 4 bits of the division flag. Since the 4 pixel × 4 pixel block has no conversion block of a smaller size, no division flag information is generated.

FIG. 11 is a flowchart for explaining the flow of encoding the division flag.
In step S051, it is determined whether or not the basic block is divided into 16 pixel × 16 pixel conversion blocks. If divided, the process proceeds to step S053; otherwise, the process proceeds to step S052.

  In step S052, since the basic block is not divided into 16 pixel × 16 pixel conversion blocks, code “0” is assigned to the division flag. Thereafter, since there is no division of the transform block, the encoding process is terminated.

  In step S053, since the basic block is not divided into conversion blocks of 16 pixels × 16 pixels, the code “1” is assigned to the division flag, and subsequently, the processing in the 16 pixels × 16 pixel block is performed.

  In step S054, it is determined whether or not the target 16 pixel × 16 pixel conversion block is divided into 8 pixel × 8 pixel conversion blocks in the order from upper left to lower right. If so, the process proceeds to step S056; otherwise, the process proceeds to step S054.

  In step S055, since the 16 pixel × 16 pixel conversion block is not divided into 8 pixel × 8 pixel conversion blocks, code “0” is assigned to the division flag. Thereafter, the process proceeds to step S061.

  In step S056, since the 16 pixel × 16 pixel conversion block is not divided into 8 pixel × 8 pixel conversion blocks, code “1” is assigned to the division flag, and the process proceeds to step S057.

  In step S057, it is determined whether or not the target 8 pixel × 8 pixel conversion block is divided into 4 pixel × 4 pixel conversion blocks in the order from upper left to lower right. If so, the process proceeds to step S059; otherwise, the process proceeds to step S058.

  In step S058, since the conversion block of 8 pixels × 8 pixels is not divided into conversion blocks of 4 pixels × 4 pixels, code “0” is assigned to the division flag. Thereafter, the process proceeds to step S060.

  In step S059, since the 8 pixel × 8 pixel conversion block is not divided into 4 pixel × 4 pixel conversion blocks, code “1” is assigned to the division flag, and the process proceeds to step S060.

  In step S060, it is determined whether or not all the 8 pixel × 8 pixel conversion blocks of the processed 16 pixel × 16 pixel conversion block have been encoded, and if completed, the process proceeds to step S061. . Otherwise, the process returns to step S057 to process the next conversion block of 8 pixels × 8 pixels.

  In step S061, it is determined whether or not all 16 pixel × 16 pixel conversion blocks of the processed 32 pixel × 32 pixel conversion block have been encoded. The process is terminated. Otherwise, the process returns to step S057 to process the next conversion block of 8 pixels × 8 pixels.

  As described above, the entropy encoding unit 11 encodes the quantization parameter and the quantization coefficient, the prediction unit 8 encodes the prediction mode, the motion vector encoding unit 20 encodes the motion vector, and the block division information encoding unit 21 encodes the encoded data. The divided flag is input to the multiplexing unit 12. In the multiplexing unit 12, the respective code data are arranged in a predetermined order and output from the terminal 13 as a bit stream.

The bit stream is input to the rate control unit 14, and a new quantization parameter is generated with reference to the quantization parameter set by the quantization parameter setting unit 6. For example, a width that can be changed from the quantization parameter set by the quantization parameter setting unit 6 is set, and control is realized therein.
The simple flow of the encoding process will be described with reference to the drawings.

FIG. 9 is a flowchart showing the entire encoding process.
In step S001, a quantization parameter for determining image quality is set when processing is started.

  In step S002, the maximum size of the transform block to be used is determined using the set quantization parameter. For example, the set quantization parameter QP is compared with the threshold Th, and if the quantization parameter QP is larger than the threshold Th, the maximum size of the transform block is set as the size of the basic block. If the quantization parameter QP is equal to or smaller than the threshold Th, the maximum size of the transform block is limited to a block size in which the variable is not replaced with 0 after the transformation.

In step S003, image data for one frame is read.
In step S004, the read one-frame image data is divided into basic blocks (in this embodiment, divided into blocks of 64 pixels × 64 pixels).
In step S005, one basic block data is read, the cost at each size when block division is performed is calculated, and the transform block division method is determined so that the total cost is minimized.

In step S006, the determined block division information is encoded.
In step S007, the divided transform blocks are sequentially read, and in the case of intra prediction or moving image, motion compensation is performed to calculate a prediction value.

  In step S008, the prediction value of the calculated transform block is compared with the input, a prediction error is calculated, and the prediction error is subjected to orthogonal transform of each size. After conversion, quantization is performed using a quantization parameter, and the quantization result is encoded.

  In step S009, inverse quantization and inverse orthogonal transform are performed on the quantization result, the prediction error is reproduced, added to the prediction value calculated in step S006, and a reproduced image is generated and held.

  In step S010, it is determined whether or not the processing from step S007 to step S009 has been completed for the conversion block in the basic block being processed. If completed, the process proceeds to step S011, and if not completed, steps S007 to S009 are performed for the next conversion block.

In step S011, the encoded quantization parameter, the code data of the quantization coefficient, and other code data are output.
If it is determined in step S012 that the processing from step S005 to step S011 has been completed for all the basic blocks in the frame, the process proceeds to step S012, and if not completed, the next basic block is targeted. Step S012 is performed from S003.

  In step S013, it is determined whether or not there is a next frame of the image data. If there is a next frame, the process from step S003 is performed, and if not, the process ends.

  By determining the maximum transform block size at the time of division by the method described above, the use of a large transform block whose coefficient is forced to 0 is restricted, thereby effectively suppressing image quality degradation. Can do. Further, by performing the control based on the quantization parameter for controlling the generation of the quantization coefficient, the image quality control can be suitably performed.

  In addition, since the large conversion block is complicated to calculate, the circuit scale is large, and the power consumption is high, by limiting the conversion block with the largest conversion block size, the calculation amount is reduced and the power consumption is suppressed. Will be able to. In the present embodiment, the decoding side can reproduce an image by performing normal decoding.

  In the present embodiment, the basic block size and the block division method are not limited thereto. For example, the basic block size may be as large as 64 pixels × 64 pixels, and the division method may be a rectangle such as 8 pixels × 4 pixels.

  In the present embodiment, the maximum transform block size is determined by the quantization parameter set by the quantization parameter setting unit 6 and set so that the quantization parameter does not fluctuate greatly in the rate control 14, but the present invention is not limited to this. As shown in FIG. 7, the new quantization parameter determined by the rate control unit 22 may be input to the maximum conversion block size determination unit 23 and the maximum conversion block size may be determined in units of frames or slices based on the new quantization parameter. Absent.

  Further, in the present embodiment, the configuration of the transform block size / division determination unit 4 is shown in FIG. 2, but the present invention is not limited to this, and in order to more accurately calculate the cost in each division, FIG. You may comprise as follows. This is a method of using a prediction error in each block size before calculating the cost in each block size. That is, reference numeral 80 denotes a terminal which inputs reference frame image data necessary for motion prediction from the frame memory 18 of FIG. 81 is a 32 × 32 prediction unit that performs prediction by intra prediction or motion compensation in units of 32 pixels × 32 pixels, and 85 is a 32 × 32 motion that calculates a prediction data by performing a motion vector search with 32 pixels × 32 pixels. Compensation unit. Hereinafter, the 16 × 16 prediction unit 82 and the 16 × 16 motion compensation unit 86 are provided before the 16 × 16 conversion unit 34, and the 8 × 8 prediction unit 83 and the 8 × 8 motion compensation unit 87 are provided before the 8 × 8 conversion unit 35. Is provided with a 4 × 4 prediction unit 84 and a 4 × 4 motion compensation unit 88 before the 4 × 4 conversion unit 36. Furthermore, by sending the prediction error generated here to subsequent processing, the prediction unit 8 and the motion compensation unit 19 in FIG. 1 can be reduced.

Further, as shown in FIG. 10, it is possible to consider not only the quantization parameter but also the quantization matrix in determining the maximum transform block size. That is, the example in which the set quantization parameter QP and the threshold value Th are compared in step S002 has been described. However, not only the quantization parameter QP but also the quantization matrix QM [0-i] (i is the transform block). May be considered). In FIG. 10, there is a quantization matrix setting unit 110 that sets the quantization matrix QM [0-i], and the coefficient Qsum is calculated using the quantization matrix QM [0-i] and the quantization parameter QP set here. calculate.
Qsum = Σ (QM [m] × QP)
The coefficient Qsum is compared with the threshold Thm, and if the quantization parameter Qsum is larger than the threshold Thm, the maximum size of the transform block is set as the size of the basic block. If the quantization parameter Qsum is equal to or smaller than the threshold Thm, the maximum size of the transform block may be limited to a block size in which the variable is not replaced with 0 after the transform.

  Although the present embodiment has been described with reference to an example of encoding a moving image using inter-frame differences, it is not limited to this, and it is obvious that only intra prediction can be used to encode a still image. It is.

<Embodiment 2>
In the first embodiment, an example in which the size of the transform block is limited using the maximum transform block size has been described. However, in the present embodiment, the encoding method for block division is used using the maximum transform block size. An example of performing control together will be described.

  In FIG. 12, reference numeral 107 denotes a maximum transform block size encoding unit that encodes the maximum transform block size determined by the maximum transform block size determination unit 7. Reference numeral 121 denotes a block division information encoding unit, which is different from the block division information encoding unit 21 of the first embodiment in that the maximum block size is input from the maximum transform block size determination unit 7. Reference numeral 112 denotes a multiplexing unit.

  The processing of the block division information encoding unit 121 will be described with reference to the flowchart of FIG. In FIG. 13, steps that perform the same processes as those in FIG. 11 of the first embodiment are given the same numbers, and descriptions thereof are omitted.

  In step S151, it is determined whether or not the maximum size of the transform block input from the maximum transform block size determination unit 7 is a transform block of 16 pixels × 16 pixels. If the maximum size of the conversion block is a conversion block of 16 pixels × 16 pixels, the process proceeds to step S054; otherwise, the process proceeds to step S152.

  In step S152, it is determined whether the maximum size of the conversion block is a conversion block of 8 pixels × 8 pixels. If the maximum size of the conversion block is a conversion block of 8 pixels × 8 pixels, the process proceeds to step S059; otherwise, the process proceeds to step S055.

  Hereinafter, step S051 is performed when the maximum conversion block size is 32 pixels × 32 pixels. In this case, similarly to the first embodiment, the processing from step S051 to step S061 is performed, and the division flag is encoded.

  If it is determined in step S151 that the maximum transform block size is 16 pixels × 16 pixels, the processing from step S054 to step SS061 is performed, and the division flag is encoded. Thereby, the division flag encodes information of division (16 pixels × 16 pixels, 8 pixels × 8 pixels, and 4 pixels × 4 pixels) with respect to the conversion block of 16 pixels × 16 pixels.

  If it is determined in step S152 that the maximum transform block size is 8 pixels × 8 pixels, the process from step S054 to step SS061 is performed, and the division flag is encoded. Thereby, the division flag encodes information of division (8 pixels × 8 pixels and 4 pixels × 4 pixels) with respect to the conversion block of 8 pixels × 8 pixels.

With the above operation, for the block division of FIG. 5, if the maximum conversion block size is 8 pixels × 8 pixels,
0 Block 8A of 8 pixels × 8 pixels is not divided 0 Block AB of 8 pixels × 8 pixels is not divided 0 Block AC of 8 pixels × 8 pixels is not divided 0 Block AD of 8 pixels × 8 pixels is not divided 0 8 The block BA of 8 pixels × 8 pixels is not divided 0 The block BB of 8 pixels × 8 pixels is not divided 0 The block BC of 8 pixels × 8 pixels is not divided 0 The block BD of 8 pixels × 8 pixels is not divided 1 8 pixels × 8 pixel block CA is divided into 4 pixel × 4 pixel block 0 8 pixel × 8 pixel block CB is not divided 1 8 pixel × 8 pixel block CC is divided into 4 pixel × 4 pixel block 0 8 The block CD of 8 pixels × 8 pixels is not divided 0 The block DA of 8 pixels × 8 pixels is not divided 0 The block DB of 8 pixels × 8 pixels is not divided Block DC 0 8 pixels × 8 pixels is the block DD 0 8 pixels × 8 pixels are not divided is not divided. Compared to the encoding of the division flag in FIG. 5 described in the first embodiment,
1 Block of 32 pixels x 32 pixels is divided into 16 pixels x 16 pixel blocks 1 Block A of 16 pixels x 16 pixels is divided into 8 pixels x 8 pixel blocks 1 Block B of 16 pixels x 16 pixels is 8 1 × 16 pixel × 16 pixel block C is divided into 8 × 8 pixel block 1 16 × 16 pixel block D is divided into 8 × 8 pixel block The sign can be omitted.

  The maximum transform block size is encoded by the maximum transform block size encoding unit 107. The encoding method is not particularly limited. For example, if the maximum transform block size is 32 pixels × 32 pixels, an index “0” is assigned, if it is 16 pixels × 16 pixels, an index “1” is assigned, and if it is 8 pixels × 8 pixels, an index “2” is assigned, It can be expressed by a 2-bit code.

  The encoded maximum block size and the division flag are output from the terminal 13 as a bit stream by the multiplexing unit 112 aligning the respective code data in a predetermined order.

  With the method described above, by determining a code representing block division according to the maximum transform block size at the time of division, it is possible to efficiently suppress deterioration in image quality with a small amount of code as in the first embodiment. be able to.

  Although the maximum block size is determined by the quantization parameter, it is not limited to this and may be set individually.

  In the first and second embodiments, the example in which the maximum block size is not set to the basic block size and the high frequency is forcibly set to 0 has been described. However, the present invention is not limited to this. It is also possible to use x16 pixels.

In the present embodiment, the maximum conversion block size is always set. However, the present invention is not limited to this, and the configuration shown in FIG. 14 can be adopted. In FIG. 14, a terminal 151 is a terminal for inputting whether or not encoding is performed using a maximum transform block size set by a user (not shown). The maximum conversion block size determination unit 158 is different from the maximum conversion block size determination unit 7 of the first embodiment in that the determination of the maximum conversion block size is controlled according to the instruction of the terminal 151. Reference numeral 157 denotes a maximum transform block size encoding unit, which is different from the maximum transform block size encoding unit 157 in that it receives an input from a terminal 151. Reference numeral 122 denotes a multiplexing unit. The encoding unit 112 in FIG. 14 is to receive code data of a profile. When encoding is performed by setting the maximum transform block size from the terminal 151, the same as described above, The maximum transform block size is determined, and the maximum transform block size is encoded and transmitted. When the maximum conversion block size is not set from the terminal 151, the maximum conversion block size determination unit 158 fixes the maximum conversion block size to the basic block size of 32 pixels × 32 pixels. Further, the maximum transform block size encoding unit 157 does not operate, and the maximum transform block size is not encoded. By doing so, compatibility with the conventional method can be maintained.

  Moreover, you may implement | achieve by including in the header information showing the attribute of a bit stream whether it encodes using the largest conversion block size. For example, H.M. The SEI code in the H.264 encoding method may be used for description, or a profile using the present invention may be used. In FIG. 15, reference numeral 160 denotes a profile setting unit. When encoding is performed by setting the maximum transform block size from the terminal 151, a profile using the maximum transform block size code (for example, an extension profile) is used, and encoding is not performed. In this case, the basic profile is used. Codes for identifying these profiles are generated and input to the multiplexing unit 122, and the codes for identifying the profiles are embedded in the bitstream.

  This makes it clear at an early stage whether or not to perform control based on the maximum block size at an early stage of the bitstream. If it is determined whether or not control is possible at an early stage of the bitstream, it is possible to save power in the conversion unit and perform it at an early stage.

<Embodiment 3>
FIG. 16 is a block diagram showing a configuration of an image decoding apparatus according to the third embodiment of the present invention. In the present embodiment, a description will be given by taking the decoding of the encoded data generated in the first embodiment as an example.

  In FIG. 16, 201 is a terminal for inputting the bit stream generated in the first embodiment, and 202 is each processed by the reverse processing of the multiplexing unit 12 described in FIG. 1 of the first embodiment. It is a separation unit that separates into codes. A quantization parameter decoding unit 203 receives the quantization parameter code from the separation unit 202 and decodes it. A maximum transform block size setting unit 204 calculates the maximum transform block size from the decoded quantization parameter. Reference numeral 205 denotes an entropy decoding unit (frequency decoding unit) that decodes code data of quantized coefficients. Reference numeral 206 denotes an inverse quantization unit (inverse frequency conversion unit) that performs inverse quantization using the quantization parameter decoded by the quantization parameter decoding unit 203. An inverse orthogonal transform unit 207 decodes coefficient data obtained by inverse quantization and reproduces prediction error data. Reference numeral 208 denotes a prediction unit that decodes the code related to the prediction separated by the separation unit 202, reads the decoded image from the frame memory 213, and performs prediction. It performs the same function as the prediction unit 8 described in FIG. 1 of the first embodiment. A motion vector decoding unit 209 decodes a motion vector code, reproduces the motion vector, and inputs the decoded motion vector to the motion compensation unit 210. A motion compensation unit 210 performs motion compensation from the frame memory 213 based on the reproduced motion vector. A prediction error adding unit 211 generates a decoded image by adding the prediction data generated by the prediction unit 208 and the prediction error data reproduced by the inverse orthogonal transform unit 207. Reference numeral 212 denotes a basic block synthesizing unit that collects reproduced image data of each transform block obtained by decoding into basic blocks. A frame memory 213 holds a decoded image. Reference numeral 214 denotes a block division information decoding unit that performs decoding performed by the block division information encoding unit 21 illustrated in FIG. 1 according to the first embodiment. Reference numeral 215 denotes a terminal that outputs the decoded image data to the outside.

  In the above configuration, a bit stream is input from the terminal 201 and input to the separation unit 202. The separation unit 202 sends a code related to the quantization parameter to the quantization parameter decoding unit 203. The encoded data of the quantized coefficient is sent to the entropy decoding unit 205. Encoded data related to the prediction mode, the prediction method, and the like are sent to the prediction unit 208. Encoded data representing block division information is input to the block division information decoding unit 214. First, the encoded data of the quantization parameter is read and decoded, and the quantization parameter QP is decoded. If the value of the quantization parameter QP decoded as in the maximum transform block size determination unit 7 of the first embodiment is QL, the maximum transform block size setting unit 204 sets the transform block size to a maximum of 32 pixels × 32 pixels. And Further, if the value of the input quantization parameter QP is QS, the maximum block size of the conversion is limited to 8 pixels × 8 pixels. The maximum value of the transform block size is input to the entropy decoding unit 205, the inverse quantization unit 206, the inverse orthogonal transform unit 207, the motion compensation unit 210, and the basic block synthesis unit 212.

  The encoded data of the quantized coefficient is input to the entropy decoding unit 205, performs decoding, and decodes the quantization result. The quantization result is input to the inverse quantization unit 206, and inverse quantization is performed using the quantization parameter decoded by the quantization parameter decoding unit 203 to obtain coefficient data. The coefficient data is input to the inverse orthogonal transform unit 207, performs the same transform as the transform of the inverse orthogonal transform unit described in FIG. 1 of the first embodiment, and reproduces the prediction error. The reproduced prediction error is input to the prediction error adding unit 211.

  Details of the inverse orthogonal transform unit 207 will be described with reference to FIG. In FIG. 6, reference numeral 260 denotes a terminal for inputting transformed block division information from the transformed block division decoding unit 214 of FIG. Reference numeral 261 denotes a terminal for inputting the maximum size of the transform block from the maximum transform block size setting unit 204 in FIG. Similarly to the controller 50 in FIG. 2, the controller 262 controls whether or not to operate the 32 × 32 conversion unit 265 and the 16 × 16 conversion unit 266 in accordance with the input from the terminal 261.

  263 is a terminal for inputting prediction error data from the prediction unit 8 of FIG. Reference numerals 264 and 269 denote selectors that select an input destination or an output destination based on the transform block size of the transform block division information input from the terminal 260.

  Reference numeral 265 denotes a 32 × 32 inverse transform unit that performs inverse DCT transform on the input 32 pixel × 32 pixel transform block to obtain 32 × 32 image data. Reference numeral 266 denotes a 16 × 16 inverse conversion unit that performs inverse DCT conversion on the input 16 pixel × 16 pixel conversion block to obtain 16 × 16 image data. Reference numeral 267 denotes an 8 × 8 inverse transform unit that performs inverse DCT transform on the input 8 pixel × 8 pixel transform block to obtain 8 × 8 image data. Reference numeral 268 denotes a 4 × 4 inverse transform unit that performs inverse DCT transform on the input 4 pixel × 4 pixel transform block to obtain 4 × 4 image data.

A terminal 270 outputs coefficient data obtained by each conversion. In the above configuration, the operation of orthogonal transform processing will be described.
Prior to processing, the maximum size of the transform block is input from the terminal 261 and input to the controller 262. If the maximum size of the transform block is 32, the controller 262 sets the 32 × 32 inverse transform unit 265 and the 16 × 16 inverse transform unit 266 to operate. If the maximum size of the transform block is 8, the 32 × 32 inverse transform unit 265 and the 16 × 16 inverse transform unit 266 are set to a paused state. At this time, the power supplied to the hardware is also cut off.

  The transformed block division information decoded from the terminal 260 is input to the selector 264 and the selector 269. If the set transform block size is 32, the selector 264 sets the output destination to the 32 × 32 inverse transform unit 265, and the selector 269 sets the input destination to the 32 × 32 inverse transform unit 265. Similarly, if the input transform block size is 16, the selector 264 sets the output destination to the 16 × 16 inverse transform unit 266, and the selector 269 sets the input destination to the 16 × 16 inverse transform unit 266. If the input transform block size is 8, the selector 264 sets the output destination to the 8 × 8 inverse transform unit 267 and the selector 269 sets the input destination to the 8 × 8 inverse transform unit 267. If the input conversion block size is 4, the selector 264 sets the output destination to the 4 × 4 inverse conversion unit 268, and the selector 69 sets the input destination to the 4 × 4 conversion unit 268.

  The coefficient data of the transform block input from the terminal 263 is input to the inverse transform unit having the corresponding size by the selector 264. The prediction error of the conversion block obtained by conversion in each inverse conversion unit is output from the terminal 270 via the selector 269.

  Returning to FIG. 16, the prediction unit 208 receives and decodes codes such as a prediction method and a prediction mode. Thereby, if it is intra prediction, the pixel used for prediction will be selected from the frame memory 213 in prediction mode, and prediction data will be reproduced | regenerated. In the case of motion compensation, the motion vector code data is decoded by the motion vector decoding unit 209, and the motion compensation unit 210 generates prediction data from the frame memory 213 based on the motion vector.

On the other hand, the division flag indicating the block division separated by the separation unit 202 is decoded by the block division information decoding unit 214, and the division state in the basic block is acquired.
Prediction data is input to the prediction error adding unit 211 and added to the above-described prediction error to obtain a reproduced image. The obtained reproduced image is synthesized as a basic block by the basic block synthesis unit 212 based on the division status in the basic block. Thereafter, the data is stored in the frame memory 213 in units of basic blocks. The reproduced image is output to the outside via the terminal 215.

FIG. 18 is a flowchart showing an image decoding process in the image decoding apparatus according to the third embodiment.
In step S201, the quantization parameter value QP of the first block of the frame is decoded.

  In step S202, if the value of the decoded quantization parameter QP is QL, the block size of the conversion is set to a maximum of 32 pixels × 32 pixels. Further, if the value of the input quantization parameter QP is QS, the maximum block size of the conversion is set to 8 pixels × 8 pixels.

In step S203, the block division information in the basic block is decoded by a method to be described later, and the size and position of the transform block are acquired.
In step S204, prediction data is generated by intra prediction or motion compensation.

  In step S205, the code data of the coefficient is decoded, and the quantization result of the prediction error is acquired.

  In step S206, the quantization result of the obtained prediction error is inversely quantized and subjected to inverse transformation to reproduce the prediction error. The prediction data generated in step S204 is added to the reproduced prediction error to restore the image data.

  In step S207, the restored block image data is synthesized with the basic block based on the block division information acquired in step S202.

  In step S208, it is determined whether or not the processing has been completed for all transform blocks in the basic block. If the decoding process for all the transform blocks has been completed, the process proceeds to step S209. Otherwise, the process proceeds to step S204 to process the next conversion block.

  In step S209, it is determined whether or not the processing has been completed for all basic blocks in the frame. If the decoding process for all the basic blocks has been completed, the process proceeds to step S211. Otherwise, the process proceeds to step S203 to process the next basic block.

In step S210, the decoded image data for one frame is output.
In step S211, it is determined whether there is a frame to be decoded next. If the decoding process for all the frames has been completed, the image decoding process is terminated. Otherwise, the process proceeds to step S201 to process the next frame.

FIG. 19 shows a flowchart of the division flag decoding process.
In step S251, it is determined whether the input division flag is “0” or “1”. If “0”, the process proceeds to step S252. Otherwise, the process proceeds to step S253.

  In step S252, the basic block is not divided, and the conversion block processing is performed with 32 pixels × 32 pixels, and the division flag decoding processing ends.

  In step S253, since the division flag indicates “1”, it is assumed that the division block is divided into 16 pixels × 16 pixels.

  In step S254, it is determined whether the subsequently input division flag is “0” or “1”. If “0”, the process proceeds to step S255, and if not, the process proceeds to step S256.

  In step S255, it is assumed that the conversion block of 16 pixels × 16 pixels is not divided, and the conversion block processing is performed with 16 pixels × 16 pixels, and the process proceeds to step S261.

In step S256, since the division flag indicates “1”, it is assumed that the division block is divided into 8 pixels × 8 pixels.
In step S257, it is determined whether the subsequently input division flag is “0” or “1”. If “0”, the process proceeds to step S258; otherwise, the process proceeds to step S259.

In step S258, the converted block of 8 pixels × 8 pixels is not divided, and the process of the converted block is performed with 8 pixels × 8 pixels, and the process proceeds to step S260.
In step S259, since the division flag indicates “1”, it is assumed that the division block is divided into 4 pixel × 4 pixel conversion blocks.
In step S260, it is determined whether or not the decoding process is completed for the presence or absence of a 4 pixel × 4 pixel conversion block in the 8 pixel × 8 pixel conversion block. If completed, the process proceeds to step S261; otherwise, the process proceeds to step S257 in order to process the next 8 pixel × 8 pixel conversion block.

In step S261, it is determined whether or not the decoding process is completed with respect to the presence or absence of a conversion block of 8 pixels × 8 pixels or less in the conversion block of 16 pixels × 16 pixels. If completed, the division flag decoding process ends. Otherwise, the process proceeds to step S254 to process the next 16 pixel × 16 pixel conversion block.
With the configuration and operation described above, in the bit stream generated in the first embodiment, the maximum transform block size is set from the quantization parameter, so that the amount of computation and power consumption can be reduced even on the decoding side.
Furthermore, in the inverse quantization unit 206, the inverse orthogonal transform unit 207, and the motion compensation unit 210, hardware resources such as a memory can be effectively used by adaptively securing the buffer size. The motion compensation unit 210 adaptively secures a buffer for reading from the frame memory 213 by acquiring the maximum transform block size.

<Embodiment 4>
FIG. 20 is a block diagram showing a configuration of an image decoding apparatus according to the fourth embodiment of the present invention. In the present embodiment, an example of decoding the encoded data generated in the second embodiment will be described.

In FIG. 20, blocks having the same functions as those in FIG. 16 of the third embodiment are assigned the same numbers, and descriptions thereof are omitted.
Reference numeral 302 denotes a demultiplexing unit that demultiplexes each code by the reverse process of the multiplexing unit 112 described in FIG. 14 of the first embodiment. A maximum transform block size decoding unit 304 decodes a code indicating the maximum transform block size separated by the separation unit 302. Reference numeral 314 denotes a block division information decoding unit that obtains information on the maximum conversion block size from the maximum conversion block size decoding unit and decodes the block division information.

The operation of the image decoding process in the above configuration will be described.
A bit stream is input from the terminal 201 and input to the separation unit 302. The separation unit 302 inputs header information, a quantization parameter code, a maximum transform block size code, a coefficient data code, a motion vector code, and a block division information code, which are separated and output to the subsequent stage. The code of the maximum transform block size is input to the maximum transform block size decoding unit 304 and decoded. In the second embodiment, the size is encoded as an index. This is decoded, and the maximum conversion block size is acquired as 32 pixels × 32 pixels / 8 pixels × 8 pixels. The acquired information on the maximum transform block size is input to the related units and the block division information decoding unit 314.

  The block division information decoding unit 314 inputs the maximum transformation block size from the maximum transformation block size decoding unit 304 and inputs a division flag representing the block division information from the separation unit.

  FIG. 21 is a flowchart showing the detailed processing. In FIG. 21, the steps having the same functions as those in FIG. 19 are given the same numbers, and the description thereof is omitted.

  In step S301, it is determined whether or not the maximum transform block size is 16 pixels × 16 pixels. If so, the process proceeds to step S254, and if not, the process proceeds to step S302.

  In step S302, it is determined whether or not the maximum conversion block size is 8 pixels × 8 pixels. If so, the process proceeds to step S257, and if not, the process proceeds to step S251.

  With the above operation, if the maximum conversion block size is 32 pixels × 32 pixels, the processing from step S251 to step S261 is performed. If the maximum conversion block size is 16 pixels × 16 pixels, the processing from step S254 to step S261 is performed. If the maximum conversion block size is 8 pixels × 8 pixels, the processing from step S257 to step S261 is performed.

  With the above configuration and operation, in the bitstream generated in the second embodiment, the maximum transform block size is set from the quantization parameter, so that the amount of computation and power consumption can be reduced even on the decoding side. That is, since the maximum transform block size is encoded, the same effect can be obtained for a short code as compared with the code of the first embodiment.

  Note that, in the second embodiment, a method has been described in which whether or not encoding is performed using the maximum transform block size is included in the header information indicating the attribute of the bitstream. This code can also be decoded by taking the configuration shown in FIG. That is, the code | symbol which identifies the profile embedded in the bit stream is isolate | separated by the separator 402, and it inputs into the profile decoding part 453. FIG. The profile decoding unit 453 decodes a profile using the maximum transform block size code (for example, an extended profile) or a basic profile when encoding is not performed. If the profile is an extended profile, the maximum transform block size decoding unit 404 is operated to obtain the maximum transform block size. Otherwise, the same effect can be obtained by setting the maximum transform block size to 32 and outputting it.

<Embodiment 5>
FIG. 20 is a block diagram showing a configuration of an image decoding apparatus according to the fourth embodiment of the present invention. In the present embodiment, an example of decoding the encoded data generated in the second embodiment will be described.

  In FIG. 20, blocks having the same functions as those in FIG. 16 of the third embodiment are assigned the same numbers, and descriptions thereof are omitted.

  Reference numeral 302 denotes a demultiplexing unit that demultiplexes each code by the reverse process of the multiplexing unit 112 described in FIG. 14 of the first embodiment. A maximum transform block size decoding unit 304 decodes a code indicating the maximum transform block size separated by the separation unit 302. Reference numeral 314 denotes a block division information decoding unit that obtains information on the maximum conversion block size from the maximum conversion block size decoding unit and decodes the block division information.

The operation of the image decoding process in the above configuration will be described.
A bit stream is input from the terminal 201 and input to the separation unit 302. The separation unit 302 inputs header information, a quantization parameter code, a maximum transform block size code, a coefficient data code, a motion vector code, and a block division information code, which are separated and output to the subsequent stage. The code of the maximum transform block size is input to the maximum transform block size decoding unit 304 and decoded. In the second embodiment, the size is encoded as an index. This is decoded, and the maximum conversion block size is acquired as 32 pixels × 32 pixels / 8 pixels × 8 pixels. The acquired information on the maximum transform block size is input to the related units and the block division information decoding unit 314.

  The block division information decoding unit 314 inputs the maximum transformation block size from the maximum transformation block size decoding unit 304 and inputs a division flag representing the block division information from the separation unit.

FIG. 21 is a flowchart showing the detailed processing. In FIG. 21, the steps having the same functions as those in FIG. 19 are given the same numbers, and the description thereof is omitted.
In step S301, it is determined whether or not the maximum transform block size is 16 pixels × 16 pixels. If so, the process proceeds to step S254, and if not, the process proceeds to step S302.

  In step S302, it is determined whether or not the maximum conversion block size is 8 pixels × 8 pixels. If so, the process proceeds to step S257, and if not, the process proceeds to step S251.

  With the above operation, if the maximum conversion block size is 32 pixels × 32 pixels, the processing from step S251 to step S261 is performed. If the maximum conversion block size is 16 pixels × 16 pixels, the processing from step S254 to step S261 is performed. If the maximum conversion block size is 8 pixels × 8 pixels, the processing from step S257 to step S261 is performed.

  With the above configuration and operation, in the bitstream generated in the second embodiment, the maximum transform block size is set from the quantization parameter, so that the amount of computation and power consumption can be reduced even on the decoding side. That is, since the maximum transform block size is encoded, the same effect can be obtained for a short code as compared with the code of the first embodiment.

  Note that, in the second embodiment, a method has been described in which whether or not encoding is performed using the maximum transform block size is included in the header information indicating the attribute of the bitstream. This code can also be decoded by taking the configuration shown in FIG. That is, the code | symbol which identifies the profile embedded in the bit stream is isolate | separated by the separator 402, and it inputs into the profile decoding part 453. FIG. The profile decoding unit 453 decodes a profile using the maximum transform block size code (for example, an extended profile) or a basic profile when encoding is not performed. If the profile is an extended profile, the maximum transform block size decoding unit 404 is operated to obtain the maximum transform block size. Otherwise, the same effect can be obtained by setting the maximum transform block size to 32 and outputting it.

<Embodiment 6>
The above-described embodiments have been described on the assumption that each processing unit described above is configured by hardware. However, the processing performed by each processing unit described above may be configured by a computer program.

  FIG. 23 is a block diagram illustrating a configuration example of computer hardware applicable to the image display device according to each of the above embodiments.

  The CPU 2301 controls the entire computer using computer programs and data stored in the RAM 2302 and the ROM 2303, and executes each process described above as performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 2301 functions as each processing unit described above.

  The RAM 2302 has an area for temporarily storing computer programs and data loaded from the external storage device 2306, data acquired from the outside via an I / F (interface) 2309, and the like. Further, the RAM 2302 has a work area used when the CPU 2301 executes various processes. That is, the RAM 2302 can be allocated as, for example, a frame memory or can provide other various areas as appropriate.

  The ROM 2303 stores setting data of the computer, a boot program, and the like. The operation unit 2304 is configured by a keyboard, a mouse, and the like, and can input various instructions to the CPU 2301 when operated by a user of the computer. A display unit 2305 displays a processing result by the CPU 2301. The display unit 2305 is configured by an impulse type display device such as a hold type display device such as a liquid crystal display or a field emission type display device.

  The external storage device 2306 is a mass information storage device represented by a hard disk drive device. The external storage device 2306 stores an OS (Operating System) and computer programs for causing the CPU 2301 to realize the functions of the above-described units. Furthermore, each image data as a processing target may be stored in the external storage device 2306.

  Computer programs and data stored in the external storage device 2306 are appropriately loaded into the RAM 2302 under the control of the CPU 2301 and are processed by the CPU 2301. The I / F 2307 can be connected to a network such as a LAN or the Internet, and other devices such as a projection device and a display device. The computer can acquire and send various information via the I / F 2307. Can be. Reference numeral 2308 denotes a bus connecting the above-described units.

  The operation having the above-described configuration is controlled by the CPU 2301 centering on the operation described in the above flowchart.

<Other embodiments>
The object of the present invention can also be achieved by supplying a storage medium storing a computer program code for realizing the above-described functions to the system, and the system reading and executing the computer program code. In this case, the computer program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the computer program code constitutes the present invention. In addition, the operating system (OS) running on the computer performs part or all of the actual processing based on the code instruction of the program, and the above-described functions are realized by the processing. .

  Furthermore, you may implement | achieve with the following forms. That is, the computer program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the above-described functions are realized by the CPU or the like provided in the function expansion card or function expansion unit performing part or all of the actual processing.

  When the present invention is applied to the above storage medium, the computer program code corresponding to the flowchart described above is stored in the storage medium.

Claims (10)

Setting means for setting encoding control parameters;
First conversion means for performing frequency conversion on pixel data of a block having a first size and converting the pixel data to a predetermined value except for a part of the conversion coefficient;
Second conversion means for frequency-converting pixel data of a block having a second size smaller than the first size;
A block size determining means for restricting the use of the first converting means based on the encoding control parameter;
Encoding means for controlling and encoding the output of the first conversion means or the output of the second conversion means based on the determined block size;
An image encoding apparatus comprising:   The block size determining means compares the value of the encoding control parameter with a threshold value, and uses only the block of the second size when the value of the encoding control parameter is smaller than the threshold value; The image coding apparatus according to claim 1, wherein the use of the second size block and the second size block is determined.   The image encoding apparatus according to claim 1, wherein the encoding unit selects a block for encoding based on a result of the block size determining unit.   The image coding apparatus according to claim 1, wherein the coding control parameter is a value calculated from a quantization step.   The image encoding apparatus according to claim 1, wherein information indicating whether to control a block size according to the encoding control parameter is encoded. An image encoding method for controlling an image encoding device, comprising:
A setting step for setting an encoding control parameter;
A first conversion step in which pixel data of a block having a first size is subjected to frequency conversion, and conversion is performed by replacing a predetermined value except for a part of the conversion coefficient;
A second conversion step of frequency-converting pixel data of a block having a second size smaller than the first size;
A block size determination step for restricting the use of the first conversion step based on the encoding control parameter;
An encoding step of controlling and encoding the output of the first conversion step or the output of the second conversion step based on the determined block size;
An image encoding method characterized by comprising: Decoding means for decoding encoded data representing a quantization step and restoring encoding control parameters;
A frequency decoding means for decoding encoded data representing a block or a coefficient in a block obtained by re-dividing the block;
First inverse transform means for inverse frequency transforming data of a block having a first size to generate pixel data;
A second inverse transform means for performing inverse frequency transform on data of a block having a second size smaller than the first size to generate pixel data;
A block size restoring means for obtaining a coded block size based on the coding control parameter;
Decoding means for decoding the image data by controlling the input to the first inverse transform means or the second inverse transform means based on the obtained block size, the result of the frequency decoding means;
An image decoding apparatus comprising: An image decoding method for controlling an image decoding device, comprising:
A decoding step of decoding encoded data representing a quantization step and restoring an encoding control parameter;
A frequency decoding step of decoding encoded data representing a block or a coefficient in a block obtained by re-dividing the block;
A first inverse transform step of inverse frequency transforming data of a block having a first size to generate pixel data;
A second inverse transformation step of inverse frequency transforming data of a block having a second size smaller than the first size to generate pixel data;
Based on the encoding control parameter, a block size restoration step for obtaining the encoded block size;
A decoding step of decoding image data by controlling an input to the first inverse transformation step or the second inverse transformation step based on the obtained block size based on the result of the frequency decoding step;
An image decoding method characterized by comprising:   A program that causes a computer to function as the image encoding device according to claim 1 by being read and executed by the computer.   A program that causes the computer to function as the image decoding device according to claim 7 by being read and executed by the computer.

Images