HK1237161B - Image decoding method and image processing apparatus - Google Patents

Description

Image decoding method and image processing device

This application is a divisional application of the invention patent application with the application date of January 12, 2012, the invention name of the invention and the application number of 201280005652.3.

Technical Field

The present technology relates to an image decoding device, an image encoding device, and a method thereof, and more specifically, improves the encoding efficiency of a quantization parameter.

Background Art

In recent years, devices that process image information into digital form for efficient information transmission and storage in the following situations have become widely used in both broadcasting and general home use: for example, formats such as MPEG are used to compress images through orthogonal transforms (such as discrete cosine transforms) and motion compensation.

Specifically, MPEG2 (ISO/IEC 13818-2) is defined as a general-purpose image coding format and is currently widely used in a wide range of applications for both professional and consumer use. By applying the MPEG2 compression format, for interlaced images with a standard resolution of, for example, 720×480 pixels, a code size (bit rate) of 4 to 8 Mbps is allocated, thereby achieving high compression and good image quality. Furthermore, for interlaced images with a high resolution of, for example, 1920×1088 pixels, a code size (bit rate) of 18 to 22 Mbps is allocated, thereby achieving high compression and good image quality.

In addition, standardization was carried out as the Joint Model of Enhanced-Compression Video Coding, and although encoding and decoding the Joint Model of Enhanced-Compression Video Coding requires a large amount of computation, it can achieve higher coding efficiency, and the Joint Model of Enhanced-Compression Video Coding became an international standard called H.264 and MPEG-4 Part 10 (hereinafter written as "H.264/AVC (Advanced Video Coding)").

MPEG and H.264/AVC use a quantization parameter proportional to the quantization step size when quantizing macroblocks to maintain a constant compression rate. MPEG uses a quantization parameter proportional to the quantization step size, while H.264/AVC uses a quantization parameter whose value increases by "6" when the quantization step size doubles. In MPEG and H.264/AVC, the quantization parameter is encoded (see Patent Document 1).

Reference List

Patent Literature

Patent Document 1: Japanese Unexamined Patent Application Publication No. 2006-094081

Summary of the Invention

Technical issues

Now, regarding the encoding process of the quantization parameter, when the decoding order is, for example, the raster scan order as shown in FIG1 , the quantization parameter SliceQPY having an initial value is used for the header macroblock of the slice. Subsequently, processing proceeds in the decoding order indicated by the arrow, and for the macroblock located on the left (mb_qp_delta), the quantization parameter of that macroblock is updated by the difference in quantization parameters. Therefore, when the decoding order changes from the block at the right edge to the block at the left edge, the difference in images becomes large, and encoding efficiency may deteriorate. Furthermore, if the difference in the macroblock located on the left is also large, encoding efficiency may deteriorate.

In addition, regarding image compression technology, research is being conducted on the standardization of HEVC (High Efficiency Video Coding) that achieves higher coding efficiency than the H.264/AVC format. With regard to this HDVC, a basic unit called a coding unit (CU: Coding Unit) is an extension of the concept of a macroblock. In the case where each block shown in FIG2 is a coding unit, the decoding order is the order of blocks whose numbers increase sequentially from "0". In the case where the decoding order is not a raster scan order in this way, since the spatial distance is large, it is conceivable that moving from block "7" to block "8" or from block "15" to block "16" will result in a large difference.

Therefore, the purpose of this technology is to improve the coding efficiency of quantization parameters.

Solution

According to an aspect of the present disclosure, there is provided an image decoding method, comprising: setting a predicted quantization parameter obtained from an average value of the quantization parameters of the block adjacent to the left and the block adjacent to the top of the current block if the quantization parameters of the block adjacent to the left and the quantization parameters of the block adjacent to the top can be used; setting a predicted quantization parameter obtained from an initial value of the quantization parameter of a slice if the quantization parameters of the block adjacent to the left and the quantization parameters of the block adjacent to the top cannot be used; and performing inverse quantization based on a quantization parameter of the current block obtained from the predicted quantization parameter and a difference, the difference being a difference between the quantization parameter of the current block obtained from a bitstream and the predicted quantization parameter.

According to another aspect of the present disclosure, an image processing device is also provided, including a circuit configured to: if the quantization parameter of the block adjacent to the left of the current block and the quantization parameter of the block adjacent to the top of the current block can be used, set a predicted quantization parameter obtained from the average value of the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top; if the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top cannot be used, set a predicted quantization parameter obtained from the initial value of the quantization parameter of the slice; and perform inverse quantization based on the quantization parameter of the current block obtained from the predicted quantization parameter and a difference, where the difference is the difference between the quantization parameter of the current block obtained from the bitstream and the predicted quantization parameter.

According to another aspect of the present disclosure, an image processing device is provided, including: a device for setting a predicted quantization parameter obtained from the average value of the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top of the current block if the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top of the current block can be used; a device for setting a predicted quantization parameter obtained from the initial value of the quantization parameter of the slice if the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top cannot be used; and a device for performing inverse quantization based on the quantization parameter of the current block obtained from the predicted quantization parameter and a difference, where the difference is the difference between the quantization parameter of the current block obtained from the bitstream and the predicted quantization parameter.

According to another aspect of the present disclosure, a non-transitory computer-readable medium is provided, which has a program that, when executed by a computer, causes the computer to execute a method, the method including: if the quantization parameter of the block adjacent to the left of the current block and the quantization parameter of the block adjacent to the top of the current block can be used, setting a predicted quantization parameter obtained from the average value of the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top; if the quantization parameter of the block adjacent to the left and the quantization parameter of the block adjacent to the top cannot be used, setting a predicted quantization parameter obtained from the initial value of the quantization parameter of the slice; and performing inverse quantization based on the quantization parameter of the current block obtained from the predicted quantization parameter and a difference, the difference being the difference between the quantization parameter of the current block obtained from the bitstream and the predicted quantization parameter.

A first aspect of the technology is an image decoding device, comprising: an information acquisition unit configured to use a quantization parameter of a decoded block that is spatially or temporally adjacent to a block to be decoded as a selection candidate, and extract difference information indicating a difference with respect to a predicted quantization parameter selected from the selection candidate from stream information; and a quantization parameter calculation unit configured to calculate the quantization parameter of the block to be decoded based on the predicted quantization parameter and the difference information.

This technology extracts difference information indicating a difference between a predicted quantization parameter selected from selection candidates, which are quantization parameters of decoded blocks spatially or temporally adjacent to a block to be decoded. Furthermore, an image decoding device excludes at least blocks for which quantization parameters are redundant or blocks for which inverse quantization using the quantization parameters is not performed from the quantization parameters of decoded blocks spatially or temporally adjacent to the block to be decoded, and obtains selection candidates. For example, for adjacent decoded blocks in a predetermined order, quantization parameters are selected for setting the predicted quantization parameter in an order indicated by identification information included in the stream information. Alternatively, selection candidates are identified in a pre-set order, and the predicted quantization parameter is set based on the identification result. Alternatively, one or the other of the following processes is selected based on the identification result included in the stream information: setting the quantization parameter to the predicted quantization parameter in the order indicated by the identification information included in the stream information; and determining selection candidates and setting the predicted quantization parameter in a pre-set order. Furthermore, the image decoding apparatus calculates the quantization parameter for the block to be decoded by adding the difference indicated by the difference information to the predicted quantization parameter. Furthermore, if no candidate is selected, the quantization parameter of the initial value in the slice is used as the predicted quantization parameter. Furthermore, the most recently updated quantization parameter is included in the selection candidate.

A second aspect of the technology is an image decoding method, which includes: taking the quantization parameter of a decoded block that is spatially or temporally adjacent to the block to be decoded as a selection candidate, and extracting difference information indicating the difference with respect to the predicted quantization parameter selected from the selection candidate from stream information; and calculating the quantization parameter of the block to be decoded based on the predicted quantization parameter and the difference information.

A third aspect of the technology is an image encoding device, which includes: a control unit configured to set a quantization parameter for a block to be encoded; an information generating unit configured to use the quantization parameters of encoded blocks that are spatially or temporally adjacent to the block to be encoded as selection candidates, select a predicted quantization parameter from the selection candidates based on the set quantization parameter, and generate difference information indicating the difference between the predicted quantization parameter and the set quantization parameter; and an encoding unit configured to include the difference information in stream information generated by encoding the block to be encoded using the set quantization parameter.

This technology excludes at least blocks whose quantization parameters are redundant or whose quantization parameters are not used from the quantization parameters of coded blocks spatially or temporally adjacent to the block to be coded, and obtains selection candidates. The selection candidates also include the most recently updated quantization parameter. The quantization parameter with the smallest difference from the quantization parameter set based on these selection candidates is selected as the predicted quantization parameter, and identification information for selecting the predicted quantization parameter from the selection candidates is generated. For example, the identification information may be the order of the blocks corresponding to the selected quantization parameters, when adjacent coded blocks are in a predetermined order. Furthermore, the predetermined order may be an order in which priority is given to one of the coded blocks adjacent to the left, adjacent to the top, and adjacent to the temporally adjacent coded blocks. Furthermore, the order of the arrays of adjacent coded blocks may be switched. Furthermore, when the order of the selected quantization parameters is used as identification information, the quantization parameters of temporally adjacent coded blocks may be recorded based on the parameter values. In addition, when a predicted quantization parameter is selected based on a predetermined result, selection candidates can be determined in a pre-set order. In addition, with respect to the image encoding device, difference information indicating the difference between the predicted quantization parameter and the set quantization parameter is generated. In addition, when no candidate is selected, difference information indicating the difference between the quantization parameter of the initial value in the slice and the set quantization parameter is generated. In addition, it is possible to select between the following processing: a processing of setting the quantization parameter whose difference with the set quantization parameter is the smallest as the predicted quantization parameter, and a processing of determining the selection candidates in a pre-set order and selecting the predicted quantization parameter based on the determination result, and generating determination information indicating the selected processing. The generated difference information, identification information, and determination information are included in the stream information generated by performing encoding processing on the block to be encoded using the set quantization parameter.

A fourth aspect of the technology is an image encoding method, which includes: setting a quantization parameter for a block to be encoded; taking the quantization parameters of encoded blocks that are spatially or temporally adjacent to the block to be encoded as selection candidates, selecting a predicted quantization parameter from the selection candidates based on the set quantization parameter, and generating difference information indicating the difference between the predicted quantization parameter and the set quantization parameter; and including the difference information in stream information generated by encoding the block to be encoded using the set quantization parameter.

Advantageous Effects of the Invention

This technology uses the quantization parameters of previously encoded blocks that are spatially or temporally adjacent to the block to be encoded as selection candidates, and selects a predicted quantization parameter from the selection candidates based on the quantization parameter set for the block to be encoded. Difference information is generated indicating the difference between the predicted quantization parameter and the quantization parameter set for the block to be encoded. This prevents the difference in quantization parameters from becoming excessively large, and improves quantization parameter encoding efficiency.

Furthermore, when decoding stream information including difference information, a predicted quantization parameter is selected from the quantization parameters of decoded blocks that are spatially or temporally adjacent to the block to be decoded, and the quantization parameter of the block to be decoded is calculated based on the predicted quantization parameter and the difference information. Therefore, even if stream information is generated with increased coding efficiency for the quantization parameter, the quantization parameter can be restored based on the predicted quantization parameter and the difference information when decoding the stream information, allowing for accurate decoding.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a case where the decoding order is the raster scan order.

FIG. 2 is a diagram showing a case where the decoding order is not the raster scan order.

FIG3 is a diagram showing the configuration of an image encoding device.

FIG4 is a diagram showing the configuration of an information generating unit.

FIG5 is a diagram exemplarily showing a hierarchical structure of coding units.

FIG6 is a flowchart showing the operation of the image encoding device.

FIG7 is a flowchart showing the prediction process.

FIG8 is a flowchart showing the intra prediction process.

FIG9 is a flowchart showing the inter-frame prediction process.

FIG10 is a diagram for describing the operation of the information generating unit.

FIG. 11 is a diagram exemplarily showing the operation of the information generating unit.

FIG12 is a flowchart showing processing regarding quantization parameters in encoding.

FIG13 is a diagram exemplarily showing a sequence parameter set.

FIG14 is a flowchart showing a frame encoding process.

FIG15 is a diagram exemplarily showing a picture parameter set.

FIG. 16 is a diagram exemplarily showing a slice header.

FIG17 is a flowchart showing the slice encoding process.

FIG18 is a diagram showing the configuration of an image decoding device.

FIG19 is a diagram showing the configuration of a quantization parameter calculation unit.

FIG20 is a flowchart showing the operation of the image decoding device.

FIG21 is a flowchart showing the predicted image generation process.

FIG22 is a flowchart showing processing regarding quantization parameters in decoding.

FIG23 is a flowchart for describing other operations of the image encoding device.

FIG. 24 is a diagram showing an operation example in the case of implicitly predicting a quantization parameter.

FIG. 25 is an example of a flowchart for the case where the quantization parameter is implicitly predicted.

FIG. 26 shows another operation example in the case of implicitly predicting the quantization parameter.

FIG27 is a diagram exemplarily showing a procedure.

FIG28 is a flowchart for describing other operations of the image decoding device.

FIG29 is a diagram illustrating a schematic configuration of a computer device.

FIG30 is a diagram illustrating a schematic configuration of a television receiver.

FIG31 is a diagram illustrating a schematic configuration of a cellular phone.

FIG32 is a diagram illustrating a schematic configuration of a recording/playback apparatus.

FIG. 33 is a diagram illustrating a schematic configuration of an imaging apparatus.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present invention will be described. Note that the description will be made in the following order.

1. Configuration of Image Coding Apparatus

2. Operation of Image Coding Apparatus

3. Generation of identification information and difference information based on quantization parameters

4. Configuration of Image Decoding Device

5. Operation of Image Decoding Device

6. Other Operations of the Image Coding and Decoding Devices

7. Examples of software processing

8. Examples of applications in electronic devices

<1. Configuration of Image Coding Apparatus>

3 shows the configuration of an image encoding device. The image encoding device 10 includes an analog/digital conversion unit (A/D conversion unit) 11, a picture reordering buffer 12, a subtraction unit 13, an orthogonal transform unit 14, a quantization unit 15, a lossless encoding unit 16, a storage buffer 17, and a rate control unit 18. In addition, the image encoding device 10 also includes an inverse quantization unit 21, an inverse orthogonal transform unit 22, an addition unit 23, a deblocking filter 24, a frame memory 26, an intra-frame prediction unit 31, a motion prediction/compensation unit 32, and a predicted image/optimal mode selection unit 33.

The A/D conversion unit 11 converts an analog image signal into digital image data and outputs the converted digital image data to the screen rearrangement buffer 12 .

The picture resetting buffer 12 resetting the frames of the image data outputted from the A/D conversion unit 11. The picture resetting buffer 12 resetting the frames according to the GOP (Group of Pictures) structure related to the encoding process, and outputs the image data after resetting to the subtraction unit 13, the rate control unit 18, the intra prediction unit 31, and the motion prediction/compensation unit 32.

The image data output by the picture rearrangement buffer 12 and the predicted image data selected at the predicted image/optimal mode selection unit 33 described later are supplied to the subtraction unit 13. The subtraction unit 13 calculates prediction error data, which is the difference between the image data output by the picture rearrangement buffer 12 and the predicted image data supplied by the predicted image/optimal mode selection unit 33, and outputs it to the orthogonal transform unit 14.

The orthogonal transform unit 14 performs orthogonal transform processing such as discrete cosine transform (DCT) or Karhunen-Loéve transform on the prediction error data output by the subtraction unit 13 . The orthogonal transform unit 14 outputs transform coefficient data obtained by performing the orthogonal transform processing to the quantization unit 15 .

The transform coefficient data output by the orthogonal transform unit 14 and the quantization parameter (quantization scale) from the information generating unit 19 described later are supplied to the quantization unit 15. The quantization unit 15 quantizes the transform coefficient data and outputs the quantized data to the lossless encoding unit 16 and the inverse quantization unit 21. In addition, the quantization unit 15 changes the bit rate of the quantized data based on the quantization parameter set in the rate control unit 18.

The quantized data output by the quantization unit 15, the identification information and difference information from the information generation unit 19 described later, the prediction mode information from the intra prediction unit 31, and the prediction mode information and difference motion vector information from the motion prediction/compensation unit 32, etc. are supplied to the lossless encoding unit 16. In addition, information indicating whether the optimal mode is intra prediction or inter prediction is supplied by the predicted image/optimal mode selection unit 33. Note that the prediction mode information includes block size information and prediction mode of the motion prediction unit, etc., depending on whether it is intra prediction or inter prediction.

The lossless encoding unit 16 performs lossless encoding processing on the quantized data, for example, by using variable length coding, arithmetic coding, etc., to generate stream information and output the stream information to the storage buffer 17. In addition, when the optimal mode is intra-frame prediction, the lossless encoding unit 16 losslessly encodes the prediction mode information provided by the intra-frame prediction unit 31. In addition, when the optimal mode is inter-frame prediction, the lossless encoding unit 16 losslessly encodes the prediction mode information and difference motion vector provided by the motion prediction/compensation unit 32. In addition, the lossless encoding unit 16 losslessly encodes, for example, information related to the quantization parameter (such as difference information). The lossless encoding unit 16 includes the losslessly encoded information in the stream information.

The storage buffer 17 stores the encoded stream from the lossless encoding unit 16. Furthermore, the storage buffer 17 outputs the stored encoded stream at a transmission speed according to a transmission path.

The rate control unit 18 monitors the available capacity of the storage buffer 17 and sets a quantization parameter so that the bit rate of the quantized data decreases when there is little available capacity, and increases when there is sufficient capacity. Furthermore, the rate control unit 18 uses the image data provided by the picture resetting buffer 12 to detect the complexity of the image, such as activity, which is information indicating changes in pixel values. Based on the result of the detection of the image complexity, the rate control unit 18 sets the quantization parameter so that, for example, coarse quantization is achieved for image portions where pixel change values are small, and fine quantization is achieved for other portions. The rate control unit 18 outputs the set quantization parameter to the information generation unit 19.

The information generation unit 19 outputs the quantization parameter provided by the rate control unit 18 to the quantization unit 15. The information generation unit 19 also uses the quantization parameters of previously encoded blocks that are spatially or temporally adjacent to the block to be encoded as selection candidates. The information generation unit 19 selects a quantization parameter from the selection candidates based on the quantization parameter set in the rate control unit 18 and uses it as the predicted quantization parameter. Furthermore, the information generation unit 19 generates identification information corresponding to the selected quantization parameter (i.e., identification information for selecting the predicted quantization parameter from the selection candidates) and difference information indicating the difference between the predicted quantization parameter and the set quantization parameter.

4 shows the configuration of the information generating unit. The information generating unit 19 includes a quantization parameter memory unit 191 and a difference calculating unit 192. The information generating unit 19 outputs the quantization parameter provided by the rate control unit 18 to the quantization unit 15. In addition, the information generating unit 19 provides the quantization parameter provided by the rate control unit 18 to the quantization parameter memory unit 191 and the difference calculating unit 192.

The quantization parameter memory unit 191 stores the supplied quantization parameters. The difference calculation unit 192 reads out the quantization parameters of coded blocks that are spatially or temporally adjacent to the block to be coded as selection candidates from the quantization parameters of the coded blocks stored in the quantization parameter memory unit 191. Furthermore, at least blocks in which the quantization parameters are redundant and blocks in which quantization using the quantization parameters is not performed, such as blocks in which all the transform coefficient data to be quantized by the quantization unit 15 is "0," are excluded from the selection candidates. Furthermore, the difference calculation unit 192 excludes blocks (hereinafter referred to as "skipped blocks") that are determined to be skipped based on information from the motion prediction/compensation unit 32 and the predicted image/optimal mode selection unit 33, described later, from the selection candidates.

The difference calculation unit 192 selects a quantization parameter as a predicted quantization parameter from the quantization parameters based on the quantization parameter of the block to be encoded (i.e., the quantization parameter provided by the rate control unit 18). The difference calculation unit 192 also generates difference information indicating the difference between the identification information used to select the predicted quantization parameter from the selection candidates and the quantization parameter of the block to be encoded, and outputs the difference information to the lossless encoding unit 16.

3 , the inverse quantization unit 21 performs inverse quantization processing on the quantized data supplied from the quantization unit 15. The inverse quantization unit 21 outputs the transform coefficient data obtained by performing the inverse quantization processing to the inverse orthogonal transform unit 22.

The inverse orthogonal transform unit 22 performs inverse transform processing on the transform coefficient data supplied from the inverse quantization unit 21 , and outputs the obtained data to the addition unit 23 .

The addition unit 23 adds the data supplied from the inverse orthogonal transform unit 22 and the predicted image data supplied from the predicted image/optimal mode selection unit 33 to generate decoded image data, and outputs the decoded image data to the deblocking filter 24 and the frame memory 26. Note that the decoded image data is used as image data of a reference image.

The deblocking filter 24 performs filtering processing to reduce block distortion that occurs when encoding an image. The deblocking filter 24 performs filtering processing to remove block distortion from the decoded image data supplied from the addition unit 23 and outputs the decoded image data after filtering processing to the frame memory 26.

The frame memory 26 stores the decoded image data after the filtering process provided by the deblocking filter 24. The decoded image data stored in the frame memory 26 is supplied to the motion prediction/compensation unit 32 as reference image data.

The intra-frame prediction unit 31 uses the input image data of the image to be encoded, which is provided by the screen re-arrangement buffer 12, and the reference image data, which is provided by the addition unit 23, to perform prediction using the intra-frame prediction process among all candidate intra-frame prediction modes, and determines the optimal intra-frame prediction mode. The intra-frame prediction unit 31 calculates a cost function value for each intra-frame prediction mode, and based on the calculated cost function value, selects the intra-frame prediction mode with the best encoding efficiency as the optimal intra-frame prediction mode. The intra-frame prediction unit 31 outputs the predicted image data generated in the optimal intra-frame prediction mode and the cost function value of the optimal intra-frame prediction mode to the predicted image/optimal mode selection unit 33. In addition, the intra-frame prediction unit 31 outputs prediction mode information indicating the intra-frame prediction mode to the lossless encoding unit 16.

The motion prediction/compensation unit 32 uses the input image data of the image to be encoded, provided by the screen re-alignment buffer 12, and the reference image data provided by the frame memory 26 to perform predictions using all candidate inter-frame prediction modes and determine the optimal inter-frame prediction mode. The motion prediction/compensation unit 32 calculates, for example, a cost function value for each inter-frame prediction mode and, based on the calculated cost function value, selects the inter-frame prediction mode with the highest encoding efficiency as the optimal inter-frame prediction mode. The motion prediction/compensation unit 32 outputs the predicted image data generated using the optimal inter-frame prediction mode and the cost function value of the optimal inter-frame prediction mode to the predicted image/optimal mode selection unit 33. Furthermore, the motion prediction/compensation unit 32 outputs prediction mode information related to the optimal inter-frame prediction mode to the lossless encoding unit 16 and the information generation unit 19.

The predicted image/optimal mode selection unit 33 compares the cost function value provided by the intra-frame prediction unit 31 with the cost function value provided by the motion prediction/compensation unit 32, and selects the mode whose cost function value is smaller than the other cost function value as the optimal mode in which the encoding efficiency is the best. In addition, the predicted image/optimal mode selection unit 33 outputs the predicted image data generated in the optimal mode to the subtraction unit 13 and the addition unit 23. In addition, the predicted image/optimal mode selection unit 33 outputs information indicating whether the optimal mode is the intra-frame prediction mode or the inter-frame prediction mode to the lossless encoding unit 16 and the information generation unit 19. Note that the predicted image/optimal mode selection unit 33 switches between intra-frame prediction and inter-frame prediction in units of slices.

<2. Operation of Image Coding Device>

Regarding the image encoding device, for example, encoding processing is performed with a macroblock size that extends beyond the macroblock size of the H.264/AVC format. FIG5 exemplarily shows a hierarchical structure of coding units. Note that FIG5 shows a case where the maximum size is 128 pixels × 128 pixels and the hierarchical depth (depth) is "5". For example, when the hierarchical depth is "0", a 2N × 2N (N = 64 pixels) block is coding unit CU0. In addition, when the split flag = 1, coding unit CU0 is divided into four independent N × N blocks, where the N × N block is a block in a lower hierarchical level. In other words, the hierarchical depth is "1", and a 2N × 2N (N = 32 pixels) block is coding unit CU1. Similarly, when the split flag = 1, coding unit CU1 is divided into four independent blocks. In addition, when the depth is the deepest hierarchical level "4", a 2N × 2N (N = 4 pixels) block is coding unit CU4, and 8 pixels × 8 pixels is the minimum size for coding unit CU. Also, with HEVC, a prediction unit (PU: Prediction Unit) which is a basic unit for dividing a coding unit and prediction, and a transform unit (TU: Transform Unit) which is a basic unit for transform and quantization are defined.

Next, the operation of the image encoding device will be described with reference to the flowchart in Fig. 6. In step ST11, the A/D conversion unit 11 performs A/D conversion on an input image signal.

In step ST12, the picture rearrangement buffer 12 performs image rearrangement. The picture rearrangement buffer 12 stores the image data supplied from the A/D conversion unit 11 and performs rearrangement from the order for displaying pictures to the order for encoding.

In step ST13, the subtraction unit 13 generates prediction error data. The subtraction unit 13 calculates the difference between the image data of the image reset in step ST12 and the predicted image data selected by the predicted image/optimal mode selection unit 33 to generate the prediction error data. The amount of prediction error data is smaller than the amount of original image data. Therefore, the amount of data can be compressed compared to the case where the image is encoded as is.

In step ST14, the orthogonal transform unit 14 performs an orthogonal transform process. The orthogonal transform unit 14 performs an orthogonal transform on the prediction error data supplied from the subtraction unit 13. Specifically, the orthogonal transform unit 14 performs an orthogonal transform such as a discrete cosine transform or a Karhunen-Loéve transform on the prediction error data to output transform coefficient data.

In step ST15, the quantization unit 15 performs quantization processing. The quantization unit 15 quantizes the transform coefficient data. Rate control shown in the processing in step ST25 described later is performed at the time of quantization.

In step ST16 , the inverse quantization unit 21 performs inverse quantization processing. The inverse quantization unit 21 inversely quantizes the transform coefficient data quantized by the quantization unit 15 in a property corresponding to the property of the quantization unit 15 .

In step ST17 , the inverse orthogonal transform unit 22 performs inverse orthogonal transform processing. The inverse orthogonal transform unit 22 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 21 , with properties corresponding to those of the orthogonal transform unit 14 .

In step ST18, the addition unit 23 generates reference image data. The addition unit 23 adds the predicted image data supplied from the predicted image/optimal mode selection unit 33 to the data after inverse orthogonal transformation at the position corresponding to the predicted image to generate decoded data (reference image data).

In step ST19, the deblocking filter 24 performs filtering processing. The deblocking filter 24 filters the decoded image data output by the adding unit 23 and removes block distortion.

In step ST20, the frame memory 26 stores the reference image data. The frame memory 26 stores the decoded image data (reference image data) after the filtering process.

In step ST21, the intra-frame prediction unit 31 and the motion prediction/compensation unit 32 each perform prediction processing. In other words, the intra-frame prediction unit 31 performs intra-frame prediction processing for the intra-frame prediction mode, while the motion prediction/compensation unit 32 performs motion prediction/compensation processing for the inter-frame prediction mode. The prediction processing will be described below with reference to FIG. 7 , in which prediction processing is performed for each candidate prediction mode, and cost function values are calculated for each candidate prediction mode. Furthermore, based on the calculated cost function values, the optimal intra-frame prediction mode and the optimal inter-frame prediction mode are selected, and the predicted image and cost function generated in the selected prediction mode, as well as the prediction mode information, are provided to the predicted image/optimal mode selection unit 33.

In step ST22, the predicted image/optimal mode selection unit 33 selects predicted image data. Based on the cost function values output by the intra-frame prediction unit 31 and the motion prediction/compensation unit 32, the predicted image/optimal mode selection unit 33 determines the optimal mode with the highest coding efficiency. Specifically, the predicted image/optimal mode selection unit 33 determines, for example, the coding unit with the highest coding efficiency at each hierarchical level shown in FIG. 5 , the block size of the prediction unit in that coding unit, and whether to perform intra-frame prediction or inter-frame prediction. Furthermore, the predicted image/optimal mode selection unit 33 outputs the predicted image data of the determined optimal mode to the subtraction unit 13 and the addition unit 23. As described above, this predicted image data is used for the calculations in steps ST13 and ST18.

In step ST23, the lossless encoding unit 16 performs lossless encoding. The lossless encoding unit 16 performs lossless encoding on the quantized data output by the quantization unit 15. In other words, lossless encoding such as variable-length encoding or arithmetic encoding is performed on the quantized data to be compressed. In addition, the lossless encoding unit 16 performs lossless encoding on the prediction mode information corresponding to the predicted image data selected in step ST22, and includes the lossless encoded data such as the prediction mode information in the stream information generated by the lossless encoding of the quantized data.

In step ST24, the storage buffer 17 performs storage processing. The storage buffer 17 stores the stream information output by the lossless encoding unit 16. The stream information stored in this storage buffer 17 is appropriately read out and transmitted to the decoding side via the transmission path.

In step ST25, the rate control unit 18 performs rate control. The rate control unit 18 controls the rate of the quantization operation of the quantization unit 15 while storing the stream information in the buffer 17 so that overflow or underflow does not occur in the storage buffer 17.

Next, the prediction process in step ST21 in FIG. 6 will be described with reference to the flowchart in FIG. 7 .

In step ST31, the intra-frame prediction unit 31 performs intra-frame prediction processing. The intra-frame prediction unit 31 performs intra-frame prediction on the image of the prediction unit to be encoded in all candidate intra-frame prediction modes. Note that the decoded image data referenced in the intra-frame prediction uses the decoded image data before being subjected to the deblocking filtering process by the deblocking filter 24. Due to this intra-frame prediction processing, intra-frame prediction is performed in all candidate intra-frame prediction modes, and cost function values are calculated for all candidate intra-frame prediction modes. Then, based on the calculated cost function values, one intra-frame prediction mode with the best encoding efficiency is selected from all intra-frame prediction modes.

In step ST32, the motion prediction/compensation unit 32 performs inter-frame prediction processing. The motion prediction/compensation unit 32 performs inter-frame prediction processing for all candidate inter-frame prediction modes using the decoded image data after deblocking filtering stored in the frame memory 26. Due to this inter-frame prediction processing, prediction processing is performed for all candidate inter-frame prediction modes, and cost function values are calculated for all candidate inter-frame prediction modes. Then, based on the calculated cost function value, one inter-frame prediction mode with the best encoding efficiency is selected from all inter-frame prediction modes.

The intra prediction process in step ST31 in FIG. 7 will be described with reference to the flowchart in FIG. 8 .

In step ST41, the intra prediction unit 31 performs intra prediction for each prediction mode. The intra prediction unit 31 generates predicted image data for each intra prediction mode by using the decoded image data before and after the block filtering process.

In step ST42, the intra prediction unit 31 calculates a cost function value in each prediction mode. As specified in JM (Joint Model) which is reference software in the H.264/AVC format, the cost function value is calculated based on a technique in High Complexity Mode or Low Complexity Mode.

In other words, in the High Complexity Mode, lossless encoding processing is tentatively performed for all candidate prediction modes, and a cost function value represented by the following Expression (1) is calculated for each prediction mode.

Cost(Mode∈Ω)＝D+λ·R...(1)

Ω represents the entire set of candidate prediction modes for encoding the image of the prediction unit. D represents the difference energy (distortion) between the decoded image and the input image when encoding is performed in the prediction mode. R is the amount of generated code including orthogonal transform coefficients, prediction mode information, etc., and λ is the Lagrange multiplier given as a function of the quantization parameter QP. In other words, in the high complexity mode, as in the process of step ST42, lossless encoding processing is experimentally performed for all candidate prediction modes, and the cost function value represented by the above expression (1) is calculated for each prediction mode.

On the other hand, in the low complexity mode, prediction images are generated for all candidate prediction modes and header bits including different motion vectors and prediction mode information and the like are generated, and a cost function value represented by the following expression (2) is calculated.

Cost(Mode∈Ω)＝D+QP2Quant(QP)·Header_Bit...(2)

Ω represents the entire set of candidate prediction modes for encoding the image of the prediction unit. D represents the difference energy (distortion) between the decoded image and the input image when encoding is performed in the prediction mode. Header_Bit is the header bit for the prediction mode, and QP2Quant is a function given as a function of the quantization parameter QP. In other words, in Low Complexity Mode, as in the process of step ST42, by using the generated prediction image and header bits such as motion vectors and prediction mode information, the cost function value represented by the above expression (2) is calculated for each prediction mode.

In step ST43, the intra prediction unit 31 determines the optimal intra prediction mode. The intra prediction unit 31 selects the intra prediction mode with the smallest cost function value based on the cost function value calculated in step ST42, and the selected intra prediction mode is determined as the optimal prediction mode.

Next, the inter prediction process of step ST32 in FIG. 7 will be described with reference to the flowchart in FIG. 9 .

In step ST51, the motion prediction/compensation unit 32 performs motion detection processing. The motion prediction/compensation unit 32 detects a motion vector and proceeds to step ST52.

In step ST52, the motion prediction/compensation unit 32 performs motion compensation processing. The motion prediction/compensation unit 32 performs motion compensation by using the reference image data based on the motion vector detected in step ST51, and generates predicted image data.

In step ST53, the motion prediction/compensation unit 32 calculates the cost function value. The motion prediction/compensation unit 32 calculates the cost function value as described above by using the input image data of the predicted image to be encoded and the predicted image data generated in step ST52, and the like, and proceeds to step ST54.

The motion prediction/compensation unit 32 determines the optimal inter-frame prediction mode. The motion prediction/compensation unit 32 performs the processing from step ST51 to step ST53 for each inter-frame prediction mode. The motion prediction/compensation unit 32 identifies the reference index, the block size of the coding unit, and the block size of the prediction unit in the coding unit for which the cost function value calculated for each prediction mode is the minimum, and decodes the optimal inter-frame prediction mode. Note that the cost function value when inter-frame prediction is performed in skip mode is also used to determine the mode for which the cost function value is the minimum.

In addition, when the prediction image/optimal mode selection unit 33 has selected the optimal inter-frame prediction mode as the optimal prediction mode, the motion prediction/compensation unit 32 generates prediction image data so that the prediction image data of the optimal inter-frame prediction mode can be provided to the subtraction unit 13 and the addition unit 23.

<3. Generation Operation of Identification Information and Difference Information Based on Quantization Parameter>

In the above-described image encoding process, the image encoding device 10 sets a quantization parameter so that appropriate quantization is performed for each block according to the complexity of the image. Furthermore, the image encoding device 10 generates identification information and difference information and includes this information in the stream information to improve the encoding efficiency of the quantization parameter used in the quantization process of step ST15.

Next, description will be made with respect to generation of identification information and difference information. The rate control unit 18 sets a quantization parameter by using, for example, a code amount control format prescribed by TM5 in MPEG2.

Regarding the code amount control format specified by TM5 in MPEG2, the processing of steps 1 to 3 is shown.

In step 1, the amount of code to be allocated to each picture in a GOP (Group of Pictures) is distributed to uncoded pictures including the picture to be allocated based on the number of allocated bits R. This distribution is repeated in the order of the coded pictures in the GOP. At this time, the amount of code to be allocated to each picture is allocated using the following two assumptions.

The first assumption is that the product of the average quantization scale code used when encoding each picture and the amount of code generated will be constant for each picture type unless the scene changes.

Therefore, after encoding each picture, parameters _Xi , _XP , and _XB (global complexity measure) representing the complexity of the picture are updated by Expressions (3) to (5). The relationship between the quantization scale code and the amount of generated code can be estimated by these parameters.

X _I = S _I · Q _I ...(3)

_XP = _SP ·Q _P ...(4)

X _B = S _B · Q _B ...(5)

Here, S _I , S _P , and S _B are code bits generated when encoding a picture, and Q _I , Q _P , and Q _B are average quantization scale codes when encoding a picture. Furthermore, the initial value is the value expressed by the following expressions (6), (7), and (8) using bit_rate [bits/sec] as the target code amount.

X _I =160×bit_rate/115...(6)

X _P =160×bit_rate/115...(7)

X _B =160×bit_rate/115...(8)

The second assumption is that, with the quantization scale code of the I picture as a reference, the overall image quality will be constantly optimized when the ratios _KP and _KB of the quantization scale codes for the P picture and the B picture are, for example, as specified in Expression (9).

K _P = 1.0; K _B = 1.4...(9)

In other words, the quantization scale code of the B picture is constantly set to 1.4 times the quantization scale code of the I picture and the P picture. This assumes that by making the B picture to be quantized somewhat coarser than the I picture and the P picture and thereby adding the code amount saved by the B picture to the I picture and the P picture, the image quality of the I picture and the P picture will be improved, and the image quality of the B picture that refers to these pictures will also be improved.

Based on the above two assumptions, the code amount ( _TI , _TP , _TB ) allocated to each picture in a GOP is the value indicated in Expressions (10), (11), and (12). Note that picture_rate indicates the number of pictures displayed per second in the sequence.

【Mathematical formula 1】

Now, _NP and _NB are the number of uncoded P and B pictures in a GOP. In other words, under the above-mentioned image quality optimization conditions, the following estimation is made for the uncoded pictures in the GOP regarding the number of times the amount of code generated by the pictures to be allocated and those pictures of different picture types: the number of times the amount of code generated by the pictures to be allocated will be the same as the amount of code generated for the pictures to be allocated. Next, the number of pictures to be coded is calculated to be equivalent to the estimated amount of code generated by all uncoded pictures. For example, _{NP, NP} /XI, _{KP ,} which is the second term in the denominator of the first argument in the expression related to _TI, indicates how many I pictures are equivalent to the _NP uncoded pictures in the GOP. Furthermore, this is obtained by multiplying _NP by the fraction _SP /SI of the amount of code generated by an I picture relative to the amount of code generated by a _P picture, and expressing it as described above by _XI , _XP , and _XB .

The bit amount for the allocated picture is obtained by dividing the allocated code amount R for the uncoded picture by the number of pictures. Note that, however, a lower limit is set for this value in consideration of the overhead code amount for the header and the like.

Based on the allocated code amount thus obtained, each time each picture is encoded following steps 1 and 2, the code amount R to be allocated to the unencoded pictures in the GOP is updated by expression (13).

R＝RS _I,P,B ...(13)

Furthermore, when encoding the first picture of a GOP, R is updated by the following expression (14).

【Mathematical formula 2】

Where N is the number of pictures in a GOP. In addition, the initial value of R is 0 at the beginning of the sequence.

Next, step 2 will be described. In step 2, a quantization scale code is obtained for actually matching the allocated code amount ( _TI , _TP , _TB ) for each picture obtained in step 1 with the actual code amount. The quantization scale code is obtained by performing feedback control in units of macroblocks for each picture type based on the capacities of the three types of independently constructed virtual buffers.

First, before encoding the j-th macroblock, the occupancy of the virtual buffer is obtained by Expression (15) to Expression (17).

【Mathematical formula 3】

d ₀ ^I , d ₀ ^P and d ₀ ^B are the initial occupancies of the virtual buffer, B _j is the amount of bits generated from the head of the picture to the j-th macroblock, and MB _cnt is the number of macroblocks in a single picture.

The virtual buffer occupancies (dMB _cnt ^I , dMB _cnt ^P , dMB _cnt ^B ) at the end of encoding of each picture are used as initial values (d ₀ ^I , d ₀ ^P , d ₀ ^B ) of the virtual buffer occupancies of the next picture of the same picture type, respectively.

Next, the reference quantization scale code Q _j of the j-th macroblock is calculated by Expression (18).

【Mathematical formula 4】

r is a parameter that controls the response speed of the feedback group, is called a reaction parameter, and is obtained by expression (19).

【Mathematical formula 5】

Note that the initial value of the virtual buffer at the beginning of the sequence is obtained by expression (20).

【Mathematical formula 6】

Next, step 3 will be described. The activity is obtained according to Expression (21) to Expression (23) by using the luminance signal pixel value of the original image, for example, by using the pixel values of a total of eight blocks of four 8×8 blocks of the frame DCT mode and four 8×8 blocks of the field DCT coding mode.

【Mathematical formula 7】

var_sblk in expression (21) is the sum of the squares of the differences between the image data of each pixel and its average value, so the more complex the images of these 8×8 blocks, the larger the value. _Pk in expressions (22) and (23) is the in-block pixel value of the luminance signal of the original image. The reason for assuming the minimum value (min) in expression (22) is to make quantization finer even when there is a locally smooth portion in the 16×16 macroblock. In addition, the normalized activity _Nactj whose value is in the range of 0.5 to 2 is obtained by expression (24).

【Mathematical formula 8】

avg_act is the average value of the activity of the picture encoded immediately before. Quantization scale code mquant _j considering visual properties is obtained by expression (25) based on the reference quantization scale code Q _j .

【Mathematical formula 9】

mquant _j ＝Q _j ×Nact _j …(25)

The rate control unit 18 outputs the quantization scale code mquant _j calculated as described above as a quantization parameter. Furthermore, for macroblocks located at slice boundaries, the quantization parameter is generated using the same technique as for macroblocks located at locations other than slice boundaries. Note that the quantization parameter is not limited to being determined based on the activity as described above, and may be determined to minimize the cost function value.

Note that the rate control method specified in TM5 of MPEG2 described above describes a case where processing is performed in units of macroblocks. Therefore, by performing similar processing in units of blocks whose quantization parameters can be switched, it is possible to set a quantization parameter for each of the blocks whose quantization parameters can be switched.

Next, the operation of generating information for improving the encoding efficiency of the quantization parameter will be described. The information generation unit 19 selects quantization parameters that are spatially or temporally adjacent to the block to be encoded as selection candidates. The information generation unit 19 also selects a quantization parameter from the selection candidates based on the quantization parameter set for the block to be encoded, and uses the selected quantization parameter as the predicted quantization parameter. The information generation unit 19 also generates identification information for selecting the predicted quantization parameter from the selection candidates, and difference information indicating the difference between the predicted quantization parameter and the quantization parameter set for the block to be encoded.

Figure 10 is a diagram for describing the operation of the information generation unit, showing a frame to be encoded and an encoded frame that is temporally closest in display order. The quantization parameter of the block to be encoded in the frame to be encoded is, for example, "QP_0." In addition, the quantization parameter of the block adjacent to the left is, for example, "QP_A." Similarly, the quantization parameters of the blocks adjacent to the top, to the upper right, to the upper left, and to the lower left are, for example, "QP_B," "QP_C," "QP_D," and "QP_E." In addition, the quantization parameter of the temporally adjacent blocks is "QP_T." Note that when encoding the block to be encoded in the frame to be encoded, the quantization parameters "QP_A" to "QP_E" and "QP_T" are stored in the quantization parameter memory unit 191. In addition, each block is the minimum unit block with respect to which the quantization parameter can be changed.

The difference calculation unit 192 selects the quantization parameters of the coded blocks adjacent to the block to be coded as selection candidates, selects the quantization parameter with the smallest difference from the quantization parameter set for the block to be coded from the selection candidates, and uses the selected quantization parameter as the predicted quantization parameter. The difference calculation unit 192 generates identification information for selecting the predicted parameter from the selection candidates and difference information indicating the difference between the predicted quantization parameter and the quantization parameter of the block to be coded.

Fig. 11 is a diagram showing an operation example of the information generating unit. Note that a case where a quantization parameter is not set for a block because the block is a skip block or has no residual information is indicated by "-".

When the block to be encoded is block BK0, the quantization parameters of the encoded block are "QP_A=32", "QP_B=40", "QP_C=40", "QP_D=35", "QP_E=-", and "QP_T=31". Here, the information generation unit 19 excludes blocks for which quantization parameters are not set due to being skip blocks or blocks without residual information, and blocks for which quantization parameters are redundant from the candidates. Therefore, the selection candidates are encoded blocks with quantization parameters of "QP_A=32", "QP_B=40", "QP_D=35", and "QP_T=31". In addition, the information generation unit 19 sets identification information, such as an index number, for the selection candidates in advance. The identification information may be set only for adjacent encoded blocks, or the identification information may be set for the quantization parameters of adjacent encoded blocks.

When setting identification information to adjacent coded blocks, the information generation unit 19 sets index numbers in the order of arrangement when the adjacent coded blocks are in a predetermined arrangement order. The predetermined arrangement order is, for example, an array order in which priority is given to one of the coded blocks adjacent to the left, the coded blocks adjacent above, and the coded blocks adjacent in time. In addition, the information generation unit 19 is capable of switching the order of the arrays. In the case where the order of the arrays can be switched, information indicating which array order is included in the stream information. In addition, the lossless encoding unit 16 and the information generation unit 19 set and losslessly encode the identification information so that a smaller amount of code is used when encoding the identification information of the blocks given priority.

The difference calculation unit 192 selects the candidate with the smallest difference with the quantization parameter of the block to be encoded from the selection candidates, and uses the identification information set for the selected candidate to generate identification information for selecting a predicted quantization parameter from the selection candidates. Furthermore, the difference calculation unit 192 generates difference information indicating the difference between the predicted quantization parameter, which is the selected candidate quantization parameter, and the quantization parameter of the block to be encoded. For example, when giving priority to the encoded blocks adjacent to the left in FIG. 11 , the information generation unit 19 sets "0 (index number): block with QP_A," "1: block with QP_B," "2: block with QP_D," and "3: block with QP_T." Furthermore, if the quantization parameter of the block to be encoded is "33," for example, the difference calculation unit 192 sets the index number of the block with the smallest difference with the quantization parameter of the block to be encoded as the identification information "0 (index number)." Furthermore, the difference calculation unit 192 generates difference information “1 (=33−32)” indicating the difference between the predicted quantization parameter and the quantization parameter of the block to be encoded.

By assigning identification information to the blocks to be encoded in this manner, the encoding efficiency of the quantization parameter can be improved. For example, if the left block is given priority and the block order is quantization parameters "QP_A," "QP_B," "QP_C," "QP_D," "QP_E," and "QP_T," the amount of data for an image in which there are more blocks to be encoded that are similar to the image of the left block is small. Furthermore, if the upper block is given priority and the block order is quantization parameters "QP_B," "QP_A," "QP_C," "QP_D," "QP_E," and "QP_T," the amount of data for an image in which there are more blocks to be encoded that are similar to the image of the upper block is small. Furthermore, if temporally adjacent blocks are given priority and the block order is quantization parameters "QP_T," "QP_A," "QP_B," "QP_C," "QP_D," and "QP_E," the amount of data for an image in which there are more blocks to be encoded that are similar to the temporally adjacent image (i.e., more stationary objects) is small.

When setting identification information for the quantization parameters of adjacent coded blocks, information generation unit 19 sets index numbers for the adjacent coded blocks in a predetermined array order. For example, information generation unit 19 sets index numbers in the order of quantization parameters with smaller parameter values. In other words, in the example shown in FIG11 , information generation unit 19 sets index numbers such as "0 (index number): 32 (quantization parameter)", "1:40", "2:35", and "3:31".

The difference calculation unit 192 selects the candidate with the smallest difference with the quantization parameter of the block to be encoded from the selection candidates, and uses the identification information set for the selected candidate to generate identification information for selecting the predicted quantization parameter from the selection candidates. Furthermore, the difference calculation unit 192 generates difference information indicating the difference between the predicted quantization parameter and the quantization parameter of the block to be encoded. For example, if the quantization parameter of the block to be encoded is "33," the difference calculation unit 192 generates difference information "1 (=33-32)" as identification information.

Furthermore, in a case where there is no selection candidate, the information generating unit 19 generates difference information indicating a difference between the quantization parameter SliceQPY of the initial value in the slice and the set quantization parameter.

12 is a flowchart showing the processing of the quantization parameter in encoding. In step ST61, the image encoding device 10 generates information for obtaining the minimum size of the quantization parameter unit (MinQpUnitSize). The minimum size of the quantization parameter unit is the minimum size in which the quantization parameter can be adaptively switched.

The image encoding device 10 uses, for example, a difference regarding the transform unit minimum size (MinTransformUnitSize) as information for obtaining the quantization parameter unit minimum size (MinQpUnitSize).

The minimum quantization parameter unit size (MinQpUnitSize) is determined by Expression (26).

MinQpUnitSize=1<<(log2_min_transform_unit_size_minus2+log2_min_qp_unit_size_offset+2)...(26)

Note that “log2_min_transform_unit_size_minus2” is a parameter for determining the minimum size of a transform unit (MinTransformUnitSize).

The minimum transform unit size (MinTransformUnitSize) is determined by expression (27).

MinTransformUnitSize＝1<<(log2_min_transform_unit_size_minus2+2)...(27)

As can be clearly understood from Expressions (26) and (27), the difference between the minimum quantization parameter unit size (MinQpUnitSize) and the minimum transform unit size (MinTransformUnitSize) is equal to "log2_min_qp_unit_size_offset". Note that the quantization parameter is used in units of transform units (TUs). In other words, the quantization parameter is constant within a transform unit.

Furthermore, the minimum quantization parameter unit size (MinQpUnitSize) may be determined based on the coding unit size. In this case, the image encoding device 10 uses, for example, information specifying the minimum size of the coding unit CU (log2_min_coding_block_size_minus3) and the maximum size of the coding unit CU (log2_diff_max_min_coding_block_size). Note that the maximum size of the coding unit CU, "log2MaxCUSize," is as shown in Expression (28).

log2MaxCUSize=log2_min_coding_block_size_minus 3+3+log2_diff_max_min_coding_block_size...(28)

The logarithmic value of the minimum size of the quantization parameter unit (log2MinQpUnitSize) is determined by Expression (29).

log2MinQpUnitSize=log2_min_coding_block_size_minus 3+3+log2_diff_max_min_coding_block_size-log2_min_qp_unit_size_offset...(29)

Therefore, "log2_min_qp_unit_size_offset" is set to a larger value to reduce the minimum size of the quantization parameter unit. For example, if the minimum size of the coding unit CU is "8×8" and the maximum size is "64×64", setting "log2_min_qp_unit_size_offset" to "1" makes the minimum size of the quantization parameter unit "32×32". In addition, setting "log2_min_qp_unit_size_offset" to "2" makes the minimum size of the quantization parameter unit "16×16".

In step ST62, the image encoding device 10 performs a process of including the generated information in the stream information. The image encoding device 10 includes "log2_min_qp_unit_size_offset" and "log2_min_qp_unit_size_offset" as a parameter that determines the minimum size of the transform unit (MinTransformUnitSize) in the stream information, and proceeds to step ST63. In addition, in the case where the minimum size of the quantization parameter unit is determined according to the coding unit size, "log2_min_coding_block_size_minus3", "log2_diff_max_min_coding_block_size", and "log2_min_qp_unit_size_offset" are included in the stream information. The image encoding device 10 includes the generated information in a sequence parameter set (SPS: Sequence Parameter Set) whose syntax is defined as, for example, RBSP (Raw Byte Sequence Payload). Note that FIG13 exemplarily shows a sequence parameter set.

In step ST63, the image encoding device 10 determines whether there is a frame to be encoded. If there is a frame to be encoded, the image encoding device 10 proceeds to step ST64 and performs the frame encoding process shown in Figure 14. If there is no frame to be encoded, the encoding process is ended.

In the frame encoding process of Figure 14, in step ST71, the image encoding device 10 determines whether there is a slice to be encoded. If there is a slice to be encoded, the image encoding device 10 proceeds to step ST72. If there is no slice to be encoded, the encoding process of the frame is ended.

In step ST72, the image encoding device 10 decides the quantization parameter of the slice to be encoded. The image encoding device 10 decides the quantization parameter of the initial value in size as the target code amount, and proceeds to step ST73.

In step ST73, the image encoding device 10 calculates "slice_qp_delta." The quantization parameter SliceQPY, which is the initial value in the slice, and "pic_init_qp_minus26" set in advance by the user or the like have the relationship shown in Expression (30). Therefore, the image encoding device 10 calculates "slice_qp_delta" as the quantization parameter decided in step ST72 and proceeds to step ST74.

SliceQPY＝26+pic_init_qp_minus26+slice_qp_delta...(30)

In step ST74, the image encoding device 10 includes "slice_qp_delta" and "pic_init_qp_minus26" in the stream information. The image encoding device 10 includes the calculated "slice_qp_delta" in, for example, a header slice of the stream information. Furthermore, the image encoding device 10 includes the set "pic_init_qp_minus26" in, for example, a picture parameter set of the stream information. By including "slice_qp_delta" and "pic_init_qp_minus26" in the stream information in this way, the image decoding device that decodes the stream information can calculate the quantization parameter SliceQPY of the initial value in the slice by performing the calculation of Expression (30). Note that FIG. 15 exemplarily shows a sequence parameter set, and FIG. 16 shows a slice header.

In step ST75, the image encoding device 10 performs a slice encoding process. Fig. 17 is a flowchart showing the slice encoding process.

In step ST81 of FIG17 , the image encoding device 10 determines whether there is a coding unit CU to be encoded. If there is a coding unit for which encoding processing has not yet been performed in the slice to be encoded, the image encoding device 10 proceeds to step ST82. If encoding processing is completed for all coding units in the slice, the image encoding device 10 ends the slice encoding process.

In step ST82, the image encoding device 10 determines whether a transform unit TU exists in the coding unit CU to be encoded. If a transform unit exists, the image encoding device 10 proceeds to step ST83, and if not, the image encoding device 10 proceeds to step ST87. For example, if all coefficients to be quantized using the quantization parameter are "0" or if a block is skipped, the process proceeds to step ST87.

In step ST83, the image encoding device 10 determines the quantization parameter of the coding unit CU to be encoded. The rate control unit 18 of the image encoding device 10 determines the quantization parameter according to the complexity of the image of the coding unit as described above, or the rate control unit 18 of the image encoding device 10 determines the quantization parameter so that the cost function value is small, and proceeds to step ST84.

In step ST84, the image coding device 10 sets identification information to the selection candidate. The information generation unit 19 of the image coding device 10 selects the quantization parameters of the encoded blocks that are spatially or temporally adjacent to the coding unit to be encoded as selection candidates. Furthermore, in the case of blocks for which no quantization parameters are set due to being skipped blocks or having no residual information, or in the case of blocks whose quantization parameters are equal to those of another candidate, the information generation unit 19 excludes these quantization parameters from the selection candidates. The image coding device 10 sets identification information (e.g., index (ref_qp_block_index)) to the selection candidate and proceeds to step ST85.

In step ST85, the image encoding device 10 generates identification information and difference information. The information generation unit 19 of the image encoding device 10 selects the candidate whose difference with the quantization parameter of the coding unit to be encoded is the smallest from the selection candidates, and uses it as the predicted quantization parameter. The information generation unit 19 generates identification information as identification information for selecting the predicted quantization parameter from the selection candidates by using the index (ref_qp_block_index) of the selected candidate. In addition, the information generation unit 19 uses the difference (qb_qp_delta) between the predicted quantization parameter and the quantization parameter of the coding unit to be encoded as the difference information, and proceeds to step ST86. Now, with respect to the predicted quantization parameter indicated by the index (ref_qp_block_index) of the determined candidate as "ref_qp (ref_qp_block_index)", the quantization parameter (CurrentQP) of the coding unit to be encoded presents the relationship indicated in Expression (31).

CurrentQP=qb_qp_delta+ref_qp(ref_qp_block_index)...(31)

In step ST86, the image encoding device 10 includes the identification information and the difference information in the stream information. The lossless encoding unit 16 of the image encoding device 10 losslessly encodes the identification information and the difference information generated by the information generating unit 19, includes them in the stream information, and proceeds to step ST87.

In step ST87 , the image encoding device 10 quantizes the coding unit using the decided quantization parameter by the quantization unit 15 , and returns to step ST81 .

In this manner, the image coding device 10 selects, from among the quantization parameters of previously encoded blocks spatially or temporally adjacent to the block to be encoded, the candidate with the smallest difference from the quantization parameter of the block to be encoded as the predicted quantization parameter. Furthermore, the image coding device 10 generates identification information corresponding to the selected quantization parameter. Furthermore, the image coding device 10 generates difference information indicating the difference between the predicted quantization parameter and the quantization parameter of the block to be encoded. The image coding device 10 includes the generated identification information and difference information in the stream information. Therefore, since the candidate with the smallest difference is selected as the predicted quantization parameter, the difference between the predicted quantization parameter and the quantization parameter of the block to be encoded can be prevented from becoming excessive. Consequently, the image coding device 10 can improve the encoding efficiency of the quantization parameters.

<4. Configuration of Image Decoding Device>

Next, an image decoding device that performs decoding processing on stream information output by the image encoding device will be described. The encoded stream generated by encoding the input image is supplied to the image decoding device via a predetermined transmission path, recording medium, etc. and decoded.

FIG18 shows the configuration of an image decoding device that performs a decoding process on stream information. The image decoding device 50 includes a storage buffer 51, a lossless decoding unit 52, an inverse quantization unit 53, an inverse orthogonal transform unit 54, an addition unit 55, a deblocking filter 56, a picture re-alignment buffer 57, and a digital/analog conversion unit (D/A conversion unit) 58. The image decoding device 50 also includes a quantization parameter calculation unit 59, a frame memory 61, an intra-frame prediction unit 71, a motion compensation unit 72, and a selector 73.

The storage buffer 51 stores the transmitted stream information. The lossless decoding unit 52 decodes the stream information supplied from the storage buffer 51 by a format corresponding to the encoding format of the lossless encoding unit 16 in FIG. 3 .

The lossless decoding unit 52 functions as an information acquisition unit and acquires various types of information from the stream information. For example, the lossless decoding unit 52 outputs the prediction mode information obtained by decoding the stream information to the intra-frame prediction unit 71 and the motion compensation unit 72. Furthermore, the lossless decoding unit 52 outputs the difference motion vector, threshold value, or threshold value generation information obtained by decoding the stream information to the motion compensation unit 72. Furthermore, the lossless decoding unit 52 outputs information related to the quantization parameter obtained by decoding the stream information (e.g., difference information, etc.) to the quantization parameter calculation unit 59. Furthermore, the lossless decoding unit 52 outputs the quantized data obtained by decoding the stream information to the inverse quantization unit 53.

The inverse quantization unit 53 inversely quantizes the quantized data decoded at the lossless decoding unit 52 in a format corresponding to the quantization format of the quantization unit 15 in FIG 3. The inverse orthogonal transform unit 54 inversely orthogonally transforms the output of the inverse quantization unit 53 in a format corresponding to the orthogonal transform format of the orthogonal transform unit 14 in FIG 3 and outputs it to the addition unit 55.

The addition unit 55 adds the data after the inverse orthogonal transform to the predicted image data supplied from the selector 73 to generate decoded image data and outputs it to the deblocking filter 56 and the intra prediction unit 71 .

The deblocking filter 56 performs filter processing on the decoded image data supplied from the addition unit 55 , removes block distortion, supplies to and stores in the frame memory 61 , and outputs it to the screen rearrangement buffer 57 .

The picture rearrangement buffer 57 rearranges the image. In other words, the order of the frames rearranged in the order for encoding by the picture rearrangement buffer 12 in FIG. 3 is rearranged into the original order for display and output to the D/A conversion unit 58.

The D/A conversion unit 58 D/A-converts the image data supplied from the screen rearrangement buffer 57 to display an image by outputting to a display not shown.

The quantization parameter calculation unit 59 restores the quantization parameter based on the information supplied from the lossless decoding unit 52, and outputs it to the inverse quantization unit 53. FIG19 shows a configuration of a quantization parameter calculation unit having a calculation unit 591 and a quantization parameter memory unit 592. As shown in FIG.

The calculation unit 591 uses the information provided by the lossless decoding unit 52 and the quantization parameter stored in the quantization parameter memory unit 592 to restore the quantization parameter used for quantization in the encoding process to which the block to be decoded is subjected, and outputs it to the inverse quantization unit 53. The calculation unit 591 also stores the quantization parameter of the block to be decoded in the quantization parameter memory unit 592.

The calculation unit 591 performs calculation of Expression (30) using, for example, “pic_init_qp_minus26” extracted from the parameter set and “slice_qp_delta” extracted from the slice header, calculates the quantization parameter SliceQPY, and outputs to the inverse quantization unit 53 .

The calculation unit 591 also uses the identification information and difference information provided by the lossless decoding unit 52, as well as the quantization parameters of the decoded blocks stored in the quantization parameter memory unit 592, to calculate the quantization parameters for the block to be decoded. The calculation unit 591 outputs the calculated quantization parameters to the inverse quantization unit 53. In this case, the calculation unit 591 reads the quantization parameters of decoded blocks that are spatially or temporally adjacent to the block to be decoded from the quantization parameters of the decoded blocks stored in the quantization parameter memory unit 592. The calculation unit 591 sets selection candidates in the same manner as the difference calculation unit 192. For example, the calculation unit 591 excludes blocks in which at least the quantization parameters are redundant or in which inverse quantization using the quantization parameters has not been performed, and excludes them as selection candidates. Furthermore, the calculation unit 591 sets identification information, i.e., an index (ref_qp_block_index), for the quantization parameters of each candidate in the same manner as the difference calculation unit 192. In other words, the calculation unit 591 sets the index (ref_qp_block_index) for the adjacent decoded blocks in a predetermined array order. The calculation unit 591 calculates Expression (31) by using the quantization parameter "ref_qp (ref_qp_block_index)" corresponding to the identification information (i.e., predicted quantization parameter) provided by the lossless decoding unit 52 and the difference (qb_qp_delta) indicated by the difference information provided by the lossless decoding unit 52. The calculation unit 591 outputs the calculated quantization parameter (CurrentQP) to the inverse quantization unit 53 as the quantization parameter to be decoded. In addition, if there is no selection candidate, the calculation unit 591 outputs the quantization parameter of the initial value in the slice to the inverse quantization unit 53.

Furthermore, when information specifying the array order of blocks is extracted from the stream information, the calculation unit 591 sets an index (ref_qp_block_index) for the decoded block having the specified array order. Therefore, even if the array order is changed by the image encoding device 10, the quantization parameter used by the image encoding device 10 can be restored.

Referring back to FIG. 18 , the frame memory 61 stores the decoded image data after the filtering process provided by the deblocking filter 56 .

The intra prediction unit 71 generates predicted image data based on the prediction mode information supplied from the lossless decoding unit 52 and the decoded image data supplied from the addition unit 55 , and outputs the generated predicted image data to the selector 73 .

The motion compensation unit 72 reads the reference image data from the frame memory 61 based on the prediction mode information and the motion vector supplied from the lossless decoding unit 52, and performs motion compensation to generate predicted image data. The motion compensation unit 72 outputs the generated predicted image data to the selector 73. In addition, the motion compensation unit 72 generates the predicted image data while switching the filtering properties according to the size of the motion vector.

The selector 73 selects the intra prediction unit 71 in the case of intra prediction and selects the motion compensation unit 72 in the case of inter prediction based on the prediction mode information supplied from the lossless decoding unit 52. The selector 73 outputs the predicted image data generated by the selected intra prediction unit 71 or motion compensation unit 72 to the addition unit 55.

The selector 73 selects the intra prediction unit 71 in the case of intra prediction and selects the motion compensation unit 72 in the case of inter prediction based on the prediction mode information supplied from the lossless decoding unit 52. The selector 73 outputs the predicted image data generated by the selected intra prediction unit 71 or motion compensation unit 72 to the addition unit 55.

<5. Operation of Image Decoding Device>

Next, the operation of the image decoding device 50 will be described with reference to the flowchart in FIG. 20 .

In step ST91, the storage buffer 51 stores the stream information supplied thereto. In step ST92, the lossless decoding unit 52 performs lossless decoding processing. The lossless decoding unit 52 decodes the stream information supplied by the storage buffer 51. In other words, it obtains the quantized data for each picture encoded by the lossless encoding unit 16 in FIG. 3 . Furthermore, the lossless decoding unit 52 losslessly encodes the prediction mode information included in the stream information, and if the obtained prediction mode information is information related to the intra-frame prediction mode, it outputs the prediction mode information to the intra-frame prediction unit 71. Furthermore, if the prediction mode information is information related to the inter-frame prediction mode, the lossless decoding unit 52 outputs the prediction mode information to the motion compensation unit 72. Furthermore, the lossless decoding unit 52 outputs the difference motion vector, threshold value, or threshold value generation information obtained by decoding the stream information to the motion compensation unit 72.

In step ST93, the inverse quantization unit 53 performs inverse quantization processing. The inverse quantization unit 53 inversely quantizes the quantized data decoded by the inverse decoding unit 52 with properties corresponding to those of the quantization unit 15 in FIG3 .

In step ST94, the inverse orthogonal transform unit 54 performs inverse orthogonal transform processing. The inverse orthogonal transform unit 54 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 53, with properties corresponding to those of the orthogonal transform unit 14 in FIG.

In step ST95, the addition unit 55 generates decoded image data. The addition unit 55 adds the data obtained by performing the inverse orthogonal transform process to the predicted image data selected in step ST99 described later, and generates decoded image data. Thus, the original image is decoded.

In step ST96, the deblocking filter 56 performs filtering processing. The deblocking filter 56 performs filtering processing on the decoded image data output by the adding unit 55, and removes block distortion included in the decoded image.

In step ST97, the frame memory 61 performs storage processing on the decoded image data. Note that the decoded image data stored in the frame memory 61 and the decoded image data output by the addition unit 55 are used as reference image data to generate predicted image data.

In step ST98 , the intra prediction unit 71 and the motion compensation unit 72 perform prediction processing. The intra prediction unit 71 and the motion compensation unit 72 each perform prediction processing corresponding to the prediction mode information supplied from the lossless decoding unit 52 .

In other words, when prediction mode information of intra prediction is supplied from the lossless decoding unit 52, the intra prediction unit 71 performs intra prediction processing based on the prediction mode information and generates predicted image data. Furthermore, when prediction mode information of inter prediction is supplied from the lossless decoding unit 52, the motion compensation unit 72 performs motion compensation based on the prediction mode information and generates predicted image data.

In step ST99, the selector 73 selects the predicted image data. The selector 73 selects the predicted image supplied by the intra-prediction unit 71 and the predicted image data supplied by the motion compensation unit 72, and supplies the selected predicted image data to the addition unit 55 so as to be added to the output of the inverse orthogonal transform unit 54 in step ST95 as described above.

In step ST100, the picture rearrangement buffer 57 performs image rearrangement. In other words, in the picture rearrangement buffer 57, the order of frames rearranged for encoding by the picture rearrangement buffer 12 of the image encoding device 10 in FIG. 3 is rearranged to the original order for display.

In step ST101, the D/A conversion unit 58 D/A converts the image data from the screen rearrangement buffer 57. The image is output to a display not shown and the image is displayed.

Next, the generation of the predicted image in step ST98 in FIG. 20 will be described with reference to the flowchart in FIG. 21 .

In step ST111, the lossless decoding unit 52 determines whether the current block is intra-coded. If the prediction mode information obtained by performing lossless coding is intra-frame prediction mode information, the lossless decoding unit 52 supplies the prediction mode information to the intra-frame prediction unit 71 and proceeds to step ST112. If the prediction mode information is not intra-frame prediction mode information, the lossless decoding unit 52 supplies the prediction mode information to the motion compensation unit 72 and proceeds to step ST113.

In step ST112, the intra prediction unit 71 performs intra prediction processing. The intra prediction unit 71 performs intra prediction by using the decoded image data before the deblocking filtering processing and the prediction mode information supplied from the addition unit 55, and generates predicted image data.

In step ST113, the motion compensation unit 72 performs inter-prediction image generation processing. The motion compensation unit 72 reads out reference image data from the frame memory 61 based on information (such as prediction mode information) supplied from the lossless decoding unit 52 and generates prediction image data.

FIG22 is a flowchart illustrating the processing related to quantization parameters during decoding. In step ST121, the image decoding device 50 extracts information for obtaining the minimum size of the quantization parameter unit. The image decoding device 50 extracts information for obtaining the minimum size of the quantization parameter unit, such as "log2_min_qp_unit_size_offset," from the stream information and proceeds to step ST122.

In step ST122, the image decoding device 50 calculates the minimum quantization parameter unit size. The image decoding device 50 calculates Expression (26) by using "log2_min_qp_unit_size_offset" and the parameter "log2_min_transform_unit_size_minus2" that determines the minimum transform unit size (MinTransformUnitSize), and calculates the minimum quantization parameter unit size (MinQpUnitSize). Furthermore, the image decoding device 50 can calculate the minimum quantization parameter unit size (MinQpUnitSize) by calculating Expression (29).

In step ST123, the image decoding device 50 determines whether there is a frame to be decoded. If there is a frame to be decoded, the image decoding device 50 proceeds to step ST124, and if there is no frame to be decoded, ends this process.

In step ST124, the image decoding device 50 determines whether there is a slice to be decoded. If there is a slice to be decoded, the image decoding device 50 proceeds to step ST125, and if there is no slice to be decoded, returns to step ST123.

In step ST125, the image decoding device 50 extracts information about the quantization parameter used to obtain the initial value in the slice. The lossless decoding unit 52 of the image decoding device 50 extracts, for example, "pic_init_qp_minus26" from the picture parameter set (PPS). Furthermore, it extracts "slice_qp_delta" from the slice header and proceeds to step ST126.

In step ST126, the image decoding device 50 calculates the quantization parameter of the initial value in the slice. The quantization parameter calculation unit 59 of the image decoding device 50 calculates the quantization parameter SliceQPY by performing calculation of Expression (30) using "pic_init_qp_minus26" and "slice_qp_delta", and proceeds to step ST127.

In step ST127, the image decoding device 50 determines whether there is a coding unit CU to be decoded. If there is a coding unit to be decoded, the image decoding device 50 proceeds to step ST128, and if there is no coding unit to be decoded, returns to step ST124.

In step ST128, the image decoding device 50 sets identification information to the selection candidate. The quantization parameter calculation unit 59 of the image decoding device 50 sets identification information to the selection candidate in the same manner as the information generation unit 19 of the image encoding device 10. In other words, the quantization parameter calculation unit 59 uses the quantization parameters of decoded coding units that are spatially or temporally adjacent to the coding unit to be decoded as selection candidates. In addition, if the quantization parameter is not set due to being a skip block or having no residual information, or if the quantization parameter is equal to another candidate, it is excluded from the selection candidate. The quantization parameter calculation unit 59 sets the same identification information (e.g., index (ref_qp_block_index)) as the image encoding device 10 to the quantization parameter of the candidate and proceeds to step ST129.

In step ST129, the image decoding device 50 obtains the identification information and the difference information. The lossless decoding unit 52 of the image decoding device 50 extracts the identification information and the difference information, namely, the index (ref_qp_block_index) and the difference (qb_qp_delta), included in the stream information at the image encoding device 10. The lossless decoding unit 52 provides the extracted identification information and difference information to the quantization parameter calculation unit 59 and proceeds to step ST130.

In step ST130, the image decoding device 50 calculates a quantization parameter using the identification information and the difference information. The quantization parameter calculation unit 59 of the image decoding device 50 calculates Expression (31) by using the quantization parameter "ref_qp (ref_qp_block_index)" corresponding to the index (ref_qp_block_index) as the identification information and (qb_qp_delta) as the difference information. In other words, the quantization parameter of the coding unit to be decoded is calculated by adding the difference to the predicted quantization parameter. The quantization parameter calculation unit 59 outputs the quantization parameter (CurrentQP) of the coding unit to be decoded to the inverse quantization unit 53, and returns to step ST124.

Therefore, by using the identification information and difference information included in the stream information, the quantization parameter associated with the block to be decoded can be restored even if the quantization parameter for each block in the block is not included in the stream information. In other words, even if the encoding efficiency of the quantization parameter is improved by using the identification information and difference information at the image encoding device 10, the quantization parameter associated with each block in the block can be restored and the decoding process can be correctly performed to generate a decoded image at the image decoding device 50.

<6. Other Operations of the Image Encoding Device and the Image Decoding Device>

Through the above-described operations of the image encoding and decoding devices, the quantization parameters of previously encoded blocks that are spatially or temporally adjacent to the block to be encoded are selected as selection candidates. Furthermore, a quantization parameter selected from the selection candidates based on the quantization parameter set for the block to be encoded is used as a predicted quantization parameter. Furthermore, by including identification information for selecting the predicted quantization parameter from the selection candidates and difference information indicating the difference between the predicted quantization parameter and the quantization parameter set for the block to be encoded in the stream information, the coding efficiency of the quantization parameter is improved.

However, the selection candidates are not limited to the quantization parameters of coded blocks that are spatially or temporally adjacent to the block to be coded, and the most recently updated quantization parameters may be included in the selection candidates. As described later, even if the blocks that are spatially or temporally adjacent to the coded block are not involved in inverse quantization, the quantization parameter of a block located adjacent to the block to be coded may be set as the predicted quantization parameter. Furthermore, the quantization parameter may be a quantization parameter that is explicitly or implicitly predicted as a selection candidate, and difference information indicating the difference between the predicted quantization parameter and the quantization parameter of the block to be coded may be generated.

Next, as another operation of the image encoding device and the image decoding device, the following case will be described, in which the minimum quantization parameter unit size (MinQpUnitSize) is determined according to the coding unit size and a predicted quantization parameter is explicitly or implicitly selected from candidate quantization parameters. Note that the following description will focus on parts that differ from the above-described image encoding device and image decoding device.

When selecting a predicted quantization parameter explicitly or implicitly from candidate quantization parameters, the image encoding device includes "qp_explicit_flag" information to distinguish whether the quantization parameter is determined explicitly or implicitly. Furthermore, an image encoding device and an image decoding device can be configured in which whether the quantization parameter is determined explicitly or implicitly is decoded in advance.

Implicitly determining a quantization parameter means that the image decoding device can select a predicted quantization parameter equivalent to that of the image encoding device, without providing identification information for selecting a predicted quantization parameter from selection candidates to the image decoding device from the image encoding device. Specifically, there are methods such as selecting a quantization parameter from selection candidates based on a predetermined priority order and determining the predicted quantization parameter, using a random value of the quantization parameter of the selection candidate as the predicted quantization parameter, weighting the quantization parameter of the selection candidate according to the distance from the current block, and using a random value of the weighted quantization parameter as the predicted quantization parameter.

Explicitly determining the quantization parameter means that by providing identification information for selecting a predicted quantization parameter from selection candidates from the image encoding device to the image decoding device, the image decoding device can select the same predicted quantization parameter as that of the image encoding device. Specifically, there are methods such as a method in which the image encoding device calculates index information for specifying the selection candidate, includes the calculated index information in stream information, and uses the quantization parameter of the selection candidate indicated in the index information as the predicted quantization parameter at the image decoding device; a method in which the index information is not included for blocks for which quantization is not performed; and the like.

FIG23 is a flowchart for describing another operation of the image encoding device, illustrating the slice encoding process. In step ST141, the image encoding device 10 determines whether there are coding units (CUs) to be encoded. If there are coding units in the slice to be encoded that have not yet been encoded, the image encoding device 10 proceeds to step ST142. If the encoding process is complete for all coding units in the slice, the image encoding device 10 ends the slice encoding process.

In step ST142, the image coding device 10 splits the coding unit CU. The image coding device 10 splits the coding unit CU as shown in FIG5 , determines the size of the coding unit with the smallest cost function value, and proceeds to step ST143. Furthermore, in order to determine the size of the coding unit with the smallest cost function value, the image coding device 10 includes, for example, "split_coding_unit_flag," a "coding tree syntax" equivalent to the split flag in FIG5 , in the stream information.

In step ST143, the image encoding device 10 determines whether the coding unit to be encoded involves inverse quantization. If the coding unit CU to be encoded is a block in a mode that does not require decoding by inverse quantization using a quantization parameter, such as a block in skip mode, I_PCM mode, or direct mode (CBP (Coding Block Pattern) = 0), the image encoding device 10 returns to step ST141, and in the case of a block in which inverse quantization is performed, proceeds to step ST144.

In step ST144, the image encoding device 10 determines whether the size of the coding unit CU is "log2MinQpUnitSize" or larger. If the size of the coding unit CU is "log2MinQpUnitSize" or larger, the image encoding device 10 proceeds to step ST145. In addition, if the size of the coding unit CU is not "log2MinQpUnitSize" or larger, the image encoding device 10 proceeds to step ST152.

In step ST145, the image encoding device 10 determines the quantization parameter QP for the coding unit CU to be encoded. The rate control unit 18 of the image encoding device 10 determines the quantization parameter according to the complexity of the image of the coding unit as described above, or determines the quantization parameter so that the cost function value is small, and proceeds to step ST146.

In step ST146, the image coding device 10 determines whether the discrimination information "qp_explicit_flag" is "1," which identifies whether the quantization parameter is to be predicted implicitly or explicitly. If the discrimination information "qp_explicit_flag" is "1" and the quantization parameter is to be predicted explicitly, the image coding device 10 proceeds to step ST147. Alternatively, if the discrimination information "qp_explicit_flag" is "0" and the quantization parameter is to be predicted implicitly, the image coding device 10 proceeds to step ST149. The image coding device 10 compares the cost function value when the discrimination information "qp_explicit_flag" is set to "1" with the cost function value when the discrimination information "qp_explicit_flag" is set to "0," for example. Based on the comparison result, the image coding device 10 sets the value of the discrimination information "qp_explicit_flag" to achieve higher encoding efficiency. In addition, in a case where the identification information “qp_explicit_flag” can be set by the user, the image encoding device 10 sets the identification information “qp_explicit_flag” according to a user instruction.

In step ST147, the image encoding device 10 generates identification information. The image encoding device 10 selects a candidate from the selection candidates so that the difference in quantization parameters for the coding unit to be encoded at the information generation unit 19 as described above is minimized, and uses it as the predicted quantization parameter. The image encoding device 10 uses, for example, the quantization parameters of encoded blocks spatially or temporally adjacent to the block to be encoded, the most recently updated quantization parameters, and the processing procedures set at the header block of the slice as selection candidates. The image encoding device 10 selects a candidate from the selection candidates whose difference in quantization parameters for the coding unit to be encoded is minimized, and uses it as the predicted quantization parameter. In addition, the information generation unit 19 uses the index (ref_qp_block_index) of the selected candidate as identification information for selecting the predicted quantization parameter from the selection candidates and proceeds to step ST148.

In step ST148, the image encoding device 10 includes the identification information in the stream information. The image encoding device 10 includes the identification information generated in step ST147, and proceeds to step ST150.

In step ST149, the image encoding device 10 implicitly determines the predicted quantization parameter dQP. In other words, the image encoding device 10 predicts the quantization parameter using the same method as the image decoding device 50. The method used to predict the quantization parameter includes, for example, decoding the predicted quantization parameter based on a priority order determined in advance. Furthermore, a random value of a plurality of candidate quantization parameters may be used as the predicted quantization parameter. Furthermore, a method may be used in which the candidate quantization parameters are weighted and selected based on the distance from the current block, and the random value of the weighted quantization parameter is used as the predicted quantization parameter. The image encoding device 10 calculates the predicted quantization parameter and proceeds to step ST150.

In step ST150, the image encoding device 10 generates difference information. The image encoding device 10 calculates the difference between the predicted quantization parameter indicated by the identification information generated in step ST147 and the quantization parameter decided in step ST145, or calculates the difference between the predicted quantization parameter decided in step ST149 and the quantization parameter decided in step ST145. The image encoding device 10 generates difference information indicating the calculated difference and proceeds to step ST151.

In step ST151, the image encoding device 10 includes the difference information and the identification information in the stream information. The image encoding device 10 includes the difference information generated in step ST151 and the identification information "qp_explicit_flag" used in step ST146 in the stream information. The image encoding device 10 includes the identification information in one of the sequence parameter set, picture parameter set, slice header, etc., and proceeds to step ST152.

In step ST152, the image encoding device 10 quantizes the coding unit CU. The image encoding device 10 quantizes the coding unit CU using the decided quantization parameter and returns to step ST141.

Fig. 24 is an example of operation in the case of implicitly predicting the quantization parameter, and Fig. 25 is an example of a flowchart in the case of explicitly predicting the quantization parameter. Note that the case of three selection candidates is shown for ease of description.

As shown in (A) of FIG24 , the quantization parameter of the block to be encoded in the frame to be encoded is, for example, "QP_0." Furthermore, three candidates are the quantization parameter "QP_A" of the coded block adjacent to the left, the quantization parameter "QP_B" of the adjacent coded block, and the quantization parameter "QP_LS" of the decoded coding unit.

In FIG25 , in step ST161, the image encoding device 10 determines whether the quantization parameters "QP_A" and "QP_B" can be referenced. If the coded block adjacent to the left and the coded block adjacent to the top are not blocks in a mode that does not require inverse quantization using quantization parameters for decoding, such as blocks in skip mode, I_PCM mode, or direct mode (CBP (Coded Block Pattern) = 0), the image encoding device 10 determines that reference can be made and proceeds to step ST162. Furthermore, if at least one of the quantization parameters "QP_A" and "QP_B" is in a mode that does not require inverse quantization, the process proceeds to step ST163.

In step ST162, the image encoding device 10 uses the average value of the quantization parameters "QP_A" and "QP_B" as the predicted quantization parameter dQP. In other words, as shown in FIG24(B), when the quantization parameters "QP_A" and "QP_B" can be used as a reference, the average value "(QP_A+QP_B+1)/2" of the quantization parameters "QP_A" and "QP_B" is used as the predicted quantization parameter dQP.

In step ST163, the image encoding device 10 determines whether the quantization parameter "QP_A" can be referenced. If the coded block adjacent to the left is not in a mode in which inverse quantization is not required, the image encoding device 10 determines that the block can be referenced and proceeds to step ST164. If the coded block adjacent to the left is in a mode in which inverse quantization is not required, the image encoding device 10 determines that the block cannot be referenced and proceeds to step ST165.

In step ST164, the image encoding device 10 uses the quantization parameter "QP_A" as the predicted quantization parameter dQP. In other words, when the quantization parameter "QP_A" can be referenced but the quantization parameter "QP_B" cannot be referenced, as shown in (C) of FIG. 24 , the quantization parameter "QP_A" is used as the predicted quantization parameter dQP. Note that in FIG. 24 and FIG. 26 described later, blocks in a mode that does not require inverse quantization, that is, blocks that cannot be referenced, are indicated by shading.

In step ST165, the image encoding device 10 determines whether the quantization parameter "QP_B" can be referenced. If the coded block adjacent to the upper side is not in a mode in which inverse quantization is not required, the image encoding device 10 determines that the block can be referenced and proceeds to step ST166. If the coded block adjacent to the upper side is in a mode in which inverse quantization is not required, the image encoding device 10 determines that the block cannot be referenced and proceeds to step ST167.

In step ST166, the image encoding device 10 uses the quantization parameter "QP_B" as the predicted quantization parameter dQP. In other words, when the quantization parameter "QP_B" can be referenced but the quantization parameter "QP_A" cannot be referenced, as shown in (D) of FIG. 24 , the quantization parameter "QP_B" is used as the predicted quantization parameter dQP.

In step ST167, the image encoding device 10 uses the quantization parameter "QP_LS" as the predicted quantization parameter dQP. As shown in (E) of FIG. 24 , when the coded block adjacent to the left and the coded block adjacent to the top are in a mode that does not require inverse quantization, the quantization parameter "QP_LS" is used as the predicted quantization parameter dQP.

FIG26 shows another operation example of implicitly predicting a quantization parameter. For example, as shown in FIG24 (E), when the coded block adjacent to the left and the coded block adjacent to the top are in a mode that does not require inverse quantization, a predetermined quantization parameter can be generated by increasing the number of selection candidates. For example, as shown in FIG26 (A), the quantization parameter "QP_C" of the coded block adjacent to the top right, the quantization parameter "QP_D" of the coded block adjacent to the top left, and the quantization parameter "QP_E" of the coded block adjacent to the bottom left are added to the selection candidates.

In the case where the quantization parameters "QP_C", "QP_D" and "QP_E" can be referenced as shown in (B) in Figure 26, the image encoding device 10 uses the average value "(QP_C+QP_D+1)/2" or median (Median) of the quantization parameters "QP_C" and "QP_D" as the predicted quantization parameter dQP.

In the case where quantization parameters “QP_C” and “QP_D” can be referenced as shown in (C) of FIG. 26 , the image encoding device 10 uses the average value “(QP_C+QP_D+1)/2” of the quantization parameters “QP_C” and “QP_D” as the predicted quantization parameter dQP.

In the case where the quantization parameters “QP_D” and “QP_E” can be referenced as shown in (D) of FIG. 26 , the image encoding device 10 uses the average value “(QP_D+QP_E+1)/2” of the quantization parameters “QP_D” and “QP_E” as the predicted quantization parameter dQP.

In the case where quantization parameters “QP_C” and “QP_E” can be referenced as shown in (E) of FIG. 26 , the image encoding device 10 uses the average value “(QP_C+QP_E+1)/2” of the quantization parameters “QP_C” and “QP_E” as the predicted quantization parameter dQP.

In the case where the quantization parameters "QP_C", "QP_D", and "QP_E" cannot be referenced as shown in (F) of FIG. 26 , the image encoding device 10 uses the quantization parameter "QP_LS" as the predicted quantization parameter dQP. Note that FIG. 27 shows a procedure for performing the operations of (B) to (D) of FIG. 24 and (B) to (F) of FIG. 26 .

Furthermore, in the case where the number of quantization parameters that can be referenced is one, as shown in (G) to (I) in FIG. 26 , the quantization parameter that can be referenced can be used as the predicted quantization parameter dQP.

Therefore, the image coding device 10 considers quantization parameters (e.g., those of previously encoded blocks spatially or temporally adjacent to the block to be coded) as selection candidates and selects a predicted quantization parameter from the selection candidates based on the set quantization parameter. Furthermore, the image coding device 10 generates identification information for selecting the predicted quantization parameter from the selection candidates. Furthermore, the image coding device 10 generates difference information indicating the difference between the predicted quantization parameter and the quantization parameter set for the block to be coded. The image coding device 10 includes the generated identification information and difference information in the stream information. By performing this process, the difference between the quantization parameter of the block to be coded and the predicted quantization parameter can be prevented from becoming excessively large. Consequently, the image coding device 10 can improve the coding efficiency of the quantization parameter.

Furthermore, when the quantization parameter is implicitly predicted, the image decoding device 50 can use the same predicted quantization parameter as the image encoding device 10 without including identification information for selecting the predicted quantization parameter from the selection candidates in the stream information. Furthermore, by including identification information in the stream information, it is possible to adaptively switch between explicit prediction of the predicted quantization parameter and implicit prediction of the predicted quantization parameter.

Fig. 28 is a flowchart for describing other operations of the image decoding device. In step ST127 in Fig. 22 , when it is determined that there is a coding unit to be decoded, the image decoding device 50 performs the processing starting from step ST171 and decodes the coding unit.

In step ST171, the image decoding device 50 extracts information. The image decoding device 50 extracts information from the stream information for decoding the coding unit. For example, the image decoding device 50 extracts information such as "coding tree syntax" that can determine the size of the coding unit, "log2_min_qp_unit_size_offset" that can determine the minimum size of the quantization parameter unit, and "qp_explicit_flag" identification information, and then proceeds to step ST172.

In step ST172, the image decoding device 50 splits the coding unit CU. The image decoding device 50 splits the coding unit CU based on "split_coding_unit_flag" and the like included in the stream information, and proceeds to step ST173.

In step ST173, the image decoding device 50 determines whether the coding unit CU to be decoded involves inverse quantization. If the coding unit CU to be decoded is in a mode in which inverse quantization is performed using a quantization parameter, the image decoding device 50 proceeds to step ST174, and if inverse quantization using a quantization parameter is unnecessary for a block, the decoding process ends.

In step ST174, the image decoding device 50 determines whether the size of the coding unit CU is "log2MinQpUnitSize" or larger. If the size of the coding unit CU is "log2MinQpUnitSize" or larger, the image decoding device 50 proceeds to step ST175. In addition, if the size of the coding unit CU is not "log2MinQpUnitSize" or larger, the image decoding device 50 proceeds to step ST180.

In step ST175, the image decoding device 50 determines whether the identification information "qp_explicit_flag" is "1". If the identification information "qp_explicit_flag" included in the stream information is "1" and the quantization parameter is to be explicitly predicted, the image decoding device 50 proceeds to step ST176. On the other hand, if the identification information "qp_explicit_flag" is "0" and the quantization parameter is to be implicitly predicted, the image decoding device 50 proceeds to step ST178.

In step ST176 , the image decoding device 50 extracts the index (ref_qp_block_index) from the stream information and proceeds to step ST177 .

In step ST177 , the image decoding device 50 determines the predicted quantization parameter dQP. The image decoding device 50 selects the same quantization parameter as the image encoding device 10 from the selection candidate quantization parameters based on the index (ref_qp_block_index), determines the selected quantization parameter as the predicted quantization parameter dQP, and proceeds to step ST179 .

In step ST178, the image decoding device 50 implicitly determines the predicted quantization parameter dQP. The image decoding device 50 predicts the quantization parameter using the same method as the image encoding device 10. The method used to predict the quantization parameter can, for example, be based on a previously determined order of priority. Alternatively, a random value of a candidate quantization parameter can be selected as the predicted quantization parameter. Alternatively, a method can be used in which the candidate quantization parameters are weighted according to the distance from the current block and the weighted random value of the quantization parameter is used as the predicted quantization parameter. The image decoding device 50 predicts the quantization parameter and proceeds to step ST179.

In step ST179, the image decoding device 50 calculates the quantization parameter QP of the current coding unit CU. The image decoding device 50 obtains the difference information "qb_qp_delta" from the stream information, adds the difference information to the predicted quantization parameter dQP, calculates the quantization parameter of the coding unit to be decoded, and proceeds to step ST180.

In step ST180, the image decoding device 50 inversely quantizes the coding unit. The image decoding device 50 inversely quantizes the coding unit by using the decoded quantization parameter.

Therefore, the image decoding device 50 can decode the image by using the same quantization parameter as that used by the image encoding device.

<7. Software Processing Examples>

The series of processing described above can be performed by configuring hardware, software, or a combination of the two. In the case of processing by software, a program in which the processing sequence is recorded is installed in a memory inside a computer built into dedicated hardware and executed. Alternatively, the program can be installed in a general-purpose computer with which various types of processing can be performed.

29 is a diagram illustrating a schematic configuration of a computer device that executes the above-described series of processes by means of a program. A CPU 801 of the computer device 80 executes various types of processes according to a program stored in a ROM 802 or recorded in a recording unit 808.

Programs executed by the CPU 801, data, and the like are stored as appropriate in the RAM 803. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804.

An input/output interface 805 is also connected to the CPU 801 via the bus 804. An input unit 806 such as a touch screen, keyboard, mouse, microphone, etc. and an output unit 807 composed of a display, etc. are also connected to the CPU 801. The CPU 801 performs various types of processing according to commands input by the input unit 806. Then, the CPU 801 outputs the results of the processing to the output unit 807.

The recording unit 808 connected to the input/output interface 805 is composed of, for example, a hard disk, and records programs executed by the CPU 801 and various types of data. The communication unit 809 communicates with external devices via a cable or wireless communication medium (such as a network such as the Internet or a local area network or digital broadcasting). In addition, the computer device 80 can acquire a program via the communication unit 809 and record it in the ROM 802 or the recording unit 808.

When a removable medium 85 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted on the drive 810, the removable medium is driven and the programs, data, etc. recorded therein are obtained. The obtained programs and data are transferred to the ROM 802 or the RAM 803 or the recording unit 808 as needed.

The CPU 801 reads out and executes a program for performing the series of processes described above, and performs encoding processing on an image signal recorded in the recording unit 808 or the removable medium 85 or an image signal supplied via the communication unit 809 , or performs decoding processing on stream information.

<8. Application of electronic equipment>

Furthermore, in the above description, the H.264/AVC format is used as the encoding format/decoding format, but the present technology is also applicable to image encoding devices/image decoding devices using encoding formats/decoding formats that perform other motion prediction/compensation processing.

In addition, the present technology can be applied to image encoding devices and image decoding devices used when receiving stream information such as MPEG, H.26x, etc. obtained by encoding processing via a network medium (such as satellite broadcasting, cable TV (television), the Internet, cellular phones, etc.), or image encoding devices and image decoding devices used when processing on storage media such as optical disks or magnetic disks and flash memories.

Next, description will be made regarding electronic devices to which the above-described image encoding device 10 and image decoding device 50 are applied.

30 exemplarily shows a schematic configuration of a television device to which the present technology is applied. The television device 90 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, and an external interface unit 909. In addition, the television device 90 includes a control unit 910, a user interface unit 911, and the like.

The tuner 902 performs demodulation by selecting a desired channel from the broadcast signal received by the antenna 901 , and outputs the obtained stream to the demultiplexer 903 .

The demultiplexer 903 extracts packets of video and audio of a program to be viewed from the stream and outputs the data of the extracted packets to the decoder 904. In addition, the demultiplexer 903 supplies data packets such as an EPG (Electronic Program Guide) to the control unit 910. Note that if scrambling has been performed, descrambling is performed at the demultiplexer or the like.

The decoder 904 performs a decoding process on the packet, and outputs the video data generated by being subjected to the decoding process to the video signal processing unit 905 and outputs the audio signal to the audio signal processing unit 907 .

The video signal processing unit 905 performs video processing on the video data based on noise reduction and user settings. Based on the processing and an application provided via the network, the video signal processing unit 905 generates video data and image data for displaying the program on the display unit 906. Furthermore, the video signal processing unit 905 generates video data for displaying, for example, a menu screen for selecting items, and superimposes it on the program video data. Based on the video data generated in this manner, the video signal processing unit 905 generates a drive signal and drives the display unit 906.

The display unit 906 drives a display device (for example, a liquid crystal display device) based on the driving signal from the video signal processing unit 905 so that the video of the program is displayed.

The audio signal processing unit 907 subjects the audio data to predetermined processing such as noise reduction, and performs audio output by performing D/A conversion processing and amplification processing on the audio data after the processing and supplying to the speaker 908 .

The external interface unit 909 is an interface to be connected to an external device or a network, and performs data transmission and reception of, for example, video data or audio data.

The user interface unit 911 is connected to the control unit 910. The user interface unit 911 is configured with an operation switch or a remote control signal receiver or the like, and supplies an operation signal to the control unit 910 according to a user operation.

The control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like. The memory stores programs to be executed by the CPU, various data necessary for the CPU to execute processing, EPG data, data obtained via a network, and the like. The programs stored in the memory are read out and executed by the CPU at a predetermined timing (for example, when the television device 90 is activated). The CPU controls each component so that the television device 90 operates by executing the programs according to user operations.

Note that, with the television apparatus 90 , the bus 912 is provided to connect the tuner 902 , the demultiplexer 903 , the video signal processing unit 905 , the audio signal processing unit 907 , the external interface unit 909 , and the control unit 910 .

With the television apparatus thus configured, the function of the image decoding device (image decoding method) of the present application is provided to the decoder 904. Therefore, even if processing is performed in the image encoding process on the broadcast station side to reduce the amount of encoding required for transmitting the quantization parameter, the television apparatus can correctly restore the quantization parameter and generate a decoded image.

FIG31 exemplarily shows a schematic configuration of a cellular phone to which the present technology is applied. The cellular phone 92 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording and reproduction unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933.

Furthermore, the antenna 921 is connected to the communication unit 922, and the speaker 924 and the microphone 925 are connected to the audio codec 923. Furthermore, an operation unit 932 is connected to the control unit 931.

The cellular phone 92 performs various operations such as transmission and reception of audio signals in various modes such as an audio call mode or a data communication mode, transmission and reception of electronic mail and image data, image capturing, data recording, and the like.

In audio call mode, the audio signal generated by the microphone 925 is converted into audio data and compressed with data by the audio codec 923, and then provided to the communication unit 922. The communication unit 922 demodulates the audio data and converts the audio data to generate a transmission signal. In addition, the communication unit 922 provides the transmission signal to the antenna 921 for transmission to a base station (not shown). In addition, the communication unit 922 amplifies, frequency converts, and demodulates the received signal received by the antenna 921, and provides the obtained audio data to the audio codec 923. The audio codec 923 decompresses the audio data, converts it into an analog audio signal, and outputs it to the speaker 924.

Furthermore, in data communication mode, when performing email transmission, the control unit 931 receives text data input by operating the operation unit 932 and displays the input text on the display unit 930. Furthermore, the control unit 931 generates email data based on user instructions at the operation unit 932 and provides it to the communication unit 922. The communication unit 922 performs modulation processing, frequency conversion processing, and the like on the email data, and transmits the resulting transmission signal via the antenna 921. Furthermore, the communication unit 922 amplifies, frequency converts, and demodulates the received signal received by the antenna 921, and recovers the email data. This email data is provided to the display unit 930 to display the contents of the email.

Note that the cellular phone 92 can store received e-mail data in a storage medium in the recording/reproducing unit 929. The storage medium is any readable/writable storage medium. For example, the storage medium is a semiconductor memory such as RAM or built-in flash memory, a removable medium such as a hard disk, a magnetic disk, an MO disk, an optical disk, a USB memory, a memory card, or the like.

In the case where image data is transmitted in the data communication mode, the image data generated at the camera unit 926 is supplied to the image processing unit 927. The image processing unit 927 performs encoding processing on the image data and generates stream information.

The demultiplexing unit 928 multiplexes the stream information generated by the image processing unit 927 and the audio data provided by the audio codec 923 according to a predetermined format and supplies the multiplexed data to the communication unit 922. The communication unit 922 performs demodulation processing, frequency conversion processing, and the like on the multiplexed data, and transmits the resulting transmission signal via the antenna 921. Furthermore, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like on the received signal received by the antenna 921, and restores the multiplexed data. This multiplexed data is supplied to the demultiplexing unit 928. The demultiplexing unit 928 demultiplexes the multiplexed data, supplies the stream information to the image processing unit 927, and supplies the audio data to the audio codec 923.

The image processing unit 927 decodes the encoded data and generates image data. The image data is provided to the display unit 930 to display the received image. The audio codec 923 converts the audio data into an analog audio signal and provides it to the speaker 924 to output the received audio.

In the cellular phone thus configured, the image processing unit 927 includes the functions of the present technology. Therefore, for example, when encoding and transmitting images, data can be reduced. Furthermore, when decoding received images, quantization parameters can be restored, and decoded images can be generated.

FIG32 exemplarily illustrates a schematic configuration of a recording and playback device to which the present technology is applied. The recording/playback device 94 records, for example, the audio and video data of a received broadcast program onto a recording medium, and provides the received data to the user according to the timing of a user instruction. Furthermore, an arrangement is provided such that the recording/playback device 94 can acquire, for example, audio and video data from another device for recording onto a recording medium. Furthermore, an arrangement is provided such that the recording/playback device 94 can decode the audio and video data recorded on the recording medium to output an image display and audio output on a monitor device.

The recording/reproducing device 94 has a tuner 941, an external interface unit 942, an encoder 943, an HDD (hard disk drive) unit 944, a disk drive 945, a selector 946, a decoder 947, an OSD (on screen display) unit 948, a control unit 949 and a user interface unit 950.

The tuner 941 selects a station of a desired channel from a broadcast signal received by an antenna (not shown), and outputs an encoded stream obtained by demodulating the received signal of the desired channel to the selector 946 .

The external interface unit 942 is configured with at least any one of an IEEE1394 interface, a network interface unit, a USB interface, a flash memory interface, etc. The external interface unit 942 is an interface connected to an external device, a network, a memory card, etc., and receives data such as video data and audio data for recording.

The encoder 943 performs encoding processing if the video data and the audio data supplied from the external interface unit 942 are not encoded in a predetermined format and outputs stream information to the selector 946 .

The HDD unit 944 records content data such as video or audio, various programs, other data, and the like in a built-in hard disk, and also reads out the content data from the hard disk at the time of reproduction.

The disk drive 945 records or reproduces signals on a mounted optical disk, such as a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD+R, DVD+RW, etc.) or a Blu-ray disk.

The selector 946 selects a stream from the tuner 941 or the encoder 943 when recording video and audio, and supplies it to either the HDD unit 944 or the disk drive 945. In addition, the selector 946 supplies the stream output by the HDD unit 944 or the disk drive 945 to the decoder 947 when reproducing video and audio.

The decoder 947 performs decoding processing on the stream. The decoder 947 supplies the generated video data by performing the decoding processing to the OSD unit 948. In addition, the decoder 947 outputs the generated audio data by performing the decoding processing.

The OSD unit 948 generates video data to display, for example, a menu screen for selecting an item, and superimposes it on the video data output by the decoder 947 , and outputs it.

The user interface unit 950 is connected to the control unit 949. The user interface unit 950 is configured with an operation switch, a remote control signal receiver, or the like and supplies an operation signal to the control unit 949 according to a user operation.

The control unit 949 is configured using a CPU or a memory. The memory stores programs executed by the CPU and various data necessary for the CPU to perform processing. The program stored in the memory is read out and executed by the CPU at a predetermined timing (such as when the recording/reproducing device 94 is activated). The CPU controls each component so that the recording/reproducing device 94 operates by executing the program according to user operations.

In the recording/reproducing apparatus thus configured, the functions of the present application are provided to the encoder 943. Therefore, for example, when encoding and recording an image, the amount of data can be reduced. Furthermore, when decoding the recorded image, the quantization parameter can be restored, and a decoded image can be generated.

33 exemplarily shows a schematic configuration of an imaging device to which the present technology is applied. The imaging device 96 images a subject to display an image of the subject on a display unit and to record it in a recording medium as image data.

The imaging device 96 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, an image data processing unit 964, a display unit 965, an external interface unit 966, a memory unit 967, a media drive 968, an OSD unit 969, and a control unit 970. In addition, a user interface unit 971 is connected to the control unit 970. In addition, the image data processing unit 964 is connected to the external interface unit 966, the memory unit 967, the media drive 968, the OSD unit 969, the control unit 970, and the like via a bus 972.

The optical block 961 is configured with a focusing lens, a film mechanism, etc. The optical block 961 forms an optical image of a subject on an imaging surface of the imaging unit 962. The imaging unit 962 is configured by using a CCD or CMOS image sensor, and an electric signal corresponding to the optical image is generated by photoelectric conversion and supplied to the camera signal processing unit 963.

The camera signal processing unit 963 performs various camera signal processing on the electrical signal provided by the imaging unit 962 , such as KNEE correction, Gamma correction, color correction, etc. The camera signal processing unit 963 provides the image data processed by the camera to the image data processing unit 964 .

The image data processing unit 964 performs encoding processing on the image data supplied by the camera signal processing unit 963. The image data processing unit 964 supplies the stream information generated by the encoding processing to the external interface unit 966 and the media drive 968. Furthermore, the image data processing unit 964 performs decoding processing on the stream information supplied by the external interface unit 966 and the media drive 968. The image data processing unit 964 supplies the generated image data by performing the decoding processing to the display unit 965. Furthermore, the image data processing unit 964 performs processing to supply the image data supplied by the camera signal processing unit 963 to the display unit 965, and performs processing to superimpose the data for display acquired from the OSD unit 969 on the image data and supplies it to the display unit 965.

The OSD unit 969 generates data for display, such as a menu screen or an icon formed of a symbol, text, or shape, and outputs it to the image data processing unit 964 .

For example, the external interface unit 966 is configured with USB input and output terminals and, in the case of printing an image, is connected to a printer. Furthermore, a drive is connected to the external interface unit 966 as needed, removable media such as a magnetic disk or optical disk is installed as appropriate, and programs read from the removable media are installed as needed. Furthermore, the external interface unit 966 has a network interface for connecting to a predetermined network (such as a LAN or the Internet). For example, based on instructions from the user interface unit 971, the control unit 970 reads stream information from the memory unit 967 so that it can be provided to other connected devices from the external interface unit 966 via the network. Furthermore, the control unit 970 obtains stream information and image data provided by other devices via the network through the external interface unit 966 and provides them to the image data processing unit 964.

For example, as for the recording medium driven by the media drive 968, any removable medium that is readable and writable, such as a magnetic disk, an MO disk, an optical disk, and a semiconductor memory, can be used. In addition, regarding the recording medium, the type of removable medium is also optional, and it can be a magnetic tape device, a magnetic disk, or a memory card. Of course, this can also be a contactless IC card or the like.

Furthermore, an arrangement is made in which the media drive 968 and the recording medium are combined and configured with, for example, a non-portable storage medium such as a built-in type hard disk drive or an SSD (Solid State Drive) or the like.

The control unit 970 is configured using a CPU memory. The memory stores programs to be stored by the CPU and various types of data necessary for the CPU to perform processing. The program stored in the memory is read out and executed by the CPU at a predetermined timing (such as when the imaging device 96 is activated). The CPU controls each component so that the operation of the imaging device 96 by executing the program corresponds to the user's operation.

With this configuration, the image data processing unit 964 is equipped with the functions of the present invention. Therefore, when encoding and recording the imaged image in the memory unit 967 or a recording medium, the amount of data to be recorded can be reduced. Furthermore, when decoding the recorded image, the quantization parameter can be restored, and a decoded image can be generated.

Furthermore, the present technology is not to be construed as being limited to the above-described embodiments. The embodiments are disclosed illustratively, and it is clearly understood that those skilled in the art can make variations and substitutions to the embodiments without departing from the essence of the present technology. In other words, the claims should be considered to determine the essence of the present technology.

Furthermore, the image decoding device and the image encoding device according to the present technology can assume the following configurations.

(1) An image decoding device comprising:

an information acquisition unit configured to take quantization parameters of decoded blocks spatially or temporally adjacent to the block to be decoded as selection candidates, and extract difference information indicating a difference with respect to predicted quantization parameters selected from the selection candidates from the stream information; and

The quantization parameter calculation unit is configured to calculate the quantization parameter of the block to be decoded according to the predicted quantization parameter and the difference information.

(2) The image decoding device according to (1), wherein the quantization parameter calculation unit sets the quantization parameters in the order indicated by the identification information included in the stream information as the predicted quantization parameters when adjacent decoded blocks are in a predetermined order.

(3) The image decoding device according to (1), wherein the quantization parameter calculation unit determines the selection candidates in an order set in advance, and sets the predicted quantization parameter based on the determination result.

(4) An image decoding device according to (1), wherein the quantization parameter calculation unit selects one or the other of the following processes based on the determination information included in the stream information: a process of setting the quantization parameters in the order indicated by the identification information included in the stream information as predicted quantization parameters, and a process of determining the selection candidates in the order set in advance and setting the predicted quantization parameters based on the determination result.

(5) An image decoding device according to any one of (1) to (4), wherein the quantization parameter calculation unit obtains selection candidates by excluding at least a block in which the quantization parameter is redundant or a block in which inverse quantization using the quantization parameter is not performed from adjacent decoded blocks.

(6) The image decoding device according to any one of (1) to (5), wherein, when there is no selection candidate, the quantization parameter calculation unit uses the quantization parameter of the initial value in the slice as the predicted quantization parameter.

(7) The image decoding device according to any one of (1) to (6), wherein the quantization parameter calculation unit includes a most recently updated quantization parameter in the selection candidates.

(8) The image decoding device according to any one of (1) to (7), wherein the quantization parameter calculation unit calculates the quantization parameter of the block to be decoded by adding the difference indicated by the difference information to the predicted quantization parameter.

(9) An image decoding method comprising:

A process of taking quantization parameters of decoded blocks spatially or temporally adjacent to a block to be decoded as selection candidates and extracting difference information indicating a difference between predicted quantization parameters selected from the selection candidates from stream information; and

The process of calculating the quantization parameter of the block to be decoded according to the predicted quantization parameter and the difference information.

(10) An image encoding device comprising:

a control unit configured to set a quantization parameter for a block to be encoded;

an information generating unit configured to use quantization parameters of encoded blocks spatially or temporally adjacent to the block to be encoded as selection candidates, select a predicted quantization parameter from the selection candidates according to the set quantization parameter, and generate difference information indicating a difference between the predicted quantization parameter and the set quantization parameter; and

The encoding unit is configured to include the difference information in stream information generated by performing an encoding process on a block to be encoded using the set quantization parameter.

(11) The image encoding device according to (10), wherein the information generating unit selects a quantization parameter with a minimum difference with respect to the set quantization parameter as the predicted quantization parameter.

(12) The image encoding device according to (11), wherein the information generating unit generates identification information indicating the order of blocks corresponding to the selected quantization parameter when adjacent encoded blocks are in a predetermined order;

And wherein the encoding unit includes the identification information in the stream information.

(13) The image encoding device according to (12), wherein the information generation unit adopts the following arrangement order: wherein priority is given to one block among the encoded block adjacent to the left, the encoded block adjacent to the top, and the encoded block adjacent in time.

(14) The image encoding device according to any one of (12) or (13), wherein the information generating unit is capable of switching the arrangement order of adjacent encoded blocks.

(15) The image encoding device according to (10), wherein the information generating unit determines the selection candidates in an order set in advance, and selects the predicted quantization parameter based on a determination result.

(16) The image encoding device according to (10), wherein the information generating unit is capable of selecting between the following processes and generating determination information indicating the selected process: a process of selecting a quantization parameter having a minimum difference with respect to the set quantization parameter as the predicted quantization parameter; and a process of determining selection candidates in an order set in advance and selecting the predicted quantization parameter based on the determination result.

And wherein the encoding unit includes the determination information in the stream information.

(17) An image encoding device according to any one of (10) to (16), wherein the information generation unit obtains selection candidates by excluding at least a block in which a quantization parameter is redundant or a block in which quantization using the quantization parameter is not performed from adjacent encoded blocks.

(18) An image encoding device according to any one of (10) to (17), wherein, when there is no selection candidate, the information generation unit generates difference information indicating the difference between the quantization parameter of the initial value in the slice and the set quantization parameter.

(19) The image encoding device according to any one of (10) to (18), wherein the information generating unit includes a most recently updated quantization parameter in the selection candidates.

(20) An image encoding method comprising:

A process for setting quantization parameters for a block to be encoded;

A process of selecting quantization parameters of coded blocks spatially or temporally adjacent to the block to be coded as selection candidates, selecting a predicted quantization parameter from the selection candidates according to the set quantization parameter, and generating difference information indicating a difference between the predicted quantization parameter and the set quantization parameter; and

A process of including the difference information in stream information generated by performing an encoding process on the block to be encoded using the set quantization parameter.

Industrial Applicability

According to the present technology, an image decoding device, an image encoding device, and a method thereof use quantization parameters of previously encoded blocks spatially or temporally adjacent to a block to be encoded as selection candidates, and select a predicted quantization parameter from among the selection candidates based on the set quantization parameter. Difference information indicating the difference between the predicted quantization parameter and the quantization parameter set for the block to be encoded is generated. This prevents the difference in quantization parameters from becoming excessively large, and improves quantization parameter encoding efficiency. Furthermore, when decoding stream information including difference information, a predicted quantization parameter is selected from the quantization parameters of previously decoded blocks spatially or temporally adjacent to the block to be decoded, and the quantization parameter for the block to be decoded is calculated based on the predicted quantization parameter and the difference information. Therefore, even if stream information is generated with improved quantization parameter encoding efficiency, when decoding the stream information, the quantization parameter can be restored based on the predicted quantization parameter and the difference information, allowing for accurate decoding. Therefore, the present technology is suitable for devices that transmit/receive stream information obtained by encoding in block units via network media (such as satellite broadcasting, cable TV, the Internet, cellular phones, etc.) and devices that implement the present technology on storage media such as optical disks, disk flash memories, etc.

Reference Signs List

10 Image Coding Device

11 A/D conversion unit

12, 57 screen reset buffer

13 Subtraction Unit

14 Orthogonal Transformation Unit

15 Quantization Unit

16 lossless coding units

17, 51 Storage Buffer

18 Rate Control Unit

19 Information Generation Unit

21, 53 Inverse quantization unit

22, 54 Inverse orthogonal transform unit

23, 55 Addition unit

24, 56 Deblocking filter

26, 61 frame memory

31, 71 intra prediction unit

32 Motion prediction/compensation units

33 Prediction image/optimal mode selection unit

50 Image decoding device

52 Lossless Decoding Unit

58 D/A conversion units

59 Quantization parameter calculation unit

62, 73 selectors

72 Motion Compensation Unit

80 Computer devices

90 TV equipment

92 cellular phones

94 Recording/reproducing device

96 Imaging Equipment

191 Quantization parameter memory unit

192 Difference calculation unit

591 computing units

592 quantization parameter memory cells

Claims (19)

1. An image decoding method, comprising: If the quantization parameters of the block adjacent to the left of the current block and the quantization parameters of the block adjacent above the current block are available, then set the predicted quantization parameter obtained from the average of the quantization parameters of the block adjacent to the left and the block adjacent above. If the quantization parameters of the block adjacent to the left and the block adjacent to the top cannot be used, then set the predicted quantization parameters obtained from the initial values of the slice's quantization parameters; and Inverse quantization is performed based on the quantization parameters of the current block obtained from the predicted quantization parameters and the difference between the quantization parameters of the current block and the predicted quantization parameters, which are obtained from the bit stream. 2. The image decoding method according to claim 1, wherein when the block adjacent to the left and the block adjacent to the top cannot be used, the quantization parameters of the preceding block are obtained to set the prediction quantization parameters. 3. The image decoding method according to claim 1, wherein the block is an encoding unit, and the encoding unit has a hierarchical structure defined by the hierarchical depth. 4. The image decoding method according to claim 3, wherein the inverse quantization is performed by a transform unit included in the encoding unit based on the quantization parameters of the current block. 5. An image processing apparatus, comprising: The circuit is configured to: If the quantization parameters of the block adjacent to the left of the current block and the quantization parameters of the block adjacent above the current block are available, then set the predicted quantization parameter obtained from the average of the quantization parameters of the block adjacent to the left and the block adjacent above. If the quantization parameters of the block adjacent to the left and the block adjacent to the top cannot be used, then set the predicted quantization parameters obtained from the initial values of the slice's quantization parameters; and Inverse quantization is performed based on the quantization parameters of the current block obtained from the predicted quantization parameters and the difference between the quantization parameters of the current block and the predicted quantization parameters, which are obtained from the bit stream. 6. The image processing apparatus according to claim 5, wherein when the block adjacent to the left and the block adjacent to the top cannot be used, the quantization parameters of the preceding block are obtained to set the predictive quantization parameters. 7. The image processing apparatus of claim 5, wherein the block is a coding unit, and the coding unit has a hierarchical structure defined by a hierarchical depth. 8. The image processing apparatus of claim 7, wherein the circuit is further configured to: perform inverse quantization based on the quantization parameters of the current block by means of a transform unit included in the encoding unit. 9. The image processing apparatus according to claim 5, wherein the circuit includes a video signal processing circuit. The circuit is further configured to decode the current block according to the quantization parameters of the current block, and to acquire video data. The video signal processing circuit is configured to perform video processing on the video data. 10. The image processing apparatus according to claim 9, wherein the video processing performed on the video data includes noise reduction of the video data. 11. The image processing apparatus of claim 10, wherein the circuit further comprises a display circuit configured to drive a display device to display the video data. 12. The image processing apparatus of claim 9, wherein the circuit further comprises a display circuit configured to drive a display device to display the video data. 13. The image processing apparatus of claim 5, wherein the circuitry includes a demultiplexer circuitry configured to extract encoded video data and encoded audio data from the bitstream, wherein the encoded video data includes bitstream information about the difference between a quantization parameter of the current block and a predicted quantization parameter. 14. The image processing apparatus of claim 13, wherein the circuit further comprises a tuner circuit configured to demodulate by selecting a desired channel from a broadcast signal received at the antenna, and to obtain a bitstream comprising the encoded video data and the encoded audio data. 15. The image processing apparatus of claim 13, wherein the circuit further comprises an audio signal processing circuit configured to perform predetermined processing on audio data obtained by decoding the encoded audio data. 16. The image processing apparatus of claim 15, wherein the predetermined processing performed on the audio data by the audio signal processing circuit includes noise reduction of the audio data. 17. The image processing apparatus of claim 16, wherein the circuit further comprises an audio codec circuit configured to convert the audio data into an analog audio signal. 18. The image processing apparatus of claim 15, wherein the circuit further comprises an audio codec circuit configured to convert the audio data into an analog audio signal. 19. An image processing apparatus, comprising: An apparatus for setting a predicted quantization parameter obtained from the average of the quantization parameters of the block adjacent to the left and the block adjacent to the top if the quantization parameters of the block adjacent to the left and the block adjacent to the top are available. A means for setting predicted quantization parameters derived from the initial values of the quantization parameters of a slice if the quantization parameters of the block adjacent to the left and the block adjacent to the top cannot be used; and A means for performing inverse quantization based on the quantization parameters of the current block obtained from the predicted quantization parameters and the difference between the quantization parameters of the current block and the predicted quantization parameters, obtained from the bit stream.