HK1250109B - Image processing device and image processing method - Google Patents

Description

Image processing device and image processing method

This application is a divisional application of the invention patent application with application number 201280007291.6, PCT international application date January 18, 2012, and invention name “Image processing device and image processing method”.

Technical Field

The present invention relates to an image processing device and an image processing method.

Background Art

In H.264/AVC (a standard specification for image coding schemes), when quantizing image data in the High Profile or higher profile, a different quantization step size can be used for each component of the orthogonal transform coefficients. The quantization step size (or quantization scale) for each component of the orthogonal transform coefficients can be set based on a quantization matrix (also called a scaling list) defined with the same size as the unit of the orthogonal transform and a standard step size value.

Figure 38 shows the four categories of default quantization matrices predefined in H.264/AVC. Matrix SL1 is the default 4×4 quantization matrix used for intra prediction mode. Matrix SL2 is the default 4×4 quantization matrix used for inter prediction mode. Matrix SL3 is the default 8×8 quantization matrix used for intra prediction mode. Matrix SL4 is the default 8×8 quantization matrix used for inter prediction mode. Users can also define their own quantization matrices that are different from the default matrices shown in Figure 38 in the sequence parameter set or picture parameter set. Note that if a quantization matrix is not specified, a flat quantization matrix with equal quantization step sizes for all components can be used.

High Efficiency Video Coding (HEVC), whose standardization is being promoted as the next-generation image coding scheme to succeed H.264/AVC, introduces the concept of coding units (CUs), which correspond to conventional macroblocks (see Non-Patent Document 1 below). The range of coding unit sizes is specified in the sequence parameter set as a pair of power-of-two values called the largest coding unit (LCU) and the smallest coding unit (SCU). Furthermore, the SPLIT_FLAG flag is used to specify a specific coding unit size within the range specified by the LCU and SCU.

In HEVC, a coding unit can be divided into one or more orthogonal transform units, or in other words, one or more transform units (TUs). Any of 4×4, 8×8, 16×16, and 32×32 transform unit sizes can be used. Therefore, a quantization matrix can also be specified for each of these candidate transform unit sizes. The following non-patent document 2 proposes specifying multiple quantization matrix candidates for one transform unit size in a picture, and adaptively selecting a quantization matrix for each block from the perspective of rate-distortion (RD) optimization.

Reference List

Non-patent literature

Non-Patent Document 1: JCTVC-B205, "Test Model under Consideration", Second Meeting of the Joint Video Coding Group of ITU-T SG16WP3 and ISO/IEC JTC1/SC29/WG11 (JCT-VC): Geneva, Switzerland, July 21-28, 2010

Non-Patent Document 2: VCEG-AD06, "Adaptive Quantization Matrix Selection on KTA Software", ITU - Telecommunication Standardization Sector Study Group 16 Question 6 Video Coding Experts Group (VCEG) 30th Meeting: Hangzhou, China, October 23-24, 2006.

Summary of the Invention

Technical issues

However, depending on the characteristics of each image included in the video, the quantization matrix adapted for quantization and inverse quantization is different. For this reason, if an attempt is made to encode a video whose image characteristics change from time to time with an optimized quantization matrix, the frequency of quantization matrix updates will increase. In H.264/AVC, quantization matrices are defined in picture parameter sets (PPS) or sequence parameter sets (SPS). Therefore, if the frequency of quantization matrix updates increases, the proportion of the coded stream occupied by the SPS or PPS will increase. This means that due to the increased overhead, coding efficiency will decrease. For HEVC, where the quantization matrix size is further increased and where several different quantization matrices can be defined for each picture, there is a risk that this decrease in coding efficiency with the update of the quantization matrix may become more significant.

Therefore, it is necessary to provide a mechanism that can alleviate the decrease in encoding efficiency accompanying the update of the quantization matrix.

Technical solutions to the problem

According to an embodiment of the present invention, there is provided an image processing device, comprising: an acquisition unit configured to acquire quantization matrix parameters from a coding stream, the coding stream being a coding stream in which the quantization matrix parameters defining the quantization matrix are set in a parameter set different from a sequence parameter set and a picture parameter set; a setting unit configured to set a quantization matrix used when inverse quantizing data decoded from the coding stream based on the quantization matrix parameters acquired by the acquisition unit; and an inverse quantization unit configured to inverse quantize the data decoded from the coding stream using the quantization matrix set by the setting unit.

The image processing device can typically be implemented as an image decoding device for decoding an image.

According to an embodiment of the present invention, there is provided an image processing method, comprising: acquiring quantization matrix parameters from a coding stream, wherein the coding stream is a coding stream in which the quantization matrix parameters defining the quantization matrix are set in a parameter set different from a sequence parameter set and a picture parameter set; setting a quantization matrix used when inverse quantizing data decoded from the coding stream based on the acquired quantization matrix parameters; and inverse quantizing the data decoded from the coding stream using the set quantization matrix.

According to an embodiment of the present invention, an image processing device is provided, comprising: a quantization unit configured to quantize data using a quantization matrix; a setting unit configured to set quantization matrix parameters, which define the quantization matrix used when the quantization unit quantizes the data; and an encoding unit configured to encode the quantization matrix parameters set by the setting unit in a parameter set different from a sequence parameter set and a picture parameter set.

The image processing apparatus may typically be implemented as an image encoding apparatus for encoding an image.

According to an embodiment of the present invention, there is provided an image processing method, comprising: quantizing data using a quantization matrix; setting quantization matrix parameters, which define the quantization matrix used when quantizing the data; and encoding the set quantization matrix parameters in a parameter set different from a sequence parameter set and a picture parameter set.

Advantageous Effects of the Invention

According to the image processing apparatus and the image processing method of the present invention, it is possible to alleviate the decrease in encoding efficiency accompanying the updating of the quantization matrix.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a block diagram showing an example structure of an image encoding device according to an embodiment;

FIG2 is a block diagram showing an example of a specific configuration of a grammar processing section shown in FIG1 ;

FIG3 is an exemplary diagram showing example parameters included in a quantization matrix parameter set in an embodiment;

FIG4 is an explanatory diagram showing example parameters included in a slice header in the embodiment;

5 is a flow chart illustrating an example process of parameter set insertion processing according to an embodiment;

FIG6 is an explanatory diagram for explaining a difference in a stream structure between the technology according to the embodiment and the related art;

7 is a block diagram showing an example structure of an image decoding device according to an embodiment;

FIG8 is a block diagram showing an example of a specific configuration of a grammar processing section shown in FIG7;

9 is a flowchart illustrating an example flow of a quantization matrix generation process according to an embodiment;

10 is a flowchart showing an example of a specific flow of processing in the copy mode according to the embodiment;

11 is a flowchart showing an example of a specific flow of processing in the axis designation mode according to the embodiment;

12 is a flowchart illustrating an example flow of a process for setting a quantization matrix for a segment according to an embodiment;

13 is a first explanatory diagram showing a first example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

14 is a second explanatory diagram showing a first example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

15 is a third explanatory diagram showing a first example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

16 is a first explanatory diagram showing a second example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

17 is a second explanatory diagram showing a second example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

18 is a third explanatory diagram showing a second example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

19 is a fourth explanatory diagram showing a second example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

20 is a fifth explanatory diagram showing a second example of an illustrative pseudo code expressing the syntax of a quantization matrix parameter set;

FIG21 is an explanatory diagram showing an example of a quantization scale setting area defined for quantizing a quantization matrix;

FIG22 is an explanatory diagram showing an example of quantization scales set in the respective quantization scale setting areas shown in FIG21;

FIG23 is an explanatory diagram for explaining 11 types of VLC tables prepared in LCEC;

FIG24 is an explanatory diagram showing an example of an encoded stream constructed according to the first technique using APS;

FIG25 is an explanatory diagram showing an example of an APS syntax defined according to the first technique using APS;

FIG26 is an explanatory diagram showing an example of a slice header syntax defined according to the first technique using APS;

27 is an explanatory diagram showing an example of an APS syntax defined according to an example modification of the first technique using APS;

FIG28 is an explanatory diagram showing an example of an encoded stream constructed according to the second technique using APS;

FIG29 is an explanatory diagram showing an example of an encoded stream constructed according to the third technique using APS;

FIG30 is an explanatory diagram showing an example of an APS syntax defined according to the third technique using APS;

FIG31 is an explanatory diagram showing an example of a slice header syntax defined according to the third technique using APS;

FIG32 is a table listing parameter characteristics for each of several typical encoding tools;

FIG33 is an explanatory diagram for explaining an example of an encoded stream constructed according to an example modification of the third technique using APS;

FIG34 is a block diagram showing an example of a schematic configuration of a television;

FIG35 is a block diagram showing an example of a schematic configuration of a mobile phone;

FIG36 is a block diagram showing an example of a schematic configuration of a recording and playback device;

FIG37 is a block diagram showing an example of a schematic configuration of an imaging device; and

FIG. 38 is an explanatory diagram showing a default quantization matrix predefined in H.264/AVC.

DETAILED DESCRIPTION

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted by the same reference numerals, and repeated explanation is omitted.

Additionally, the description will be made in the following order.

1. Example Configuration of Image Encoding Device According to Embodiment

1-1. Example overall configuration

1-2. Example Configuration of Syntax Processing Unit

1-3. Example parameter structure

2. Processing Flow During Encoding According to the Embodiment

3. Example Configuration of Image Decoding Device According to Embodiment

3-1. Example overall configuration

3-2. Example Configuration of Syntax Processing Unit

4. Processing Flow During Decoding According to an Embodiment

4-1. Generating Quantization Matrix

4-2. Setting the quantization matrix for the segment

5. Syntax Examples

5-1. First Example

5-2. Second Example

6. Various example configurations of parameter sets

6-1. First Technology

6-2. Example Modification of First Technology

6-3. Second technology

6-4. Third Technology

6-5. Example Modification of the Third Technology

7. Application

8. Conclusion

<1. Example Configuration of Image Encoding Device According to Embodiment>

This section describes an example configuration of an image encoding device according to an embodiment.

[1-1. Example overall configuration]

FIG1 is a block diagram illustrating an example structure of an image encoding device according to an embodiment. Referring to FIG1 , the image encoding device 10 includes an analog-to-digital (A/D) conversion section 11, a recording buffer 12, a syntax processing section 13, a subtraction section 14, an orthogonal transform section 15, a quantization section 16, a lossless encoding section 17, an accumulation buffer 18, a rate control section 19, an inverse quantization section 21, an inverse orthogonal transform section 22, an addition section 23, a deblocking filter 24, a frame memory 25, a selector 26, an intra-frame prediction section 30, a motion estimation section 40, and a mode selection section 50.

The A/D conversion section 11 converts an image signal input in an analog format into image data in a digital format, and outputs a sequence of digital image data to the recording buffer 12 .

The recording buffer 12 records images included in the sequence of image data input from the A/D conversion section 11. After recording images in a group of pictures (GOP) structure according to encoding processing, the recording buffer 12 outputs the recorded image data to the syntax processing section 13.

The image data output from the recording buffer 12 to the syntax processing unit 13 is mapped into a bitstream in units of so-called Network Abstraction Layer (NAL) units. The image data stream consists of one or more sequences. The leading picture in a sequence is called an Instantaneous Decoding Refresh (IDR) picture. Each sequence consists of one or more pictures, and each picture also consists of one or more slices. In H.264/AVC and HEVC, these slices are the basic units of video encoding and decoding. The data for each slice is identified as a Video Coding Layer (VCL) NAL unit.

The syntax processing unit 13 sequentially identifies NAL units in the image data stream input from the recording buffer 12 and inserts non-VCL NAL units storing header information into the stream. The non-VCL NAL units inserted into the stream by the syntax processing unit 13 include a sequence parameter set (SPS) and a picture parameter set (PPS). Furthermore, in this embodiment, the syntax processing unit 13 inserts a quantization matrix parameter set (QMPS), a non-VCL NAL unit other than the SPS and PPS, into the stream. The syntax processing unit 13 also adds a slice header (SH) to the beginning of a slice. The syntax processing unit 13 then outputs the image data stream, including VCL NAL units and non-VCL NAL units, to the subtraction unit 14, the intra prediction unit 30, and the motion estimation unit 40. The specific configuration of the syntax processing unit 13 will be further described below.

The subtraction unit 14 is supplied with the image data output from the syntax processing unit 13 and the predicted image data selected by the mode selection unit 50 described below. The subtraction unit 14 calculates prediction error data, which is the difference between the image data output from the syntax processing unit 13 and the predicted image data input from the mode selection unit 50, and outputs the calculated prediction error data to the orthogonal transformation unit 15.

The orthogonal transform section 15 performs an orthogonal transform on the prediction error data input from the subtraction section 13. The orthogonal transform performed by the orthogonal transform section 15 may be, for example, a discrete cosine transform (DCT) or a Karhunen-Loeve transform. The orthogonal transform section 15 outputs the transform coefficient data obtained by the orthogonal transform process to the quantization section 16.

The quantization unit 16 quantizes the transform coefficient data input from the orthogonal transform unit 15 using a quantization matrix and outputs the quantized transform coefficient data (hereinafter referred to as quantized data) to the lossless encoding unit 17 and the inverse quantization unit 21. The bit rate of the quantized data is controlled based on a rate control signal from the rate control unit 19. The quantization matrix used by the quantization unit 16 is defined in a quantization matrix parameter set and can be specified in the slice header for each slice. In this case, if no quantization matrix is specified, a flat quantization matrix with an equal quantization step size for all components is used.

The lossless encoding section 17 generates an encoded stream by performing a lossless encoding process on the quantized data input from the quantization section 16. The lossless encoding performed by the lossless encoding section 17 may be, for example, variable length encoding or arithmetic coding. Furthermore, the lossless encoding section 17 multiplexes information on intra-frame prediction or information on inter-frame prediction input from the mode selection section 50 into the header of the encoded stream. The lossless encoding section 17 then outputs the encoded stream generated in this manner to the accumulation buffer 18.

The accumulation buffer 18 temporarily buffers the encoded stream input from the lossless encoding section 17 using a storage medium such as a semiconductor memory. The accumulation buffer 18 then outputs the encoded stream thus buffered to a transmission section (not shown) (such as a communication interface with a peripheral device or a connection interface) at a rate according to the bandwidth of the transmission channel.

The rate control section 19 monitors the free space in the accumulation buffer 18. The rate control section 19 then generates a rate control signal based on the free space in the accumulation buffer 18 and outputs the generated rate control signal to the quantization section 16. For example, when there is not much free space in the accumulation buffer 18, the rate control section 19 generates a rate control signal for reducing the bit rate of the quantized data. Alternatively, when there is sufficient free space in the accumulation buffer 18, the rate control section 19 generates a rate control signal for increasing the bit rate of the quantized data.

The inverse quantization section 21 performs an inverse quantization process using a quantization matrix on the quantized data input from the quantization section 16. The inverse quantization section 21 then outputs transform coefficient data obtained by the inverse quantization process to the inverse orthogonal transform section 22.

The inverse orthogonal transform section 22 performs an inverse orthogonal transform process on the transform coefficient data input from the inverse quantization section 21 , thereby restoring the prediction error data.

The addition unit 23 generates decoded image data by adding the restored prediction error data input from the inverse orthogonal transform unit 22 to the predicted image data input from the mode selection unit 50. The addition unit 23 then outputs the decoded image data generated in this way to the deblocking filter 24 and the frame memory 25.

The deblocking filter 24 applies filtering to reduce blocking artifacts generated during image encoding. The deblocking filter 24 removes blocking artifacts by filtering the decoded image data input from the adding unit 23 and outputs the filtered decoded image data to the frame memory 25 .

The frame memory 25 stores the decoded image data input from the addition section 23 and the decoded image data after filtering input from the deblocking filter 24 using a storage medium.

The selector 26 reads unfiltered decoded image data to be used for intra prediction from the frame memory 25, and supplies the read decoded image data as reference image data to the intra prediction section 30. In addition, the selector 26 reads filtered decoded image data to be used for inter prediction from the frame memory 25, and supplies the read decoded image data to the motion estimation section 40 as reference image data.

The intra-frame prediction section 30 performs intra-frame prediction processing in each intra-frame prediction mode based on the image data to be encoded input from the syntax processing section 13 and the decoded image data provided by the selector 26. For example, the intra-frame prediction section 30 evaluates the prediction results of each intra-frame prediction mode using a predetermined cost function. The intra-frame prediction section 30 then selects the intra-frame prediction mode that produces the smallest cost function value, that is, the intra-frame prediction mode that produces the highest compression ratio, as the optimal intra-frame prediction mode. In addition, the intra-frame prediction section 30 outputs information about intra-frame prediction, such as prediction mode information indicating the optimal intra-frame prediction mode, predicted image data, and cost function value, to the mode selection section 50.

The motion estimation unit 40 performs inter-frame prediction processing (prediction processing between frames) based on the image data to be encoded input from the syntax processing unit 13 and the decoded image data provided by the selector 26. For example, the motion estimation unit 40 evaluates the prediction results of each prediction mode using a predetermined cost function. The motion estimation unit 40 then selects the prediction mode that produces the smallest cost function value, that is, the prediction mode that produces the highest compression ratio, as the optimal prediction mode. The motion estimation unit 40 generates predicted image data based on the optimal prediction mode. The motion estimation unit 40 outputs information regarding inter-frame prediction, such as prediction mode information indicating the selected optimal inter-frame prediction mode, predicted image data, and cost function value, to the mode selection unit 50.

The mode selection section 50 compares the cost function value for intra-frame prediction input from the intra-frame prediction section 30 with the cost function value for inter-frame prediction input from the motion estimation section 40. The mode selection section 50 then selects the prediction method with the smaller cost function value between intra-frame prediction and inter-frame prediction. If intra-frame prediction is selected, the mode selection section 50 outputs information about the intra-frame prediction to the lossless encoding section 17 and also outputs predicted image data to the subtraction section 14 and the addition section 23. If inter-frame prediction is selected, the mode selection section 50 outputs information about the inter-frame prediction to the lossless encoding section 17 and also outputs predicted image data to the subtraction section 14 and the addition section 23.

[1-2. Example Configuration of Syntax Processing Unit]

Fig. 2 is a block diagram showing an example of a specific configuration of the syntax processing section 13 of the image encoding device 10 shown in Fig. 1. Referring to Fig. 2 , the syntax processing section 13 includes a setting section 110, a parameter generating section 120, and an inserting section 130.

(1) Setting department

The settings section 110 stores various settings for the encoding process performed by the image encoding device 10. For example, the settings section 110 stores information about the profile for each sequence in the image data, the encoding mode for each picture, data regarding the GOP structure, and the like. Furthermore, in this embodiment, the settings section 110 stores settings regarding the quantization matrix used by the quantization section 16 (and the inverse quantization section 21). The quantization matrix used by the quantization section 16 can be predetermined for each slice, typically based on offline image analysis.

For example, in an example application such as a digital video camera, there are no compression artifacts in the input image, so a quantization matrix with a reduced quantization step size can be used even in the high range. The quantization matrix changes on a picture-by-picture or frame-by-frame basis. In the case of an input image with low complexity, the use of a flat quantization matrix with a smaller quantization step size makes it possible to improve the image quality perceived by the user. On the other hand, in the case of an input image with high complexity, it is desirable to use a larger quantization step size to suppress the increase in rate. In this case, the use of a flat quantization matrix has the potential to identify artifacts in the low range signal as block noise. For this reason, it is advantageous to reduce noise by using a quantization matrix in which the quantization step size increases from the low range to the high range.

In an example application such as a recorder for recompressing broadcast content encoded in MPEG-2, MPEG-2 compression artifacts such as mosquito noise are present in the input image itself. Mosquito noise is noise generated as a result of quantizing a high-range signal with a large quantization step size, and the frequency components of this noise themselves become extremely high frequencies. When recompressing these input images, it is desirable to use a quantization matrix with a large quantization step size in the high range. In addition, compared to progressive signals, in interlaced signals, the correlation of the signals in the horizontal direction is higher than the correlation of the signals in the vertical direction due to the influence of interlaced scanning. For this reason, it is also advantageous to use different quantization matrices depending on whether the image signal is a progressive signal or an interlaced signal. In both cases, the optimal quantization matrix can be varied on a picture-by-picture basis or on a frame-by-frame basis, depending on the image content.

(2) Parameter generation unit

The parameter generation section 120 generates parameters that define settings for the encoding process held by the setting section 110 , and outputs the generated parameters to the insertion section 130 .

For example, in this embodiment, the parameter generation unit 120 generates quantization matrix parameters for defining the quantization matrix to be used by the quantization unit 16. A set of quantization matrix parameters generated by the parameter generation unit 120 is included in a quantization matrix parameter set (QMPS). Each QMPS is assigned a QMPS ID, which is an identifier for distinguishing each QMPS from another. Typically, multiple quantization matrices are defined in one QMPS. The types of quantization matrices are distinguished from each other by the matrix size, as well as the corresponding prediction method and signal component. For example, up to six quantization matrices (Y/Cb/Cr components in intra-frame prediction/inter-frame prediction) can be defined within one QMPS for each of the sizes 4×4, 8×8, 16×16, and 32×32.

More specifically, the parameter generation unit 120 may convert each quantization matrix into a linear array using zigzag scanning, and encode the value of each element in the linear array using differential pulse code modulation (DPCM) format, similar to the quantization matrix encoding process in H.264/AVC. In this case, the linear array of DPCM differential data becomes the quantization matrix parameters. In this description, this mode for generating quantization matrix parameters is referred to as the full scan mode.

In addition, the parameter generation unit 120 may generate quantization matrix parameters in a mode other than the full scan mode to reduce the amount of code for the quantization matrix parameters. For example, instead of the full scan mode, the parameter generation unit 120 may generate quantization matrix parameters in the copy mode or axis designation mode described below.

The copy mode is a mode that can be selected in situations where the quantization matrix for a given segment is similar to or equal to an already defined quantization matrix. In the copy mode, the parameter generation unit 120 includes the QMPS ID of the QMPS, in which the copy source quantization matrix, as well as the size and type of the copy source quantization matrix, are defined as quantization matrix parameters. Note that in this specification, the type of quantization matrix is specified for the combination of a prediction method and a signal component corresponding to a given quantization matrix. In the case where there is a difference between the defined quantization matrix and the copy source quantization matrix, the parameter generation unit 120 may also include residual data for generating a residual matrix representing the residual error for each component in the QMPS.

The processing for the axis designation mode is further divided according to two designation methods: the difference method and the interpolation method. In the difference method, the parameter generation unit 120 only specifies the values of the elements in the quantization matrix corresponding to the vertical axis of the leftmost column, the horizontal axis of the top row, and the diagonal axes along the diagonal lines of the transform unit. In the interpolation method, the parameter generation unit 120 only specifies the values of the elements in the quantization matrix corresponding to the four corners located at the top left (DC component), top right, bottom left, and bottom right of the transform unit. The values of the remaining elements can be interpolated using any technique such as linear interpolation, cubic interpolation, or Lagrange interpolation. Similarly, in the axis designation mode, in the case where there is a difference between the defined quantization matrix and the interpolated quantization matrix, the parameter generation unit 120 may also include residual data for generating a residual matrix representing the residual of each component in the QMPS.

(3) Insertion

The insertion unit 130 inserts header information such as the SPS, PPS, and QMPS, each including a parameter group generated by the parameter generation unit 120, and a slice header into the stream of image data input from the recording buffer 12. As previously described, the QMPS is a non-VCL NAL unit different from the SPS and PPS. The QMPS includes the quantization matrix parameters generated by the parameter generation unit 120. The insertion unit 130 then outputs the image data stream with the inserted header information to the subtraction unit 14, the intra-frame prediction unit 30, and the motion estimation unit 40.

[1-3. Example parameter structure]

(1) Quantization matrix parameter set

Fig. 3 is an explanatory diagram showing example parameters included in each QMPS in this embodiment. Referring to Fig. 3 , each QMPS includes "QMPS ID," "generation mode presence flag," "generation mode," and different quantization matrix parameters for each mode.

"QMPS ID" is an identifier used to distinguish each QMPS from another. The QMPS ID can be an integer in the range of 0 to 31. Specifying an unused QMPS ID indicates that a new QMPS will be defined. Re-specifying a QMPS ID already in use in a sequence indicates that the already defined QMPS will be updated.

The "generation mode presence flag" is a flag indicating whether the "generation mode" (which is a classification indicating the mode of the quantization matrix generation process) is present in the QMPS. If the generation mode presence flag indicates "0: not present," the quantization matrix is defined in the full scan mode in the QMPS. Meanwhile, if the generation mode presence flag indicates "1: present," the "generation mode" is present in the QMPS.

The "generation mode" is a classification that can take any value, for example, "0: copy," "1: axis specification," or "2: full scan." In the following pseudo-grammar, the generation mode is represented by a variable called "pred_mode."

In the case of copy mode (i.e., pred_mode=0), the QMPS may include "Source ID," "Copy Source Size," "Copy Source Category," "Residual Flag," and "Residual Data" as quantization matrix parameters. "Source ID" is the QMPS ID that specifies the QMPS in which the copy source quantization matrix is defined. "Copy Source Size" is the size of the copy source quantization matrix. "Copy Source Category" is the category of the copy source quantization matrix (Intra-Y, Intra-Cb, ..., Intra-Cr). "Residual Flag" is a flag indicating whether a residual exists. "Residual Data" is data used to generate a residual matrix representing the residual when a residual exists. Residual data can be omitted when the residual flag indicates "0: not present."

Note that the case where the source ID of a given QMPS is equal to the QMPS ID of the QMPS itself can be interpreted as specifying the default quantization matrix shown by the example in FIG38 , etc. By doing so, since it is no longer necessary to include an independent flag for specifying the default quantization matrix in the QMPS, the code amount of the QMPS can be reduced.

In the case of axis designation mode (i.e., pred_mode=1), the QMPS may include a "designation method flag", as well as "reference axis data" or "corner data", a "residual flag", and "residual data" as quantization matrix parameters. The designation method flag is a flag that indicates how the value of the element used as a reference for generating the quantization matrix is specified along the reference axis, and the flag can take a value of, for example, "0: differential" or "1: interpolation". In the case where the designation method is "0: differential", the element values corresponding to the reference axes of the quantization matrix (these are the vertical axis, horizontal axis, and diagonal axis) are specified by the reference axis data. In the case where the designation method is "1: interpolation", the values of the elements corresponding to the four corners located at the upper left, upper right, lower left, and lower right of the quantization matrix are specified by the corner data. The values of the elements on the three reference axes can be generated by interpolating from the values of these four corners. The residual flag and residual data are similar to those in the case of the copy mode.

In the case of full scan mode (ie, pred_mode=2), the QMPS may include a linear array of DPCM difference data as quantization matrix parameters.

Note that each QMPS may include different generation modes for each type of quantization matrix and quantization matrix parameters corresponding to each mode. In other words, as an example, in a single QMPS, a quantization matrix for a given class may be defined in full scan mode, a quantization matrix for another class may be defined in axis designation mode, and the remaining quantization matrices may be defined in copy mode.

(2) Fragment header

FIG4 is an explanatory diagram partially illustrating example parameters included in each slice header in this embodiment. Referring to FIG4 , each slice header may include a "slice category," a "PPS ID," a "QMPS ID presence flag," and a "QMPS ID." The "slice category" is a classification indicating the encoding category of the slice and takes values corresponding to, for example, a P-slice, a B-slice, or an I-slice. The "PPS ID" is the ID of the picture parameter set (PPS) referenced for the slice. The "QMPS ID presence flag" is a flag indicating whether a QMPS ID is present in the slice header. The "QMPS ID" is the QMPS ID of the quantization matrix parameter set (QMPS) referenced for the slice.

<2. Processing Flow During Encoding According to the Embodiment>

(1) Parameter set insertion processing

FIG5 is a flowchart showing an example flow of a parameter set insertion process by the inserting section 130 of the syntax processing section 13 according to the present embodiment.

Referring to Figure 5 , the insertion unit 130 first continuously retrieves NAL units from the image data stream input from the recording buffer 12 and identifies a single picture (step S100 ). Next, the insertion unit 130 determines whether the identified picture is the leading picture of the sequence (step S102 ). If the identified picture is the leading picture of the sequence, the insertion unit 130 inserts the SPS into the stream (step S104 ). Next, the insertion unit 130 also determines whether there is a change in the PPS for the identified picture (step S106 ). If there is a change in the PPS, or if the identified picture is the leading picture of the sequence, the insertion unit 130 inserts the PPS into the stream (step S108 ). Next, the insertion unit 130 also determines whether there is a change in the QMPS (step S110 ). If there is a change in the QMPS, or if the identified picture is the leading picture of the sequence, the insertion unit 130 inserts the QMPS into the stream (step S112 ). After that, if the insertion unit 130 detects the end of the stream, the insertion unit 130 ends the process. On the other hand, if the stream has not ended, the insertion unit 130 repeats the above process for the next picture (step S114).

Note that although this flowchart only shows the insertion of SPS, PPS, and QMPS for simplicity, the inserting unit 130 may also insert other header information such as supplemental enhancement information (SEI) and slice headers into the stream.

(2) Description of convection structure

FIG. 6 is an explanatory diagram for explaining the difference in flow structure between the technology according to this embodiment and the related art.

The left side of FIG6 shows a stream ST1 as an example generated according to the prior art. Since the start of the stream ST1 is the start of a sequence, the first SPS (1) and the first PPS (1) are inserted at the start of the stream ST1. One or more quantization matrices can be defined in the SPS (1) and the PPS (1). Then, it is assumed that it becomes necessary to update the quantization matrix after several subsequent slice headers and slice data. Thus, the second PPS (2) is inserted into the stream ST1. The PPS (2) also includes parameters other than the quantization matrix parameters. Then, it is assumed that it becomes necessary to update the PPS after several subsequent slice headers and slice data. Thus, the third PPS (3) is inserted into the stream ST1. The PPS (3) also includes quantization matrix parameters. The quantization process (and inverse quantization process) for the subsequent slices is performed using the quantization matrix defined in the PPS specified by the PPS ID in the slice header.

The right side of FIG6 shows a stream ST2 as an example generated according to the above-described technique of this embodiment. Since the beginning of the stream ST2 is the beginning of the sequence, the first SPS (1), the first PPS (1), and the first QMPS (1) are inserted at the beginning of the stream ST2. In the stream ST2, one or more quantization matrices can be defined in the QMPS (1). The sum of the lengths of the PPS (1) and the QMPS (1) in the stream ST2 is approximately equal to the length of the PPS (1) in the stream ST1. Then, if it becomes necessary to update the quantization matrix after several subsequent slice headers and slice data, a second QMPS (2) is inserted into the stream ST2. Since the QMPS (2) does not contain parameters other than the quantization matrix parameters, the length of the QMPS (2) is shorter than the length of the PPS (2) in the stream ST1. Then, if it becomes necessary to update the PPS after several subsequent slice headers and slice data, a second PPS (2) is inserted into the stream ST2. Since PPS(2) in stream ST2 does not include quantization matrix parameters, the length of PPS(2) in stream ST2 is shorter than the length of PPS(3) in stream ST1. Quantization processing (and inverse quantization processing) for subsequent slices is performed using the quantization matrix defined in the QMPS specified by the QMPSID in the slice header.

FIG6 shows a comparison of streams ST1 and ST2, showing that the technique described in the embodiment of the present invention can reduce the overall code size of the stream. In particular, in the case of a large-scale quantization matrix or in the case of defining more quantization matrices for each picture, the above technique can more effectively reduce the code size.

<3. Example Configuration of Image Decoding Device According to Embodiment>

[3-1. Example overall configuration]

This section describes an example configuration of an image decoding device according to an embodiment.

[3-1. Example overall configuration]

FIG7 is a block diagram showing an example structure of an image decoding device 60 according to an embodiment. Referring to FIG7 , the image decoding device 60 is configured with a syntax processing section 61, a lossless decoding section 62, an inverse quantization section 63, an inverse orthogonal transform section 64, an addition section 65, a deblocking filter 66, a recording buffer 67, a digital-to-analog (D/A) conversion section 68, a frame memory 69, selectors 70 and 71, an intra-frame prediction section 80, and a motion compensation section 90.

The syntax processing section 61 acquires header information such as the SPS, PPS, QMPS, and slice header from the encoded stream input via the transmission channel, and identifies, based on the acquired header information, various settings for the decoding process performed by the image decoding device 60. For example, in this embodiment, the syntax processing section 61 sets the quantization matrix to be used during the inverse quantization process of the inverse quantization section 63 based on the quantization matrix parameters included in the QMPS. The specific configuration of the syntax processing section 61 will be further described later.

The lossless decoding section 62 decodes the encoded stream input from the syntax processing section 61 according to the encoding method used at the time of encoding. The lossless decoding section 62 then outputs the decoded quantized data to the inverse quantization section 63. In addition, the lossless decoding section 62 outputs information on intra-frame prediction included in the header information to the intra-frame prediction section 80, and outputs information on inter-frame prediction to the motion compensation section 90.

The inverse quantization section 63 inversely quantizes the quantized data (i.e., quantized transform coefficient data) decoded by the lossless decoding section 62 using the quantization matrix set by the syntax processing section 61. The question of which quantization matrix should be used for each block in a given slice can be determined based on the QMPS ID specified in the slice header, the size of each block (transform unit), the prediction method used for each block, and the signal component.

The inverse orthogonal transform section 64 generates prediction error data by performing inverse orthogonal transform on the transform coefficient data input from the inverse quantization section 63 according to the orthogonal transform method used at the time of encoding.

The addition section 65 generates decoded image data by adding the prediction error data input from the inverse orthogonal transform section 64 and the predicted image data input from the selector 71 . The addition section 65 then outputs the generated decoded image data to the deblocking filter 66 and the frame memory 69 .

The deblocking filter 66 removes blocking artifacts by filtering the decoded image data input from the adding section 65 , and outputs the filtered decoded image data to the recording buffer 67 and the frame memory 69 .

The recording buffer 67 generates a time-series sequence of image data by recording the image input from the deblocking filter 66. Then, the recording buffer 67 outputs the generated image data to the D/A conversion section 68.

The D/A converter 68 converts the digital image data input from the recording buffer 67 into an analog image signal. The D/A converter 68 then displays an image by outputting the analog image signal to a display (not shown) connected to the image decoding device 60, for example.

The frame memory 69 stores the unfiltered decoded image data input from the addition section 65 and the filtered decoded image data input from the deblocking filter 66 using a storage medium.

The selector 70 switches the output destination of the image data from the frame memory 69 between the intra prediction section 80 and the motion compensation section 90 for each block in the image according to the mode information acquired by the lossless decoding section 62. For example, in the case where the intra prediction mode is specified, the selector 70 outputs the unfiltered decoded image data supplied from the frame memory 69 to the intra prediction section 80 as reference image data. In addition, in the case where the inter prediction mode is specified, the selector 70 outputs the filtered decoded image data supplied from the frame memory 69 to the motion compensation section 90 as reference image data.

The selector 71 switches the output source of the predicted image data to be supplied to the addition section 65 between the intra prediction section 80 and the motion compensation section 90 for each block in the image according to the mode information acquired by the lossless decoding section 62. For example, in the case where the intra prediction mode is specified, the selector 71 supplies the predicted image data output from the intra prediction section 80 to the addition section 65. In the case where the inter prediction mode is specified, the selector 71 supplies the predicted image data output from the motion compensation section 90 to the addition section 65.

The intra prediction section 80 performs intra-picture prediction on pixel values based on the information on intra prediction input from the lossless decoding section 62 and the reference image data from the frame memory 69 and generates predicted image data. The intra prediction section 80 then outputs the generated predicted image data to the selector 71.

The motion compensation section 90 performs motion compensation processing based on the information on inter prediction input from the lossless decoding section 62 and the reference image data from the frame memory 69 and generates predicted image data. The motion compensation section 90 then outputs the generated predicted image data to the selector 71.

[3-2. Example Configuration of Syntax Processing Unit]

Fig. 8 is a block diagram showing an example of a specific configuration of the syntax processing section 61 of the image decoding device 60 shown in Fig. 7. Referring to Fig. 8 , the syntax processing section 61 includes a parameter acquisition section 160 and a setting section 170.

(1) Parameter acquisition unit

The parameter acquisition unit 160 identifies header information such as the SPS, PPS, QMPS, and slice header from the image data stream and acquires parameters included in the header information. For example, in this embodiment, the parameter acquisition unit 160 acquires quantization matrix parameters that define the quantization matrix from the QMPS. As previously mentioned, the QMPS is a non-VCL NAL unit, unlike the SPS and PPS. The parameter acquisition unit 160 then outputs the acquired parameters to the setup unit 170. The parameter acquisition unit 160 also outputs the image data stream to the lossless decoding unit 62.

(2) Setting department

The setting unit 170 applies settings for the processing in each unit shown in FIG. 7 based on the parameters acquired by the parameter acquisition unit 160. For example, the setting unit 170 identifies the range of coding unit sizes from the LCU and SCU value pairs and also sets the coding unit size based on the value of split_flag. Image data is decoded using these coding units as processing units. Furthermore, the setting unit 170 also sets the transform unit size. Using these transform units as processing units, the inverse quantization performed by the inverse quantization unit 63 and the inverse orthogonal transform performed by the inverse orthogonal transform unit 64 are performed.

In this embodiment, the setting unit 170 sets the quantization matrix based on the quantization matrix parameters acquired from the QMPS by the parameter acquisition unit 160. More specifically, based on the quantization parameters included in the QMPS, the setting unit 170 generates a plurality of quantization matrices having different sizes and types in the full scan mode, the copy mode, and the axis designation mode. Quantization matrix generation can be performed each time a QMPS is detected in the image data stream.

For example, in full scan mode, the setting section 170 decodes the linear array of differential data included in the quantization matrix parameters in DPCM format and then converts the decoded linear array into a two-dimensional quantization matrix according to a zigzag scan pattern.

In copy mode, the setup unit 170 copies the (previously generated) quantization matrix specified by the source ID, copy source size, and copy source type included in the quantization matrix parameters. If the size of the new quantization matrix is smaller than the size of the copy source quantization matrix, the setup unit 170 generates the new quantization matrix by reducing the elements in the copy quantization matrix. If the size of the new quantization matrix is larger than the size of the copy source quantization matrix, the setup unit 170 generates the new quantization matrix by interpolating the elements in the copy quantization matrix. If a residual component exists, the setup unit 170 adds the residual component to the new quantization matrix.

In addition, in the case where the source ID included in the quantization matrix parameters in a given QMPS is equal to the QMPS ID of the QMPS, the setting section 170 regards the new quantization matrix as a default quantization matrix.

In addition, in the axis designation mode, the setup unit 170 identifies the designation method flag included in the quantization matrix parameters. Then, in the case of the differential method, the setup unit 170 generates values for the quantization matrix elements corresponding to the vertical, horizontal, and diagonal axes based on the reference axis data included in the quantization matrix parameters, and generates values for the other elements by interpolation. Furthermore, in the case of the interpolation method, the setup unit 170 generates values for the quantization matrix elements corresponding to the four corners based on the angle data included in the quantization matrix parameters. After generating values for the elements along the reference axes by interpolation, it also generates values for the remaining elements by interpolation. Then, if a residual component exists, the setup unit 170 adds the residual component to the new quantization matrix.

Thereafter, when the QMPS ID is specified in the slice header, the setting section 170 sets the quantization matrix generated for the QMPS identified by the specified QMPS ID as the quantization matrix to be used by the inverse quantization section 63 .

<4. Processing Flow During Decoding According to the Embodiment>

[4-1. Generating Quantization Matrix]

(1) Quantization matrix generation processing

Fig. 9 is a flowchart showing an example flow of a quantization matrix generation process by the syntax processing section 61 according to the present embodiment. The quantization matrix generation process in Fig. 9 is a process that can be performed every time a QMPS is detected in an image data stream.

Referring to Figure 9 , the parameter acquisition unit 160 first acquires a QMPS ID from the QMPS (step S200). If the acquired QMPS ID is an unused ID in the stream, the setup unit 170 generates a new quantization matrix to be associated with the QMPS ID according to the process described below. On the other hand, if the QMPS ID is already in use, the setup unit 170 updates the quantization matrix stored in association with the QMPS ID to a matrix generated according to the process described below. The parameter acquisition unit 160 then acquires a generation mode presence flag from the QMPS (step S202).

The subsequent processing from step S206 to step S240 is repeated for each quantization matrix (step S204 ). Note that the category of the quantization matrix corresponds to a combination of the size and type (ie, prediction method and signal component) of the quantization matrix.

In step S206, the setup unit 170 determines whether (the type of) a generation mode exists in the QMPS based on the generation mode presence flag (step S206). If a generation mode does not exist at this time, the setup unit 170 generates a quantization matrix using a full scan method, similar to the quantization matrix decoding process in H.264/AVC (step S208). On the other hand, if a generation mode exists, the parameter acquisition unit 160 acquires the generation mode from the QMPS (step S210). The setup unit 170 then performs different processes depending on the generation mode (steps S212 and S214).

For example, if the copy mode is indicated, the setup unit 170 performs processing in the copy mode illustrated in FIG. 10 (step S220). Alternatively, if the axis designation mode is indicated, the setup unit 170 performs processing in the axis designation mode illustrated in FIG. 11 (step S240). Furthermore, if the full scan mode is indicated, the setup unit 170 generates a quantization matrix using a full scan method, similar to the quantization matrix decoding process in H.264/AVC (step S208).

Then, when the quantization matrices for the quantization matrices of all the classes are generated, the quantization matrix generation process shown in FIG. 9 ends.

(2) Processing in copy mode

FIG. 10 is a flowchart showing an example of a specific flow of processing in the copy mode in step S220 of FIG. 9 .

Referring to FIG10 , the parameter acquisition unit 160 first acquires the source ID from the QMPS (step S221). Next, the setup unit 170 determines whether the QMPS ID acquired in step S200 of FIG9 (the QMPS ID of the current QMPS) is equal to the source ID. If the QMPS ID of the current QMPS is equal to the source ID, the setup unit 170 generates a new quantization matrix as a default quantization matrix (step S223). On the other hand, if the QMPS ID of the current QMPS is not equal to the source ID, the process proceeds to step S224.

In step S224, the parameter acquisition unit 160 acquires the copy source size and copy source type from the QMPS (step S224). The setup unit 170 then copies the quantization matrix specified by the source ID, copy source size, and copy source type (step S225). The setup unit 170 then compares the copy source size with the size of the quantization matrix to be generated (step S226). If the copy source size is not equal to the size of the quantization matrix to be generated, the setup unit 170 generates a new quantization matrix by interpolating or reducing the elements of the copied quantization matrix according to the size difference (step S227).

In addition, the parameter acquisition unit 160 acquires the residual flag from the QMPS (step S228). Next, the setting unit 170 determines whether residual data exists based on the value of the residual flag (step S229). If residual data exists, the setting unit 170 adds the residual to the new quantization matrix generated in step S223 or steps S225 to S227 (step S230).

(3) Processing in axis designation mode

FIG. 11 is a flowchart showing an example of a specific flow of processing in the axis designation mode in step S240 of FIG. 9 .

Referring to FIG. 11 , the parameter acquisition unit 160 first acquires a designated method flag from the QMPS (step S241). The setting unit 170 then determines the designated method based on the value of the designated method flag (step S242). If the difference method is designated, the process proceeds to step S243. On the other hand, if the interpolation method is designated, the process proceeds to step S246.

In the case of the differential method, the setting unit 170 generates values for the elements of the quantization matrix corresponding to the vertical, horizontal, and diagonal axes based on the reference axis data included in the quantization matrix parameters (steps S243, S244, and S245). Meanwhile, in the case of the interpolation method, the setting unit 170 generates values for the elements of the quantization matrix corresponding to the four corners based on the corner data included in the quantization matrix parameters (step S246). Next, the setting unit 170 generates values for the elements along the reference axes (vertical, horizontal, and diagonal) connecting the four corners through interpolation (step S247). The setting unit 170 then interpolates the values of the remaining elements based on the values of the elements along the reference axes (step S248).

Furthermore, the parameter acquisition unit 160 acquires the residual flag from the QMPS (step S249). Next, the setting unit 170 determines whether residual data exists based on the value of the residual flag (step S250). If residual data exists, the setting unit 170 adds the residual to the new quantization matrix generated in step S248 (step S251).

[4-2. Setting the quantization matrix for the fragment]

Fig. 12 is a flowchart showing an example flow of a process of setting a quantization matrix for a slice by the syntax processing section 61 according to the present embodiment. The process in Fig. 12 can be performed every time a slice header is detected in an image data stream.

First, the parameter acquisition unit 160 acquires the QMPS ID presence flag from the slice header (step S261). Next, the parameter acquisition unit 160 determines whether a QMPS ID exists in the slice header based on the value of the QMPS ID presence flag (step S262). If a QMPS ID exists, the parameter acquisition unit 160 also acquires the QMPS ID from the QMPS (step S263). The setting unit 170 then sets the quantization matrix generated for the QMPS identified by the acquired QMPS ID for the slices following the slice header (step S264). On the other hand, if a QMPS ID does not exist in the slice header, the setting unit 170 sets the flattened quantization matrix for the slices following the slice header (step S265).

<5. Syntax Examples>

[5-1. First Example]

Figures 13 to 15 show a first example of an illustrative pseudocode expressing the syntax of a QMPS according to this embodiment. Line numbers are given at the left edge of the pseudocode. In addition, underlined variables in the pseudocode indicate that the parameter corresponding to the variable is specified within the QMPS.

The QuantizationMatrixParameterset() function on line 1 in Figure 13 expresses the syntax of a single QMPS. Lines 2 and 3 specify the QMPS ID (quantization_matrix_paramter_id) and the generation mode presence flag (pred_present_flag). The syntax from lines 6 to 56 then loops for each quantization matrix size and type. If the generation mode exists (pred_present_flag = 1), the syntax from lines 7 to 53 in the loop is inserted into the QMPS.

In the case of the Generate mode, the syntax from lines 9 to 16 is for the Copy mode. Lines 9 to 16 specify the source ID, copy source size, and copy source type. The pred_matrix() function on line 12 indicates that the quantization matrix specified by the source ID, copy source size, and copy source type will be copied. Line 13 specifies the residual flag. The residual_matrix() function on line 15 indicates that, in the case of the residual component, residual data is specified in the QMPS.

The syntax from lines 18 to 50 is the syntax for the axis designation mode and is described in FIG14. On line 18, a designation method flag is specified. In the case where the designation method is the differential (DPCM) method, the values of the elements along the vertical axis are specified from lines 21 to 25, the values of the elements along the horizontal axis are specified from lines 26 to 34, and the values of the elements along the diagonal axis are specified from lines 35 to 40. The reference axis data in this case is a linear array of DPCM differential data. Note that in the case where the values of the elements along the horizontal axis can be copied, the syntax for the elements along the horizontal axis can be omitted (line 27, line 28). In the case where the designation method is the interpolation method, the values of the upper left (DC component), upper right, lower left, and lower right elements are specified as angle data from lines 42 to 45, respectively.

The process on line 52 is the syntax for full scan mode. The process on line 55 is the syntax for the case where there is no generation mode. In both cases, the quantization matrix is specified in the full scan method using the function qmatrix(), which represents the quantization matrix syntax in H.264/AVC.

The residual_matrix() function on line 1 in FIG. 15 is used to specify the residual data used on line 15 in FIG. 13 and line 49 in FIG. 14 . In the example in FIG. 15 , the residual data is specified using either the DPCM method or the run-length method. In the case of the DPCM method, the difference (delta_coef) from the last element is specified for each element in the linear array from lines 4 to 8. In the case of the run-length method, the length of the element group in the portion where the values are continuously zero (run) and the values (data) of the non-zero elements are repeatedly specified from lines 11 to 18.

[5-2. Second Example]

16 to 20 show a second example of illustrative pseudo code expressing the syntax of QMPS according to the present embodiment.

The QuantizationMatrix parameterSet() function on line 1 in Figure 16 expresses the syntax of a single QMPS. Line 2 specifies the QMPS ID (quantization_matrix_paramter_id). In addition, line 6 specifies the generation mode presence flag (pred_present_flag), except when specifying only the default quantization matrix.

In the second example, lines 7 through 10 of the QuantizationMatrixParameterSet() function specify four quantization scales (Qscale0 through Qscale3). These quantization scales are parameters that can be used to quantize the values of each element in the quantization matrix and further reduce the rate. More specifically, four quantization scale setting areas A1 through A4, as shown in Figure 21, are defined in an 8×8 quantization matrix. Quantization scale setting area A1 is used for groups of elements corresponding to low-range signals (including DC components). Quantization scale setting areas A2 and A3 are used for groups of elements corresponding to signals in the mid-range. Quantization scale setting area A4 is used for groups of elements corresponding to high-range signals. For each of these areas, a quantization scale can be set for quantizing the values of elements in the quantization matrix. For example, referring to Figure 22, for quantization scale setting area A1, the first quantization scale (Qscale0) is "1." This indicates that values in the quantization matrix are not quantized in groups of elements corresponding to low-range signals. Meanwhile, for quantization scale setting area A2, the second quantization scale (Qscale1) is "2." For the quantization scale setting area A3, the third quantization scale (Qscale2) is "3." For the quantization scale setting area A4, the fourth quantization scale (Qscale3) is "4." As the quantization scale becomes larger, the error generated by quantization increases. However, generally, a certain degree of error is acceptable for high-range signals. Thus, in cases where high coding efficiency is desired, by setting such a quantization scale for quantizing the quantization matrix, the amount of code required to define the quantization matrix can be effectively reduced without significantly degrading image quality. In the case where the quantization matrix is quantized, the value of each element in the residual data or differential data shown in the example of FIG. 3 can be substantially quantized or inversely quantized by the quantization step set in the quantization scale setting area to which each element belongs.

Note that the layout of the quantization scale setting areas shown in FIG21 is merely an example. For example, a different number of quantization scale setting areas can be defined for each size of the quantization matrix (e.g., more quantization scale setting areas can be defined for larger sizes). Furthermore, the positions of the quantization scale setting area boundaries are not limited to the example shown in FIG21 . Typically, the scanning pattern when linearizing the quantization matrix is a zigzag scan. For this reason, diagonal area boundaries from the upper right to the lower left are preferably used, as shown in FIG21 . However, depending on factors such as the correlation between elements in the quantization matrix and the scanning pattern used, area boundaries along the vertical or horizontal direction can also be used. Furthermore, the layout of the quantization scale setting areas (number of areas, boundary positions, etc.) can be adaptively selected from the perspective of coding efficiency. For example, when defining a nearly flat quantization matrix, a smaller number of quantization scale setting areas can be selected.

In Fig. 16 , the syntax from lines 13 to 76 that follow loops for each quantization matrix size and type. In the case where the generation mode exists (pred_present_flag=1), the syntax from lines 14 to 66 (see Fig. 18 ) in the loop is inserted into the QMPS.

The syntax from lines 16 to 22 in the case where there is a generation mode is the syntax for the copy mode. From lines 16 to 18, the source ID, copy source size, and copy source type are specified. On line 19, the residual flag is specified. The function residual_matrix() on line 21 indicates that residual data is specified in the QMPS in the case where there is a residual component. The residual data here can be quantized according to the values of the four quantization scales (Qscale0 to Qscale3) described above. The syntax from lines 23 to 56 is the syntax for the axis designation mode and is described in Figure 17. The residual data in the axis designation mode can also be quantized according to the values of the four quantization scales (Qscale0 to Qscale3) described above (line 55).

The syntax from lines 57 to 66 in Figure 18 is for full scan mode. Additionally, the syntax from lines 68 to 75 is for when generate mode is not present. In both cases, the quantization matrix is specified using the full scan method using the qmatrix() function. However, in the second example, to further improve coding efficiency, the VLC tables used for entropy-encoded differential data (delta_coef) in the DPCM method or run values (run) and non-zero element values (data) in the run-length method are adaptively switched. vlc_table_data on lines 61 and 71 specifies the number of VLC tables selected for differential data (delta_coef) in the DPCM method or non-zero element values (data) in the run-length method. vlc_table_run on lines 63 and 73 specifies the number of VLC tables selected for run values (run) in the run-length method.

The function qmatrix() on line 1 of Figure 19 is a syntax for specifying a quantization matrix using the full scan method. Lines 3 to 8 of Figure 19 indicate syntax for the DPCM method, and the differential data (delta_coef) on line 5 is encoded using the VLC table specified by vlc_table_data. Additionally, lines 10 to 21 indicate syntax for the run length method, and the run values (run) on line 12 are encoded using the VLC table specified by vlc_table_run. The non-zero element values (data) on line 13 are encoded using the VLC table specified by vlc_tabla_data.

Figure 23 shows a list of codewords from 11 variable-length coding (VLC) tables that can be selected in the Low Complexity Entropy Coding (LCEC) method. The "x" in each codeword in Figure 23 represents a given suffix. For example, if the value "15" is encoded using the Exponential Golomb code, the 9-bit codeword "000010000" is obtained, while if the same value is encoded using VLC4, the 5-bit codeword "11111" is obtained. This improves coding efficiency when encoding many large values by selecting a VLC table with a larger number of suffixes in short codewords. For example, among the 11 VLC tables in Figure 23, VLC4 has a 4-bit suffix in a 5-bit codeword. Furthermore, VLC9 has a 4-bit suffix in a 6-bit codeword. Therefore, these VLC tables are suitable for encoding many large values.

Returning to the quantization matrix syntax, since the differential data of the linear array of the quantization matrix has multiple consecutive zeros, the run values in the run-length method generate multiple large values rather than small values such as 0, 1, or 2. On the other hand, the non-zero element values and differential data values in the run-length method only rarely form large values. Therefore, by switching the VLC table (DPCM/run-length) and class value (run/non-zero element in the case of the run-length method) for each differential data specification method using the above syntax, the amount of code required to define the quantization matrix is greatly reduced.

The residual_matrix() function on line 1 of Figure 20 also implements adaptive switching of VLC tables. Specifically, vlc_table_data on line 7 specifies the table number of the VLC table selected for the differential data (delta_coef) in the DPCM method. The differential data (delta_coef) on line 10 is encoded using the VLC table specified in line 7. vlc_table_data on line 15 specifies the table number of the VLC table selected for the non-zero element values (data) in the run-length method. vlc_table_run on line 16 specifies the table number of the VLC table selected for the run values (run) in the run-length method. The run values (run) on line 19 are encoded using the VLC table specified by vlc_table_run. The non-zero element values (data) on line 20 are encoded using the VLC table specified by vlc_table_data.

By using various features of the QMPS syntax, the amount of code required to define the quantization matrix is effectively reduced, and coding efficiency can be improved. However, the syntax described here is merely an example. In other words, a portion of the syntax shown here can be reduced or omitted, the order of parameters can be changed, or other parameters can be added to the syntax. In addition, when the quantization matrix is defined in an SPS or PPS instead of a QMPS, several features of the syntax described in this section can also be implemented. In this case, the amount of code required to define the quantization matrix in the SPS or PPS can be reduced.

<6. Various example configurations of parameter sets>

The above describes several specific examples of the syntax of a Quantization Matrix Parameter Set (QMPS) for storing quantization matrix parameters. A QMPS can essentially be a dedicated parameter set containing only quantization matrix parameters, but it can also be a shared parameter set that also includes parameters related to coding tools other than the quantization matrix. For example, the "Adaptation Parameter Set (APS)" (JCTVC-F747r3, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 6th Meeting: Torino, IT, July 14-22, 2011) introduces a new parameter set called the Adaptation Parameter Set (APS) and proposes storing parameters related to the Adaptive Loop Filter (ALF) and Sample Adaptive Offset (SAO) in the APS. By additionally including quantization matrix parameters in this APS, the aforementioned QMPS can be essentially configured. Thus, in this section, several techniques for configuring a QMPS by using the Adaptation Parameter Set (APS) proposed by the APS (JCTVC-F747r3) will be described.

[6-1. First technique]

The first technique is a technique of listing all target parameters in one APS and indicating each parameter using an APS ID (an identifier uniquely representing an APS). Fig. 24 shows an example of a coded stream configured according to the first technique.

24 , SPS 801, PPS 802, and APS 803 are inserted at the beginning of picture P0, which is located at the beginning of the sequence. PPS 802 is identified by PPS ID "P0." APS 803 is identified by APS ID "A0." APS 803 includes parameters related to ALF, parameters related to SAO, and quantization matrix parameters (hereinafter referred to as QM parameters). Slice header 804 attached to slice data in picture P0 includes a reference to PPS ID "P0," indicating that the parameters in PPS 802 are referenced for decoding the slice data. Similarly, slice header 804 includes a reference to APS ID "A0," indicating that the parameters in APS 803 are referenced for decoding the slice data.

APS 805 is inserted into picture P1 following picture P0. APS 805 is identified by APS ID "A1." APS 805 includes parameters related to ALF, SAO, and QM. The ALF and SAO parameters included in APS 805 have been updated from APS 803, but the QM parameters have not been updated. The slice header 806 attached to the slice data in picture P1 includes a reference to APS ID "A1," indicating that the parameters in APS 805 are referenced to decode the slice data.

APS 807 is inserted into picture P2, which follows picture P1. APS 807 is identified by APS ID "A2." APS 807 includes parameters related to ALF, SAO, and QM. The ALF and QM parameters included in APS 807 have been updated from APS 805, but the SAO parameters have not been updated. Slice header 808 attached to the slice data in picture P2 includes a reference to APS ID "A2," indicating that the parameters in APS 807 are referenced to decode the slice data.

FIG25 illustrates an example of the APS syntax defined according to the first technique. Line 2 of FIG25 specifies the APS ID, which uniquely identifies the APS. This APS ID is a parameter set identifier used in place of the QMPS ID described in FIG3 . Lines 13 to 17 specify parameters related to the ALF. Lines 18 to 23 specify parameters related to the SAO. Lines 24 to 28 specify parameters related to the QM. "aps_qmatrix_flag" on line 24 is a quantization matrix presence flag indicating whether QM-related parameters are set in the APS. "qmatrix_param()" on line 27 is a function that specifies the quantization matrix parameters shown in the examples of FIG13 to 20 . The quantization matrix presence flag on line 24 indicates that QM-related parameters are set in the APS (aps_qmatrix_flag = 1). The qmatrix_param() function can be used to set the quantization matrix parameters in the APS.

In the case of implementing the first technique, the parameter acquisition unit 160 shown in FIG8 determines whether the quantization matrix parameters are set in the APS by referring to the quantization matrix presence flag included in the APS. Then, if the quantization matrix parameters are set in the APS, the parameter acquisition unit 160 acquires the quantization matrix parameters from the APS.

FIG26 is an explanatory diagram showing an example of the slice header syntax defined according to the first technique. In line 5 of FIG26 , a reference PPS ID for referencing parameters included in the PPS is specified from among the parameters to be set for the slice. In line 8, a reference APS ID for referencing parameters included in the APS is specified from among the parameters to be set for the slice. This reference APS ID is a reference parameter used in place of the (reference) QMPS ID described in FIG4 .

According to the first technique, the aforementioned quantization matrix parameter set can be implemented at low cost by extending the Adaptation Parameter Set (APS) proposed by JCTVC-F747r3. Furthermore, a quantization matrix presence flag can be used to update only some of the parameters associated with various coding tools that may be included in the APS, or alternatively, to exclude some of the quantization matrix parameters from being updated. In other words, since the quantization matrix parameters can be included in the APS only when they are needed, the quantization matrix parameters can be efficiently transmitted in the APS.

[6-2. Example modification of the first technique]

The modified technique according to the example below can also be implemented to further reduce the code amount of the quantization matrix parameters in the APS.

FIG27 illustrates an example of an APS syntax defined according to an example modification of the first technique. In the syntax illustrated in FIG27 , parameters related to the QM are specified on lines 24 to 33. "aps_qmatrix_flag" on line 24 is a quantization matrix presence flag indicating whether parameters related to the QM are set in the APS. "ref_aps_id_present_flag" on line 25 is a past reference ID presence flag indicating whether a past reference ID will be used as a parameter related to the QM in the APS. When the past reference ID presence flag indicates that a past reference ID will be used (ref_aps_id_present_flag = 1), the past reference ID "ref_aps_id" is set on line 27. The past reference ID is an identifier for referencing the APS ID of an APS encoded or decoded before the current APS. When a past reference ID is used, the quantization matrix parameters are not set in the reference source (latter) APS. In this case, the setup unit 170 shown in FIG8 reuses the quantization matrix set based on the quantization matrix parameters in the reference target APS indicated by the past reference ID as the quantization matrix corresponding to the reference source APS. Note that it is possible to prohibit the use of a past reference ID that references the APS ID of the reference source APS (referred to as self-reference). Alternatively, the setup unit 170 may set a default quantization matrix as the quantization matrix corresponding to the self-reference APS. If a past reference ID is not used (ref_aps_id_present_flag = 0), the function "qmatrix_param()" on line 31 may be used to set the quantization matrix parameters in the APS.

In this way, by using the past reference ID to reuse the quantization matrix that has already been encoded or decoded, it is avoided to repeatedly set the same quantization matrix parameters in the APS. As a result, the code amount of the quantization matrix parameters in the APS can be reduced. Note that although Figure 27 shows an example in which the APS ID is used to reference the past APS, the method of referencing the past APS is not limited to this example. For example, other parameters such as the number of APSs between the reference source APS and the reference target APS can also be used to reference the past APS. In addition, instead of using the past reference ID existence flag, it is possible to switch the reference to the past APS and set new quantization matrix parameters depending on whether the past reference ID indicates a given value (-1, for example).

[6-3. Second technique]

The second technique is a technique of storing parameters in different APSs (different NAL units) for each parameter, and referencing each parameter using an APS ID that uniquely identifies each APS. Fig. 28 shows an example of a coded stream configured according to the second technique.

28 , SPS 811, PPS 812, APS 813a, APS 813b, and APS 813c are inserted at the beginning of picture P0 at the beginning of the sequence. PPS 812 is identified by PPS ID "P0." APS 813a is an APS for parameters related to ALF and is identified by APS ID "A00." APS 813b is an APS for parameters related to SAO and is identified by APS ID "A10." APS 813c is an APS for parameters related to QM and is identified by APS ID "A20." The slice header 814 attached to the slice data in picture P0 includes a reference to PPS ID "P0," indicating that the parameters in PPS 812 are referenced to decode the slice data. Similarly, the slice header 814 includes reference APS_ALF ID “A00”, reference APS_SAO ID “A10”, and reference APS_QM ID “A20”, and these indicate that parameters in the APSs 813a, 813b, and 813c are referenced to decode the slice data.

APS 815a and APS 815b are inserted into picture P1, which follows picture P0. APS 815a is an APS for parameters related to ALF and is identified by APS ID "A01." APS 815b is an APS for parameters related to SAO and is identified by APS ID "A11." Since the parameters related to QM are not updated from picture P0, no APS for QM parameters is inserted. The slice header 816 attached to the slice data in picture P1 includes reference APS_ALF ID "A01," reference APS_SAO ID "A11," and reference APS_QM ID "A20." These indicate that the parameters in APSs 815a, 815b, and 815c are referenced to decode the slice data.

APS 817a and APS 817c are inserted into picture P2, which follows picture P1. APS 817a is an APS for parameters related to ALF and is identified by APS ID "A02." APS 817c is an APS for parameters related to QM and is identified by APS ID "A21." Since SAO parameters are not updated from picture P1, no APS for SAO parameters is inserted. The slice header 818 attached to the slice data in picture P2 includes reference APS_ALF ID "A02," reference APS_SAO ID "A11," and reference APS_QM ID "A21." These indicate that the parameters in APSs 817a, 817b, and 817c are referenced to decode the slice data.

The APS for QM-related parameters in the second technique (e.g., APSs 813c and 817c) is approximately equivalent to the aforementioned QMPS. The APS ID of the APS for QM-related parameters is used instead of the QMPS ID described using FIG. 3 . According to the second technique, since a different APS is used for each parameter, redundant parameters are not transmitted for parameters that do not need to be updated. This allows for optimization of coding efficiency. However, in the second technique, as the number of parameter categories to be included in the APS increases, the number of NAL unit types (nal_unit_type), identifiers used to identify APS categories, increases. The HEVC standard specification reserves a limited number of NAL unit types (nal_unit_type) for extension. Therefore, it is advantageous to consider avoiding the expansion of a structure with multiple NAL unit types used for the APS.

[6-4. Third Technology]

The third technique includes quantization matrix parameters and other parameters in the APS, and groups these parameters using identifiers defined independently from the APS ID. In this specification, the identifier assigned to each group and defined independently from the APS ID is referred to as a sub-identifier (SUB ID). Each parameter is referenced using the sub-identifier in the slice header. Figure 29 shows an example of a coded stream configured according to the third technique.

29 , an SPS 821, a PPS 822, and an APS 823 are inserted at the beginning of picture P0 at the beginning of the sequence. PPS 822 is identified by PPS ID "P0." APS 823 includes parameters related to ALF, SAO, and QM. The ALF parameters belong to a single group and are identified by a sub-identifier for ALF, SUB_ALF ID "AA0." The SAO parameters belong to a single group and are identified by a sub-identifier for SAO, SUB_SAO ID "AS0." The QM parameters belong to a single group and are identified by a sub-identifier for QM, SUB_QM ID "AQ0." The slice header 824 attached to the slice data in picture P0 includes references to SUB_ALF ID "AA0," SUB_SAO ID "AS0," and SUB_QM ID "AQ0." This means that parameters regarding ALF belonging to SUB_ALF ID “AA0”, parameters regarding SAO belonging to SUB_SAO ID “AS0”, and parameters regarding QM belonging to SUB_QM ID “AQ0” are referenced to decode the segment data.

APS 825 is inserted into picture P1, which follows picture P0. APS 825 includes parameters related to ALF and SAO. ALF parameters are identified by SUB_ALF ID "AA1." SAO parameters are identified by SUB_SAO ID "AS1." Since QM parameters are not updated from picture P0, QM parameters are not included in APS 825. The slice header 826 attached to the slice data in picture P1 includes a reference to SUB_ALF ID "AA1," a reference to SUB_SAO ID "AS1," and a reference to SUB_QM ID "AQ0." This indicates that the ALF parameters associated with SUB_ALF ID "AA1" in APS 825, the SAO parameters associated with SUB_SAO ID "AS1," and the QM parameters associated with SUB_QM ID "AQ0" in APS 823 are referenced for decoding the slice data.

APS 827 is inserted into picture P2, which follows picture P1. APS 827 includes parameters related to the ALF and QM. The ALF parameters are identified by the SUB_ALF ID "AA2." The QM parameters are identified by the SUB_QM ID "AQ1." Since the SAO parameters are not updated from picture P1, the SAO parameters are not included in APS 827. The slice header 828 attached to the slice data in picture P2 includes a reference to the SUB_ALF ID "AA2," a reference to the SUB_SAO ID "AS1," and a reference to the SUB_QM ID "AQ1." This indicates that the ALF parameters associated with the SUB_ALF ID "AA2," the QM parameters associated with the SUB_QM ID "AQ1," and the SAO parameters associated with the SUB_SAO ID "AS1" in APS 825 are referenced for decoding the slice data.

FIG30 illustrates an example of the APS syntax defined according to the third technique. Lines 2 through 4 of FIG30 specify three group presence flags: "aps_adaptive_loop_filter_flag," "aps_sample_adaptive_offset_flag," and "aps_qmatrix_flag." The group presence flags indicate whether the APS includes parameters belonging to each group. Although the APS ID is omitted from the syntax in the example of FIG30 , it may be added to the syntax to identify the APS. Lines 12 through 17 specify parameters related to the ALF. "sub_alf_id" on line 13 is a supplementary identifier for the ALF. Lines 18 through 24 specify parameters related to the SAO. "sub_sao_id" on line 19 is a supplementary identifier for the SAO. Lines 25 through 30 specify parameters related to the QM. "sub_qmatrix_id" on line 26 is a supplementary identifier for the QM. “qmatrix_param()” on line 29 is a function that specifies the quantization matrix parameters shown by way of example in FIGS. 13 to 20 .

FIG31 is an explanatory diagram showing an example of the slice header syntax defined according to the third technique. In line 5 of FIG31 , a reference PPS ID for referencing parameters included in the PPS is specified from the parameters to be set for the slice. In line 8, a reference SUB_ALF ID for referencing parameters related to the ALF is specified from the parameters to be set for the slice. In line 9, a reference SUB_SAO ID for referencing parameters related to the SAO is specified from the parameters to be set for the slice. In line 10, a reference SUB_QM ID for referencing parameters related to the QM is specified from the parameters to be set for the slice.

When implementing the third technique, the parameter generation unit 130 of the syntax processing unit 13 of the image encoding device 10 appends a new SUB_QM ID as an auxiliary identifier to the updated set of quantization matrix parameters each time the quantization matrix parameters are updated. The insertion unit 130 then inserts the quantization matrix parameters with the appended SUB_QM ID into the APS along with other parameters. The parameter acquisition unit 160 of the syntax processing unit 61 of the image decoding device 60 uses the reference SUB_QM ID specified in the slice header to acquire the quantization matrix parameters to be set for each slice from the APS.

According to the third technique, by grouping parameters in an APS using auxiliary identifiers, redundant parameters are not transmitted for parameters in a group that do not need to be updated. This optimizes coding efficiency. Furthermore, since the number of APS types does not increase even when the number of parameter types increases, the large number of NAL unit types is not consumed as in the second technique described above. Consequently, the third technique does not compromise flexibility for future expansion.

In the examples of Figures 29 to 31, the parameters included in the APS are grouped according to the coding tools related to ALF, SAO, and QM. However, this is merely one example of grouping parameters. The APS may include parameters related to other coding tools. For example, parameters related to the adaptive interpolation filter (AIF), such as filter coefficients for the AIF, are one example of parameters that may be included in the APS. Hereinafter, various criteria for grouping parameters to be included in the APS will be described with reference to Figure 32.

The table shown in FIG32 lists “parameter contents”, “update frequency”, and “data size” as characteristics of each parameter in a general encoding tool.

Adaptive loop filtering (ALF) is a type of filtering (typically a Wiener filter) that applies two-dimensional filtering to the decoded image using adaptively determined filter coefficients to minimize the error between the decoded image and the original image. ALF parameters include the filter coefficients applied to each block and an on/off flag for each coding unit (CU). The data size of ALF filter coefficients is significantly larger than that of other types of parameters. For this reason, ALF parameters are typically transmitted for high-rate I pictures, while transmission of ALF parameters for low-rate B pictures can be omitted. This is because transmitting large ALF parameters for low-rate pictures is inefficient from a performance perspective. In most cases, the ALF filter coefficients change for each picture. Because the filter coefficients depend on the image content, the likelihood of reusing previously set filter coefficients is low.

Sample Adaptive Offset (SAO) is a tool for improving the image quality of decoded images by adding an adaptively determined offset value to each pixel value in the decoded image. SAO parameters include an offset pattern and an offset value. The data size of SAO parameters is smaller than that of ALF parameters. Generally speaking, SAO parameters also change for each picture. However, since SAO parameters do not change significantly even with slight changes in image content, it is possible to reuse previously set parameter values.

The quantization matrix (QM) is a matrix whose elements are the quantization scale used when quantizing transform coefficients transformed from image data through an orthogonal transform. The parameters regarding the QM, or in other words, the quantization matrix parameters, are described in detail in this specification. The data size of the parameters regarding the QM is larger than that of the parameters regarding the SAO. As a general rule, the quantization matrix is required for all pictures, but if the image content does not change significantly, it is not necessary to update the quantization matrix for each picture. For this reason, the quantization matrix can be reused for the same picture type (such as I/P/B pictures) or for each GOP.

Adaptive interpolation filtering (AIF) adaptively changes the filter coefficients of the interpolation filter used during motion compensation for each sub-pixel position. AIF parameters include the filter coefficients for each sub-pixel position. The data size of AIF parameters is smaller than the three parameters mentioned above. Generally speaking, AIF parameters vary for each picture. However, since pictures of the same type tend to have similar interpolation properties, AIF parameters can be reused for the same picture type (such as I/P/B pictures).

Based on the above parameter qualities, the following three criteria may be used to group the parameters included in the APS:

Criterion A) Grouping by Coding Tool

Criterion B) Grouping by update frequency

Criterion C) Grouping by the likelihood of parameter reuse.

Criteria A groups parameters according to their associated coding tools. The parameter set structures illustrated in Figures 29 through 31 are based on Criteria A. Because parameter quality is typically determined based on its associated coding tool, grouping parameters by coding tool allows for timely and efficient parameter updates based on their varying qualities.

Standard B groups parameters according to their update frequency. As shown in Figure 32, as a general rule, parameters related to ALF, SAO, and AIF can all be updated for each picture. Therefore, these parameters can be grouped into a single group, while parameters related to QM can be grouped into another group. In this case, there are fewer groups compared to Standard A. Consequently, there are fewer auxiliary identifiers that need to be specified in the slice header, and the amount of code in the slice header can be reduced. Furthermore, since the update frequencies of parameters belonging to the same group are similar, the likelihood of redundantly transmitting parameters that are not updated in order to update other parameters is low.

Criterion C groups parameters based on their potential for reuse. While ALF parameters are unlikely to be reused, SAO and AIF parameters are somewhat likely to be reused. QM parameters are highly likely to be reused across multiple pictures. Therefore, by grouping parameters based on their potential for reuse, redundant transmission of reused parameters in APS can be avoided.

[6-5. Example modification of the third technique]

With the third technique described above, as shown in the example of FIG31 , the number of groups into which parameters are grouped in the APS results in the same number of reference SUB IDs being specified in the slice header. The amount of code required for the reference SUB IDs is approximately proportional to the product of the number of slice headers and the number of groups. A modified technique, as shown in the example below, can also be implemented to further reduce this ratio.

In an example modification of the third technique, a combination ID associated with a combination of auxiliary identifiers is defined in an APS or other parameter set. The parameters included in the APS can then be referenced from the slice header using the combination ID. FIG33 shows an example of a coded stream configured according to an example modification of the third technique.

33 , SPS 831, PPS 832, and APS 833 are inserted at the beginning of picture P0, which is located at the beginning of the sequence. PPS 832 is identified by PPS ID "P0." APS 833 includes parameters related to ALF, SAO, and QM. Parameters related to ALF are identified by SUB_ALF ID "AA0." Parameters related to SAO are identified by SUB_SAO ID "AS0." Parameters related to QM are identified by SUB_QM ID "AQ0." Furthermore, APS 833 includes a combination ID "C00" = {AA0, AS0, AQ0} as a definition of a combination. The slice header 834 attached to the slice data in picture P0 includes the combination ID "C00." This means that parameters regarding ALF belonging to SUB_ALF ID "AA0", parameters regarding SAO belonging to SUB_SAO ID "AS0", and parameters regarding QM belonging to SUB_QM ID "AQ0" respectively associated with combination ID "C00" are referenced to decode the segment data.

APS 835 is inserted into picture P1, which follows picture P0. APS 835 includes parameters related to ALF and parameters related to SAO. Parameters related to ALF are identified by SUB_ALF ID "AA1." Parameters related to SAO are identified by SUB_SAO ID "AS1." Since parameters related to QM are not updated from picture P0, they are not included in APS 835. In addition, APS 835 includes combination IDs "C01" = {AA1, AS0, AQ0}, "C02" = {AA0, AS1, AQ0}, and "C03" = {AA1, AS1, AQ0} as definitions of combinations. The slice header 836 attached to the slice data in picture P1 includes the combination ID "C03." This means that parameters regarding ALF belonging to SUB_ALF ID "AA1", parameters regarding SAO belonging to SUB_SAO ID "AS1", and parameters regarding QM belonging to SUB_QM ID "AQ0", which are respectively associated with combination ID "C03", are referenced to decode the segment data.

APS 837 is inserted into picture P2, which follows picture P1. APS 837 includes parameters related to the ALF. These parameters are identified by the SUB_ALF ID "AA2." Since the SAO and QM parameters are not updated from picture P1, APS 837 does not include SAO and QM parameters. Furthermore, APS 837 includes combination IDs "C04" = {AA2, AS0, AQ0} and "C05" = {AA2, AS1, AQ0} as definitions of combinations. The slice header 838 attached to the slice data in picture P2 includes the combination ID "C05." This indicates that the ALF parameters associated with SUB_ALF ID "AA2," the SAO parameters associated with SUB_SAO ID "AS1," and the QM parameters associated with SUB_QM ID "AQ0," respectively associated with combination ID "C05," are referenced for decoding the slice data.

Note that in this example modification, combination IDs may not be defined for all combinations of auxiliary identifiers, and thus may be defined only for combinations of auxiliary identifiers actually referenced in the segment header. In addition, combinations of auxiliary identifiers may be defined in an APS different from the APS storing the corresponding parameters.

In implementing this exemplary modification, the parameter generation unit 130 of the syntax processing unit 13 of the image encoding device 10 generates a combination ID as a supplementary parameter, which is associated with a combination of auxiliary identifiers that group various parameters, including the quantization matrix parameter. The insertion unit 130 then inserts the combination ID generated by the parameter generation unit 130 into the APS or other parameter set. The parameter acquisition unit 160 of the syntax processing unit 61 of the image decoding device 60 acquires the combination ID specified in the slice header of each slice and uses the auxiliary identifier associated with the combination ID to separately acquire the quantization matrix parameter in the APS.

In this way, by using a combination ID associated with a combination of auxiliary identifiers to reference parameters in the APS, the amount of code required to reference each parameter from the slice header can be reduced.

7. Example Application

The image encoding device 10 and image decoding device 60 according to the above-described embodiments can be applied to various electronic devices, such as transmitters and receivers for satellite broadcasting, cable broadcasting such as cable television, Internet distribution, and distribution to client devices via cellular communications; recording devices for recording images onto media such as optical disks, magnetic disks, or flash memory; and playback devices for playing back images from these storage media. Four example applications will be described below.

[7-1. First Example Application]

34 is a block diagram showing an exemplary schematic configuration of a television employing the above-described embodiment. A television 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, an external interface 909, a control unit 910, a user interface 911, and a bus 912.

The tuner 902 extracts a signal of a desired channel from the broadcast signal received via the antenna 901 and demodulates the extracted signal. The tuner 902 then outputs the encoded bit stream obtained by demodulation to the demultiplexer 903. That is, the tuner 902 serves as a transmission device of the television 900 for receiving an encoded stream in which an image is encoded.

The demultiplexer 903 separates the video stream and audio stream of the program to be viewed from the encoded bit stream, and outputs the separated streams to the decoder 904. In addition, the demultiplexer 903 extracts auxiliary data such as an electronic program guide (EPG) from the encoded bit stream, and supplies the extracted data to the control section 910. In addition, when the encoded bit stream is scrambled, the demultiplexer 903 can perform descrambling.

The decoder 904 decodes the video stream and audio stream input from the demultiplexer 903. The decoder 904 then outputs the video data generated by the decoding process to the video signal processing section 905. In addition, the decoder 904 outputs the audio data generated by the decoding process to the audio signal processing section 907.

The video signal processing unit 905 plays back the video data input from the decoder 904 and causes the display unit 906 to display the video. The video signal processing unit 905 may also cause the display unit 906 to display an application screen provided via the network. Furthermore, the video signal processing unit 905 may perform other processing, such as noise removal according to settings on the video data. Furthermore, the video signal processing unit 905 may generate a graphical user interface (GUI) image, such as a menu, button, or cursor, and superimpose the generated image on the output image.

The display section 906 is driven by the driving signal supplied from the video signal processing section 905 , and displays a video or an image on a display screen of a display device such as a liquid crystal display, a plasma display, or an OLED display.

The audio signal processing section 907 performs playback processing such as D/A conversion and amplification on the audio data input from the decoder 904, and outputs audio from the speaker 908. In addition, the audio signal processing section 907 may perform other processing such as noise removal on the audio data.

The external interface 909 is an interface for connecting the television 900 to an external device or a network. For example, a video stream or an audio stream received via the external interface 909 can be decoded by the decoder 904. That is, the external interface 909 also serves as a transmission device of the television 900 for receiving an encoded stream in which an image is encoded.

The control unit 910 includes a processor such as a central processing unit (CPU), and memory such as random access memory (RAM) and read-only memory (ROM). The memory stores programs to be executed by the CPU, program data, EPG data, data obtained via a network, and the like. When the television 900 is turned on, for example, the CPU reads and executes the programs stored in the memory. By executing the programs, the CPU controls the operation of the television 900 based on operation signals input from, for example, the user interface 911.

The user interface 911 is connected to the control unit 910. The user interface 911 includes, for example, buttons and switches used by the user to operate the television 900, and a remote control signal receiver. The user interface 911 detects user operations via these components, generates operation signals, and outputs the generated operation signals to the control unit 910.

The bus 912 connects the tuner 902 , the demultiplexer 903 , the decoder 904 , the video signal processing section 905 , the audio signal processing section 907 , the external interface 909 , and the control section 910 to one another.

In the television 900 thus configured, the decoder 904 includes the function of the image decoding device 60 according to the above-described embodiment. Therefore, it is possible to alleviate a decrease in encoding efficiency for a video decoded by the television 900 or improve encoding efficiency.

[7-2. Second Example Application]

FIG35 is a block diagram showing an exemplary schematic configuration of a mobile phone employing the above-described embodiment. The mobile phone 920 includes an antenna 921, a communication section 922, an audio codec 923, a speaker 924, a microphone 925, a camera section 926, an image processing section 927, a multiplexing/demultiplexing (mux/demux) section 928, a recording and playback section 929, a display section 930, a control section 931, an operation section 932, and a bus 933.

The antenna 921 is connected to the communication section 922. The speaker 924 and the microphone 925 are connected to the audio codec 923. The operation section 932 is connected to the control section 931. The bus 933 connects the communication section 922, the audio codec 923, the camera section 926, the image processing section 927, the mux/demux section 928, the recording and playback section 929, the display 930, and the control section 931 to each other.

The mobile phone 920 performs operations such as transmitting and receiving audio signals, transmitting and receiving emails or image data, capturing images, recording data, etc. in various operation modes including an audio communication mode, a data communication mode, an imaging mode, and a video phone mode.

In audio communication mode, an analog audio signal generated by microphone 925 is provided to audio codec 923. Audio codec 923 converts the analog audio signal into audio data, performs A/D conversion on the converted audio data, and compresses the audio data. Audio codec 923 then outputs the compressed audio data to communication unit 922. Communication unit 922 encodes and modulates the audio data to generate a transmission signal. Communication unit 922 then transmits the generated transmission signal to a base station (not shown) via antenna 921. Furthermore, communication unit 922 amplifies a wireless signal received via antenna 921, converts the frequency of the wireless signal, and acquires the received signal. Communication unit 922 then demodulates and decodes the received signal to generate audio data, which it then outputs to audio codec 923. Audio codec 923 decompresses and D/A converts the audio data to generate an analog audio signal. Audio codec 923 then supplies the generated audio signal to speaker 924 for audio output.

In data communication mode, for example, the control unit 931 generates text data constituting an email based on user operations via the operation unit 932. Furthermore, the control unit 931 causes the text to be displayed on the display unit 930. Furthermore, the control unit 931 generates email data based on a transmission instruction from the user via the operation unit 932 and outputs the generated email data to the communication unit 922. The communication unit 922 encodes and modulates the email data to generate a transmission signal. The communication unit 922 then transmits the generated transmission signal to a base station (not shown) via the antenna 921. Furthermore, the communication unit 922 amplifies the wireless signal received via the antenna 921, converts the frequency of the wireless signal, and acquires the received signal. The communication unit 922 then demodulates and decodes the received signal to reconstruct the email data and outputs the reconstructed email data to the control unit 931. The control unit 931 causes the display unit 930 to display the contents of the email and also causes the email data to be stored in the storage medium of the recording and playback unit 929.

The recording and playback section 929 includes a storage medium that can be read and written arbitrarily. For example, the storage medium may be a built-in storage medium such as RAM, or a flash memory, or an externally mounted storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.

In addition, in the imaging mode, the camera section 926 captures an image of a subject, generates image data, and outputs the generated image data to, for example, the image processing section 927. The image processing section 927 encodes the image data input from the camera section 926 and causes the encoded stream to be stored on the storage medium of the recording and playback section 929.

In videophone mode, for example, the mux/demux unit 928 multiplexes the video stream encoded by the image processing unit 927 and the audio stream input from the audio codec 923, and outputs the multiplexed stream to the communication unit 922. The communication unit 922 encodes and modulates the stream to generate a transmission signal. The communication unit 922 then transmits the generated transmission signal to a base station (not shown) via the antenna 921. Furthermore, the communication unit 922 amplifies the wireless signal received via the antenna 921, converts the frequency of the wireless signal, and obtains the received signal. The transmission signal and the received signal may include an encoded bit stream. The communication unit 922 then demodulates and decodes the received signal to reconstruct the stream and output the reconstructed stream to the mux/demux unit 928. The mux/demux unit 928 separates the video stream and the audio stream from the input stream, outputs the video stream to the image processing unit 927, and the audio stream to the audio codec 923. The image processing unit 927 decodes the video stream to generate video data. The video data is supplied to the display unit 930, and a series of images are displayed on the display unit 930. The audio codec 923 decompresses and D/A converts the audio stream and generates an analog audio signal. The audio codec 923 then supplies the generated audio signal to the speaker 924 so that the audio is output.

In the mobile phone 920 thus configured, the image processing section 927 has the functions of the image encoding device 10 and the image decoding device 60 according to the above-described embodiment. Therefore, it is possible to alleviate a decrease in encoding efficiency for videos encoded and decoded by the mobile phone 920 or improve encoding efficiency.

[7-3. Third Example Application]

FIG36 is a block diagram illustrating an exemplary schematic configuration of a recording and playback device employing the above-described embodiment. The recording and playback device 940 encodes and records, for example, audio and video data of a received broadcast program onto a recording medium. The recording and playback device 940 can also encode and record, for example, audio and video data acquired from another device onto a recording medium. Furthermore, the recording and playback device 940 plays back data recorded on the recording medium via a monitor and speakers, for example, in response to user instructions. In this case, the recording and playback device 940 decodes the audio and video data.

The recording and playback device 940 includes a tuner 941 , an external interface 942 , an encoder 943 , a hard disk drive (HDD) 944 , a disk drive 945 , a selector 946 , a decoder 947 , an on-screen display (OSD) 948 , a control section 949 , and a user interface 950 .

The tuner 941 extracts a signal of a desired channel from a broadcast signal received via an antenna (not shown) and demodulates the extracted signal. The tuner 941 then outputs a coded bit stream obtained by demodulation to the selector 946. That is, the tuner 941 functions as a transmission device of the recording and playback device 940.

The external interface 942 is an interface for connecting the recording and playback device 940 to an external device or a network. The external interface 942 may be, for example, an IEEE 1394 interface, a network interface, a USB interface, a flash memory interface, or the like. For example, audio data and video data received via the external interface 942 are input to the encoder 943. That is, the external interface 942 serves as a transmission device of the recording and playback device 940.

In the case where the video data and audio data input from the external interface 942 are not encoded, the encoder 943 encodes the video data and audio data, and then outputs the encoded bit stream to the selector 946 .

The HDD 944 records a coded bit stream, which is compressed content data such as video or audio, various programs and other data, on an internal hard disk. In addition, the HDD 944 reads the data from the hard disk when playing back video and audio.

The disk drive 945 records or reads data with respect to a recording medium inserted. The recording medium inserted into the disk drive 945 may be, for example, a DVD disk (such as a DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD+, or DVD+RW disk), a Blu-ray Disc (registered trademark), or the like.

When recording video and audio, the selector 946 selects an encoded bit stream input from the tuner 941 or the encoder 943 and outputs the selected encoded bit stream to the HDD 944 or the disk drive 945. In addition, when playing back video and audio, the selector 946 outputs the encoded bit stream input from the HDD 944 or the disk drive 945 to the decoder 947.

The decoder 947 decodes the encoded bit stream and generates audio data and video data. The decoder 947 then outputs the generated video data to the OSD 948. In addition, the decoder 904 outputs the generated audio data to an external speaker.

The OSD 948 plays back the video data input from the decoder 947 and displays the video. In addition, the OSD 948 may superimpose a GUI image such as a menu, a button, or a cursor on the displayed video.

The control unit 949 includes a processor such as a CPU and a memory such as a RAM or ROM. The memory stores programs executed by the CPU, program data, and the like. When the recording and playback device 940 is activated, for example, the CPU reads and executes the program stored in the memory. By executing the program, the CPU controls the operation of the recording and playback device 940 based on an operation signal input from, for example, the user interface 950.

The user interface 950 is connected to the control unit 949. The user interface 950 includes, for example, buttons and switches used by the user to operate the recording and playback device 940, and a remote control signal receiver. The user interface 950 detects the user's operation via these structural elements, generates an operation signal, and outputs the generated operation signal to the control unit 949.

In the recording and playback device 940 configured in this manner, the encoder 943 includes the functions of the image encoding device 10 according to the above-described embodiment. In addition, the decoder 947 has the functions of the image decoding device 60 according to the above-described embodiment. Therefore, it is possible to alleviate a decrease in the encoding efficiency of a video encoded and decoded by the recording and playback device 940, or to improve the encoding efficiency.

[7-4. Fourth Example Application]

37 is a block diagram showing an exemplary schematic configuration of an imaging device employing the above-described embodiment. The imaging device 960 captures an image of a subject, generates an image, encodes the image data, and records the image data on a recording medium.

The imaging device 960 includes an optical block 961 , an imaging section 962 , a signal processing section 963 , an image processing section 964 , a display section 965 , an external interface 966 , a memory 967 , a media drive 968 , an OSD 969 , a control section 970 , a user interface 971 , and a bus 972 .

The optical block 961 is connected to the imaging section 962. The imaging section 962 is connected to the signal processing section 963. The display section 965 is connected to the image processing section 964. The user interface 971 is connected to the control section 970. The bus 972 connects the image processing section 964, the external interface 966, the memory 967, the media drive 968, the OSD 969, and the control section 970 to each other.

The optical block 961 includes a focusing lens, an aperture stop mechanism, and the like. The optical block 961 forms an optical image of a subject on the imaging surface of the imaging section 962. The imaging section 962 includes an image sensor such as a CCD or CMOS sensor, and photoelectrically converts the optical image formed on the imaging surface into an image signal, which is an electrical signal. The imaging section 962 then outputs the image signal to the signal processing section 963.

The signal processing section 963 performs various camera signal processes such as knee correction, gamma correction, color correction, etc. on the image signal input from the imaging section 962 . The signal processing section 963 outputs the processed image data to the image processing section 964 .

The image processing unit 964 encodes the image data input from the signal processing unit 963 and generates encoded data. The image processing unit 964 then outputs the generated encoded data to the external interface 966 or the media drive 968. The image processing unit 964 also decodes the encoded data input from the external interface 966 or the media drive 968 and generates image data. The image processing unit 964 then outputs the generated image data to the display unit 965. In addition, the image processing unit 964 can also output the image data input from the signal processing unit 963 to the display unit 965 so that the image is displayed. In addition, the image processing unit 964 can superimpose the display data obtained from the OSD 969 on the image to be output to the display unit 965.

The OSD 969 generates a GUI image such as a menu, a button, or a cursor, and outputs the generated image to the image processing section 964 .

The external interface 966 is configured as, for example, a USB input/output terminal. The external interface 966 connects the imaging device 960 to a printer, for example, when printing an image. Furthermore, a drive is connected to the external interface 966 as needed. Removable media such as a magnetic disk or optical disk is inserted into the drive, and programs read from the removable media can be installed in the imaging device 960. Furthermore, the external interface 966 may be configured as a network interface for connecting to a network such as a LAN or the Internet. In other words, the external interface 966 functions as a transmission device for the image capturing device 960.

The recording medium to be inserted into the media drive 968 may be any removable medium that can read and write, such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. In addition, the recording medium may be permanently installed in the media drive 968 to constitute a non-portable storage unit such as an internal hard disk drive or a solid-state drive (SSD).

The control unit 970 includes a processor such as a CPU and a memory such as RAM or ROM. The memory stores programs executed by the CPU, program data, and the like. When the imaging device 960 is activated, for example, the CPU reads and executes the program stored in the memory. By executing the program, the CPU controls the operation of the imaging device 960 based on an operation signal input from, for example, the user interface 971.

The user interface 971 is connected to the control section 970. The user interface 971 includes buttons, switches, and the like used by the user to operate, for example, the imaging device 960. The user interface 971 detects the user's operation via these structural elements, generates an operation signal, and outputs the generated operation signal to the control section 970.

In the imaging device 960 configured in this manner, the image processing section 964 has the functions of the image encoding device 10 and the image decoding device 60 according to the above-described embodiment. Therefore, it is possible to alleviate a decrease in encoding efficiency for video encoded and decoded by the imaging device 960 or improve encoding efficiency.

8. Conclusion

The image encoding device 10 and image decoding device 60 according to the embodiment have been described above using Figures 1 to 37. According to the embodiment, quantization matrix parameters defining the quantization matrix used when quantizing and inverse quantizing an image are inserted into a quantization matrix parameter set (QMPS) that is different from the sequence parameter set and the picture parameter set. This eliminates the need to encode parameters other than the quantization matrix parameters when updating the quantization matrix, and vice versa. Consequently, the decrease in coding efficiency associated with updating the quantization matrix is mitigated, or coding efficiency is improved. Specifically, in the case of a quantization matrix with a larger size, or in the case of a large number of quantization matrices defined for each picture, the techniques disclosed in this specification become more effective in reducing the amount of code.

Furthermore, according to an embodiment of the present invention, instead of directly defining the quantization matrix, the QMPS may include parameters specifying a copy of a previously generated quantization matrix. In this case, the parameters specifying the quantization matrix itself (e.g., a differential data array in DPCM format) are omitted from the QMPS, further reducing the amount of code required to define the quantization matrix.

Furthermore, according to an embodiment of the present invention, a QMPS ID is assigned to each QMPS. In copy mode, the QMPS ID of the QMPS that defines the copy-source quantization matrix can be specified as the source ID. Furthermore, the size and type of the copy-source quantization matrix can be specified as the copy-source size and copy-source type. This allows the quantization matrix in any QMPS from among the quantization matrices in multiple previously generated QMPSs to be flexibly designated as the copy-source quantization matrix. Quantization matrices of different sizes or types can also be copied and reused.

In addition, according to an embodiment of the present invention, a parameter specifying a residual component of the quantization matrix to be copied may be included in the QMPS. Thus, a quantization matrix that is not completely equal to a previously generated quantization matrix may be newly generated at a low rate.

In addition, in axis-specific mode, instead of scanning all elements of the quantization matrix, the QMPS can only include the values of the elements corresponding to the three reference axes or four corners of the quantization matrix. Thus, in this case, the quantization matrix can also be defined with a small amount of code.

In addition, according to an embodiment of the present invention, the quantization matrix used for each slice is set based on the quantization matrix parameters in the QMPS identified by the QMPS ID specified in the slice header. Therefore, since the quantization matrix can be flexibly switched for each slice, even in the case where the image characteristics change over time, the optimal quantization matrix at each time point can be used to encode or decode the video.

Note that this disclosure describes an example in which quantization matrix parameter sets are multiplexed into the header of the coded stream and transmitted from the encoding side to the decoding side. However, the technique for transmitting quantization matrix parameter sets is not limited to this example. For example, the information in each parameter set can be transmitted or recorded as separate data associated with the coded bitstream, rather than multiplexed into the coded bitstream. Here, the term "associated" means that the images (including partial images such as slices or blocks) included in the bitstream can be associated with the information corresponding to these images during decoding. In other words, the information can also be transmitted over a transmission channel separate from the images (or bitstream). Alternatively, the information can be recorded in a separate recording medium (or a separate recording area within the same recording medium) from the images (or bitstream). Furthermore, the information and the images (or bitstream) can be associated with each other in arbitrary units, such as multiple frames, a single frame, or a portion within a frame.

The preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings. However, the technical scope of the present invention is not limited to the examples described. It will be apparent to those skilled in the art that various modifications and substitutions may be made as long as they fall within the scope of the technical concept described in the claims, and it will be understood that such modifications or substitutions clearly fall within the technical scope of the present invention.

Additionally, the technology of the present invention can also be configured as follows.

(1) An image processing device comprising:

an acquisition unit configured to acquire a quantization matrix parameter from a coded stream, the quantization matrix parameter defining the quantization matrix being set in a parameter set different from a sequence parameter set and a picture parameter set;

a setting section configured to set a quantization matrix used when inverse quantizing data decoded from an encoded stream, based on the quantization matrix parameters acquired by the acquisition section; and

The inverse quantization section is configured to inversely quantize data decoded from the encoded stream using the quantization matrix set by the setting section.

(2) The image processing apparatus according to (1), wherein

The parameter set including the quantization matrix parameters is a common parameter set, which can be used to set other coding parameters related to coding tools other than the quantization matrix, and

When a quantization matrix parameter is set in the common parameter set, the acquisition section acquires the quantization matrix parameter.

(3) The image processing apparatus according to (2), wherein

The acquisition section determines whether a quantization matrix parameter is set in the common parameter set by referring to a flag included in the common parameter set.

(4) The image processing apparatus according to (2) or (3), wherein

The common parameter set is an adaptation parameter set.

(5) The image processing apparatus according to (4), wherein

In a case where a second adaptation parameter set decoded after the first adaptation parameter set includes a reference to the first adaptation parameter set, the setting unit reuses the quantization matrix set based on the quantization matrix parameters obtained from the first adaptation parameter set as the quantization matrix corresponding to the second adaptation parameter set.

(6) The image processing apparatus according to (5), wherein

In a case where a reference to a third adaptation parameter set is included in a third adaptation parameter set, the setting section sets a default quantization matrix as a quantization matrix corresponding to the third adaptation parameter set.

(7) The image processing apparatus according to (1), wherein

In a case where a copy parameter for instructing to copy a first quantization matrix of the first parameter set is included in the second parameter set, the setting section sets the second quantization matrix by copying the first quantization matrix.

(8) The image processing apparatus according to (7), wherein

Each parameter set including the quantization matrix parameters has an identifier for identifying each parameter set, and

The copy parameters include an identifier of a parameter set of a copy source.

(9) The image processing apparatus according to (8), wherein

Each parameter set includes quantization matrix parameters that define quantization matrices of multiple categories, and

The copy parameters include a category parameter specifying a category of the first quantization matrix.

(10) The image processing apparatus according to (8), wherein

The setting section sets a default quantization matrix as a third quantization matrix for the third parameter set in a case where an identifier of a parameter set of a copy source included in the third parameter set is equal to an identifier of the third parameter set.

(11) The image processing apparatus according to (7), wherein

In a case where a size of the second quantization matrix is larger than a size of the first quantization matrix, the setting section sets the second quantization matrix by interpolating elements of the copied first quantization matrix.

(12) The image processing apparatus according to (7), wherein

In a case where a size of the second quantization matrix is smaller than a size of the first quantization matrix, the setting section sets the second quantization matrix by reducing elements of the copied first quantization matrix.

(13) The image processing apparatus according to (7), wherein

In a case where a residual designation parameter designating a residual component of the copied quantization matrix is included in the second parameter set, the setting section sets the second quantization matrix by adding the residual component to the copied first quantization matrix.

(14) The image processing apparatus according to (1), wherein

Each parameter set including the quantization matrix parameters has a parameter set identifier for identifying each parameter set, and

The inverse quantization section uses, for each slice, a quantization matrix set by the setting section based on quantization matrix parameters included in a parameter set identified by a parameter set identifier specified in a slice header.

(15) The image processing device according to any one of (7) to (14), wherein

The parameter set containing the quantization matrix parameters may also include other coding parameters related to coding tools other than the quantization matrix.

(16) The image processing apparatus according to (15), wherein

The quantization matrix parameters and the other encoding parameters are grouped by an auxiliary identifier defined separately from a parameter identifier identifying each parameter set, and

The acquisition section acquires a quantization matrix parameter using the auxiliary identifier.

(17) The image processing apparatus according to (16), wherein

defining a combined identifier associated with a combination of multiple auxiliary identifiers in a parameter set or in another parameter set, and

The acquisition section acquires a combination identifier specified in a slice header of each slice, and acquires a quantization matrix parameter using an auxiliary identifier associated with the acquired combination identifier.

(18) An image processing method comprising:

Acquire a quantization matrix parameter from a coded stream, wherein the quantization matrix parameter defining the quantization matrix is set in a parameter set different from a sequence parameter set and a picture parameter set;

setting a quantization matrix used when inverse quantizing data decoded from an encoded stream based on the acquired quantization matrix parameter; and

The data decoded from the encoded stream is inversely quantized using the set quantization matrix.

(19) An image processing device comprising:

a quantization unit configured to quantize data using a quantization matrix;

a setting section configured to set a quantization matrix parameter that defines a quantization matrix used when the quantization section quantizes the data; and

The encoding unit is configured to encode the quantization matrix parameters set by the setting unit in a parameter set different from the sequence parameter set and the picture parameter set.

(20) An image processing method comprising:

quantize the data using a quantization matrix;

Setting a quantization matrix parameter, the quantization matrix parameter defining a quantization matrix to be used when quantizing the data; and

The set quantization matrix parameters are encoded in a parameter set different from the sequence parameter set and the picture parameter set.

Reference Signs List

10 Image Processing Device (Image Coding Device)

16 Quantitative Department

120 Parameter Generation Unit

130 Insertion

60 Image Processing Device (Image Decoding Device)

63 Inverse Quantization Unit

160 Parameter Acquisition Unit

170 Settings

Claims (36)

1. An image processing apparatus, comprising: At least one processor; At least one memory including computer program code, the memory and computer program code being configured to cause the device to perform at least the following steps when working in conjunction with a processor: The first quantization matrix and the second quantization matrix are set as quantization matrices for quantizing the first transform coefficient data of the first image; The second quantization matrix is set by copying the first quantization matrix to the second quantization matrix; The first transform coefficient data is quantized using the second quantization matrix to generate the first quantized transform coefficient data; and Encode the first quantized transform coefficient data into a first encoded stream; Set a third quantization matrix; Decode the second quantized transform coefficient data from the second encoded stream; The fourth quantization matrix is set by copying the third quantization matrix to the fourth quantization matrix; and The fourth quantization matrix is used to inverse quantize the second quantized transform coefficient data decoded from the second encoded stream to generate inverse quantized transform coefficient data. 2. The device according to claim 1, wherein the first quantization matrix corresponds to the value of the first identification information, and the second quantization matrix corresponds to the value of the second identification information. 3. The device of claim 2, wherein when the first quantization matrix is copied to the second quantization matrix, the memory includes computer program code configured to cause the device to generate a value of first identification information that is different from the value of the second identification information when working with the processor. 4. The device of claim 3, wherein the memory includes computer program code configured to cause the device, when working with the processor, to determine that the value of the first identification information is different from the value of the second identification information, wherein the second quantization matrix is set by copying the first quantization matrix based on the determination that the value of the first identification information is different from the value of the second identification information. 5. The device of claim 1, wherein the memory includes computer program code configured to cause the device to decode third identification information from a second encoded stream when working with a processor, the third identification information identifying whether the third quantization matrix is copied to the fourth quantization matrix. 6. The device of claim 5, wherein the memory includes computer program code configured to: cause the device, when working with a processor, to determine that third identification information identifies that the third quantization matrix is copied to the fourth quantization matrix, wherein the fourth quantization matrix is set by copying the third quantization matrix to the fourth quantization matrix based on the determination that the third identification information identifies that the third quantization matrix is copied to the fourth quantization matrix. 7. The device of claim 1, wherein the memory includes computer program code configured to cause the device to set the second default quantization matrix to the fourth quantization matrix when working with the processor. 8. The device according to claim 1, wherein the first quantization matrix, the second quantization matrix, the third quantization matrix and the fourth quantization matrix are configured based on a combination of prediction modes and color components. 9. The device of claim 8, wherein the combination of prediction mode and color components comprises the following combinations: Intra-frame prediction and luminance component (Y), intra-frame prediction and first color difference component (Cb), intra-frame prediction and second color difference component (Cr), inter-frame prediction and luminance component (Y), inter-frame prediction and first color difference component (Cb), and inter-frame prediction and second color difference component (Cr). 10. The apparatus of claim 1, wherein the first quantization matrix and the second quantization matrix are generated for first transform coefficient data of different sizes, and the third quantization matrix and the fourth quantization matrix are generated for second quantized transform coefficient data of different sizes. 11. The device of claim 1, wherein the memory includes computer program code configured to cause the device to execute when working in conjunction with a processor: An orthogonal transformation of the image data used to generate the first transform coefficient data; and Inverse orthogonal transform of the inverse quantized transform coefficient data. 12. The device of claim 1, wherein the memory includes computer program code configured to cause the device to: when working in conjunction with the processor: The first quantized transform coefficient data is encoded using a first coding unit derived from the first maximum coding unit, and the second coded stream is decoded using a second coding unit derived from the second maximum coding unit. 13. An image processing method, comprising: The first quantization matrix and the second quantization matrix are set as quantization matrices for quantizing the first transform coefficient data of the first image; The second quantization matrix is set by copying the first quantization matrix to the second quantization matrix; The first transform coefficient data is quantized using the second quantization matrix to generate the first quantized transform coefficient data; and Encode the first quantized transform coefficient data into a first encoded stream; Set a third quantization matrix; Decode the second quantized transform coefficient data from the second encoded stream; The fourth quantization matrix is set by copying the third quantization matrix to the fourth quantization matrix; and The fourth quantization matrix is used to inverse quantize the second quantized transform coefficient data decoded from the second encoded stream to generate inverse quantized transform coefficient data. 14. The method of claim 13, wherein the first quantization matrix corresponds to the value of the first identification information, and the second quantization matrix corresponds to the value of the second identification information. 15. The method of claim 13, further comprising, in the case of copying the first quantization matrix to the second quantization matrix, generating a value of first identification information that is different from the value of the second identification information. 16. The method of claim 15, further comprising determining that the value of the first identification information is different from the value of the second identification information, wherein the second quantization matrix is set by copying the first quantization matrix based on the determination that the value of the first identification information is different from the value of the second identification information. 17. The method of claim 13, further comprising decoding third identification information from a second encoded stream, wherein the third identification information identifies whether the third quantization matrix is copied to the fourth quantization matrix. 18. The method of claim 17, further comprising determining that third identification information identifies that the third quantization matrix is copied to the fourth quantization matrix, wherein the fourth quantization matrix is set by copying the third quantization matrix to the fourth quantization matrix based on the determination that the third identification information identifies that the third quantization matrix is copied to the fourth quantization matrix. 19. The method of claim 18, further comprising setting the second default quantization matrix as the fourth quantization matrix. 20. The method of claim 19, wherein the first quantization matrix, the second quantization matrix, the third quantization matrix, and the fourth quantization matrix are configured based on a combination of prediction modes and color components. 21. The method of claim 20, wherein the combination of prediction mode and color components comprises the following combinations: Intra-frame prediction and luminance component (Y), intra-frame prediction and first color difference component (Cb), intra-frame prediction and second color difference component (Cr), inter-frame prediction and luminance component (Y), inter-frame prediction and first color difference component (Cb), and inter-frame prediction and second color difference component (Cr). 22. The method of claim 13, wherein the first quantization matrix and the second quantization matrix are generated for first transform coefficient data of different sizes, and the third quantization matrix and the fourth quantization matrix are generated for second quantized transform coefficient data of different sizes. 23. The method of claim 13, further comprising: Perform an orthogonal transformation on the image data to generate first transform coefficient data; and Perform an inverse orthogonal transform on the dequantized transform coefficients. 24. The method of claim 13, further comprising: The first quantized transform coefficient data is encoded using a first coding unit derived from the first largest coding unit. The second encoded stream is decoded using the second encoded unit derived from the second largest encoded unit. 25. A computer-readable medium storing a computer program, which, when executed by a processor, performs: The first quantization matrix and the second quantization matrix are set as quantization matrices for quantizing the first transform coefficient data of the first image; The second quantization matrix is set by copying the first quantization matrix to the second quantization matrix; The first transform coefficient data is quantized using the second quantization matrix to generate the first quantized transform coefficient data; and Encode the first quantized transform coefficient data into a first encoded stream; Set a third quantization matrix; Decode the second quantized transform coefficient data from the second encoded stream; The fourth quantization matrix is set by copying the third quantization matrix to the fourth quantization matrix; and The fourth quantization matrix is used to inverse quantize the second quantized transform coefficient data decoded from the second encoded stream to generate inverse quantized transform coefficient data. 26. The medium of claim 25, wherein the first quantization matrix corresponds to the value of the first identification information, and the second quantization matrix corresponds to the value of the second identification information. 27. The medium according to claim 26, further storing a program in which, when executed by a processor in the case of copying the first quantization matrix to the second quantization matrix, a value of first identification information different from the value of the second identification information is generated. 28. The medium according to claim 27, further storing a program that, when executed by a processor, determines that the value of the first identification information is different from the value of the second identification information, wherein the second quantization matrix is set by copying the first quantization matrix based on the determination that the values of the first identification information and the second identification information are different. 29. The medium according to claim 25 further stores a program that, when executed by a processor, decodes third identification information from a second encoded stream, the third identification information identifying whether the third quantization matrix is copied to the fourth quantization matrix. 30. The medium according to claim 29, further storing a program that, when executed by a processor, determines that third identification information identifies that the third quantization matrix is copied to the fourth quantization matrix, wherein the fourth quantization matrix is set by copying the third quantization matrix to the fourth quantization matrix based on the determination that the third identification information identifies that the third quantization matrix is copied to the fourth quantization matrix. 31. The medium according to claim 25 further stores a program that, when executed by a processor, sets the second default quantization matrix to the fourth quantization matrix. 32. The medium of claim 25, wherein the first quantization matrix, the second quantization matrix, the third quantization matrix, and the fourth quantization matrix are configured based on a combination of prediction modes and color components. 33. The medium of claim 32, wherein the combination of prediction mode and color components comprises the following combinations: Intra-frame prediction and luminance component (Y), intra-frame prediction and first color difference component (Cb), intra-frame prediction and second color difference component (Cr), inter-frame prediction and luminance component (Y), inter-frame prediction and first color difference component (Cb), and inter-frame prediction and second color difference component (Cr). 34. The medium of claim 25, wherein the first quantization matrix and the second quantization matrix are generated for first transform coefficient data of different sizes, and the third quantization matrix and the fourth quantization matrix are generated for second quantized transform coefficient data of different sizes. 35. The medium according to claim 25, further storing a program that, when executed by a processor, performs: An orthogonal transformation of the image data used to generate the first transform coefficient data; and Inverse orthogonal transform of the inverse quantized transform coefficient data. 36. The medium according to claim 25, further storing a program that, when executed by a processor, performs: Encode the first quantized transform coefficient data using a first coding unit derived from the first largest coding unit; and The second encoded stream is decoded using the second encoded unit derived from the second largest encoded unit.