WO2019194420A1 - Image coding method and device on basis of transform indicator - Google Patents

Abstract

A picture decoding method performed by a decoding device according to the present invention comprises the steps of: decoding a unified transform indicator (UTI) index for a current block from a bitstream; decoding transform coefficients for the current block from the bitstream; deriving a transform type for the current block on the basis of the UTI index; performing inverse transform on the basis of the derived transform type and the transform coefficients for the current block; and generating a reconstruction picture on the basis of residual samples for the current block derived via the inverse transform, wherein the UTI index indicates at least one of a transform kernel for a first transform and a transform kernel for a second transform.

Description

Image coding method and apparatus therefor based on transform indicator

The present invention relates to image coding technology, and more particularly, to an image coding method and apparatus based on a transform indicator in an image coding system.

Recently, the demand for high resolution and high quality images such as high definition (HD) images and ultra high definition (UHD) images is increasing in various fields. The higher the resolution and the higher quality of the image data, the more information or bit rate is transmitted than the existing image data. Therefore, the image data can be transmitted by using a medium such as a conventional wired / wireless broadband line or by using a conventional storage medium. In the case of storage, the transmission cost and the storage cost are increased.

Accordingly, a high efficiency image compression technique is required to effectively transmit, store, and reproduce high resolution, high quality image information.

An object of the present invention is to provide a method and apparatus for improving image coding efficiency.

Another technical problem of the present invention is to provide a method and apparatus for improving conversion efficiency.

Another technical problem of the present invention is to provide a method and apparatus for increasing the efficiency of residual coding through transformation.

Another technical problem of the present invention is to provide an image coding method and apparatus based on a transform indicator.

Another technical problem of the present invention is to provide a method and apparatus for improving image coding efficiency based on a Unified Transform Indicator (UTI) index.

According to an embodiment of the present invention, a picture decoding method performed by a decoding apparatus is provided. The method includes decoding an unified transform indicator (UTI) index for a current block from a bitstream, decoding transform coefficients for the current block from the bitstream, and based on the UTI index. Deriving a transform type for a current block, performing an inverse transform based on the derived transform type and the transform coefficients for the current block, and a register for the current block derived through the inverse transform Generating a reconstructed picture based on the dual samples, wherein the UTI index indicates at least one of a transform kernel for a primary transform and a transform kernel for a secondary transform.

According to another embodiment of the present invention, a picture encoding method performed by an encoding device is provided. The method comprises: deriving residual samples for a current block, performing a transform on the residual samples for the current block to derive transform coefficients, an integrated transform indicating a transform type of the performed transform Deriving an indicator (UTI) index and encoding information about the transform coefficients and image information including the UTI index, wherein the UTI index is used for transform kernel and secondary transform for the primary transform. It is characterized by indicating at least one of the conversion kernel for.

According to another embodiment of the present invention, a decoding apparatus for decoding a picture is provided. The decoding apparatus decodes a Unified Transform Indicator (UTI) index for the current block from the bitstream, decodes transform coefficients for the current block from the bitstream, and applies the current block to the current block based on the decoded UTI index. An entropy decoding unit that derives a transform type for the transform, an inverse transform unit that performs inverse transform based on the derived transform type and the transform coefficients for the current block, and a register for the current block derived through the inverse transform And an adder for generating a reconstructed picture based on the dual samples, wherein the UTI index indicates at least one of a transform kernel for the first transform and a transform kernel for the second transform.

According to yet another embodiment of the present invention, an encoding apparatus for encoding a picture is provided. The encoding apparatus may be configured to derive residual samples for a current block, perform a transform on the residual samples for the current block, and to derive transform coefficients, and to indicate a transform type of the performed transform. Deriving a transform indicator (UTI) index, and includes an entropy encoding unit for encoding the information about the transform coefficients and the image information including the UTI index, wherein the UTI index is a transform kernel and secondary transform for the primary transform It is characterized by indicating at least one of the conversion kernel for.

According to the present invention, the overall video / video compression efficiency can be improved.

According to the present invention, the amount of data to be transmitted for residual processing can be reduced through efficient conversion, and the residual coding efficiency can be improved.

According to the present invention, image coding efficiency may be improved by performing image coding based on a transform indicator.

According to the present invention, the conversion may be performed based on the Unified Transform Indicator (UTI) index to increase image coding efficiency.

1 is a diagram schematically illustrating a configuration of a video / video encoding apparatus to which the present invention can be applied.

2 is a diagram schematically illustrating a configuration of a video / video decoding apparatus to which the present invention may be applied.

3 schematically illustrates a multiple conversion scheme according to an embodiment.

4 exemplarily shows intra directional modes of 65 prediction directions.

5A and 5B are flowcharts illustrating a coding process of transform coefficients, according to an exemplary embodiment.

6 is a flowchart illustrating a process of coding a transform coefficient according to an embodiment.

7 is a flowchart illustrating a coding process of transform coefficients, according to an exemplary embodiment.

8A and 8B are flowcharts illustrating a coding order of transform coefficients and UTI indexes, according to an exemplary embodiment.

9 is a flowchart illustrating an operation of an encoding apparatus according to an embodiment.

10 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment.

11 is a flowchart illustrating an operation of a decoding apparatus according to an embodiment.

12 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment.

According to an embodiment of the present invention, a picture decoding method performed by a decoding apparatus is provided. The method includes decoding an unified transform indicator (UTI) index for a current block from a bitstream, decoding transform coefficients for the current block from the bitstream, and based on the UTI index. Deriving a transform type for a current block, performing an inverse transform based on the derived transform type and the transform coefficients for the current block, and a register for the current block derived through the inverse transform Generating a reconstructed picture based on the dual samples, wherein the UTI index indicates at least one of a transform kernel for a primary transform and a transform kernel for a secondary transform.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the invention to the specific embodiments. The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the spirit of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. The terms "comprise" or "having" in this specification are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features It is to be understood that the numbers, steps, operations, components, parts or figures do not exclude in advance the presence or possibility of adding them.

On the other hand, each configuration in the drawings described in the present invention are shown independently for the convenience of description of the different characteristic functions, it does not mean that each configuration is implemented by separate hardware or separate software. For example, two or more of each configuration may be combined to form one configuration, or one configuration may be divided into a plurality of configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of the present invention without departing from the spirit of the present invention.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant description of the same components is omitted.

In the present specification, a video may mean a series of images over time. A picture generally refers to a unit representing one image in a specific time zone, and a slice is a unit constituting a part of a picture in coding. One picture may be composed of a plurality of slices, and if necessary, the picture and the slice may be mixed with each other. Also, in some cases, an "image" may mean a concept including a still image and a video, which is a set of a series of still images over time. In addition, “video” does not necessarily mean a set of a series of still images over time, and in some embodiments, may be interpreted as a concept in which still images are included in video.

A pixel or a pel may refer to a minimum unit constituting one picture (or image). Also, 'sample' may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a value of a pixel, and may only represent pixel / pixel values of the luma component, or only pixel / pixel values of the chroma component.

A unit represents the basic unit of image processing. The unit may include at least one of a specific region of the picture and information related to the region. The unit may be used interchangeably with terms such as block or area in some cases. In a general case, an M × N block may represent a set of samples or transform coefficients composed of M columns and N rows.

FIG. 1 is a diagram schematically illustrating a configuration of a video / video encoding apparatus to which the present invention may be applied. Hereinafter, the encoding / decoding device may include a video encoding / decoding device and / or an image encoding / decoding device, and the video encoding / decoding device is used as a concept including the image encoding / decoding device, or the image encoding / decoding device is It may be used in a concept including a video encoding / decoding device.

Referring to FIG. 1, the (video) encoding apparatus 100 may include a picture partitioning module 105, a prediction module 110, a residual processing module 120, and an entropy encoding unit ( The entropy encoding module 130 may include an adder 140, a filtering module 150, and a memory 160. The residual processor 120 may include a substractor 121, a transform module 122, a quantization module 123, a rearrangement module 124, and a dequantization module 125. ) And an inverse transform module 126.

The picture divider 105 may divide the input picture into at least one processing unit.

As an example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively split from the largest coding unit (LCU) according to a quad-tree binary-tree (QTBT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad tree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, the quad tree structure may be applied first, and the binary tree structure and the ternary tree structure may be applied later. Alternatively, the binary tree structure / tunary tree structure may be applied first. The coding procedure according to the present invention may be performed based on the final coding unit that is no longer split. In this case, the maximum coding unit may be used as the final coding unit immediately based on coding efficiency according to the image characteristic, or if necessary, the coding unit is recursively divided into coding units of lower depths and optimized. A coding unit of size may be used as the final coding unit. Here, the coding procedure may include a procedure of prediction, transform, and reconstruction, which will be described later.

As another example, the processing unit may include a coding unit (CU) prediction unit (PU) or a transform unit (TU). The coding unit may be split from the largest coding unit (LCU) into coding units of deeper depths along the quad tree structure. In this case, the maximum coding unit may be used as the final coding unit immediately based on coding efficiency according to the image characteristic, or if necessary, the coding unit is recursively divided into coding units of lower depths and optimized. A coding unit of size may be used as the final coding unit. If a smallest coding unit (SCU) is set, the coding unit may not be split into smaller coding units than the minimum coding unit. Here, the final coding unit refers to a coding unit that is the basis of partitioning or partitioning into a prediction unit or a transform unit. The prediction unit is a unit partitioning from the coding unit and may be a unit of sample prediction. In this case, the prediction unit may be divided into sub blocks. The transform unit may be divided along the quad tree structure from the coding unit, and may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient. Hereinafter, a coding unit may be called a coding block (CB), a prediction unit is a prediction block (PB), and a transform unit may be called a transform block (TB). A prediction block or prediction unit may mean a specific area in the form of a block within a picture, and may include an array of prediction samples. In addition, a transform block or a transform unit may mean a specific area in a block form within a picture, and may include an array of transform coefficients or residual samples.

The prediction unit 110 performs prediction on a block to be processed (hereinafter, may mean a current block or a residual block), and generates a predicted block including prediction samples for the current block. can do. The unit of prediction performed by the prediction unit 110 may be a coding block, a transform block, or a prediction block.

The prediction unit 110 may determine whether intra prediction or inter prediction is applied to the current block. As an example, the prediction unit 110 may determine whether intra prediction or inter prediction is applied on a CU basis.

In the case of intra prediction, the prediction unit 110 may derive a prediction sample for the current block based on reference samples outside the current block in the picture to which the current block belongs (hereinafter, referred to as the current picture). In this case, the prediction unit 110 may (i) derive the prediction sample based on the average or interpolation of neighboring reference samples of the current block, and (ii) the neighbor reference of the current block. The prediction sample may be derived based on a reference sample present in a specific (prediction) direction with respect to the prediction sample among the samples. In case of (i), it may be called non-directional mode or non-angle mode, and in case of (ii), it may be called directional mode or angular mode. In intra prediction, the prediction mode may have, for example, 33 directional prediction modes and at least two non-directional modes. The non-directional mode may include a DC prediction mode and a planner mode (Planar mode). The prediction unit 110 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

In the case of inter prediction, the prediction unit 110 may derive the prediction sample for the current block based on the sample specified by the motion vector on the reference picture. The prediction unit 110 may apply one of a skip mode, a merge mode, and a motion vector prediction (MVP) mode to derive a prediction sample for the current block. In the skip mode and the merge mode, the prediction unit 110 may use the motion information of the neighboring block as the motion information of the current block. In the skip mode, unlike the merge mode, the difference (residual) between the prediction sample and the original sample is not transmitted. In the MVP mode, the motion vector of the current block may be derived using the motion vector of the neighboring block as a motion vector predictor.

In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture. A reference picture including the temporal neighboring block may be called a collocated picture (colPic). The motion information may include a motion vector and a reference picture index. Information such as prediction mode information and motion information may be encoded (entropy) and output in the form of a bitstream.

When the motion information of the temporal neighboring block is used in the skip mode and the merge mode, the highest picture on the reference picture list may be used as the reference picture. Reference pictures included in a reference picture list may be sorted based on a difference in a picture order count (POC) between a current picture and a corresponding reference picture. The POC corresponds to the display order of the pictures and may be distinguished from the coding order.

The subtraction unit 121 generates a residual sample which is a difference between the original sample and the prediction sample. When the skip mode is applied, residual samples may not be generated as described above.

The transform unit 122 generates transform coefficients by transforming the residual sample in units of transform blocks. The transform unit 122 may perform the transform according to the size of the transform block and the prediction mode applied to the coding block or the prediction block that spatially overlaps the transform block. For example, if intra prediction is applied to the coding block or the prediction block that overlaps the transform block, and the transform block is a 4 × 4 residual array, the residual sample is configured to perform a discrete sine transform (DST) transform kernel. The residual sample may be transformed using a discrete cosine transform (DCT) transform kernel.

The quantization unit 123 may quantize the transform coefficients to generate quantized transform coefficients.

The reordering unit 124 rearranges the quantized transform coefficients. The reordering unit 124 may reorder the quantized transform coefficients in the form of a block into a one-dimensional vector form through a coefficient scanning method. Although the reordering unit 124 has been described in a separate configuration, the reordering unit 124 may be part of the quantization unit 123.

The entropy encoding unit 130 may perform entropy encoding on the quantized transform coefficients. Entropy encoding may include, for example, encoding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and the like. The entropy encoding unit 130 may encode information necessary for video reconstruction other than the quantized transform coefficients (for example, a value of a syntax element) together or separately according to entropy encoding or a predetermined method. The encoded information may be transmitted or stored in units of network abstraction layer (NAL) units in the form of bitstreams. The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like.

The inverse quantization unit 125 inverse quantizes the quantized values (quantized transform coefficients) in the quantization unit 123, and the inverse transformer 126 inverse transforms the inverse quantized values in the inverse quantization unit 125 to obtain a residual sample. Create

The adder 140 reconstructs the picture by combining the residual sample and the predictive sample. The residual sample and the predictive sample may be added in units of blocks to generate a reconstructed block. Although the adder 140 has been described in a separate configuration, the adder 140 may be part of the predictor 110. The adder 140 may also be called a reconstruction module or a restore block generator.

The filter unit 150 may apply a deblocking filter and / or a sample adaptive offset to the reconstructed picture. Through deblocking filtering and / or sample adaptive offset, the artifacts of the block boundaries in the reconstructed picture or the distortion in the quantization process can be corrected. The sample adaptive offset may be applied on a sample basis and may be applied after the process of deblocking filtering is completed. The filter unit 150 may apply an adaptive loop filter (ALF) to the reconstructed picture. ALF may be applied to the reconstructed picture after the deblocking filter and / or sample adaptive offset is applied.

The memory 160 may store reconstructed pictures (decoded pictures) or information necessary for encoding / decoding. Here, the reconstructed picture may be a reconstructed picture after the filtering process is completed by the filter unit 150. The stored reconstructed picture may be used as a reference picture for (inter) prediction of another picture. For example, the memory 160 may store (reference) pictures used for inter prediction. In this case, pictures used for inter prediction may be designated by a reference picture set or a reference picture list.

FIG. 2 is a diagram schematically illustrating a configuration of a video / video decoding apparatus to which the present invention can be applied. Hereinafter, the video decoding apparatus may include an image decoding apparatus.

Referring to FIG. 2, the (video) decoding apparatus 200 may include an entropy decoding module 210, a residual processing module 220, a prediction module 230, and an adder. , 240, a filtering module 250, and a memory 260. The residual processor 220 may include a rearrangement module 221, a dequantization module 222, and an inverse transform module 223. In addition, although not shown, the video decoding apparatus 200 may include a receiver that receives a bitstream including video information. The receiver may be configured as a separate module or may be included in the entropy decoding unit 210.

When the bitstream including the video / image information is input, the (video) decoding apparatus 200 may restore the video / image / picture in response to a process in which the video / image information is processed in the (video) encoding apparatus. .

For example, the video decoding apparatus 200 may perform video decoding using a processing unit applied in the video encoding apparatus. Thus, the processing unit block of video decoding may be, for example, a coding unit, and in another example, a coding unit, a prediction unit, or a transform unit. The coding unit may be split along the quad tree structure, binary tree structure and / or ternary tree structure from the largest coding unit.

The prediction unit and the transform unit may be further used in some cases, in which case the prediction block is a block derived or partitioned from the coding unit and may be a unit of sample prediction. At this point, the prediction unit may be divided into subblocks. The transform unit may be divided along the quad tree structure from the coding unit, and may be a unit for deriving a transform coefficient or a unit for deriving a residual signal from the transform coefficient.

The entropy decoding unit 210 may parse the bitstream and output information necessary for video reconstruction or picture reconstruction. For example, the entropy decoding unit 210 decodes information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, quantized values of syntax elements necessary for video reconstruction, and residual coefficients. Can be output.

More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in a bitstream, and decodes syntax element information and decoding information of neighboring and decoding target blocks or information of symbols / bins decoded in a previous step. The context model is determined using the context model, the probability of occurrence of a bin is predicted according to the determined context model, and arithmetic decoding of the bin is performed to generate a symbol corresponding to the value of each syntax element. can do. In this case, the CABAC entropy decoding method may update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model.

The information related to the prediction among the information decoded by the entropy decoding unit 210 is provided to the prediction unit 230, and the residual value on which the entropy decoding has been performed by the entropy decoding unit 210, that is, the quantized transform coefficient, is used as a reordering unit ( 221 may be input.

The reordering unit 221 may rearrange the quantized transform coefficients in a two-dimensional block form. The reordering unit 221 may perform reordering in response to coefficient scanning performed by the encoding apparatus. Here, the rearrangement unit 221 has been described in a separate configuration, but the rearrangement unit 221 may be part of the inverse quantization unit 222.

The inverse quantization unit 222 may dequantize the quantized transform coefficients based on the (inverse) quantization parameter and output the transform coefficients. In this case, information for deriving a quantization parameter may be signaled from the encoding apparatus.

The inverse transform unit 223 may inversely transform transform coefficients to derive residual samples.

The prediction unit 230 may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The unit of prediction performed by the prediction unit 230 may be a coding block, a transform block, or a prediction block.

The prediction unit 230 may determine whether to apply intra prediction or inter prediction based on the information about the prediction. In this case, a unit for determining which of intra prediction and inter prediction is to be applied and a unit for generating a prediction sample may be different. In addition, the unit for generating a prediction sample in inter prediction and intra prediction may also be different. For example, whether to apply inter prediction or intra prediction may be determined in units of CUs. In addition, for example, in inter prediction, a prediction mode may be determined and a prediction sample may be generated in PU units, and in intra prediction, a prediction mode may be determined in PU units and a prediction sample may be generated in TU units.

In the case of intra prediction, the prediction unit 230 may derive the prediction sample for the current block based on the neighbor reference samples in the current picture. The prediction unit 230 may derive the prediction sample for the current block by applying the directional mode or the non-directional mode based on the neighbor reference samples of the current block. In this case, the prediction mode to be applied to the current block may be determined using the intra prediction mode of the neighboring block.

In the case of inter prediction, the prediction unit 230 may derive the prediction sample for the current block based on the sample specified on the reference picture by the motion vector on the reference picture. The prediction unit 230 may apply any one of a skip mode, a merge mode, and an MVP mode to derive a prediction sample for the current block. In this case, motion information required for inter prediction of the current block provided by the video encoding apparatus, for example, information about a motion vector, a reference picture index, and the like may be obtained or derived based on the prediction information.

In the skip mode and the merge mode, the motion information of the neighboring block may be used as the motion information of the current block. In this case, the neighboring block may include a spatial neighboring block and a temporal neighboring block.

The prediction unit 230 may construct a merge candidate list using motion information of available neighboring blocks, and may use information indicated by the merge index on the merge candidate list as a motion vector of the current block. The merge index may be signaled from the encoding device. The motion information may include a motion vector and a reference picture. When the motion information of the temporal neighboring block is used in the skip mode and the merge mode, the highest picture on the reference picture list may be used as the reference picture.

In the skip mode, unlike the merge mode, the difference (residual) between the prediction sample and the original sample is not transmitted.

In the MVP mode, the motion vector of the current block may be derived using the motion vector of the neighboring block as a motion vector predictor. In this case, the neighboring block may include a spatial neighboring block and a temporal neighboring block.

For example, when the merge mode is applied, a merge candidate list may be generated by using a motion vector of a reconstructed spatial neighboring block and / or a motion vector corresponding to a Col block, which is a temporal neighboring block. In the merge mode, the motion vector of the candidate block selected from the merge candidate list is used as the motion vector of the current block. The information about the prediction may include a merge index indicating a candidate block having an optimal motion vector selected from candidate blocks included in the merge candidate list. In this case, the prediction unit 230 may derive the motion vector of the current block by using the merge index.

As another example, when the Motion Vector Prediction (MVP) mode is applied, a motion vector predictor candidate list may be generated using a motion vector of a reconstructed spatial neighboring block and / or a motion vector corresponding to a Col block, which is a temporal neighboring block. Can be. That is, the motion vector of the reconstructed spatial neighboring block and / or the Col vector, which is a temporal neighboring block, may be used as a motion vector candidate. The prediction information may include a prediction motion vector index indicating an optimal motion vector selected from the motion vector candidates included in the list. In this case, the prediction unit 230 may select the predicted motion vector of the current block from the motion vector candidates included in the motion vector candidate list using the motion vector index. The prediction unit of the encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and may encode the output vector in a bitstream form. That is, MVD may be obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the prediction unit 230 may obtain a motion vector difference included in the information about the prediction, and derive the motion vector of the current block by adding the motion vector difference and the motion vector predictor. The prediction unit may also obtain or derive a reference picture index or the like indicating a reference picture from the information about the prediction.

 The adder 240 may reconstruct the current block or the current picture by adding the residual sample and the predictive sample. The adder 240 may reconstruct the current picture by adding the residual sample and the predictive sample in block units. Since the residual is not transmitted when the skip mode is applied, the prediction sample may be a reconstruction sample. Although the adder 240 has been described in a separate configuration, the adder 240 may be part of the predictor 230. The adder 240 may also be called a reconstruction module or a reconstruction block generator.

The filter unit 250 may apply the deblocking filtering sample adaptive offset, and / or ALF to the reconstructed picture. In this case, the sample adaptive offset may be applied in units of samples and may be applied after deblocking filtering. ALF may be applied after deblocking filtering and / or sample adaptive offset.

The memory 260 may store reconstructed pictures (decoded pictures) or information necessary for decoding. Here, the reconstructed picture may be a reconstructed picture after the filtering process is completed by the filter unit 250. For example, the memory 260 may store pictures used for inter prediction. In this case, pictures used for inter prediction may be designated by a reference picture set or a reference picture list. The reconstructed picture can be used as a reference picture for another picture. In addition, the memory 260 may output the reconstructed picture in an output order.

Meanwhile, as described above, prediction is performed in order to increase compression efficiency in performing video coding. Through this, a predicted block including prediction samples of the current block, which is a coding target block, may be generated. Wherein the predicted block comprises prediction samples in the spatial domain (or pixel domain). The predicted block is derived identically in the encoding apparatus and the decoding apparatus, and the encoding apparatus decodes information (residual information) about the residual between the original block and the predicted block, not the original sample value itself of the original block. Signaling to an apparatus may increase image coding efficiency. The decoding apparatus may derive a residual block including residual samples based on the residual information, generate the reconstructed block including reconstructed samples by adding the residual block and the predicted block, and generate reconstructed blocks. A reconstructed picture may be generated.

The residual information may be generated through a transform and quantization procedure. For example, the encoding apparatus derives a residual block between the original block and the predicted block, and performs transform procedure on residual samples (residual sample array) included in the residual block to derive transform coefficients. The quantized transform coefficients may be derived by performing a quantization procedure on the transform coefficients to signal related residual information to the decoding device (via a bitstream). Here, the residual information may include information such as value information of the quantized transform coefficients, position information, a transform scheme, a transform kernel, and a quantization parameter. The decoding apparatus may perform an inverse quantization / inverse transformation procedure and derive residual samples (or residual blocks) based on the residual information. The decoding apparatus may generate a reconstructed picture based on the predicted block and the residual block. The encoding apparatus may then dequantize / inverse transform the quantized transform coefficients for reference for inter prediction of the picture to derive a residual block, and generate a reconstructed picture based thereon.

3 schematically illustrates a multiple conversion scheme according to the present invention.

Referring to FIG. 3, the transform unit may correspond to a transform unit in the encoding apparatus of FIG. 1, and the inverse transform unit may correspond to an inverse transform unit in the encoding apparatus of FIG. 1 or an inverse transform unit in the decoding apparatus of FIG. 2. .

The transform unit may derive (primary) transform coefficients by performing a first-order transform based on the residual samples (residual sample array) in the residual block (S310). In this case, the first transform may include a multiple transform set. Multiple transform sets may be referred to as adaptive multiple core transforms as the case may be. Furthermore, the adaptive multiple core transform may be referred to as adaptive multiple transform (AMT), explicit multiple transform (EMT), or multiple transform transformation (MTS).

The adaptive multi-core transform may represent a method of additionally using a discrete cosine transform (DCT) type 2, a discrete sine transform (DST) type 7, a DCT type 8, and / or a DST type 1. That is, the multi-core transform is a frequency domain of a residual signal (or residual block) of the spatial domain based on a plurality of transform kernels selected from the DCT type 2, the DST type 7, the DCT type 8, and the DST type 1. A transform method of converting to transform coefficients (or first-order transform coefficients) may be represented. Here, the first-order transform coefficients may be referred to as temporary transform coefficients from a transform unit point of view.

In other words, when the conventional transform method is applied, transform coefficients may be generated by applying the spatial domain to the frequency domain transform for the residual signal (or the residual block) based on the DCT type 2. On the contrary, when the adaptive multi-core transform is applied, the residual signal (or the residual block) on the DCT type 2, the DST type 7, the DCT type 8, the DST type 1, and / or the DST type 5 is applied. The transform from the spatial domain to the frequency domain may be applied to generate transform coefficients (or first order transform coefficients). Here, DCT type 2, DST type 7, DCT type 8, DST type 1, and / or DST type 5 may be referred to as a conversion type, a conversion kernel, or a conversion core. The transformation kernel according to an example may include information about a transformation algorithm applicable for the corresponding transformation.

For reference, the DCT / DST conversion types may be defined based on base functions, and the base functions may be represented as the following table.

TABLE 1

When the adaptive multi-core transform is performed, a vertical transform kernel and a horizontal transform kernel for a target block among the transform kernels may be selected, and a vertical transform on the target block is performed based on the vertical transform kernel. The horizontal transformation may be performed on the target block based on the horizontal transformation kernel. Here, the horizontal transform may indicate a transform for horizontal components of the target block, and the vertical transform may indicate a transform for vertical components of the target block. The vertical transform kernel / horizontal transform kernel may be adaptively determined based on the prediction mode of the target block (CU or subblock) enclosing the residual block and / or the transform index indicating the transform subset.

The transform unit may derive the (secondary) transform coefficients by performing a quadratic transformation based on the (primary) transform coefficients (S320). If the first order transform was from spatial domain to frequency domain, then the second order transform can be viewed as a frequency domain to frequency domain transformation. The secondary transform may include a non-separable transform. In this case, the secondary transform may be referred to as a non-separable secondary transform (NSST) or a mode-dependent non-separable secondary transform (MDNSST). The non-separated quadratic transform transforms (first-order) transform coefficients derived through the first-order transform based on a non-separable transform matrix to transform coefficients (or Transform to generate quadratic transform coefficients). Here, the transformation may be applied at once without applying the vertical and horizontal transformations (or independently of the horizontal and vertical transformations) to the (primary) transformation coefficients based on the non-separated transformation matrix. In other words, the non-separated second-order transform generates transform coefficients (or second-order transform coefficients) by transforming together the vertical and horizontal components of the (primary) transform coefficients without separating them based on the non-separated transform matrix. It can represent the conversion method.

The non-separated secondary transform may be applied to a top-left region of a block composed of (primary) transform coefficients (hereinafter, referred to as a transform coefficient block). For example, when the width W and the height H of the transform coefficient block are both 8 or more, an 8x8 non-separated secondary transform may be applied to the upper left 8x8 region of the transform coefficient block. Further, when the width W and height H of the transform coefficient block are both 4 or more, and the width W or height H of the transform coefficient block is less than 8, 4 × 4 non-separated secondary A transform may be applied to the upper left min (8, W) × min (8, H) region of the transform coefficient block. However, the embodiment is not limited thereto. For example, even if only the width W or the height H of the transform coefficient block is less than 8, a 4 × 4 non-separated quadratic transform is applied to the upper left end of the transform coefficient block. It may also be applied to the min (8, W) × min (8, H) region.

Specifically, for example, when a 4x4 input block is used, non-separated quadratic transformation may be performed as follows.

The 4x4 input block X may be represented as follows.

[Equation 1]

는 다음과 같이 나타내어질 수 있다. If X is represented in vector form, then

Can be expressed as follows.

[Equation 2]

In this case, the second order non-separation transform can be calculated as follows.

[Equation 3]

는 변환 계수 벡터를 나타내고, T는 16×16 (비분리) 변환 매트릭스를 나타낸다. here,

Denotes a transform coefficient vector, and T denotes a 16x16 (non-separated) transform matrix.

가 도출될 수 있으며, 상기

는 스캔 순서(수평, 수직, 대각(diagonal) 등)를 통하여 4×4 블록으로 재구성(re-organized)될 수 있다. 다만, 상술한 계산은 예시로서 비분리 2차 변환의 계산 복잡도를 줄이기 위하여 HyGT(Hypercube-Givens Transsform) 등이 비분리 2차 변환의 계산을 위하여 사용될 수도 있다. 16 × 1 transform coefficient vector through Equation 3

Can be derived and said

Can be re-organized into 4x4 blocks through the scan order (horizontal, vertical, diagonal, etc.). However, the above-described calculation may be used to calculate a hypercube-givens transsform (HyGT) or the like for the calculation of the non-separated secondary transform as an example to reduce the computational complexity of the non-separated secondary transform.

In the meantime, the non-separated secondary transform may be mode dependent, and a transform kernel (or a transform core or a transform type) may be selected. Here, the mode may include an intra prediction mode and / or an inter prediction mode.

As described above, the non-separated secondary transform may be performed based on an 8 × 8 transform or a 4 × 4 transform determined based on the width W and the height H of the transform coefficient block. That is, the non-separated quadratic transform may be performed based on an 8 × 8 subblock size or a 4 × 4 subblock size. For example, for the mode based transform kernel selection, 35 sets of non-separated secondary transform kernels, 3 for non-separated secondary transform, for both 8x8 subblock size and 4x4 subblock size, Can be configured. That is, 35 transform sets may be configured for an 8 × 8 subblock size, and 35 transform sets may be configured for a 4 × 4 subblock size. In this case, 35 transform sets for 8 × 8 subblock sizes may include three 8 × 8 transform kernels, and in this case, 35 transform sets for 4 × 4 subblock sizes, three 4 × for each 4 translation kernels may be included. However, the transform subblock size, the number of sets and the number of transform kernels in the set may be used as an example, other than 8 × 8 or 4 × 4, or n sets are configured and k sets are included in each set. Conversion kernels may be included.

The translation set may be called an NSST set, and the translation kernel within the NSST set may be called an NSST kernel. Selection of a specific set of the transform sets may be performed based on, for example, the intra prediction mode of the target block (CU or subblock).

For reference, for example, the intra prediction mode includes two non-directinoal or non-angular intra prediction modes and 65 directional or angular intra prediction modes. Can include them. The non-directional intra prediction modes may include a planar intra prediction mode of 0 and a DC intra prediction mode of 1, and the directional intra prediction modes may include 65 intra prediction modes of 2 to 66. . However, as an example, the present invention may be applied to a case where the number of intra prediction modes is different. In some cases, the intra prediction mode 67 may further be used, and the intra prediction mode 67 may represent a linear model (LM) mode.

4 exemplarily shows intra directional modes of 65 prediction directions.

Referring to FIG. 4, an intra prediction mode having horizontal directionality and an intra prediction mode having vertical directionality may be distinguished from an intra prediction mode 34 having a left upward diagonal prediction direction. H and V in FIG. 3 mean horizontal directionality and vertical directionality, respectively, and numbers of -32 to 32 represent a displacement of 1/32 on a sample grid position. Intra prediction modes 2 to 33 have horizontal orientation, and intra prediction modes 34 to 66 have vertical orientation. Intra prediction mode 18 and intra prediction mode 50 indicate a horizontal intra prediction mode and a vertical intra prediction mode, respectively, and an intra prediction mode 2 indicates a left downward diagonal intra prediction mode, The 34th intra prediction mode may be referred to as a left upward diagonal intra prediction mode, and the 66th intra prediction mode may be referred to as a right upward diagonal intra prediction mode.

In this case, the mapping between the 35 transform sets and the intra prediction modes may be represented, for example, in the following table. For reference, when the LM mode is applied to the target block, the quadratic transformation may not be applied to the target block.

TABLE 2

On the other hand, if it is determined that a particular set is to be used, one of k transform kernels in the particular set may be selected via a non-separated secondary transform index. The encoding apparatus may derive a non-separated secondary transform index pointing to a specific transform kernel based on a rate-distortion (RD) check, and may signal the non-separated secondary transform index to the decoding apparatus. The decoding apparatus may select one of k transform kernels in a specific set based on the non-separated secondary transform index. For example, NSST index value 0 can point to the first non-isolated secondary transform kernel, NSST index value 1 can point to the second non-isolated secondary transform kernel, and NSST index value 2 is the third non-isolated secondary transform kernel. Can point to Alternatively, the NSST index value 0 may indicate that the first non-separated secondary transform is not applied to the target block, and the NSST index values 1 to 3 may indicate the three transform kernels.

Referring back to FIG. 3, the transform unit may perform the non-separated secondary transform based on the selected transform kernels and obtain (secondary) transform coefficients. The transform coefficients may be derived as quantized transform coefficients through the quantization unit as described above, and may be encoded and transferred to the decoding apparatus and to the inverse quantization / inverse transform unit in the encoding apparatus.

On the other hand, when the secondary transform is omitted as described above, the (primary) transform coefficients, which are outputs of the primary (separated) transform, may be derived as quantized transform coefficients through the quantization unit as described above, and may be encoded. The decoding apparatus may be transferred to an inverse quantization / inverse transform unit in the signaling and encoding apparatus.

The inverse transform unit may perform a series of procedures in a reverse order of the procedure performed in the above-described transform unit. The inverse transform unit receives the (inverse quantized) transform coefficients, performs a second order (inverse) transform, and derives (first order) transform coefficients (S350), and performs a first order (inverse) on the (first order) transform coefficients. The transformation may be performed to obtain a residual block (residual samples). Here, the first transform coefficients may be referred to as modified transform coefficients from an inverse transform unit. As described above, the encoding apparatus and the decoding apparatus may generate a reconstructed block based on the residual block and the predicted block, and generate a reconstructed picture based on the residual block and the predicted block.

Meanwhile, as described above, when the secondary (inverse) transform is omitted, a residual block (residual samples) may be obtained by receiving the (dequantized) transform coefficients and performing the first (separated) transform. . As described above, the encoding apparatus and the decoding apparatus may generate a reconstructed block based on the residual block and the predicted block, and generate a reconstructed picture based on the residual block and the predicted block.

5A and 5B are flowcharts illustrating a coding process of transform coefficients, according to an exemplary embodiment.

Each of the steps disclosed in FIGS. 5A and 5B may be performed by the encoding apparatus 100 or the decoding apparatus 200 disclosed in FIGS. 1 and 2, and more specifically, the entropy encoding unit 130 and the disclosure illustrated in FIG. 1. It may be performed by the entropy decoding unit 210 disclosed in FIG.

In this specification, terms or sentences for defining specific information or concepts are used. For example, in the present specification, the "non-transform-skip mode" of the non-transform coefficients of the transform coefficients for the current block (hereinafter, may also mean a target block or residual block). A flag for indicating whether the number is less than two or a flag indicating whether a non-zero transform coefficient exists among the transform coefficients for the current block is indicated by the cbf flag. However, the "cbf flag" may be replaced with various terms such as coded_block_flag, cbf, and the term or sentence used to define a specific information or concept in the present specification is to be interpreted limited to its name when interpreting throughout the specification. Rather, it is necessary to pay attention to and interpret various operations, functions, and effects as the term means.

5A illustrates an example of an encoding process of transform coefficients.

The encoding apparatus 100 according to an embodiment may include a flag indicating whether at least one non-zero transform coefficient among the transform coefficients for the current block exists or the number of nonzero transform coefficients among the transform coefficients for the current block. It may be determined whether or not the cbf flag indicating a flag indicating 1 is 1 or less (S500). If the cbf flag indicates 1, it may indicate that there is at least one non-zero transform coefficient among the transform coefficients for the current block, and in some cases, a non-zero transform coefficient among the transform coefficients for the current block is present. Two or more may be present. Conversely, if the cbf flag indicates 0, it may indicate that there are no non-zero transform coefficients among the transform coefficients for the current block, and in some cases, a non-zero transform coefficient among the transform coefficients for the current block. May be present in fewer than two.

In one embodiment, the cbf flag is a flag for the luma block (eg cbf_luma flag, cbf_luma [x0] [y0] [tranfoDepth]) and a flag for the chroma block (eg cbf_cb flag and cbf_cr flag, or cbf_cb [x0] [y0] [trafoDepth] and cbf_cr [x0] [y0] [trafoDepth]). With respect to cbf_luma [x0] [y0] [tranfoDepth], cbf_cb [x0] [y0] [trafoDepth], and cbf_cr [x0] [y0] [trafoDepth], the array indexes x0 and y0 are the top-left of the current picture. ) Denotes the position of the upper left luma / chroma sample of the target block with respect to the luma / chroma sample, and the array index trafoDepth may refer to a level at which the coding block is divided for transform coding purposes. If the coding block and the transform block are defined to be identical to the coding block, the blocks in which trafoDepth indicates 0 may be regarded as trafoDepth.

According to an embodiment, when the cbf flag indicates 1 in S500, the encoding apparatus 100 may encode information about transform coefficients for the current block (S510). In one example, the information about the transform coefficients for the current block includes information on the position of the last non-zero transform coefficient, group flag information indicating whether a non-zero transform coefficient is included in a subblock of the current block, EMT_CU_Flag (or MTS_CU_Flag), EMT_TU (or MTS_TU), EMT_TU index (or MTS_TU index), transform coefficients (value of) for the current block, and the like.

The encoding apparatus 100 according to an embodiment may determine whether it corresponds to a condition for performing NSST (S520). More specifically, the encoding apparatus 100 may determine whether it corresponds to a condition for encoding the NSST index. In this case, the NSST index may be referred to as, for example, a transform index.

If the cbf flag indicates 0 in S500, the encoding apparatus 100 according to an exemplary embodiment may omit the operations according to S510, S520, and S530.

The encoding apparatus 100 according to an embodiment may omit the operation according to S530 when it is determined in S520 that it does not correspond to a condition for performing NSST.

5B illustrates an example of a decoding process of transform coefficients.

The decoding apparatus 200 according to an embodiment may determine whether the cbf flag indicates 1 (S540). If the cbf flag indicates 1, it may indicate that there is at least one non-zero transform coefficient among the transform coefficients for the current block, and in some cases, a non-zero transform coefficient among the transform coefficients for the current block is Two or more may be present. Conversely, if the cbf flag indicates 0, it may indicate that there are no non-zero transform coefficients among the transform coefficients for the current block, and in some cases, a non-zero transform coefficient among the transform coefficients for the current block. May be present in fewer than two.

The decoding apparatus 200 according to an embodiment may decode information about transform coefficients for the current block when the cbf flag indicates 1 in S540 (S550).

The decoding apparatus 200 according to an embodiment may determine whether it corresponds to a condition for performing NSST (S560). More specifically, the decoding apparatus 200 may determine whether it corresponds to a condition for encoding the NSST index.

When the cbf flag indicates 0 in S540, the decoding apparatus 200 may omit the operations according to S550, S560, and S570.

In addition, if it is determined that the decoding apparatus 200 does not correspond to a condition for performing NSST, the decoding apparatus 200 according to an exemplary embodiment may omit the operation according to S570.

6 is a flowchart illustrating a process of coding a transform coefficient according to an embodiment.

Each step disclosed in FIG. 6 may be performed by the encoding apparatus 100 or the decoding apparatus 200 disclosed in FIGS. 1 and 2, and more specifically, the entropy encoding unit 130 disclosed in FIG. 1 or the disclosure described in FIG. 2. It may be performed by the entropy decoding unit 210. In addition, S600 of FIG. 6 may correspond to S500 of FIG. 5A and S540 of FIG. 5B, and S605 to S625 and S645 of FIG. 6 may correspond to S510 of FIG. 5A and S550 of FIG. 5B, and S630 of FIG. 6. And S635 may correspond to S520 and S530 of FIG. 5A and S560 and S570 of FIG. 5B. Therefore, specific details overlapping with the above-described contents in FIGS. 1, 2, 5A, and 5B will be omitted or simply described.

In the present specification, when referring to an encoding apparatus, a transform performed first on residual samples for a current block is referred to as a primary transform, and a transform performed second is referred to as a secondary transform. do. However, the primary transform may be referred to by various names such as core transform, core transform, primary transform, primary transform, adaptive multiple transform, AMT, EMT, MTS, and secondary transform may be non-separated secondary transform, NSST, MDNSST, etc. It may be referred to by various names, examples are not limited thereto.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether the cbf flag indicates 1 (S600).

When the cbf flag indicates 1 in S600, the encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether a single transform indicator (STI) indicates 1 (S605). “STI” in this specification indicates whether to apply a single transform instead of applying both transforms (eg, primary transform and secondary transform) to transform the residual coefficients. It may mean information for.

More specifically, when the cbf flag indicates 1 in S600, the encoding apparatus 100 or the decoding apparatus 200 may determine whether the STI is enabled, and when the STI is enabled, the STI sets 1 Can be determined. However, the process of determining whether the STI is activated is not a required process, and in another embodiment, it may be determined whether the STI indicates 1 directly without determining whether the STI is activated. Alternatively, if the STI indicates 1 in some cases, it may indicate that the STI is activated.

In one example, the STI may be referred to as STI coding, STI coding flag, ST flag, or the like. When the STI indicates 1, it may mean that a single transform is applied in converting the residual coefficients. When the STI indicates 0, it may mean that a single transform is not applied in converting the residual coefficients.

The type of a single transformation can vary. Examples of a single transform may be a separable transform, a non-separable transform, an approximated transform from the non-separable transform, and the like.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code (encode or decode) EMT_CU_Flag (or MTS_CU_Flag, hereinafter same) when STI indicates 0 in S605 (S610).

When the cbf flag indicates 0, the encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may skip the operations according to S605 and S610 and proceed to the operation according to S615.

The EMT (or MTS, hereinafter same) may correspond to an example of first-order transformation or core transformation, as described above with reference to FIG. 3. The EMT (or MTS) may be applied to a coding block CU whose width and height are less than or equal to 32, and in some cases, may be applied to a coding block smaller than or equal to 64. Whether to apply EMT (or MTS) may be controlled based on the EMT_CU_Flag (or MTS_CU_Flag) signaled at the CU level. If EMT_CU_Flag (or MTS_CU_Flag) indicates 0, DCT type 2 may be used to perform residual coding for the current block. Two additional flags may be signaled to identify the horizontal and vertical transforms to be used in the luma coding block in the coding block to which the EMT (or MTS) is applied.

In intra residual coding, a mode-dependent transform candidate selection process may be used due to different residual states of different intra prediction modes. Examples of the three transform subsets are shown in Table 3 below, and the relationship between the intra prediction mode and the transform subset is shown in Table 4 below.

TABLE 3

TABLE 4

Based on the subset concept, one transform (sub) set may be derived using the intra prediction mode when EMT_CU_flag (or MTS_CU_flag) indicates 1 according to Table 3. In addition, referring to Table 4, the EMT_TU_horizontal_index (or MTS_TU_horizontal_index, hereinafter same), which is the EMT_TU index (or MTS_TU_index, hereinafter same) in the horizontal direction for each intra mode, or the same as the EMT_TU_horizontal_flag (or, MTS_TU_horizontal_flag, hereinafter vertical) EMT_TU_vertical_index (or MTS_TU_vertical_index, hereinafter same) (or EMT_TU_horizontal_flag (or EMT_TU_horizontal_flag, hereinafter same)), which is an EMT_TU index (or MTS_TU index) in the direction, may be allocated. A flag for determining one of the transform candidates may be signaled.

Referring back to S610, the decoding apparatus 200 according to an embodiment decodes EMT_CU_Flag (MTS_CU_Flag), and when EMT_CU_Flag (MTS_CU_Flag) indicates 1, one transform set using an intra prediction mode based on Table 3 Can be derived.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code (encode or decode) transform coefficients for the current block (S615). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may encode or decode transform coefficients derived by performing transform on residual coefficients for the current block.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether the EMT_TU (or MTS_TU, hereinafter identical) is coded (or coded) (S620). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may determine whether the EMT_TU index (or MTS_TU index) is coded. As described above, the EMT_TU index (or MTS_TU index) is the EMT_TU_horizontal_index (MTS_TU_horizontal_index) (or EMT_TU_horizontal_flag (MTS_TU_horizontal_flag) in the vertical direction (or EMT_TU_horizontal_flag)), which is the EMT_TU index (or MTS_TU index) in the horizontal direction, as described above. It may include an EMT_TU_vertical_index (MTS_TU_vertical_index) (or an EMT_TU_vertical_flag (MTS_TU_vertical_flag)) which is an MTS_TU index.

If the encoding apparatus 100 or the decoding apparatus 200 according to an embodiment determines that the EMT_TU (or MTS_TU) is not coded (or not to be coded) in S620, the encoding apparatus 100 or the decoding apparatus 200 according to the embodiment may be added to the EMT_TU index (or, MTS_TU index). The coding may be skipped and an operation according to S630 may be performed.

If it is determined that the EMT_TU (or MTS_TU) is coded in S620, the encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code (encode or decode) the EMT_TU index (or MTS_TU index). There is (S625). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may code an EMT_TU index (or MTS_TU index) for each intra mode. In one example, the encoding apparatus 100 or the decoding apparatus 200 may have an EMT_TU_horizontal_index (MTS_TU_horizontal_index) (or EMT_TU_horizontal_flag (MTS_TU_horizontal_flag)) which is a horizontal EMT_TU index (or MTS_TU index) for each intra mode. EMT_TU_vertical_index (MTS_TU_vertical_index) (or EMT_TU_vertical_flag (MTS_TU_vertical_flag)) that is (or MTS_TU index) may be coded. An example of an EMT_TU index (or MTS_TU index) may be as shown in Table 3, and an example in which an EMT_TU index (or MTS_TU index) is allocated to each intra mode may be as shown in Table 4. In addition, as described above, a flag for determining one of two transform candidates included in the transform set of Table 3 may be further signaled.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether it corresponds to a condition for performing NSST (S630). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may determine whether it corresponds to a condition for coding (encoding or decoding) the NSST index.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code an NSST index when it is determined that the condition corresponds to a condition for performing NSST in S630 (S635). More specifically, when it is determined in S630 that the conditions for coding the NSST index, the encoding apparatus 100 or the decoding apparatus 200 may encode or decode the NSST index.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code an STI index when STI indicates 1 in S605 (S640). More specifically, when the STI is activated and the STI indicates 1, the encoding apparatus 100 or the decoding apparatus 200 may code the STI index. However, the embodiment is not limited thereto, and in another embodiment, the encoding apparatus 100 or the decoding apparatus 200 may code the STI index regardless of whether the STI indicates one. In other words, the encoding apparatus 100 or the decoding apparatus 200 may code the STI index without omitting the determination of whether the STI indicates 1.

The STI index may mean an index corresponding to a transform (or a transform kernel) to be applied to the current block among a plurality of available transform candidates.

In one example, the decoding apparatus 200 may derive the STI without separate signaling from the encoding apparatus 100. For example, a condition in which the size of the current block is the same as a predetermined value such as 4 and 8, a condition in which the current block is a square shape having the same width and height, and an intra prediction mode are predetermined as in the DC mode or the planar mode. If at least one of the conditions, which is one of the modes, is satisfied, the STI may indicate 1.

In another example, the decoding apparatus 200 may receive information on the STI from the encoding apparatus 100. That is, information about the STI may be signaled from the encoding apparatus 100 to the decoding apparatus 200. In addition, the information on the STI may be conditionally signaled in consideration of at least one of the size of the current block, the shape of the current block, and the intra prediction mode.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code transform coefficients for the current block (S645). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may code transform coefficients for the current block based on the STI index derived in S640.

However, the embodiment is not limited thereto, and for example, transform coefficients for the current block may be coded before the STI index is coded. That is, the transform coefficients for the current block are coded before the STI index is coded, so that information obtained in the process of deriving the transform coefficients for the current block may be used when deriving the STI index.

In one example, the STI index may be determined based on at least one of the size of the current block, the intra prediction mode, and the number of nonzero transform coefficients. In another example, the STI index may be conditionally encoded or decoded based on at least one of the size of the current block, the shape of the current block, the intra prediction mode, and the number of nonzero transform coefficients. In another example, whether to skip signaling of the STI index may be determined based on a distribution state of non-zero transform coefficients in the current block. More specifically, when a non-zero transform coefficient is detected in a region except the upper left 4x4 region of the current block, signaling of the STI index may be skipped.

7 is a flowchart illustrating a coding process of transform coefficients, according to an exemplary embodiment.

Each step disclosed in FIG. 7 may be performed by the encoding apparatus 100 or the decoding apparatus 200 disclosed in FIGS. 1 and 2, and more specifically, the entropy encoding unit 130 disclosed in FIG. 1 or the disclosure described in FIG. 2. It may be performed by the entropy decoding unit 210. In addition, S700 of FIG. 7 may correspond to S500 of FIG. 5A and S540 of FIG. 5B, and S705 to S725 and S745 of FIG. 7 may correspond to S510 of FIG. 5A and S550 of FIG. 5B, and S730 of FIG. 7. And S735 may correspond to S520 and S530 of FIG. 5A and S560 and S570 of FIG. 5B. Accordingly, detailed descriptions overlapping with those described above in FIGS. 1, 2, 5A, 5B, and 6 will be omitted or simplified.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether the cbf flag indicates 1 (S700).

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether the Unified Transform Indicator (UTI) indicates 1 when the cbf flag indicates 1 in S700 (S705). In the present specification, “UTI” may include an indicator of each of two transforms (eg, a primary transform and a secondary transform) for transforming residual coefficients. In one example, the UTI can include at least one of a primary transform indicator and a secondary transform indicator. The primary transform indicator may indicate a transform kernel for the primary transform and the secondary transform indicator may indicate a transform kernel for the secondary transform.

More specifically, the encoding apparatus 100 or the decoding apparatus 200 may determine whether the UTI is enabled when the cbf flag indicates 1 in S700, and if the UTI is enabled, the UTI is 1. Can be determined. However, the process of determining whether the UTI is activated is not a mandatory process, and in another embodiment, it may be determined whether the UTI indicates 1 directly without determining whether the UTI is activated. Alternatively, in some cases, when the UTI indicates 1, it may indicate that the UTI is activated.

In one example, the UTI may be referred to as UTI coding, UTI coding flag, UT flag, or the like. If the UTI indicates 1, it may indicate that transforms applied in transforming the residual coefficients are signaled based on the UTI index. If the UTI indicates 0, the transforms applied in converting the residual coefficients indicate a UTI index. May mean not based.

The type of each transformation indicated by the UTI index may vary. Examples of the types of transformations are: separable transform, non-separable transform (NSST), approximated transform from non-separated transform, primary transform, secondary transform, core transform, primary There may be transforms and the like, and the type of each transform is not limited to the examples.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code (encode or decode) EMT_CU_Flag (or, MTS_CU_Flag) when the UTI indicates 0 in S705 (S710).

When the cbf flag indicates 0, the encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may skip the operations according to S705 and S710 and proceed with the operations according to S715.

Referring back to S710, the decoding apparatus 200 according to an embodiment may decode EMT_CU_Flag and, when EMT_CU_Flag indicates 1, may derive one transform set using the intra prediction mode based on Table 3.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code (encode or decode) transform coefficients for the current block (S715). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may encode or decode transform coefficients derived by performing transform on residual coefficients for the current block.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether the EMT_TU is coded (or coded) (S720). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may determine whether the EMT_TU index is coded. As described above, the EMT_TU index may include EMT_TU_horizontal_index (or EMT_TU_horizontal_flag) that is the horizontal EMT_TU index and EMT_TU_vertical_index (or EMT_TU_vertical_flag) that is the vertical direction for each intra mode.

If it is determined that the EMT_TU is not coded (or not coded) in S720, the encoding apparatus 100 or the decoding apparatus 200 according to an embodiment skips coding for the EMT_TU index and performs an operation according to S730. can do.

If it is determined in step S720 that the EMT_TU is coded, the encoding apparatus 100 or the decoding apparatus 200 may code (encode or decode) the EMT_TU index (S725). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may code an EMT_TU index for each intra mode. In an example, the encoding apparatus 100 or the decoding apparatus 200 may code, for each intra mode, an EMT_TU_horizontal_index (or EMT_TU_horizontal_flag) that is a horizontal EMT_TU index and an EMT_TU_vertical_index (or EMT_TU_vertical_flag) that is a vertical EMT_TU index. An example of the EMT_TU index may be as shown in Table 3, and an example in which the EMT_TU index is allocated to each intra mode may be as shown in Table 4. In addition, as described above, a flag for determining one of two transform candidates included in the transform set of Table 3 may be further signaled.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may determine whether it corresponds to a condition for performing NSST (S730). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may determine whether it corresponds to a condition for coding (encoding or decoding) the NSST index.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code the NSST index when it is determined that the condition corresponds to the condition for performing NSST in S630 (S735). More specifically, when it is determined in S630 that the conditions for coding the NSST index, the encoding apparatus 100 or the decoding apparatus 200 may encode or decode the NSST index.

In one embodiment, data hiding and implicit coding methods for the transform indicator may be used. The transform indicator can include an STI index, a UTI index, an EMT_CU_Flag, an EMT_TU_Flag, an NSST index, and any index related to the transform that can be used to indicate the transform kernel. In one example, the transform indicator is not signaled and information corresponding to the transform indicator can be derived in the coding process of transform coefficients for the current block. However, the example is not limited thereto, and in another example, the transform indicator may be signaled, and the transform indicator signaled within the transform coefficient coding process for deriving information corresponding to the transform indicator may be used.

The information corresponding to the conversion indicator is, for example, information about Last_position_x, information about Last_position_y, information about Group flag, information about Significance map, information about Greater_than_1 flag, information about Greater_than_2 flag, and information about Greater_than_3 flag. It may include at least one of information on Remaining level coding, Information on Sign coding, and Information on Parity_map.

In one example, the information about Last_position_x and the information about Last_position_y may represent the x-axis position information and the y-axis position information of the last transform coefficient other than 0, respectively.

In another example, the information on the group flag may indicate whether a non-zero transform coefficient is included in a subblock (or subgroup) of the current block. The group flag may be signaled through various field names, for example, Group_Flag, group_flag, subgroup_flag, subblock_flag, coded_sub_block_flag, and the example is not limited thereto.

In another example, the information about the significance map may include information about valid coefficients among transform coefficients for the target block.

In another example, the information on the Greater_than_1 flag may include information about transform coefficients having an absolute value greater than 1 among valid transform coefficients for the target block.

In another example, the information about the Parity_map may include information about whether the corresponding transform coefficient is even for the case where the absolute value of the valid transform coefficients for the target block is 2 or more.

In another example, the information on the Greater_than_2 flag may include information about transform coefficients having an absolute value of 2 or more among valid transform coefficients for the target block. However, the example is not limited thereto. For example, the information on the Greater_than_2 flag may represent information on transform coefficients having an absolute value of 3 or more among valid transform coefficients for the target block in consideration of the information on Parity_map. . In another example, information about the Greater_than_3 flag may be derived instead of information about the Greater_than_2 flag, and the information about the Greater_than_3 flag may have an absolute value of 2 among the valid transform coefficients for the target block in consideration of the information about the Parity_map. Information on the above-described transform coefficients may be indicated.

In another example, the information on the retaining level coding may indicate information for level coding of transform coefficients classified based on the information on the Greater_than_2 flag or the information on the Greater_than_3 flag.

In another example, the information on sign coding may indicate information about a sign of transform coefficients for the current block.

Meanwhile, the information about the transform indicator may be used in the transform coefficient coding process. In order to use the information on the transform indicator in the transform coefficient coding process, for example, information on the pattern of the sign coding, information on the absolute value of the remaining level, information on the number of the Greater_than_1 flag, Last_position_X At least one of a value and a value of Last_position_Y may be used.

Meanwhile, the information on Last_position_x, information on Last_position_y, information on Group flag, information on Significance map, information on Greater_than_1 flag, information on Greater_than_2 flag, information on Greater_than_3 flag, and information on Remaining level coding For example, information about sign coding and information about Parity_map are merely examples of syntax elements that may be signaled in the residual coding process, and thus, the names or definitions may vary slightly from case to case. It will be readily understood by those skilled in the art.

For example, information about Last_position_x may be represented by last_sig_coeff_x, and information about last_sig_coeff_x may be derived by considering information about last_sig_coeff_x_prefix and information quantum about last_sig_coeff_x_suffix. In addition, the information on Last_position_y may be represented by last_sig_coeff_y, and the information on last_sig_coeff_y may be derived in consideration of information about last_sig_coeff_y_prefix and information quantum about last_sig_coeff_y_suffix.

In another example, the Greater_than_1 flag, Greater_than_2 flag, Remaining level coding, and Significance map are syntax elements for classifying the absolute values of the transform coefficients based on integers 0, 1, 2, 3, and the like. The signaling method may have a slight difference in some cases. The significance map may be referred to as sig_coeff_flag. The Greater_than_1 flag may be referred to as rem_abs_gt1_flag and may indicate whether the absolute value of the transform coefficient is greater than 1 or greater than 1. The Greater_than_2 flag may be referred to as rem_abs_gt2_flag and may indicate whether the absolute value of the transform coefficient is greater than 2, greater than 2, greater than 3, or greater than 3. The Greater_than_3 flag may be referred to as rem_abs_gt3_flag and may indicate whether the absolute value of the transform coefficient is greater than 2, greater than 2, greater than 3, or greater than 3. When rem_abs_gt1_flag indicates 1, rem_abs_gt2_flag or rem_abs_gt3_flag may be signaled. If rem_abs_gt2_flag or rem_abs_gt3_flag indicates 1, abs_remainder including information on the remaining transform coefficients not derived may be signaled. Alternatively, dec_abs_level including information about transform coefficients derived in the transform coefficient coding process may be signaled. One of ordinary skill in the art will understand that the syntax elements described above mainly classify the absolute values of the transform coefficients on the basis of 0, 1, 2, 3, etc., so that the Greater_than_1 flag, Greater_than_2 flag, Remaining level coding and Significance It will be readily understood that the correspondence between syntax elements such as map and syntax elements such as rem_abs_gt1_flag, rem_abs_gt2_flag, rem_abs_gt3_flag, abs_remainder, sig_coeff_flag, etc. may be different from those described above in some cases.

In another example, the information about the parity map may be referred to as par_level_flag. The par_level_flag may be signaled only when sig_coeff_flag indicates 1 for the corresponding transform coefficient, and may be signaled only when rem_abs_gt1_flag indicates 1 for the corresponding transform coefficient. Alternatively, rem_abs_gt1_flag may be signaled only when par_level_flag indicates 1 by changing the signaling order.

Meanwhile, the above-described data hiring may be applied conditionally. For example, data Haiing may depend on the number of nonzero transform coefficients.

In addition, in one embodiment, the NSST index and the EMT_CU_Flag or EMT_TU index may be interdependent. For example, when EMT_CU_Flag is 0 (or 1), the NSST index may not be zero. In this case, NSST index-1 may be signaled instead of NSST index.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code the UTI index when the UTI indicates 1 in S705 (S740). More specifically, when the UTI is activated and the UTI indicates 1, the encoding apparatus 100 or the decoding apparatus 200 may code the UTI index. However, the embodiment is not limited thereto, and in another embodiment, the encoding apparatus 100 or the decoding apparatus 200 may code the UTI index regardless of whether the UTI indicates 1. In other words, the encoding apparatus 100 or the decoding apparatus 200 may code the UTI index without omitting the determination of whether the UTI indicates 1.

Coding for the UTI index may depend on at least one of the size of the current block, the shape of the current block, and the intra prediction mode.

The UTI index is at least one of a primary transform (or kernel for primary transform) and a secondary transform (or transform kernel for secondary transform) to be applied to the current block among a plurality of available transform candidates. You can indicate one. In one embodiment, how to derive (or extract) the primary transform from the UTI and how to derive (or extract) the secondary transform from the UTI may be predefined.

In one example, the UTI index may indicate both a primary transform (or a transform kernel for the primary transform) and a secondary transform (or a transform kernel for the secondary transform). More specifically, the UTI index joint coding an index for indicating a primary transform (or a transform kernel for a primary transform) and an index for indicating a secondary transform (or a transform kernel for a secondary transform). )can do. An example of a UTI index in which an index for indicating a primary transform and an index for indicating a secondary transform is jointly coded is shown in Table 5 below.

TABLE 5

Referring to Table 5, the indices for indicating the primary transform are described from 0 to 5, and the indices for indicating the secondary transform are described from 0 to 3. Each index may indicate a specific conversion type. UTI indices jointly coded with an index for indicating a primary transform and an index for indicating a secondary transform are described from 0 to 31, and binary codes corresponding to each UTI index are described from 00000 to 11111. In one example, the binary code corresponding to each UTI index may be a fixed length coding (FLC) binary code, but the example is not limited thereto.

As used herein, "UTI index list" may mean a set of UTI indexes indicating transform kernel combinations. For example, UTI indexes 0 to 31 of Table 5 may constitute one UTI index list. In an embodiment, the UTI index list may be determined based on at least one of a size of a current block, a shape of the current block, and an intra prediction mode applied to the current block.

In one embodiment, when a predefined condition is satisfied, the UTI index may indicate a primary transform type. More specifically, when the size of the current block is smaller than the predefined threshold, the UTI index may indicate only the transform kernel for the primary transform. Here, the predefined threshold may include, for example, 2x2, 4x4, 8x8, and the like, but the example is not limited thereto.

In another embodiment, when a primary transform is determined based on at least one of a size of a current block, a shape of a current block, and an intra prediction mode applied to the current block, the UTI index is an index for indicating a secondary transform. Can be indicated. More specifically, when the transform kernel for the first transform is predetermined based on at least one of the size of the current block, the shape of the current block, and the intra prediction mode applied to the current block, the UTI index is used to transform the second transform. Can only point to the kernel.

In another embodiment, when a second transform is predetermined based on at least one of a size of a current block, a shape of a current block, and an intra prediction mode applied to the current block, the UTI index is used to indicate a primary transform. It can indicate an index. More specifically, when the transform kernel for the second transform is predetermined based on at least one of the size of the current block, the shape of the current block, and the intra prediction mode applied to the current block, the UTI index is used to transform the first transform. Can only point to the kernel.

In another embodiment, if it is determined that no secondary transform is indicated via the UTI index (eg, via signaling or when a secondary transform is predetermined), the UTI index is used to indicate the primary transform. It can indicate an index.

In another embodiment, if it is determined that no primary transform is indicated via the UTI index (eg, via signaling or when a primary transform is predetermined), the UTI index is used to indicate the secondary transform. It can indicate an index.

The encoding apparatus 100 or the decoding apparatus 200 according to an embodiment may code transform coefficients for the current block (S745). More specifically, the encoding apparatus 100 or the decoding apparatus 200 may code transform coefficients for the current block based on the UTI index derived in S740. However, the embodiment is not limited thereto, and for example, transform coefficients for the current block may be coded before the UTI index is coded. A more detailed description of an embodiment in which transform coefficients for the current block are coded before the UTI index is coded will be described later with reference to FIGS. 8A and 8B.

8A and 8B are flowcharts illustrating a coding order of transform coefficients and UTI indexes, according to an exemplary embodiment.

Each of the steps disclosed in FIGS. 8A and 8B may be performed by the encoding apparatus 100 or the decoding apparatus 200 disclosed in FIGS. 1 and 2, and more specifically, the entropy encoding unit 130 illustrated in FIG. 1 or FIG. It may be performed by the entropy decoding unit 210 disclosed in 2. In addition, S800 of FIG. 8A and S830 of FIG. 8B may correspond to S500 of FIG. 5A and S540 of FIG. 5B, S810 of FIG. 8A and S850 of FIG. 8B may correspond to S740 of FIG. 7, and FIG. 8A of FIG. S820 and S840 of FIG. 8B may correspond to S745 of FIG. 7. In summary, detailed descriptions that overlap with the above-described contents in FIGS. 1, 2, 5A, 5B, 6, and 7 will be omitted or simplified. In FIGS. 8A and 8B, the front and rear sides between S810 and S820 will be omitted. The relationship and the future relationship between S840 and S850 will be reviewed.

In one example, the UTI index may be determined based on at least one of the size of the current block, the intra prediction mode, and the number of nonzero transform coefficients. In another example, the UTI index may be conditionally encoded or decoded based on at least one of the size of the current block, the shape of the current block, the intra prediction mode, and the number of nonzero transform coefficients. In another example, whether to skip signaling of the UTI index may be determined based on a distribution state of non-zero transform coefficients in the current block. More specifically, when a non-zero transform coefficient is detected in a region other than the upper left 4x4 region of the current block, signaling of the UTI index may be skipped.

In one embodiment, as shown in FIG. 8A, transform coefficients for the current block may be coded (ie, encoded or decoded) (S820) after the UTI index is coded (ie, encoded or decoded) (S810). . In other words, the transform coefficients for the current block may be coded (ie, encoded or decoded) after the UTI index is coded (ie, encoded or decoded). Since the UTI index is coded before the transform coefficients for the current block are coded, the information obtained in the process of deriving the UTI index may be used when deriving the transform coefficients for the current block.

In another embodiment, as shown in FIG. 8B, the UTI index may be coded (ie, encoded or decoded) (S850) after the transform coefficients for the current block are coded (ie, encoded or decoded) (S840). have. In other words, the UTI index may be coded (ie, encoded or decoded) after the transform coefficients for the current block are coded (ie, encoded or decoded). Since transform coefficients for the current block are coded before the UTI index is coded, information obtained in the process of deriving the transform coefficients for the current block may be used when deriving the UTI index.

9 is a flowchart illustrating an operation of an encoding apparatus according to an embodiment, and FIG. 10 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment.

Each step disclosed in FIG. 9 may be performed by the encoding apparatus 100 disclosed in FIG. 1. More specifically, S900 and S910 may be performed by the transform unit 122 shown in FIG. 1, and S920 and S930 may be performed by the entropy encoding unit 130 shown in FIG. 1. In addition, operations according to S900 to S930 are based on some of the contents described above with reference to FIGS. 7 to 8B. Therefore, specific details overlapping with the above-described contents in FIGS. 1 and 7 to 8b will be omitted or simply described.

As shown in FIG. 10, the encoding apparatus according to the embodiment may include a transformer 122 and an entropy encoder 130. However, in some cases, not all of the components shown in FIG. 10 may be required components of the encoding apparatus, and the encoding apparatus may be implemented by more or fewer components than those illustrated in FIG. 10.

In the encoding apparatus according to an embodiment, the converter 122 and the entropy encoder 130 may be implemented as separate chips, or at least two or more components may be implemented through one chip.

The encoding apparatus according to an embodiment may derive residual samples for the current block (S900). More specifically, the converter 122 of the encoding apparatus may derive residual samples for the current block.

The encoding apparatus according to an embodiment may perform transform on residual samples for the current block to derive transform coefficients (S910). More specifically, the transform unit 122 of the encoding apparatus may derive transform coefficients by performing transform on residual samples of the current block.

In one example, the transform can include at least one of a primary transform (or core transform) and a secondary transform, and each of the primary transform and the secondary transform can be based on one transform type of the plurality of transform types. . The plurality of transform types of the primary transform may include the multiple transform set described above in FIG. 3, and the plurality of transform types of the secondary transform may include a non-separated secondary transform (NSST).

According to an embodiment, the encoding apparatus may derive a UTI index indicating a transform type of the performed transform (S920). More specifically, the entropy encoding unit 130 of the encoding apparatus may derive a UTI index indicating the transform type of the performed transform.

The UTI index is at least one of a primary transform (or kernel for primary transform) and a secondary transform (or transform kernel for secondary transform) to be applied to the current block among a plurality of available transform candidates. You can indicate one.

In one example, the UTI index may indicate both a primary transform (or a transform kernel for the primary transform) and a secondary transform (or a transform kernel for the secondary transform). More specifically, the UTI index joint coding an index for indicating a primary transform (or a transform kernel for a primary transform) and an index for indicating a secondary transform (or a transform kernel for a secondary transform). )can do.

In another example, when the size of the current block is smaller than a predefined threshold, the UTI index may indicate only the transform kernel for the primary transform.

In another example, when a primary transform is predetermined based on at least one of a size of a current block, a shape of a current block, and an intra prediction mode applied to the current block, the UTI index is an index for indicating a secondary transform. Can be indicated.

In another example, when a second transform is predetermined based on at least one of a size of a current block, a shape of a current block, and an intra prediction mode applied to the current block, the UTI index is an index for indicating a primary transform. Can be indicated.

According to an embodiment, the encoding apparatus may encode image information including information on transform coefficients and a UTI index (S930). More specifically, the entropy encoding unit 130 of the encoding apparatus may encode information about transform coefficients and image information including a UTI index.

According to the encoding apparatus and the operation method of the encoding apparatus disclosed in FIGS. 9 and 10, the encoding apparatus derives residual samples for the current block (S900), performs transformation on the residual samples for the current block, and performs the transformation. Coefficients may be derived (S910), a UTI index indicating a transform type of the performed transform may be derived (S920), and information about the transform coefficients and image information including the UTI index may be encoded (S930). By pointing at once the transform type of the primary transform (or core transform) and the secondary transform based on the UTI index, the coding efficiency (or coding gain) of the encoding apparatus may be increased.

11 is a flowchart illustrating an operation of a decoding apparatus according to an embodiment, and FIG. 12 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment.

Each step disclosed in FIG. 11 may be performed by the decoding apparatus 200 disclosed in FIG. 2. More specifically, S1100 to S1120 may be performed by the entropy decoding unit 210 shown in FIG. 2, S1130 may be performed by the inverse transform unit 223 shown in FIG. 2, and S1140 may be added as shown in FIG. 2. It may be performed by the unit 240. In addition, operations according to S1100 to S1140 are based on some of the contents described above with reference to FIGS. 7 to 8B. Therefore, detailed descriptions overlapping with those described above with reference to FIGS. 2 and 7 to 8B will be omitted or simply described.

As shown in FIG. 12, the decoding apparatus according to the embodiment may include an entropy decoding unit 210, an inverse transform unit 223, and an adder 240. However, in some cases, all of the components shown in FIG. 12 may not be essential components of the decoding apparatus, and the decoding apparatus may be implemented by more or fewer components than those illustrated in FIG. 12.

In the decoding apparatus according to an embodiment, the entropy decoding unit 210, the inverse transform unit 223, and the adder 240 are each implemented as separate chips, or at least two or more components are implemented through one chip. May be

The decoding apparatus according to an embodiment may decode the UTI index for the current block from the bitstream (S1100). More specifically, the entropy decoding unit 210 of the decoding apparatus may decode the UTI index for the current block from the bitstream.

The UTI index may indicate at least one of a primary transform (or a kernel for the primary transform) and a secondary transform (or a transform kernel for the secondary transform) to be applied to the current block among the plurality of valid transform candidates.

In one example, the UTI index may indicate (or both) the primary transform (or transform kernel for the primary transform) and the secondary transform (or transform kernel for the secondary transform). More specifically, the UTI index may joint code an index for indicating a primary transform (or a transform kernel for a primary transform) and an index for indicating a secondary transform (or a transform kernel for a secondary transform). .

In another example, when the size of the current block is smaller than a predefined threshold, the UTI index may indicate only the transform kernel for the primary transform.

In another example, when a primary transform is predetermined based on at least one of a size of a current block, a shape of a current block, and an intra prediction mode applied to the current block, the UTI index is an index for indicating a secondary transform. Can be indicated.

In another example, when a second transform is predetermined based on at least one of a size of a current block, a shape of a current block, and an intra prediction mode applied to the current block, the UTI index is an index for indicating a primary transform. Can be indicated.

The decoding apparatus according to an embodiment may decode transform coefficients for the current block from the bitstream (S1110). More specifically, the entropy decoding unit 210 of the decoding apparatus may decode transform coefficients for the current block from the bitstream.

The decoding apparatus according to an embodiment may derive a transform type for the current block based on the UTI index (S1120). More specifically, the entropy decoding unit 210 of the decoding apparatus may derive a transform type for the current block based on the UTI index.

In one example, the transform can include at least one of a primary transform (or core transform) and a secondary transform, and each of the primary transform and the secondary transform can be based on one transform type of the plurality of transform types. . The plurality of transform types of the primary transform may include the multiple transform set described above in FIG. 3, and the plurality of transform types of the secondary transform may include a non-separated secondary transform (NSST).

The decoding apparatus according to an embodiment may perform inverse transformation based on the derived transform type and transform coefficients for the current block (S1130). More specifically, the inverse transform unit 223 of the decoding apparatus may perform inverse transform based on the derived transform type and transform coefficients for the current block.

The decoding apparatus according to an embodiment may generate a reconstructed picture based on residual samples for the current block derived through inverse transform (S1140). More specifically, the adder 240 of the decoding apparatus may generate a reconstructed picture based on residual samples for the current block derived through inverse transform.

According to the decoding apparatus and the operating method of the decoding apparatus disclosed in FIGS. 11 and 12, the decoding apparatus decodes the UTI index for the current block from the bitstream (S1100), decodes the transform coefficients for the current block from the bitstream ( S1110), a transform type for the current block is derived based on the UTI index (S1120), an inverse transform is performed based on the derived transform type and transform coefficients for the current block (S1130), and the current block derived through the inverse transform A reconstructed picture may be generated based on the residual samples for (S1140).

The above-described method according to the present invention may be implemented in software, and the encoding device and / or the decoding device according to the present invention may perform image processing of, for example, a TV, a computer, a smartphone, a set-top box, a display device, and the like. It can be included in the device.

When the embodiments of the present invention are implemented in software, the above-described method may be implemented as a module (process, function, etc.) for performing the above-described function. The module may be stored in memory and executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor by various well known means. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium and / or other storage device.

Claims (15)

In the picture decoding method performed by the decoding apparatus, Decoding an unified transform indicator (UTI) index for the current block from the bitstream; Decoding transform coefficients for the current block from the bitstream; Deriving a transform type for the current block based on the UTI index; Performing an inverse transform based on the derived transform type and the transform coefficients for the current block; And Generating a reconstructed picture based on residual samples for the current block derived through the inverse transform, And the UTI index indicates at least one of a transform kernel for a first order transform and a transform kernel for a second order transform. The method of claim 1, Performing the inverse transform, Deriving modified transform coefficients by performing second-order inverse transform on the transform coefficients for the target block based on the transform kernel for the secondary transform indicated by the UTI index; And And performing a first order inverse transform on the modified transform coefficients based on the transform kernel for the first transform indicated by the UTI index. The method of claim 1, And the UTI index indicates both a transform kernel for the primary transform and a transform kernel for the secondary transform. The method of claim 1, The UTI index indicates one of the transform kernel combinations included in the UTI index list, wherein the UTI index list is a size of the current block, a shape of the current block, and an intra applied to the current block. And determine based on at least one of the prediction modes. The method of claim 1, And if the size of the current block is smaller than a predefined threshold, the UTI index indicates a transform kernel for the first transform. The method of claim 4, wherein When a transform kernel for the first transform is predetermined based on at least one of the size of the current block, the shape of the current block, and an intra prediction mode applied to the current block, the UTI index is determined for the second transform. And picture a transform kernel. The method of claim 4, wherein When a transform kernel for the second transform is predetermined based on at least one of the size of the current block, the shape of the current block, and an intra prediction mode applied to the current block, the UTI index is determined for the first transform. And picture a transform kernel. The method of claim 1, And the transform coefficients for the current block are decoded after the UTI index is decoded. The method of claim 1, And the transform coefficients for the current block are decoded before the UTI index. In the picture encoding method performed by the encoding device, Deriving residual samples for the current block; Performing transform on the residual samples for the current block to derive transform coefficients; Deriving a Unified Transform Indicator (UTI) index that represents the transform type of the performed transform; And Encoding information about the transform coefficients and the image information including the UTI index, And the UTI index indicates at least one of a transform kernel for a first order transform and a transform kernel for a second order transform. The method of claim 10, Performing the conversion, Performing a first order transform on the residual samples for the current block based on the transform kernel for the first order indicated by the UTI index; And And performing a second transform on transform coefficients on which the first transform is performed based on the transform kernel for the second transform indicated by the UTI index to derive modified transform coefficients. . The method of claim 10, The UTI index indicates one of transform kernel combinations included in a UTI index list, and the UTI index list includes at least one of a size of the current block, a type of the current block, and an intra prediction mode applied to the current block. The picture encoding method, characterized in that determined based on. The method of claim 10, And if the size of the current block is smaller than a predefined threshold, the UTI index indicates a transform kernel for the first transform. The method of claim 10, And the transform coefficients for the current block are encoded after the UTI index is encoded. In the decoding apparatus for decoding a picture, Decode a Unified Transform Indicator (UTI) index for the current block from the bitstream, decode transform coefficients for the current block from the bitstream, and derive a transform type for the current block based on the decoded UTI index. An entropy decoding unit; An inverse transform unit performing an inverse transform based on the derived transform type and the transform coefficients for the current block; And Including an adder for generating a reconstruction picture based on the residual samples for the current block derived through the inverse transform, And the UTI index indicates at least one of a transform kernel for a first order transform and a transform kernel for a second order transform.

Images