WO2018169267A1 - Device and method for encoding or decoding image - Google Patents

Abstract

The present invention relates to efficiently signaling division information related to quadtree plus binary tree (QTBT)-based block partitioning. An image encoding device according to one aspect of the present invention determines the greatest QT leaf node depth for indicating the highest depth among depths in which at least one leaf node of a quadtree is present in a determined block portioning structure, and signals depth information for indicating the greatest QT leaf node depth as quadtree division information on all nodes from a root node to a parent node of nodes belonging to the greatest QT leaf node depth in the determined block partitioning structure.

Description

Apparatus and method for image encoding or decoding

The present invention relates to image encoding or decoding for efficiently encoding an image. More specifically, the present invention relates to a technique for signaling partition information about Quadtree plus Binary tree (QTBT) based block partitioning.

The contents described in this section merely provide background information on the present embodiment and do not constitute a prior art.

1 is a diagram illustrating an example of a quad tree splitting structure for a CTU and a tree structure thereof. In HEVC, a coding tree unit (CTU) is a coding tree for reflecting various local characteristics in an image, and is divided into a coding unit (CU) using a quadtree structure. When the CU is determined in this way, intra and inter coding prediction is also performed in the CU unit. Each CU is again divided into a PU (Prediction Unit). After the PU is determined and the prediction process is performed, the CU is divided into a TU (Transformation Unit) for the residual block.

Recently, a quadtree plus binary tree (QTBT) structure is newly discussed. The QTBT structure removes existing CU, PU, and TU concepts, and proposes various CU partition shapes to match various local features of video data. That is, in QTBT, a CU may be defined to have a square or rectangular shape.

In addition, although the CTU size was 64 in HEVC, the necessity of larger block sizes such as 128 and 256 has been discussed as the resolution of the image increases, and it is necessary to signal such segmentation information when the block is divided into at least 4x4 in the CTU. The amount of data that is added increases.

An object of the present invention is to efficiently signal partitioning information about a quadtree plus binary tree (QTBT) based block partitioning.

According to an aspect of the present invention, determining a QuadTree plus BinaryTree (QTBT) block partitioning structure for encoding a block of image data; And encoding the block of the image data and the segmentation information representing the determined block partitioning structure based on the determined block partitioning structure. The QTBT block partitioning structure is a structure in which a binary tree is rooted from a leaf node of a quadtree. The encoding of the split information may include determining, from the determined block partitioning structure, the highest depth among depths in which at least one leaf node of the quadtree exists (hereinafter, referred to as “highest QT leaf node depth”). ; And encoding depth information indicating the highest QT leaf node depth as quadtree split information from a root node to parent nodes of nodes belonging to the highest QT leaf node depth in the determined block partitioning structure.

According to another aspect of the invention, parsing the block of the encoded image data and the segmentation information associated with the block of the image data; And decoding the block of the encoded image data for each leaf node of the determined QTBT while determining the QTBT using the split information. The block of the encoded image data is divided into a plurality of divided blocks according to a quadtree plus binarytree (QTBT) block partitioning structure, and the QTBT block partitioning structure is formed from a binary tree (leaf node) of a quadtree (quadtree). binarytree) is a rooted structure. The splitting information includes depth information indicating the highest depth among the depths in which at least one leaf node of the quadtree exists in the QTBT block partitioning structure (hereinafter, referred to as “highest QT leaf node depth”). The determining of the QTBT includes quadtree splitting for all nodes from a root node of the QTBT to a parent node of nodes belonging to the highest QT leaf node depth using depth information indicating the highest QT leaf node depth. It includes the step of performing.

According to another aspect of the present invention, an apparatus for decoding image data is provided. The decoding apparatus includes a memory and one or more processors, wherein the one or more processors are configured to parse the block of encoded image data and the partition information associated with the block of the image data and determine the QTBT using the partition information. And decoding the block of the encoded image data for each leaf node of the QTBT.

1 is a diagram illustrating an example of a quad tree splitting structure for a CTU and a tree structure thereof.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.

3 shows an example of a plurality of intra prediction modes.

4 is an exemplary diagram of neighboring blocks of a current block.

5 illustrates an image decoding apparatus according to an embodiment of the present invention.

6 is a flowchart illustrating an exemplary operation of encoding an image by the image encoding apparatus.

7 is a flowchart illustrating an exemplary operation of decoding an image by the image decoding device.

8 is a diagram illustrating an example of a QTBT splitting structure of a CTU and a tree structure thereof.

FIG. 9 is a diagram illustrating the number of bits that can be represented as BT in one CU.

10 is a diagram illustrating an exemplary QTBT splitting structure of a CTU in a tree structure.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding an identification code to the components of each drawing, it should be noted that the same components as possible, even if shown on different drawings have the same reference numerals. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus includes a block divider 210, a predictor 220, a subtractor 230, a transformer 240, a quantizer 245, an encoder 250, an inverse quantizer 260, and an inverse transform unit ( 265, an adder 270, a filter unit 280, and a memory 290. In the image encoding apparatus, each component may be implemented as a hardware chip, or may be implemented in software and implemented so that the microprocessor executes a function of software corresponding to each component.

After dividing each picture constituting the image into a plurality of coding tree units (CTUs), the block dividing unit 210 recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU (coding unit) which is a basic unit of coding. The tree structure is a quadtree (QT) in which a parent node (or parent node) is divided into four child nodes (or child nodes) of the same size, or such a QT structure and a parent node are divided into two child nodes. A QTBT (QuadTree plus BinaryTree) structure that uses a binary tree structure may be used. That is, QTBT may be used to divide a CTU into a plurality of CUs.

In a QTBT (QuadTree plus BinaryTree) structure, the CTU may be first divided into a QT structure. Quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf nodes allowed in QT. If the leaf node of the quadtree is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, it may be further partitioned into the BT structure. In BT, there may be a plurality of partition types. For example, in some examples, there may be two types of horizontally dividing a block of a node into two blocks of the same size (ie, symmetric horizontal splitting) and vertically dividing (ie, symmetric vertical splitting). In addition, there may further be a type of dividing the block of the node into two blocks having an asymmetric shape. The asymmetrical form may include dividing a block of the node into two rectangular blocks having a size ratio of 1: 3, or dividing the block of the node in a diagonal direction.

The partition information generated by the block divider 210 by dividing the CTU by the QTBT structure is encoded by the encoder 250 and transmitted to the image decoding apparatus. Examples of the present disclosure generally relate to a technique for signaling QTBT block partition information and a technique for determining block partitioning from QTBT block partition information. As such, certain techniques of the present disclosure may be performed by the encoder 250. That is, for example, the encoder 250 may perform the techniques of the present disclosure described with reference to FIGS. 8 to 10 below. In other examples, one or more other units of the encoding apparatus may be responsible for additionally or alternatively performing the techniques of this disclosure.

Hereinafter, a block corresponding to a CU (that is, a leaf node of QTBT) to be encoded or decoded is called a 'current block'.

The prediction unit 220 generates a prediction block by predicting the current block. The predictor 220 includes an intra predictor 222 and an inter predictor 224.

The intra predictor 222 predicts pixels in the current block by using pixels (reference pixels) positioned around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction, and the peripheral pixels to be used and the equations are defined differently according to each prediction mode. In particular, the intra predictor 222 may determine an intra prediction mode to be used to encode the current block. In some examples, intra prediction unit 222 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, intra predictor 222 calculates rate distortion values using rate-distortion analysis for several tested intra prediction modes, and has the best rate distortion characteristics among the tested modes. Intra prediction mode may be selected.

3 shows an example of a plurality of intra prediction modes.

As shown in FIG. 3, the plurality of intra prediction modes may include two non-directional modes (planar mode and DC mode) and 65 directional modes.

The intra predictor 222 selects one intra prediction mode from among the plurality of intra prediction modes, and predicts the current block by using a peripheral pixel (reference pixel) and an operation formula determined according to the selected intra prediction mode. Information on the selected intra prediction mode is encoded by the encoder 250 and transmitted to the image decoding apparatus.

Meanwhile, the intra prediction unit 222 may efficiently encode intra prediction mode information indicating which mode of the plurality of intra prediction modes is used as the intra prediction mode of the current block. Some of the most probable modes as the intra prediction mode of may be determined as the most probable mode (MPM). In addition, mode information indicating whether the intra prediction mode of the current block is selected from the MPM is generated and transmitted to the encoder 250. In general, when the intra prediction mode of the current block is selected from the MPMs, the first intra identification information for indicating which mode of the MPMs is selected as the intra prediction mode of the current block is transmitted to the encoder. On the other hand, if the intra prediction mode of the current block is not selected from the MPM, the second intra identification information for indicating which mode other than the MPM is selected as the intra prediction mode of the current block is transmitted to the encoder. Alternatively, the intra prediction unit 222 according to an aspect of the present invention instead of explicitly signaling which mode among the MPMs and / or non-MPMs is selected as the intra prediction mode for predicting the current block. Can group the MPMs and / or non-MPMs and signal the index of the group to which the intra mode for predicting the current block belongs.

Hereinafter, a method of constructing the MPM list will be described. Herein, an example of configuring an MPM list with six MPMs is described as an example. However, the present invention is not limited thereto, and the number of MPMs included in the MPM list may be selected within a range of 3 to 10.

First, the MPM list is constructed using the intra prediction mode of neighboring blocks of the current block. The neighboring block may be, for example, all or some of the left block L, the upper block A, the lower left block BL, the upper right block AR, and the upper left block AL of the current block. It may include.

The intra prediction mode of these neighboring blocks is included in the MPM list. In this case, the intra prediction mode of the valid blocks in the order of the left block (L), the top block (A), the bottom left block (BL), the top right block (AR), and the top left block (AL) is included in the MPM list, The candidate is configured by adding a planar mode and a DC mode to the intra prediction modes of the blocks. Alternatively, valid modes in the order of the left block (L), the top block (A), the planar mode, the DC mode, the bottom left block (BL), the top right block (AR), and the top left block (AL) may be added to the MPM list. have.

The MPM list includes only different intra prediction modes. That is, when a duplicated mode is present, only one of them is included in the MPM list.

If the number of MPMs in the list is smaller than the predetermined number (eg, 6), the MPM may be derived by adding -1 or +1 to the directional modes in the list. In addition, when the number of MPMs in the list is smaller than the predetermined number, the number of insufficient modes is added to the MPM list in the order of vertical mode, horizontal mode, diagonal mode, and the like. You may.

The inter prediction unit 224 searches for the block most similar to the current block in the coded and decoded reference picture before the current picture, and generates a prediction block for the current block using the searched block. A motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. The motion information including the information about the reference picture and the motion vector used to predict the current block is encoded by the encoder 250 and transmitted to the image decoding apparatus.

The subtractor 230 subtracts the prediction block generated by the intra predictor 222 or the inter predictor 224 from the current block to generate a residual block.

The converter 240 converts the residual signal in the residual block having pixel values of the spatial domain into a transform coefficient of the frequency domain. The transform unit 240 may convert the residual signals in the residual block using the size of the current block as a conversion unit, or divide the residual block into a plurality of smaller subblocks and convert the residual signals in a subblock-sized transform unit. You can also convert. There may be various ways of dividing the residual block into smaller subblocks. For example, it may be divided into sub-blocks of a predetermined same size, or a quadtree (QT) scheme may be used in which the residual block is a root node.

The quantization unit 245 quantizes the transform coefficients output from the transform unit 240, and outputs the quantized transform coefficients to the encoder 250.

The encoder 250 generates a bitstream by encoding the quantized transform coefficients by using an encoding method such as CABAC. In addition, the encoder 250 encodes information such as CTU size, MinQTSize, MaxBTSize, MaxBTDepth, MinBTSize, QT split flag, BT split flag, split type, etc. related to block division, so that the image decoding apparatus is the same as the image encoding apparatus. Allows you to split blocks.

The encoder 250 encodes information about a prediction type indicating whether a current block is encoded by intra prediction or inter prediction, and encodes intra prediction information or inter prediction information according to the prediction type.

The inverse quantizer 260 inversely quantizes the quantized transform coefficients output from the quantizer 245 to generate transform coefficients. The inverse transformer 265 restores the residual block by converting the transform coefficients output from the inverse quantizer 260 from the frequency domain to the spatial domain.

The adder 270 reconstructs the current block by adding the reconstructed residual block and the predicted block generated by the predictor 220. The pixels in the reconstructed current block are used as reference pixels when intra prediction of the next order of blocks.

The filter unit 280 deblocks and filters the boundary between the reconstructed blocks in order to remove blocking artifacts that occur due to encoding / decoding of blocks. When all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

Hereinafter, an image decoding apparatus will be described.

5 illustrates an image decoding apparatus according to an embodiment of the present invention.

The image decoding apparatus includes a decoder 510, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a filter 560, and a memory 570. Like the image encoding apparatus of FIG. 2, the image decoding apparatus may be implemented by each component as a hardware chip, or may be implemented by software and a microprocessor to execute a function of software corresponding to each component.

The decoder 510 decodes the bitstream received from the image encoding apparatus, extracts information related to block division, determines a current block to be decoded, and includes prediction information and residual signal information necessary for reconstructing the current block. Extract

The decoder 510 extracts information about the CTU size from a high level syntax such as a Sequence Parameter Set (SPS) or a Picture Parameter Set (PPS) to determine the size of the CTU and determine a picture of the determined size. Split into CTUs. The CTU is determined as the highest layer of the tree structure, that is, the root node, and the CTU is partitioned using a tree structure (eg, a QTBT structure) by extracting partition information about the CTU.

Examples of the present disclosure generally relate to a technique for signaling QTBT block partition information and a technique for determining block partitioning from QTBT block partition information. As such, certain techniques of the present disclosure may be performed by the decoder 510. That is, for example, the decoder 510 may perform the techniques of this disclosure described with respect to FIGS. 8 through 10 below. In other examples, one or more other units of the decryption apparatus may be responsible for additionally or alternatively performing the techniques of this disclosure.

When the decoder 510 determines the current block (current block) to be decoded by splitting the tree structure, the decoder 510 extracts information about a prediction type indicating whether the current block is intra predicted or inter predicted.

When the prediction type information indicates intra prediction, the decoder 510 extracts a syntax element for intra prediction information (intra prediction mode) of the current block. First, the decoder 510 extracts mode information (ie, an MPM flag) indicating whether an intra prediction mode of a current block is selected from the MPMs. Also, in general, when the intra mode encoding information indicates that the intra prediction mode of the current block is selected from the MPMs, the first intra identification information for indicating which mode of the MPMs is selected as the intra prediction mode of the current block is extracted. And if the intra mode encoding information indicates that the intra prediction mode of the current block is not selected among the MPMs, a second intra for indicating which mode other than the MPM is selected as the intra prediction mode of the current block. Extract identification information. Alternatively, the intra prediction unit 222 according to an aspect of the present invention may indicate an intra identification information indicating which mode among MPMs and / or non-MPMs is selected as an intra prediction mode for predicting a current block. Instead of extracting, group MPMs and / or non-MPMs and extract intra identification information (eg, group index, etc.) indicating whether an intra mode for predicting the current block belongs.

Meanwhile, the decoder 510 extracts information on the quantized transform coefficients of the current block as information on the residual signal.

The inverse quantizer 520 inversely quantizes the quantized transform coefficients, and the inverse transformer 530 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to generate a residual block for the current block.

The predictor 540 includes an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the intra prediction is the prediction type of the current block, and the inter predictor 544 is activated when the intra prediction is the prediction type of the current block.

The intra predictor 542 determines the intra prediction mode of the current block among the plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the decoder 510, and references pixels around the current block according to the intra prediction mode. Predict the current block using

To determine the intra prediction mode of the current block, the intra predictor 542 constructs an MPM list including a predetermined number of MPMs from neighboring blocks of the current block. The method of constructing the MPM list is the same as that of the intra predictor 222 of FIG. 2.

In general, when the mode information of the intra prediction (ie, the MPM flag) indicates that the intra prediction mode of the current block is selected from the MPMs, the intra prediction unit 542 may assign the first intra identification information among the MPMs in the MPM list. Select the MPM indicated by the intra prediction mode of the current block. On the other hand, if the mode information indicates that the intra prediction mode of the current block is not selected from the MPM, the intra prediction mode of the current block is determined among the remaining intra prediction modes except the MPMs in the MPM list using the second intra identification information. do.

Alternatively, the intra prediction unit 222 of the image encoding apparatus according to an aspect of the present invention, as described above, an intra prediction mode for which mode among the MPMs and / or non-MPMs predicts the current block. Instead of explicitly signaling whether or not is selected, group the MPMs and / or non-MPMs and signal the index of the group to which the intra mode to predict the current block belongs. In this case, the intra prediction unit 542 of the image decoding apparatus may determine the final intra mode (that is, the intra mode for predicting the current block) by evaluating the intra modes belonging to the corresponding group. For example, in some examples, the intra predictor 542 may generate a reconstructed block for a plurality of intra modes belonging to a group, and evaluate the reconstructed blocks to determine a final intra mode.

The inter predictor 544 determines motion information of the current block by using a syntax element for the intra prediction mode extracted from the decoder 510, and predicts the current block by using the determined motion information.

The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer and the prediction block output from the inter predictor or the intra predictor. The pixels in the reconstructed current block are utilized as reference pixels in intra prediction of the block to be subsequently decoded.

The filter unit 560 deblocks and filters the boundary between the reconstructed blocks in order to remove blocking artifacts that occur due to block-by-block decoding and stores them in the memory 570. When all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be decoded later.

6 is a flowchart illustrating an exemplary operation of encoding an image by the image encoding apparatus. In the example of FIG. 6, the image encoding apparatus determines a QTBT block partitioning structure for encoding a block of image data (S610). The QTBT block partitioning structure is a structure in which a binary tree is rooted from a leaf node of a quadtree. The image encoding apparatus generates an encoded bitstream including the block of the image data and the segmentation information representing the determined block partitioning structure based on the determined block partitioning structure (S620).

7 is a flowchart illustrating an exemplary operation of decoding an image by the image decoding device. In the example of FIG. 7, the image decoding apparatus parses a block of image data encoded from a bitstream and segmentation information associated with the block of the image data (S710). The block of the encoded image data is divided into a plurality of partitioned blocks according to a QTBT block partitioning structure, and the QTBT block partitioning structure is a structure in which a binary tree is routed from a leaf node of a quadtree. The image decoding apparatus determines the leaf nodes of the QTBT using the split information, and decodes the block of the encoded image data for each leaf node of the QTBT (S720).

The techniques disclosed below relate to a technique for signaling QTBT block partition information applied to step S620 of FIG. 6 and a technique for determining block partitioning from QTBT block partition information applied to step S720 of FIG. 7. The techniques of this disclosure may be performed by, for example, the image encoding apparatus and the image decoding apparatus shown and described with respect to FIGS. 2 and 5. That is, in one example, the encoder described with reference to FIG. 2 may perform certain techniques described below when determining block partitioning to encode a block of image data. In another example, the decoder described with respect to FIG. 5 may perform certain techniques described below when determining block partitioning to decode a block of image data. The QTBT block partitioning can be used to partition a CTU into CUs. That is, the root node of the QTBT may be a CTU, and the leaf node of the QTBT may be a CU.

Now, referring to FIGS. 8 to 10, the encoding apparatus signals the split information obtained by dividing the CTU into a plurality of CUs, and correspondingly, the decoding apparatus uses the signaled split information to convert the CTU into the plurality of CUs. Embodiments for determining partitioning to be divided are described.

First embodiment

In the QTBT structure, the CTU that is the root node may be first divided into a QT structure. The encoding apparatus uses a first flag (QT split flag, QT_split_flag) to indicate whether the root node of the QTBT is split into a QT structure. For example, the first flag = 1 indicates that the root node is divided into four blocks of the same size, and the first flag = 0 means that the root node is not QT divided. The QT division of each node may be repeated recursively. BT partitioning may be performed on blocks that are not QT partitioned. The encoding apparatus further uses a second flag (BT split flag, BT_split_flag) to indicate whether a block is split into a BT structure. The BT split may have a plurality of split types. In this case, when the block is divided into BT structures, the encoding apparatus further uses split type information indicating the BT split type of the corresponding block.

8 is a diagram illustrating an example of a QTBT splitting structure of a CTU and a tree structure thereof. In FIG. 8, the solid line indicates the division by the QT structure, and the dotted line indicates the division by the BT structure. In FIG. 8, it is assumed that BT splitting has two splitting types, that is, a splitting of a block of a node into two blocks of the same size and a splitting type vertically.

In the QTBT splitting structure illustrated in FIG. 8, the CTU corresponding to the root node of the QTBT is divided into four child nodes belonging to depth 1. Accordingly, the encoding apparatus encodes a QT split flag (QT_split_flag = 1) indicating that the CTU is QT split. The block corresponding to the first node of the depth 1 is no longer divided by the QT, but is BT divided, and the BT division type is a vertical type. Accordingly, the encoding apparatus determines that the block corresponding to the first node of depth 1 has a QT split flag (QT_split_flag) = 0 indicating that the block is not QT splitted and a BT split flag indicating that the corresponding block is BT splitted ( BT_split_flag) = 1 and BT split type information 1 indicating that the BT split type of the corresponding block is a vertical type is encoded.

In FIG. 8, the block corresponding to the first node of depth 2 is a child node of the BT partitioned parent node (that is, the first node of depth 1), and is again BT split. Note that the BT split may be recursively repeated, but child nodes of the BT split node are not QT split again. Therefore, in generating split information for child nodes of the BT split node, there is no need to explicitly signal that the block is not QT split. For example, in FIG. 8, in signaling split information of a block corresponding to the first node of depth 2, the encoding apparatus omits the QT split flag QT_split_flag, and the BT split flag BT_split_flag = 1 and the BT split type are vertical. The BT partition type information 1 indicating the type is encoded.

In FIG. 8, the block corresponding to the first node of depth 3 is a child node of the BT partitioned parent node (ie, the first node of depth 2) and is no longer BT partitioned. Accordingly, the encoding apparatus encodes only the BT split flag BT_split_flag = 0 indicating that the corresponding block is not BT split.

As a result, when the division structure of FIG. 8 is recorded by the depth-first search method, a total of 34 bits are required as follows. Here, the underlined bits relate to the BT division and the remaining bits relate to the QT division.

One ( Of QTBT Leaf  Node) 0 11 11  0 0 0  (The first node in depth 1 and Rooted  Nodes) 0 10 0 0 (The second node at depth 1 and Rooted  Nodes) 1 0 11 0 10 0 0  0 0  0 0  0 10 0 0 (The third node at depth 1 and Rooted  Nodes) 0 0 (Fourth node in depth 1)

Now, a description will be given of a method in which a decoding apparatus determines a partitioning for dividing a CTU into a plurality of CUs using split information signaled from the encoding apparatus in the present embodiment.

The decoding apparatus first extracts a first flag (QT_split_flag) related to the division of the QT. When the first flag indicates the division of the QT, the decoding apparatus divides the corresponding node into four nodes of lower depths. For the node corresponding to the leaf node of the QT, the second flag BT_split_flag and the split type information related to the splitting of the BT are extracted to split the corresponding leaf node into the BT structure.

Taking the block splitting structure of FIG. 8 as an example, the demodulation device extracts a QT splitting flag QT_split_flag corresponding to the node of the highest depth of the QTBT structure. Since the value of the extracted QT split flag QT_split_flag is 1, the node of the highest depth is divided into four nodes of the lower depth (depth 1 of QT). Then, the QT splitting flag QT_split_flag of the first node of depth 1 is extracted. Since the extracted QT split flag QT_split_flag value is 0, the first node of depth 1 is no longer split into a QT structure.

Since the first node of the depth 1 of the QT becomes the leaf node of the QT, the process proceeds to the BT having the first node of the depth 1 of the QT as the root node of the BT. The BT split flag BT_split_flag corresponding to the root node of the BT (ie, depth 0) is extracted. Since the BT split flag BT_split_flag is 1, the root node of BT is split into two nodes of depth 1. Since the root node of the BT is split, split type information indicating whether a block corresponding to the root node of the BT is split vertically or horizontally is extracted. Since the split type information is 1, the block corresponding to the root node of BT is split vertically. Then, the BT split flag (BT_split_flag) for the first node of the depth 1 split from the root node of BT is extracted. Since the BT split flag BT_split_flag is 1, split type information of a block of the first node of depth 1 is extracted. Since the partition type information of the block of the first node of depth 1 is 1, the block of the first node of depth 1 is vertically divided. Then, the BT split flag BT_split_flag of the second node of the depth 1 split from the root node of BT is extracted. Since the BT split flag BT_split_flag is 0, it is no longer split by BT.

In this way, the decoder 510 first extracts the QT split flag QT_split_flag repeatedly to split the CTU into a QT structure. The BT split flag (BT_split_flag) is extracted for the leaf node of the QT, and when the BT split flag (BT_split_flag) indicates the split, split type information is extracted. In this manner, the decoding apparatus may confirm that the CTU is divided by the encoding apparatus into a structure as shown in FIG.

Second embodiment

As the CTU size grows larger, in the QTBT splitting structure, it is highly likely that all nodes will be QT split again until the size of the split blocks decreases below a certain level. According to the first embodiment described above, as the number of times the QT division is repeated increases, the number of bits required to express these QT division information using the QT division flag is increased by n-th power of four. This embodiment aims to reduce the QT partitioning information of these nodes when all nodes from the highest depth (that is, depth 0) to a certain lower depth are commonly QT partitioned.

In the present embodiment, when the root node is recursively QT divided until the divided block reaches a certain depth, the encoding apparatus uses the QT splitting flag to express these depths instead of expressing these QT split information. Depth information that specifies can be signaled.

The encoding apparatus may signal depth information indicating the highest depth in which the leaf nodes of the QT exist among the depths of the QTBT partition structure. Hereinafter, among the depths of the QTBT splitting structure, the highest depth in which the leaf nodes of the QT exist may be referred to as the 'highest QT leaf node depth'. For example, 'top QT leaf node depth = 2' means that the root node of the QTBT belonging to the highest depth (0) of the QTBT is QT splitted, and each of the four child nodes (which belong to depth 1) of the root node are also QTBT. QT split, and further, at least some of the nodes belonging to depth 2 are leafs of QT that are not QT split. That is, in the QTBT splitting structure of the highest QT leaf node depth = 2, some of nodes belonging to the depth 2 are BT splitted, neither QT splitted nor BT splitted. Accordingly, the depth information may function as QT partitioning information from a root node to parent nodes of nodes belonging to the highest QT leaf node depth in the QTBT partitioning structure.

The depth information may be expressed, for example, in a truncated unary manner. Depth information may be expressed in a CTU header. An example codeword for each depth is as follows.

Depth Codeword (Max = 3) 0 0 One 10 2 110 3 111

In the case of using the highest QT leaf node depth as described above, the QT structure repeated from the root node can be represented by a breadth first scan method, and the remaining lower QTs or BTs are similar to those of the first embodiment. The QT split flag, the BT split flag, and the BT split type information may be used to express the depth first scan method. The example QTBT splitting structure of FIG. 8 is the leaf nodes of QT where the root node is QT split and the first and second and fourth nodes of depth 1 are no longer QT split. Thus, the highest QT leaf node depth is 1 in the example QTBT splitting structure of FIG. 8. In representing the QTBT splitting structure of FIG. 8, if the highest QT leaf node depth is separately indicated using the codeword of Table 1, a total of 35 bits are required as follows. Here, the underlined bits relate to the BT division and the remaining bits relate to the QT division.

1 0 (top QT Leaf node  Depth) 0 11 11  0 0 0  (The first node in depth 1 and Rooted  Nodes) 0 10 0 0 (The second node at depth 1 and Rooted  Nodes) 1 0 11 0 10 0 0 0 0  0 0  0 10 0 0 (The third node at depth 1 and Rooted  Nodes) 0 0  (Fourth node in depth 1)

In representing the exemplary partitioning structure of FIG. 8 with the highest QT leaf node depth of 1, it can be seen that according to the method of the second embodiment, 1 bit is additionally taken as compared to the first embodiment. However, if the highest QT leaf node depth is 2, the first embodiment needs 1 + 4 = 5 bits to represent the QT split flags of nodes of depth 0 and depth 1, while the second embodiment uses a common QT. It can be expressed as 3 bits (“110” in Table 1) indicating the depth. Also, when the highest QT leaf node depth is 3, the first embodiment requires 1 + 4 + 16 = 21 bits to represent the QT split flags of nodes of depth 0, depth 1, and depth 2, but the second In the case of the embodiment, it can be represented by 3 bits (“111” in Table 1) indicating the highest QT leaf node depth. That is, as the depth of the highest QT leaf node increases, the QT increases by n-th power of 4, so that the data amount can be reduced when the depth of the commonly used QT is expressed only as the depth, not the depth-first method. Note that as the CTU size grows to 256x256 or 128x128, it is likely that all nodes will be repeatedly QT split until the size of the split block is 32x32 or 16x16.

As described above, in representing the QT segmentation structure in the QTBT block partitioning structure, a signaling method with depth information may be extended to a slice, a picture, or a sequence level. For example, when all the CTUs included in one slice are commonly QT split recursively for all nodes from the highest depth to a depth (for example, using a code word as shown in Table 1) The depth can be expressed in the Slice Header. The depth expressed in the slice header may function as QT split information for all nodes existing between the highest depth and the depth expressed in the slice header, for all CTUs included in the slice. The remaining lower QT or BT may be expressed using the QT split flag, the BT split flag, and the BT split type information similarly to the first embodiment.

For clarity, a slice consisting of four CTUs is exemplarily described. The highest QT leaf node depth of CTU No. 1, the highest QT leaf node depth of CTU No. 2 is 2, and the highest CTU No. 3 CTU. Assume that the QT leaf node depth is 4 and the highest QT leaf node depth of CTU # 4 is 3. In this case, all four CTUs are divided into the same shape up to depth 2. That is, all the CTUs are recursively QT-divided until all the CTU blocks reach depth 2. Accordingly, the encoding apparatus may express information indicating that all CTUs of the corresponding slice are QT-split up to depth 2 in the slice header.

 Similarly, in the QTBT splitting structure of all the CTUs included in one picture, when all the nodes are recursively QT splitted to a certain depth, the depth is added to the picture-level header information (e.g., picture parameter sets). (in a picture parameter set (PPS)). Furthermore, in the QTBT splitting structure of all the CTUs included in one sequence, when all nodes are recursively QT splitted to a certain depth, the depth is determined in sequence-level header information (e.g., a sequence parameter set parameter set (SPS)).

The depth expressed in the picture-level header information, the sequence-level header information, and the sequence-level header information in this manner is referred to as "QT common depth" below. The QT common depth is used as part of the QTBT segmentation information for all CTUs included in that level. Table 2 shows an example syntax element for indicating the QT common depth in a Sequence Parameter Set (SPS). As shown in Table 2, in order to signal the QT common depth, a flag indicating that there is a QT common depth (QT_depth_signal_flag) and a parameter (QT_comm_depth_minus3) that specifies the QT common depth may be used. Similar syntax elements may be included in a Picture Parameter Set (PPS) or Slice Header.

seq_parameter_set_rbsp () {   ...   QT_depth_signal_flag if (QT_depth_signal_flag) {     QT_comm_depth_minus3   ...

In some other examples, the QT common depth represented in picture-level header information, sequence-level header information, or sequence-level header information is determined whether all CTUs of that level are actually divided into the same form up to that QT common depth. Regardless, it may be a predetermined default value. In this case, the encoding apparatus may additionally signal a difference value between the QT common depth defaulted to the SPS, PPS, or Slice and the highest QT leaf node depth of each CTU. The default value may be predetermined based on statistics related to the likelihood of matching the highest QT leafnode depth of the CTUs included in that level. Considering that as the CTU size grows to 256x256 or 128x128, all nodes are likely to be QT split until the size of the leaf nodes of the QT is 32x32 or 16x16, setting depth 3 or depth 4 to the default value. It can be efficient for bit savings. In addition, the default value may be determined based on an average of the highest QT leaf node depths of all the CTUs included in the level. Now, in the present embodiment, using the split information signaled from the encoding apparatus, the decoding apparatus may determine the CTU. A method of determining partitioning to be divided into a plurality of CUs will be described.

The decoding apparatus first extracts depth information indicating the highest depth (that is, the highest QT leaf node depth) among depths in which at least one leaf node of the quadtree exists in the QTBT structure from the encoded bitstream. Using depth information indicating the highest QT leaf node depth, the decoding apparatus performs QT splitting for all nodes from a root node of the QTBT to a parent node of nodes belonging to the highest QT leaf node depth. For example, when the highest QT leaf node depth is 2, the root node is QT splitted, and the four child nodes (which belong to depth 1) of the root node are each QT splitted.

By definition of the highest QT leaf node depth, at least some of the nodes belonging to the highest QT leaf node depth are not QT split. In addition, some of the nodes belonging to the highest QT leaf node depth are neither BT split nor QT split or BT split. In other words, some of the nodes belonging to the highest QT leaf node depth may be further QT split, BT split, or leaf nodes of QTBT (either QT split or BT split). Therefore, nodes belonging to the highest QT leaf node depth are determined in a partition structure in substantially the same manner as in the first embodiment. That is, the decoder 510 extracts a first flag QT_split_flag related to the division of the QT, for nodes belonging to the highest QT leaf node depth. If the first flag indicates that the QT is split (QT_split_flag = 1), the corresponding node is QT split again. If the first flag indicates that the QT is not divided (QT_split_flag = 0), a second flag (BT_split_flag) related to the BT division is extracted. If the second flag indicates that the BT is split (BT_split_flag = 1), the BT split type information is further extracted, and the corresponding node is split according to the extracted BT split type information. If the second flag indicates that the QT is not split (BT_split_flag = 0), the corresponding node is no longer split, and is determined as the leaf node (ie, CU) of the QTBT.

Furthermore, as described above, when the technique for signaling the QT common depth is applied to the slice, picture, or sequence level, the decoding apparatus parses the bitstream to process a sequence parameter set (SPS), a picture parameter set (PPS), or a slice. Extract depth information expressed in Header. Using the extracted depth information, the decoding apparatus may recursively split all nodes until the depth indicated by the depth information is reached.

Third embodiment

As described above, the leaf nodes of the quadtree in the QTBT partition structure may be further partitioned into the BT structure. In BT, there may be a plurality of split types, for example, a type of dividing a block of a node horizontally into two blocks of the same size (ie, symmetric horizontal splitting) and a type of splitting vertically (ie, symmetrical). There may be two kinds of symmetric vertical splitting. In this embodiment, a technique for reducing the amount of bits representing such BT is proposed. The BT partition information representation scheme proposed in this embodiment may be used together with the QT partition information representation scheme illustrated in the first or second embodiment.

FIG. 9 is a diagram illustrating the number of bits that can be represented as BT in one CU.

BT is expressed as 2 bits when it is split, where the first bit indicates whether the split has ended after QT is 0 (no split = 0) or if BT is started (1), and if the first bit is 1 the second bit is Indicates the split type of BT. For example, 0 represents symmetrical horizontal division and 1 represents symmetrical vertical division. Considering possible combinations of the parent node and the child node, when the parent node is divided into symmetrical horizontal division and symmetrical vertical division, respectively, as shown in FIG. It is noted here that if BT is started after QT ends, QT is not repeated again. In other words, the parent node is divided horizontally (up and down) and the child node is divided vertically (left and right), or the parent node is divided vertically (left and right) and the child node is horizontal. The division into (up and down) is not considered. Considering the combination of parent node and child node partitions, 16 expressions are possible. In FIG. 9, these combinations are represented by 4 bits to 6 bits using the BT partition flag and the BT partition type flag.

In FIG. 9, it is noted that child nodes are repeatedly represented in horizontally or vertically divided parent nodes. In view of this, the present invention represents a method of reducing the amount of bits representing the combination of these parent and child nodes by representing the parent node in 2 bits and the child node in a fixed length codeword of 3 bits. Adopt. Example codewords are listed in Table 3. In the case of “10 10 10” and “11 11 11”, where the parent node and the child node are divided in the same form, the leaf node is different, but when the child node is marked with a specific bit, the first two bits (2 bits of the parent node) are divided. Prediction of the method is possible.

Parent node (2 bits) Child Node (First embodiment, 2 to 4 bits) Child Node (3rd Embodiment, 3 bits) 10/11 0 0 000 0 10 001 to 110 0 11 10 0 10 11 11 0 11 10 10 10 10 111 11 11 11

For example, with respect to the block division structure as shown in FIG. 10, according to the method described in the first embodiment, 1 0 11 11 0 0 0 (No. 1 BT) and 0 10 0 0 (No. 2 BT) are used as the depth priority method. , 1 0 11 10 11 11 0 0 10 0 0 0 11 0 0 (BT 3, 4) 0 0 0 0 0 0 10 10 0 0 10 0 0 (BT 5) 0 0 (52 bits total) Can be. According to the scheme proposed in this embodiment, when a fixed bit of a parent node (2 bits) + a child node (3 bits) is expressed, 1 0 11 101 (child node 3 bits) 0 0 (No. 1 BT), 0 10 000 (BT 2), 1 0 11 100 (BT 3) 11 110 (BT 4) 0 0 0 0 0 0 0 , 0 0 0 0 , 0 10 111 0 0 0 0 (BT 5) 0 0 (50 bits in total). In the first to third embodiments described above, the following information may be additionally used to efficiently signal information about block division by the QTBT structure. The information is encoded as header information of an image, for example, may be encoded by a sequence parameter set (SPS) or a picture parameter set (PPS).

CTU size: the top layer of QTBT, that is, the block size of the root node

MinQTSize: the minimum block size of leaf nodes allowed in QT

MaxBTSize: the maximum block size of the root node allowed by BT

MaxBTDepth: the maximum depth allowed by BT

MinBTSize: the minimum block size of leaf nodes allowed in BT

In QT, a block having the same size as MinQTSize is no longer split, and thus split information (first flag) about QT corresponding to the block is not encoded. In addition, there is no BT in a block having a size larger than MaxBTSize in QT. Therefore, splitting information (second flag, splitting type information) regarding BT corresponding to the block is not encoded. In addition, when the depth of the corresponding node of the BT reaches MaxBTDepth, the block of the corresponding node is no longer split and no splitting information (second flag, split type information) regarding the BT of the corresponding node is not encoded. In addition, a block having the same size as MinBTSize in BT is no longer split and no split information (second flag, split type information) regarding BT is also encoded. In this way, the maximum or minimum block size that a loop or leaf node of the QT and BT can have at a high level such as a sequence parameter set (SPS) or a picture parameter set (PPS) can be defined to determine whether the CTU is divided or not. The amount of coding for the information indicating the partition type can be reduced.

As such, when information such as MinQTSize, MaxBTSize, MaxBTDepth, MinBTSize, etc. is additionally defined in the SPS or PPS, the decoding apparatus extracts the information and reflects this information when extracting the split information for QT and BT. Can be.

For example, blocks having the same size as MinQTSize in QT are no longer split. Therefore, the decoding apparatus does not extract the split information (QT split flag) about the QT of the block from the bitstream (that is, the QT split flag of the block does not exist in the bitstream), and sets the value to 0 automatically. do. In addition, there is no BT in a block having a size larger than MaxBTSize in QT. Therefore, the decoding apparatus does not extract the BT partition flag for the leaf node having a block larger than MaxBTSize in QT, and automatically sets the BT partition flag to 0. In addition, when the depth of the node of the BT reaches MaxBTDepth, the block of the node is no longer split. Therefore, the BT partition flag of the corresponding node is not extracted from the bitstream, and its value is automatically set to zero. Also, in BT, blocks having the same size as MinBTSize are no longer split. Therefore, the decoding apparatus does not extract the BT partition flag of the block having the same size as MinBTSize from the bitstream, and automatically sets the value to zero.

Meanwhile, in some examples, the luma component and the chroma component of the CTU may be divided into the same QTBT structure. In other examples, the luminance component and the chrominance component may each be divided into separate QTBT structures. For example, in the case of an I (Intra) slice, the luminance component and the chrominance component may be divided into different QTBT structures. That is, CTU may be divided into luminance coding blocks (CBs) and color difference CBs to perform block partitioning, and block block information may be generated for each CB and transmitted to a decoding apparatus. In this case, if the shape of the block partitions of luminance and color difference among the CTUs in one picture is the same, the same block partition information is transferred redundantly, thereby reducing the bit efficiency. In consideration of this, in some examples of the present invention, an additional syntax (split_cb_flag_in_CTU) is added to the syntax representing the CTU, and when this value is 0, the decoding apparatus adds block partition information for the luminance CB to the color difference CB. When used as block partition information, and this value is 1, different block partition information can be sent to the decoding apparatus as before. That is, all block partition information of luminance including QTBT can be utilized as it is in color difference.

In addition, unlike the luminance information, the color difference information may not require a detailed block partition. In consideration of this, in some examples, only QT partition information among all block partition information of luminance (that is, among QTBT partition information) may be used as block partition information of the color difference CB. For example, a separate syntax (split_cb_flag_for_QT_in_CTU) is added to the syntax representing the CTU, and when this value is 0, the decoding apparatus converts the QT partition information from the entire block partition information of luminance to the QT partition information of the luminance CB. When the CB is used as QT block partition information, and this value is 1, different block partition information may be sent to the decoding apparatus as before.

As described in connection with the second embodiment, when the QT common depth of the QT is expressed in any one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), and a Slice Header, in some examples, the luminance in the CTU unit The QT common depth for the color difference may refer to the QT common depth set higher than the CTU (that is, set in the SPS, PPS, or Slice Header). In some other examples, the QT common depth for luminance and color difference in the CTU unit may be expressed as a difference value from the QT common depth set above the CTU.

The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which the present embodiment belongs may make various modifications and changes without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment but to describe the present invention, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority to Korean Patent Application No. 10-2017-0031234 filed on March 13, 2017, which is hereby incorporated by reference in its entirety.

Claims (15)

As a method of encoding image data, Determining a QuadTree plus BinaryTree (QTBT) block partitioning structure for encoding a block of image data, wherein the QTBT block partitioning structure is a structure in which a binary tree is rooted from a leaf node of a quadtree; And Encoding partition information representing a block of the image data and the determined block partitioning structure based on the determined block partitioning structure; Encoding the split information, Determining, in the determined block partitioning structure, the highest depth among the depths in which at least one leaf node of the quadtree exists (hereinafter, referred to as “top QT leaf node depth”); And And encoding depth information representing the highest QT leaf node depth as quadtree split information from a root node to a parent node of nodes belonging to the highest QT leaf node depth in the determined block partitioning structure. How to encode. The method of claim 1, Depth information indicating the highest QT leaf node depth, A truncated unary codeword, characterized by specifying the highest QT leaf node depth among the depths of the QTBT. The method of claim 1, And wherein the binary tree is defined by partition types for dividing a parent node into two child nodes, wherein the partition types include symmetrical horizontal partitioning and symmetrical vertical partitioning. The method of claim 5, Encoding the split information, And encoding binary tree partitioning information representing a binary tree in the determined block partitioning structure in a manner that specifies any one of possible combinations of partitioning of the parent node and partitioning of the child node. A method of encoding video data. The method of claim 6, Possible combinations of partitioning of the parent node and partitioning of the child node, And a fixed length codeword indicating the division of the parent node and a fixed length codeword indicating the division of the child node. The method of claim 1, Determining a QTBT block partitioning structure for encoding the block of the image data, And determining the QTBT block partitioning structure separately for a luma component and a chroma component of the block of image data. The method of claim 6, Encoding the split information, Encoding a flag indicating whether the QTBT block partitioning structure determined for the luminance component and the QTBT block partitioning structure determined for the chroma component are the same; And When the QTBT block partitioning structure determined for the luminance component and the QTBT block partitioning structure determined for the chroma component are the same, any of the QTBT block partitioning structure determined for the luminance component and the QTBT block partitioning structure determined for the chroma component Encoding only partition information The method of encoding image data, further comprising. As a method of decoding image data, Parsing a block of encoded image data and partition information associated with the block of the image data, wherein the block of encoded image data is divided into a plurality of partition blocks according to a quadtree plus binarytree (QTBT) block partitioning structure, The QTBT block partitioning structure is a structure in which a binary tree is rooted from a leaf node of a quadtree; And Decoding the block of the encoded image data for each leaf node of the QTBT while determining the QTBT using the split information; The splitting information includes depth information indicating the highest depth among the depths in which at least one leaf node of the quadtree exists in the QTBT block partitioning structure (hereinafter, referred to as “highest QT leaf node depth”). Determining the QTBT uses quadrature splitting for all nodes from the root node of the QTBT to the parent node of nodes belonging to the highest QT leaf node depth using depth information indicating the highest QT leaf node depth. A method of decoding image data, comprising the step of performing. The method of claim 8, Depth information indicating the highest QT leaf node depth, Characterized by the truncated unary codeword, specifying the highest QT leaf node depth among the depths of the QTBT. The method of claim 8, The binary tree is defined by partition types for dividing a parent node into two child nodes, wherein the partition types include symmetrical horizontal partitioning and symmetrical vertical partitioning. The method of claim 10, The partitioning information includes binary tree partitioning information in the QTBT block partitioning structure, and the binary tree partitioning information is expressed by specifying one of possible combinations of partitioning of the parent node and partitioning of the child node. Characterized in that the video data is decoded. The method of claim 11, Possible combinations of partitioning of the parent node and partitioning of the child node, And a fixed length codeword indicating the division of the parent node and a fixed length codeword indicating the division of the child node. The method of claim 11, Encoding the split information, And performing binary partitioning on the leaf nodes of the quadtree by using the binary tree partitioning information. The method of claim 8, The segmentation information includes a flag indicating whether the QTBT block partitioning structure for the luminance component and the QTBT block partitioning structure for the chroma component are the same in the block of the image data The decoding of the encoded image data block for each leaf node of the QTBT may be performed when the flag indicates that the QTBT block partitioning structure for the luminance component and the QTBT block partitioning structure for the chroma component are the same. And decoding the luma component and the chroma component of the block of the encoded image data for each leaf node of the QTBT. An apparatus for decoding video data, Memory; And Includes one or more processors, The one or more processors, Parsing a block of encoded image data and partition information associated with the block of image data, wherein the block of encoded image data is divided into a plurality of partition blocks according to a quadtree plus binarytree (QTBT) block partitioning structure, The QTBT block partitioning structure is a structure in which a binary tree is rooted from a leaf node of a quadtree, and the partitioning information includes at least a leaf tree of a quadtree in the QTBT block partitioning structure. Includes depth information indicative of the highest depth of the existing depths (hereinafter referred to as "top QT leaf node depth"); And And determining the QTBT using the split information, and decoding the block of the encoded image data for each leaf node of the QTBT. Determining the QTBT, Performing quadtree splitting on all nodes from a root node of the QTBT to a parent node of nodes belonging to the highest QT leaf node depth, using depth information indicating the highest QT leaf node depth, An apparatus for decoding image data.

Images