WO2018128511A1 - Device and method for encoding or decoding image - Google Patents

Abstract

An intra-prediction technique of the present disclosure considers adding a new intra-mode, referred to as "a division intra-mode," to existing intra-modes. Based on the division intra-mode, a prediction region is diagonally divided into two regions, and each region can be predicted with a separate directional or nondirectional mode. Techniques which can be considered when intra-predicting a diagonal edge which may exist in a block to be encoded through this are explained herein.

Description

Apparatus and method for image encoding or decoding

The present invention relates to image encoding or decoding for efficiently encoding an image. More specifically, it relates to intra prediction techniques that can be useful in blocks with different textures in the form of diagonal edges or diagonal shapes.

The contents described in this section merely provide background information on the present embodiment and do not constitute a prior art.

Joint Exploration Model (JEM), a reference SW codec of the Joint Video Exploration Team (JVET), which is exploring next-generation video standard technology, provides a coding unit (CU) of a recursive Quadtree plus Binarytree (QTBT) partition structure. . Through QTBT, the redundant coding block structure concept of CU, Prediction Unit (PU), and Transform Unit (TU) in High Efficiency Video Coding (HEVC) eliminates redundancy, and is more flexible and versatile, such as non-square. A block structure with a CU structure has been attempted.

JVET, meanwhile, is investigating the need to include 360-degree video coding technology in next-generation video coding standards. 360-degree video (hereinafter referred to as '360 video') is a video taken from multiple directions with multiple cameras or omni-directional cameras, and the video in multiple directions can be compressed into one 2D video to compress and transmit video from multiple scenes. Stitching, the stitched image is compressed and transmitted to the decoding apparatus. After the decoding apparatus decodes the compressed image, the decoding apparatus maps and reproduces the image in 3D.

An example of a projection format for 360 images is EPR (Equirectangular Projection). The EPR format has the disadvantage of distorting the 3D sphere-shaped 360 image by increasing the pixels at the top and bottom of the image and severely distorting the image. The EPR format also increases the data amount and the encoding throughput in the enlarged portion. Accordingly, various projection formats have been proposed that can replace the EPR format.

When Icosahedral projection (ISP) is unfolded as a 2D image, it becomes as shown in FIG. In FIG. 1, (a) is a general native layout form, and (b) and (c) are two kinds of compact layouts. All three types illustrated in FIG. 1 include a lot of diagonal edges due to the characteristics of the icosahedron composed of triangular faces. 2 is a compact layout of an Octahedron projection (OHP) map of 360 video. As shown in FIG. 2, the area where two faces in one block exist occupies a significant portion of the image. Thus, applying one intra mode to a rectangular or square shaped CU shown by way of example in FIG. 2 will reduce spatial compression efficiency.

The present disclosure describes intra prediction techniques that can be useful in blocks with different textures in diagonal shapes or diagonal shapes. In the intra prediction technique of the present disclosure, consider adding a new intra mode referred to as a "split intra mode" to existing intra modes. In the split intra mode, the prediction region is divided into two regions in the diagonal direction so that each region can be predicted in a separate directional or non-directional mode.

According to an aspect of the present invention, there is provided a method, comprising: decoding syntax elements defining an intra mode and a residual block of a current block of image data from an encoded bitstream; Based on the decoded syntax elements defining the intra mode of the current block, the plurality of available intra modes including a plurality of directional modes, a plurality of non-directional modes, and at least one split intra mode; Determining an intra mode; Determining a prediction block for the current block based on the determined intra mode of the current block; And reconstructing the current block based on the prediction block and the residual block. The divisional intra mode is a mode in which one of the plurality of directional modes and the plurality of non-directional modes is individually applied to each of two regions divided by dividing the current block diagonally.

According to another aspect of the invention, an apparatus for decoding image data comprising a memory and one or more processors, the one or more processors are syntax that defines an intra mode and a residual block of a current block of image data from an encoded bitstream Decrypting the elements; Based on the decoded syntax elements defining the intra mode of the current block, the plurality of available intra modes including a plurality of directional modes, a plurality of non-directional modes, and at least one split intra mode; Determining an intra mode; Determining a prediction block for the current block based on the determined intra mode of the current block; And reconstruct the current block based on the prediction block and the residual block.

1 is a diagram illustrating various two-dimensional layouts of Icosahedral projection (ISP).

2 illustrates various two-dimensional layouts of Octahedron projection (OHP).

3 is a block diagram of an image encoding apparatus that may use the techniques of this disclosure.

4 is a conceptual diagram of exemplary block partitioning using a QTBT structure.

5 is a conceptual diagram illustrating a plurality of intra modes that can be used for intra prediction.

6 is an exemplary diagram of neighboring blocks of a current block.

7 illustrates an image decoding apparatus that may use the techniques of the present disclosure.

8 is a diagram illustrating a positional relationship between a current block and reference samples in intra prediction.

9A and 9B are conceptual diagrams illustrating how reference samples are used for a current block in directional intra mode.

10A and 10B are diagrams illustrating types of split intra mode applied to a square block.

11A and 11B illustrate types of split intra mode applied to a rectangular block.

12A and 12B are diagrams for describing respective reference pixels used for two regions of the current block in the split intra mode.

13A, 13B, 14A, and 14B are conceptual diagrams illustrating some exemplary schemes in which reference samples are used for two regions in split intra mode;

15A to 15C are conceptual views illustrating a method of determining prediction pixel values on a diagonal line in a split intra mode of the split_right_down type.

16 is a flowchart illustrating an example method for encoding image data according to one or more examples of the present invention.

17 is a flowchart illustrating an example method for decoding image data in accordance with one or more examples of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding an identification code to the components of each drawing, it should be noted that the same components as possible, even if shown on different drawings have the same reference numerals. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

3 is a block diagram of an image encoding apparatus that may use the techniques of this disclosure.

The image encoding apparatus includes a block divider 310, a predictor 320, a subtractor 330, a transformer 340, a quantizer 345, an encoder 350, an inverse quantizer 360, and an inverse transform unit ( 365, an adder 370, a filter unit 380, and a memory 390. In the image encoding apparatus, each component may be implemented as a hardware chip, or may be implemented in software and implemented so that the microprocessor executes a function of software corresponding to each component.

After dividing each picture constituting the image into a plurality of coding tree units (CTUs), the block dividing unit 310 recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU (coding unit) which is a basic unit of coding. As a tree structure, QuadTree (QT), which the parent node splits into four child nodes, or QTBT (QuadTree), which uses a QT structure and a binaryTree (BT) structure that the parent node splits into two child nodes. plus BinaryTree) structure can be used.

In a QTBT (QuadTree plus BinaryTree) structure, the CTU is first divided into a QT structure. The leaf nodes of the QT may then be further partitioned by BT. The partition information generated by the block divider 310 by dividing the CTU by the QTBT structure is encoded by the encoder 350 and transmitted to the image decoding apparatus.

In QT, a first flag (QT split flag, QT_split_flag) indicating whether a block of a corresponding node is split is encoded. If the first flag is 1, the block of the node is divided into four blocks of the same size. If the first flag is 1, the node is no longer divided by QT.

In BT, a second flag (BT split flag, BT_split_flag) indicating whether a block of the corresponding node is split is encoded. In BT, there may be a plurality of partition types. For example, there may be two types of partitioning a block of a node horizontally into two blocks of the same size and a type of partitioning vertically. Alternatively, there may further be a type in which blocks of the corresponding node are further divided into two blocks having an asymmetric shape. The asymmetric form may include a form in which a block of a node is divided into two rectangular blocks having a size ratio of 1: 3. When the BT has a plurality of division types in this manner, when a second flag indicating that a block is divided is encoded, partition type information indicating a partition type of the corresponding block is further encoded.

4 is an exemplary diagram of block division using a QTBT structure. 4A illustrates an example of dividing a block by a QTBT structure, and FIG. 4B illustrates a tree structure. In FIG. 4, the solid line indicates the division by the QT structure, and the dotted line indicates the division by the BT structure. In addition, with reference to the layer notation in FIG. 4B, the parenthesis indicates a layer of QT, and the parenthesis indicates a layer of BT. In the BT structure represented by a dotted line, a number represents partition type information.

In FIG. 4, the CTU, which is the highest layer of QT, is divided into four nodes of layer 1. Accordingly, the block divider 310 generates a QT split flag (QT_split_flag = 1) indicating that the CTU is split. The block corresponding to the first node of layer 1 is no longer divided by QT. Accordingly, the block divider 310 generates QT_split_flag = 0.

Thereafter, the block corresponding to the first node of layer 1 of the QT proceeds to BT. In this example, it will be described that there are two types of BT splitting a block of a node into two blocks of the same size horizontally and vertically. The first node of QT's layer 1 becomes the root node of BT (layer 0). Since the block corresponding to the root node of BT is further divided into blocks of (layer 1), the block dividing unit 310 generates a BT split flag (BT_split_flag) = 1 indicating that the split is performed by BT. Subsequently, partition type information indicating whether the corresponding block is divided horizontally or vertically is generated. In FIG. 4, since the block corresponding to the root node of BT is vertically divided, 1 indicating vertical division is generated as the partition type information. Since the first block of the block of (layer 1) divided from the root node is further split and the split type is vertical, it generates BT_split_flag = 1 and split type information 1. On the other hand, since the second block of (layer 1) split from the root node of BT is no longer split, BT_split_flag = 0 is generated.

As described above, since the upper left block corresponding to the root node of BT in FIG. 4 is vertically divided, BT split flag (BT_split_flag) = 1 indicating splitting by BT and 1 indicating vertical splitting are generated as split type information. . If the first block of blocks of (layer 1) partitioned from the root node of BT is further split and its split type is horizontal, BT_split_flag = 1 and split type information 0 will be generated. At this time, if the second block of (layer 1) partitioned from the root node of BT is further partitioned further, the partition type must be vertical. This is because, if the partition type of the second block of (layer 1) divided from the root node of the BT is horizontal, the block corresponding to the root node of the BT is QT divided. That is, it is a case where it should be expressed by QT split information rather than BT split information. However, in this example, since the block corresponding to the root node of BT has already finished QT splitting, such splitting type cannot be taken. Therefore, since the partition type of the second block of (layer 1) partitioned from the root node of BT must be vertical, and the decoder can infer the partition type information as 1, the partition type information other than BT_split_flag = 1 may be signaled. no need.

On the other hand, in order to efficiently signal the information about the block division by the QTBT structure to the image decoding apparatus, the following information may be further encoded. The information is encoded as header information of an image, for example, may be encoded by a sequence parameter set (SPS) or a picture parameter set (PPS).

CTU size: the top layer of QTBT, that is, the block size of the root node

MinQTSize: the minimum block size of leaf nodes allowed in QT

MaxBTSize: the maximum block size of the root node allowed by BT

MaxBTDepth: the maximum depth allowed by BT

MinBTSize: the minimum block size of leaf nodes allowed in BT

In QT, a block having the same size as MinQTSize is no longer split, and thus split information (first flag) about QT corresponding to the block is not encoded. In addition, there is no BT in a block having a size larger than MaxBTSize in QT. Therefore, splitting information (second flag, splitting type information) regarding BT corresponding to the block is not encoded. In addition, when the depth of the corresponding node of the BT reaches MaxBTDepth, the block of the corresponding node is no longer split and no splitting information (second flag, split type information) regarding the BT of the corresponding node is not encoded. In addition, a block having the same size as MinBTSize in BT is no longer split and no split information (second flag, split type information) regarding BT is also encoded. In this way, the maximum or minimum block size that a loop or leaf node of the QT and BT can have at a high level such as a sequence parameter set (SPS) or a picture parameter set (PPS) can be defined to determine whether the CTU is divided or not. The amount of coding for the information indicating the partition type can be reduced.

Meanwhile, the luma component and the chroma component of the CTU may be divided into the same QTBT structure. However, the present invention is not limited thereto, and the luminance component and the chrominance component may be divided using separate QTBT structures, respectively. For example, in the case of an I (Intra) slice, a luma component and a chroma component may be divided into different QTBT structures.

Hereinafter, a block corresponding to a CU to be encoded or decoded is called a 'current block'.

The prediction unit 320 predicts the current block and generates a prediction block. The predictor 320 includes an intra predictor 322 and an inter predictor 324.

The intra predictor 322 predicts pixels in the current block by using pixels (reference pixels) positioned around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction, and the peripheral pixels to be used and the equations are defined differently according to each prediction mode. In particular, the intra predictor 322 may determine an intra mode to be used to encode the current block. In some examples, intra prediction unit 322 may encode the current block using several intra modes and select an appropriate intra mode to use from the tested modes. For example, intra predictor 322 calculates rate distortion values using rate-distortion analysis for several tested intra modes, and intra with the best rate distortion characteristics of the tested modes. You can also select the mode.

5 is a conceptual diagram illustrating a plurality of intra modes that can be used for intra prediction. As shown in FIG. 5, the plurality of available intra modes may include two non-directional modes (planar mode and DC mode) and 65 directional modes. In addition, as described below, the present disclosure proposes an additional intra mode suitable for intra prediction of blocks having diagonal edges. However, the number of directional and non-directional modes, the number of all intra modes, etc. are merely exemplary, and various combinations are possible within the scope of the present invention.

The intra predictor 322 selects one intra mode from among a plurality of intra modes, and selects a current block by using a peripheral equation (which may be referred to as a reference pixel or a reference sample) determined by the selected intra mode and an expression. Predict. Information on the selected intra mode is encoded by the encoder 350 and transmitted to the image decoding apparatus.

On the other hand, the intra prediction unit 322 is used as an intra mode of the current block among the plurality of intra modes in order to efficiently encode intra mode information indicating which mode of the plurality of intra modes is used as the intra mode of the current block. Some of the more likely modes can be determined as the most probable mode (MPM). In addition, mode information indicating whether the intra mode of the current block is selected from the MPM is generated and transmitted to the encoder 350. In general, when the intra mode of the current block is selected from the MPMs, the first intra identification information for indicating which mode of the MPMs is selected as the intra mode of the current block is transmitted to the encoder. On the other hand, when the intra mode of the current block is not selected from the MPM, the second intra identification information for indicating which mode other than the MPM is selected as the intra mode of the current block is transmitted to the encoder.

Hereinafter, a method of constructing the MPM list will be described. Herein, an example of configuring an MPM list with six MPMs is described as an example. However, the present invention is not limited thereto, and the number of MPMs included in the MPM list may be selected within a range of 3 to 10.

First, the MPM list is constructed using the intra mode of neighboring blocks of the current block. The neighboring block may be, for example, all or some of the left block L, the upper block A, the lower left block BL, the upper right block AR, and the upper left block AL of the current block. It may include.

The intra mode of these neighboring blocks is included in the MPM list. In this case, the intra prediction mode of the valid blocks in the order of the left block (L), the top block (A), the bottom left block (BL), the top right block (AR), and the top left block (AL) is included in the MPM list, A candidate is formed by adding a planar mode and a DC mode to the intra modes of the blocks. Alternatively, valid modes in the order of the left block (L), the top block (A), the planar mode, the DC mode, the bottom left block (BL), the top right block (AR), and the top left block (AL) may be added to the MPM list. have. Alternatively, valid modes in the order of the left block (L), the top block (A), the planar mode, the bottom left block (BL), the top right block (AR), the top left block (AL), and the DC mode may be added to the MPM list. have.

The MPM list contains only different intra modes. That is, when a duplicated mode is present, only one of them is included in the MPM list.

If the number of MPMs in the list is smaller than the predetermined number (eg, 6), the MPM may be derived by adding -1 or +1 to the directional modes in the list. In addition, when the number of MPMs in the list is smaller than the predetermined number, the number of modes that are insufficient in the order of vertical mode, horizontal mode, diagonal mode, etc. are added to the MPM list. You may.

The inter prediction unit 324 searches a block most similar to the current block in the reference picture encoded and decoded before the current picture, and generates a prediction block for the current block by using the searched block. A motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. The motion information including the information about the reference picture and the motion vector used to predict the current block is encoded by the encoder 350 and transmitted to the image decoding apparatus.

The subtractor 330 subtracts the prediction block generated by the intra predictor 322 or the inter predictor 324 from the current block to generate a residual block.

The transformer 340 converts the residual signal in the residual block having pixel values of the spatial domain into a transform coefficient of the frequency domain. The transform unit 340 may convert the residual signals in the residual block using the size of the current block as a conversion unit, or divide the residual block into a plurality of smaller subblocks and convert the residual signals in a subblock-sized transform unit. You can also convert. There may be various ways of dividing the residual block into smaller subblocks. For example, it may be divided into sub-blocks of a predetermined same size, or a quadtree (QT) scheme may be used in which the residual block is a root node.

The quantization unit 345 quantizes the transform coefficients output from the transform unit 340, and outputs the quantized transform coefficients to the encoder 350.

The encoder 350 generates a bitstream by encoding the quantized transform coefficients by using an encoding method such as CABAC. In addition, the encoder 350 encodes information such as CTU size, MinQTSize, MaxBTSize, MaxBTDepth, MinBTSize, QT split flag, BT split flag, split type, etc. related to block division, so that the image decoding apparatus is the same as the image encoding apparatus. Allows you to split blocks.

The encoder 350 encodes information about a prediction type indicating whether a current block is encoded by intra prediction or inter prediction, and encodes intra prediction information or inter prediction information according to the prediction type.

The inverse quantizer 360 inverse quantizes the quantized transform coefficients output from the quantizer 345 to generate transform coefficients. The inverse transformer 365 restores the residual block by converting the transform coefficients output from the inverse quantizer 360 from the frequency domain to the spatial domain.

The adder 370 adds the reconstructed residual block and the predicted block generated by the predictor 320 to reconstruct the current block. The pixels in the reconstructed current block are used as reference pixels when intra prediction of the next order of blocks.

The filter unit 380 deblocks and filters the boundary between the reconstructed blocks in order to remove blocking artifacts that occur due to encoding / decoding of blocks. When all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

7 illustrates an image decoding apparatus that may use the techniques of the present disclosure.

The image decoding apparatus includes a decoder 710, an inverse quantizer 720, an inverse transformer 730, a predictor 740, an adder 750, a filter 760, and a memory 770. Like the image encoding apparatus of FIG. 2, the image decoding apparatus may be implemented by each component as a hardware chip, or may be implemented by software and a microprocessor to execute a function of software corresponding to each component.

The decoder 710 decodes the bitstream received from the image encoding apparatus, extracts information related to block division, determines a current block to be decoded, and includes prediction information and residual signal information necessary for reconstructing the current block. Extract

The decoder 710 extracts information on the CTU size from a Sequence Parameter Set (SPS) or Picture Parameter Set (PPS) to determine the size of the CTU, and divides the picture into a CTU of the determined size. The CTU is determined as the highest layer of the tree structure, that is, the root node, and the CTU is partitioned using the tree structure by extracting partition information about the CTU. For example, when splitting a CTU using a QTBT structure, first, a first flag (QT_split_flag) related to splitting of QT is extracted, and each node is divided into four nodes of a lower layer. For the node corresponding to the leaf node of the QT, the second flag BT_split_flag and the split type information related to the splitting of the BT are extracted to split the corresponding leaf node into the BT structure.

Taking the block splitting structure of FIG. 4 as an example, the QT splitting flag QT_split_flag corresponding to the node of the highest layer of the QTBT structure is extracted. Since the value of the extracted QT split flag QT_split_flag is 1, the node of the highest layer is divided into four nodes of the lower layer (layer 1 of QT). Then, the QT splitting flag QT_split_flag for the first node of layer 1 is extracted. Since the extracted QT split flag (QT_split_flag) has a value of 0, the first node of layer 1 is no longer split into a QT structure.

Since the first node of layer 1 of QT becomes a leaf node of QT, the first node of layer 1 of QT proceeds to BT, which is the root node of BT. A BT split flag (BT_split_flag) corresponding to the root node of BT, that is, (layer 0), is extracted. Since the BT split flag BT_split_flag is 1, the root node of BT is split into two nodes of (layer 1). Since the root node of the BT is split, split type information indicating whether a block corresponding to the root node of the BT is split vertically or horizontally is extracted. Since the split type information is 1, the block corresponding to the root node of BT is split vertically. Then, the BT split flag (BT_split_flag) for the first node of (layer 1) split from the root node of BT is extracted. Since the BT split flag BT_split_flag is 1, the split type information of the block of the first node of (layer 1) is extracted. Since the partition type information of the block of the first node of (layer 1) is 1, the block of the first node of (layer 1) is vertically divided. Then, the BT split flag BT_split_flag of the second node of (layer 1) divided from the root node of BT is extracted. Since the BT split flag BT_split_flag is 0, it is no longer split by BT.

For example, extracting the BT split flag (BT_split_flag = 1) for the first node of (layer 1) partitioned from the root node of BT, and extracting the partition type information (0), the first node of (layer 1) The blocks are divided horizontally. Then, the BT split flag BT_split_flag of the second node of (layer 1) divided from the root node of BT is extracted. If the BT split flag BT_split_flag is 1, the split type information is automatically set to 1 without extracting the split type information anymore, and the block of the second node of (layer 1) is vertically split.

In this way, the decoder 710 first extracts the QT split flag QT_split_flag repeatedly to split the CTU into a QT structure. The BT split flag (BT_split_flag) is extracted for the leaf node of the QT, and when the BT split flag (BT_split_flag) indicates the split, split type information is extracted. In this manner, the decoder 710 may confirm that the CTU is divided into a structure as shown in FIG.

On the other hand, if information such as MinQTSize, MaxBTSize, MaxBTDepth, MinBTSize, etc. is additionally defined in the SPS or PPS, the decoder 710 extracts the information and uses the information when extracting the split information for QT and BT. Can reflect.

For example, blocks having the same size as MinQTSize in QT are no longer split. Therefore, the decoder 710 does not extract the split information (QT split flag) about the QT of the block from the bitstream (that is, the QT split flag of the block does not exist in the bitstream), and automatically extracts the value. Set to zero. In addition, there is no BT in a block having a size larger than MaxBTSize in QT. Therefore, the decoder 710 does not extract the BT partition flag for the leaf node having a block larger than MaxBTSize in QT, and automatically sets the BT partition flag to 0. In addition, when the depth of the node of the BT reaches MaxBTDepth, the block of the node is no longer split. Therefore, the BT partition flag of the corresponding node is not extracted from the bitstream, and its value is automatically set to zero. Also, in BT, blocks having the same size as MinBTSize are no longer split. Accordingly, the decoder 710 does not extract the BT partition flag of the block having the same size as MinBTSize from the bitstream, and automatically sets the value to zero.

On the other hand, when the decoder 710 determines the current block (current block) to be decoded by splitting the tree structure, the decoder 710 extracts information about a prediction type indicating whether the current block is intra predicted or inter predicted.

When the prediction type information indicates intra prediction, the decoder 710 extracts syntax elements of intra prediction information (intra mode) of the current block. First, the decoder 710 extracts mode information (ie, an MPM flag) indicating whether an intra mode of a current block is selected from the MPMs. Also, in general, when the intra mode encoding information indicates that the intra mode of the current block is selected from the MPMs, first intra identification information for indicating which mode of the MPMs is selected as the intra mode of the current block is extracted; When the intra mode encoding information indicates that the intra mode of the current block is not selected among the MPMs, the second intra identification information for indicating which mode other than the MPM is selected as the intra mode of the current block is extracted. do.

Meanwhile, the decoder 710 extracts information on the quantized transform coefficients of the current block as information on the residual signal.

The inverse quantizer 720 inversely quantizes the quantized transform coefficients, and the inverse transformer 730 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to generate a residual block for the current block.

The predictor 740 includes an intra predictor 742 and an inter predictor 744. The intra predictor 742 is activated when the intra prediction is the prediction type of the current block, and the inter predictor 744 is activated when the intra prediction is the prediction type of the current block.

The intra predictor 742 determines the intra mode of the current block among the plurality of intra modes from the syntax element for the intra mode extracted from the decoder 710, and uses the reference pixels around the current block according to the intra mode. Predict the block.

To determine the intra mode of the current block, the intra predictor 742 constructs an MPM list including a predetermined number of MPMs from neighboring blocks of the current block. The method of constructing the MPM list is the same as that of the intra predictor 322 of FIG. 3.

In general, when the mode information of the intra prediction (ie, the MPM flag) indicates that the intra mode of the current block is selected from the MPMs, the intra prediction unit 742 may determine the first intra identification information among the MPMs in the MPM list. Select the indicated MPM as the intra mode of the current block. On the other hand, if the mode information indicates that the intra mode of the current block is not selected from the MPM, the intra mode of the current block is determined among the remaining intra modes except the MPMs in the MPM list using the second intra identification information.

The inter predictor 744 determines motion information of the current block using a syntax element for the intra mode extracted from the decoder 710, and predicts the current block using the determined motion information.

The adder 750 reconstructs the current block by adding the residual block output from the inverse transformer and the prediction block output from the inter predictor or the intra predictor. The pixels in the reconstructed current block are utilized as reference pixels in intra prediction of the block to be subsequently decoded.

The filter unit 760 deblocks and filters the boundary between the reconstructed blocks in order to remove blocking artifacts caused by block-by-block decoding, and stores them in the memory 390. When all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be decoded later.

The techniques disclosed below relate to intra coding. The techniques of this disclosure may be performed by, for example, the image encoding apparatus and the image decoding apparatus shown and described with respect to FIGS. 3 and 7. That is, in one example, the intra prediction unit described with respect to FIG. 3 may perform certain techniques described below when performing intra prediction while encoding a block of image data. In another example, the intra prediction unit described with respect to FIG. 7 may perform certain techniques described below when performing intra prediction while decoding a block of video data.

8 is a diagram illustrating a positional relationship between a current block and reference samples in intra prediction. In conventional general directional mode and non-directional modes, the left reference pixels R _0,1 , R _0,2 , R _0,3 , _... R _0,2N ) and the upper reference pixels R _1,0 , R _2,0 , R _{3,0 ...} R _{2N, 0} ) is used to determine prediction pixel values for the current block. 9A and 9B are conceptual diagrams illustrating how reference samples are used for a current block in directional intra mode. As illustrated in FIGS. 9A and 9B, the predictive block of the current block is generated singularly according to the direction of the selected intra mode.

The present disclosure is directed to intra prediction techniques that can be useful in blocks with different textures in the form of diagonal edges or diagonal shapes. In the new intra mode proposed in the present disclosure, the current block to be predicted is divided into two regions, and each region can be predicted individually according to the directional or non-directional mode illustrated in FIG. 5. The new intra mode proposed in the present disclosure may be referred to as "split intra mode" in the following description.

In split intra mode, a first type (referred to as "Split_right_down type"), in which two areas are separated by a diagonal line connecting the upper and lower right corners of the block, and two areas are divided by a diagonal line connecting the left and right ears of the block. There are two types (referred to as "Split_right_up type"). As shown in Figs. 10A, 10B, 11A and 11B, in the divided intra mode, the two regions are divided by a diagonal line crossing the current block. Two types for square blocks are illustrated in FIGS. 10A and 10B, and two types for rectangular blocks are illustrated in FIGS. 11A and 11B.

12A is a diagram for describing respective reference pixels used for two regions of the current block in the split intra mode of the Split_right_down type. In the Split_right_down type, the lower left region of the current block includes the left reference pixels R _0,1 , R _0,2 , (hatched in (a) of FIG. 12A). R _0,3 , _... R _0,2N ) are used to determine predicted pixel values, and the upper right region includes the upper reference pixels R _1,0 , R _2,0 , (hatched in (b) of FIG. 12A). R _{3,0 ...} R _{2N, 0} ) is used to determine prediction pixel values. Here, R _0,0 may or may not be used as the reference pixel for both regions.

12B is a diagram for describing respective reference pixels used for two regions of the current block in the split intra mode of the Split_right_up type. In the Split_right_up type, the upper left region is the left and upper reference pixels ({R _{0, 0} , R _{0 , 1} , R _{0 , 2} , adjacent to the corresponding block (hatched in (a) of FIG. 12B). R _{0 , 3} , _... R _{0, N} } and {R _{1, 0} , R _{2 , 0} , R _{3,0 ...} R _{N, 0} }) to determine the prediction pixel values, and the lower right region is the left and upper reference pixels R _{0, N + 1 that} are not adjacent to the corresponding block (hatched in (b) of FIG. 12B). , R _{0, N + 2} , R _{0 , N + 3} , _... R _0,2N and R _{N + 1, 0} , R _{N + 2, 0} , R _{N + 3,0} , _... R _{2N, 0} ) is used to determine prediction pixel values. Here, R _{0, N +1} and R _{N + 1,0} may or may not be used as reference pixels for both regions. Alternatively, R _{0, N +1} and R _{N + 1,0} may be used only to generate predicted pixel values on the diagonal. That is, the predicted pixel values on the diagonal may be determined from R _{0, N + 1} and R _{N + 1,0} .

In some embodiments, each mode to be used for prediction of two regions in split intra mode may be fixed. That is, one or two modes selected from the directional or non-directional modes illustrated in FIG. 5 may be assigned to each region as a default mode. The default modes assigned to the two regions may have the same direction or reverse directions. In addition, the default modes allocated to the two regions may be planar or DC mode, which is a non-directional mode. For example, in the Split_right_down type, the lower left region of the current block determines predictive pixel values in a DC mode using left reference pixels, and the upper right region predicts pixel values in a DC mode using upper reference pixels. Can be determined. In some other embodiments, the default modes assigned to the two regions may have the directionality illustrated in FIGS. 13A and 13B.

In some other embodiments, each mode to be used for prediction of the two regions in the split intra mode may be selected by the image encoding apparatus in consideration of compression efficiency, among the directional or non-directional modes illustrated in FIG. 5. .

13A, 13B, 14A, and 14B are conceptual diagrams illustrating some example ways in which reference samples are used for two regions in split intra mode. In the example of FIG. 13A, the two regions are predicted in opposite directions, and in the example of FIG. 13B, the predictions for the two regions have the same direction. In the example of FIGS. 14A and 14B, the predictions for the two regions have different directions from each other. In particular, the prediction shown by the scheme illustrated in FIG. 13B is bidirectional. That is, the left reference pixels R _0,0 , R _0,1 , R _0,2 , R _{0, 3} , _... , R _{0 , 2N} ) are directed upwards (with the same angle as the diagonal), and the upper reference pixels R _1,0 , R _2,0 , R _{3, 0} , _... , R _{2N, 0} ) are computed with one reference pixel value using two reference pixels values in the downward direction (same as the angle of the diagonal). That is, each pixel in the current block is predicted using a left reference pixel and an upper reference pixel value located on a straight line parallel to a diagonal line dividing the current block into two regions. In other words, the prediction pixel and the two reference pixels used to predict the pixel value lie on a straight line parallel to the diagonal that divides the current block into two regions. For example, the prediction pixels P _{1, 1} is predicted from the left reference pixels (R _0,2) and an upper reference pixel (R _2,0). Predicted pixel P _N , ₁ (or P _{1, N} ) is predicted from left R _{0, N +1} and top R _{N + 1,0} . In addition, P _{N, N} is predicted from the left R _0,2N and the upper R _{2N, 0} . Here, when predicting one prediction pixel value from two reference pixels, weights according to the distance from the prediction pixel position to the reference pixel position may be given to the reference pixel values. For example, the pixel value of the prediction pixel P _N , ₁ may be determined to be closer to R _{N + 1,0} than to R _{0, N + 1} .

The following four methods can be considered for signaling that the split intra mode is selected for the current block.

The first method is to treat split_right_down and split_right_up as new intra modes and signal them. For example, if the two intra modes are added to a video standard having n intra modes, split_right_down may be in ( n +1) times and split_right_up may be in ( n +2) times. Or you can reverse the order. The added two modes are signaled in the same way as other existing modes.

The second method is to treat split_right_down and split_right_up as one split_intra_mode and signal it. For example, if one intra mode is added to a video standard having n intra modes, split_intra_mode may be ( n + 1) times. The added one mode is signaled in the same manner as other existing modes. Here, when the ( n + 1) mode is selected, the 1-bit flag is additionally used to signal whether split_right_down or split_right_up is used.

The third method is to treat split_right_down and split_right_up as one split_intra_mode and signal them. For example, if one intra mode is added to a video standard having n intra modes, split_intra_mode may be ( n + 1) times. The added one mode is signaled in the same manner as other existing modes. In this case, when the ( n + 1) mode is selected, whether split_right_down or split_right_up is derived is derived from values of reference pixels by the image decoding apparatus without signaling of a separate syntax element.

 If split_right_down was selected as the type of split intra mode of a given block, the lower left region of the current block would have a different texture than the lower right region, so that the reference pixels adjacent to the lower left side were also different from the reference pixels adjacent to the upper right region. Note that it is likely to have different values. Accordingly, the apparatus for decoding an image may infer that split_right_down is used if Equation 1 is satisfied, or infer that split_right_up is selected if it is not satisfied.

In Equation 1, the left side is the left reference pixels R _{0, 1} , R _{0 , 2} , for the lower left region when split_right_down is applied. R _{0 , 3} , _... The average value of R _0,2N ) and the upper reference pixels R _1,0 , R _2,0 , for the upper right region. R _{3,0 ...} R _{2N, 0} ) represents the difference between the mean values.

In Equation 1, the right side is the left and upper reference pixels R _{0, 0} , R _{0 , 1} , R _{0 , 2} , adjacent to the corresponding block for the upper left region when split_right_up is applied. R _{0 , 3} , _... R _{0, N} & R _{1, 0} , R _{2 , 0} , R _{3,0 ...} The left and upper reference pixels (R _{0, N + 1} , R _{0 , N + 2} , not adjacent to the corresponding block for the lower right region and the average value of R _{N, 0} ) R _{0 , N + 3} , _... R _0,2N & R _{N + 1, 0} , R _{N + 2, 0} , R _{N + 3,0} , _... R _{2N, 0} ) represents the difference between the mean values.

Although 2N reference pixels are all considered in Equation 1, a similar scheme may be applied to N reference pixels selected from 2N to reduce the amount of computation. For example, a similar approach is applied for reference pixels selected one pixel across, such as "0, 2, 4, 6, 8, 10 ..." or "1, 3, 5, 7, 9 ...". It may be.

The fourth way is to replace either split_right_down or split_right_up with the existing intra mode. For example, split_right_up can be used instead of the existing DC mode. In this case, split_right_up may be one of a non-directional mode (diagonal planar). As a result, the intra mode may include two non-directional modes and 65 directional modes including the conventional (vertical / horizontal) planar mode and the diagonal planar mode. The diagonal planar mode may replace the DC mode in every encoding / decoding process including the MPM setting process described with reference to FIGS. 3 to 7.

In the split intra mode, when a mode for two regions of the current block is selected by the image encoding apparatus from among a plurality of intra modes including directional and non-directional modes, additional signaling is required for each mode applied to the two regions. . On the other hand, when the default mode is allocated to two regions for the split intra mode, only information on whether the split intra mode is applied to the current block is sufficient.

On the other hand, when generating the prediction pixel values in the split intra mode, prediction pixel values placed on a diagonal line separating two regions may be determined using reference samples of either region. That is, the pixels on the diagonal may be included in either area and may be predicted in the same mode as the other pixels in the area. Alternatively, pixels on the diagonal can be determined using both reference samples in both regions.

15A to 15C are conceptual views illustrating a method of determining prediction pixel values on a diagonal line in a split intra mode of the split_right_down type. FIG. 15A is an example of how predictive pixel values on a diagonal line are determined using reference samples for the upper right region, and FIG. 15B is an example of how predictive pixel values on diagonal line are determined using reference samples for the lower left region. 15C is an example of how the prediction pixel values on the diagonal line are determined using both reference samples for the upper right region and reference samples for the lower left region.

As described above, segmented intra mode may be particularly useful for intra prediction of 360 images in which projection formats consisting of triangular faces may be used. When a 360 image is converted to a two-dimensional image based on Icosahedral projection (ISP) and Octahedron projection (OHP), if the original pixel values on the diagonal are filled with the upper left region value, the predicted pixels on the diagonal are also the upper left reference sample. Value should be filled in, and if the original pixel values on the diagonal were filled with the lower right region value, the prediction pixels should also be filled using the lower right reference sample values. If the information about the original pixel values on the diagonal is unknown, the prediction pixels can be filled using the upper left and lower right reference sample values. Alternatively, the image encoding apparatus and the image decoding apparatus may generate prediction pixel values on a diagonal line using reference samples of a predetermined region. Alternatively, the apparatus for encoding an image may generate prediction pixel values on a diagonal line using reference samples of a specific (upper left or lower right) region and then explicitly signal information about the diagonal.

Meanwhile, the split intra mode may be activated / deactivated by a flag indicating activation / deactivation of the split intra mode. In addition, the split intra mode may be activated / deactivated by a flag indicating 360 video. In addition, the split intra mode may be activated / deactivated by a syntax element that defines the projection format used for the two-dimensional representation of 360 video. As such, split intra mode may or may not be included in the plurality of available intra modes, as desired.

16 is a flowchart illustrating an example method for encoding image data in accordance with one or more examples described above of the present invention.

The video encoding apparatus determines an intra mode for predicting a current block of video data among a plurality of available intra modes (S1610). Wherein the plurality of available intra modes includes a plurality of directional modes, a plurality of non-directional modes, and at least one split intra mode. In the split intra mode, the prediction block for the current block is predicted separately for each of the two regions (ie, the first region and the second region) divided by dividing the current block diagonally. One of a plurality of directional modes and a plurality of non-directional modes is applied to the prediction for each region. One directional or non-directional mode may be commonly applied to the two regions, and a mode different from the mode applied to the first region may be applied to the second region. The mode applied to the first region and the mode applied to the second region may be predetermined among a plurality of directional or non-directional modes. Each region of the split intra mode may be assigned one or two modes selected from a plurality of directional or non-directional modes as a default mode. Default modes assigned to the two regions may have the same directionality, reverse direction to each other, or both directions at the same time. In addition, the default modes allocated to the two regions may be planar or DC mode, which is a non-directional mode. In some other embodiments, each mode to be used for prediction of the two regions in the split intra mode may be selected by the image encoding apparatus in consideration of compression efficiency, among the directional or non-directional modes illustrated in FIG. 5. .

In some embodiments, the plurality of available intra modes include two split intra modes, a first split intra mode in which two areas are separated by a diagonal line connecting the upper and lower right corners of the current block, and the two areas are current blocks. There may be a second split intra mode separated by a diagonal line connecting the lower left and upper right ears of.

In some other examples, there may be one split intra mode in the plurality of available intra modes. The one split intra mode is a first type in which two areas are divided by a diagonal line connecting the upper left and the right lower end of the current block, and a second type in which the two areas are separated by a diagonal connecting the left and right ears of the current block. This may exist.

Next, the image encoding apparatus determines a prediction block for the current block based on the determined intra mode of the current block (S1620). In split intra modes, at least some of the reference pixels used to determine the prediction pixels for the first region are different from the reference pixels used to determine the prediction pixels for the second region.

For example, when the first split intra mode is determined to be the intra mode of the current block, among the two areas of the current block, at least some of the reference samples used to determine the prediction pixels for the first area (the lower left area) are At least some of the reference samples selected on the left block L and the bottom left block BL of the current block, and used to determine the prediction pixels for the second region (the top right region) are the top block A of the current block. ) And the upper right block (AR).

Further, when the second split intra mode is determined to be the intra mode of the current block, at least some of the reference samples used to determine the prediction pixels for the first of the two regions (the upper left region) are the current block. At least some of the reference samples selected on the left block (L) and the top block (A) of and used to determine the prediction pixels for the second region (lower region) are the lower left block BL of the current block. And the upper right block AR.

The image encoding apparatus determines a residual block based on the current block and the prediction block (S1630). The image encoding apparatus encodes syntax elements that define an intra mode and a residual block of the current block (S1640). When the split intra mode is determined as the intra mode of the current block, the image encoding apparatus may further encode syntax elements that define one of a plurality of directional modes and a plurality of non-directional modes applied to each of the two regions.

17 is a flow chart illustrating an exemplary method of decoding image data in accordance with one or more examples described above of the present invention.

The image decoding apparatus decodes syntax elements that define an intra mode and a residual block for the current block of image data from the encoded bitstream (S1710).

Next, the image decoding apparatus determines an intra mode for predicting a current block of image data, from among a plurality of available intra modes, based on the decoded syntax elements (S1720). Wherein the plurality of available intra modes includes a plurality of directional modes, a plurality of non-directional modes, and at least one split intra mode. When the split intra mode is determined as the intra mode of the current block, the image decoding apparatus further decodes syntax elements defining one of a plurality of directional modes and a plurality of non-directional modes applied to each of two regions from the encoded bitstream. can do.

Next, the image decoding apparatus determines a prediction block for the current block based on the determined intra mode of the current block (S1730).

Next, the image decoding apparatus reconstructs the current block based on the determined prediction block and the residual block (S1740).

The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which the present embodiment belongs may make various modifications and changes without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment but to describe the present invention, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is incorporated by reference in Korean Patent Application No. 10-2017-0003158 filed on Jan. 09, 2017, and Jan. 08, 2018, which is incorporated by reference in its entirety. Claim priority on No. 10-2018-0002063.

Claims (12)

As a method of decoding image data, Decoding syntax elements that define an intra mode and a residual block of a current block of image data from the encoded bitstream; Determining an intra mode of the current block from among a plurality of available intra modes based on the decoded syntax elements defining the intra mode of the current block, wherein the plurality of available intra modes are a plurality of directional modes, a plurality of available intra modes. And a non-directional mode, and a plurality of divided intra modes, wherein the divided intra mode divides the current block diagonally so as to select one of the plurality of directional modes and the plurality of non-directional modes in each of two divided regions. Individually applied mode; Determining a prediction block for the current block based on the determined intra mode of the current block; And Reconstructing the current block based on the prediction block and the residual block. The method of claim 1, The plurality of split intra modes A first divisional intra mode in which the two regions are divided by a diagonal line connecting an upper left and a lower right of the current block; And A second divisional intra mode in which the two regions are divided by a diagonal line connecting a lower left and a right upper ear of the current block; And decoding the image data. The method of claim 2, In the first split intra mode, At least some of the reference samples used to determine the prediction pixels for the first of the two areas are selected on the left block L and the bottom left block BL of the current block, At least some of the reference samples used to determine the prediction pixels for the second of the two areas are selected on the upper block A and the upper right block AL of the current block. How to decrypt. The method of claim 2, In the second split intra mode, At least some of the reference samples used to determine the prediction pixels for the first of the two areas are selected on the left block (L) and the top block (A) of the current block, Image data, characterized in that at least some of the reference samples used to determine the prediction pixels for the second of the two areas are selected on the lower left block BL and the upper right block AR of the current block. How to decrypt. The method of claim 2, In the first split intra mode and the second split intra mode, And a predefined defold mode among the plurality of directional modes and the plurality of non-directional modes is applied to each of the two regions. The method of claim 2, Determining the prediction block, In response to determining that the split intra mode is the intra mode of the current block, syntax elements defining one of the plurality of directional modes and the plurality of non-directional modes applied to each of the two regions from an encoded bitstream are generated. And decoding the image data. As a method of decoding image data, Decoding syntax elements that define an intra mode and a residual block of a current block of image data from the encoded bitstream; Determining an intra mode of the current block from among a plurality of available intra modes based on the decoded syntax elements defining the intra mode of the current block, wherein the plurality of available intra modes are a plurality of directional modes, a plurality of available intra modes. And a split intra mode, wherein the split intra mode divides the current block diagonally to divide one of the plurality of directional modes and the plurality of non-directional modes in each of two divided regions. Individually applied mode; Determining a prediction block for the current block based on the determined intra mode of the current block; And Reconstructing the current block based on the prediction block and the residual block. The method of claim 7, wherein The split intra mode is Wherein the two regions have a first type separated by a diagonal connecting the upper left and the lower right of the current block, and the two regions have a second type separated by a diagonal connecting the lower left and the upper right of the current block. And decoding the video data. The method of claim 8, Determining the prediction block, In response to determining that the split intra mode is the intra mode of the current block, decoding a syntax element that defines the type of the split intra mode from an encoded bitstream. Way. The method of claim 8, Determining the prediction block, In response to determining that the split intra mode is the intra mode of the current block, Calculating a difference between an average value of reference pixels for a first region according to the first type of the current block and an average value of reference pixels for a second region according to the first type; Calculating a difference between an average value of reference pixels for a first region according to the second type of the current block and an average value of reference pixels for a second region according to the second type; And Determining the type of the split intra mode based on the comparison of the two differences. The method of claim 7, wherein In the split intra mode, At least some of the reference pixels used to determine the prediction pixels for the first one of the two regions are different from the reference pixels used to determine the prediction pixels for the second one of the two regions. And decoding the video data. An apparatus for decoding video data, Memory; And Includes one or more processors, The one or more processors, Decoding syntax elements that define an intra mode and a residual block of a current block of image data from the encoded bitstream; Determining an intra mode of the current block from among a plurality of available intra modes based on the decoded syntax elements defining the intra mode of the current block, wherein the plurality of available intra modes are a plurality of directional modes, a plurality of available intra modes. A non-directional mode, and at least one split intra mode, wherein the split intra mode divides the current block diagonally to individually separate one of the directional mode and the plurality of non-directional modes into each of two divided regions. Mode to apply; Determining a prediction block for the current block based on the determined intra mode of the current block; And Reconstructing the current block based on the prediction block and the residual block And decode the image data.

Images