WO2020040439A1 - Intra prediction method and device in image coding system - Google Patents

Abstract

An image decoding method performed by a decoding apparatus according to the present invention comprises the steps of: obtaining a Most Probable Mode (MPM) index of a current block; constructing an MPM list on the basis of neighboring blocks of the current block, deriving an intra prediction mode for the current block on the basis of the MPM list and the MPM index; and generating predicted blocks for the current block on the basis of the intra prediction mode, wherein the MPM list is derived based on the size of the current block, and the size of the current block is calculated by means of multiplying the width of the current block and the height of the current block.

Description

Intra prediction method and apparatus in image coding system

The present invention relates to image coding technology, and more particularly, to an intra prediction method and apparatus in an image coding system.

Recently, the demand for high resolution and high quality images such as high definition (HD) and ultra high definition (UHD) images is increasing in various fields. The higher the resolution and the higher quality of the image data, the more information or bit amount is transmitted than the existing image data. Therefore, the image data can be transmitted by using a medium such as a conventional wired / wireless broadband line or by using a conventional storage medium. In the case of storage, the transmission cost and the storage cost increase.

Accordingly, a high efficiency image compression technique is required to effectively transmit, store, and reproduce high resolution, high quality image information.

An object of the present invention is to provide a method and apparatus for improving image coding efficiency.

Another technical problem of the present invention is to provide an efficient intra prediction method and apparatus.

Another technical problem of the present invention is to provide a method and apparatus for constructing an MPM list based on the size of a current block.

Another technical problem of the present invention is to provide a method and apparatus for constructing a simple MPM list using a left neighbor block and an upper neighbor block based on the size of a current block.

According to an embodiment of the present invention, an image decoding method performed by a decoding apparatus is provided. The method may include obtaining a Most Probable Mode (MPM) index of a current block, constructing an MPM list based on neighboring blocks of the current block, intra prediction of the current block based on the MPM list and the MPM index. Deriving a mode and generating a predicted block for the current block based on the intra prediction mode, wherein the MPM list is derived based on the size of the current block, and the size of the current block The width of the current block and the height of the current block is calculated as a value.

According to another embodiment of the present invention, a decoding apparatus for performing image decoding is provided. The decoding apparatus obtains an entropy decoding unit for obtaining prediction information on a current block and a Most Probable Mode (MPM) index of the current block, constructs an MPM list based on neighboring blocks of the current block, and generates the MPM list. And a prediction unit for deriving an intra prediction mode of the current block based on the MPM index and generating a predicted block for the current block based on the intra prediction mode, wherein the MPM list includes a size of the current block. The size of the current block is calculated based on the product of the width of the current block and the height of the current block.

According to another embodiment of the present invention, a video encoding method performed by an encoding apparatus is provided. The method includes constructing a Most Probable Mode (MPM) list based on neighboring blocks of the current block, deriving an intra prediction mode of the current block based on the MPM list, and deriving an MPM index of the current block. Generating a predicted block for the current block based on the intra prediction mode, and generating, encoding and outputting prediction information for the current block including the MPM index, wherein the MPM list Is derived based on the size of the current block, and the size of the current block is calculated by multiplying the width of the current block and the height of the current block.

According to another embodiment of the present invention, a video encoding apparatus is provided. The encoding apparatus constructs a Most Probable Mode (MPM) list based on neighboring blocks of the current block, derives an intra prediction mode of the current block based on the MPM list, derives an MPM index of the current block, A prediction unit for generating a predicted block for the current block based on the intra prediction mode, and an entropy encoding unit for generating, encoding and outputting prediction information for the current block including the MPM index, wherein the MPM list Is derived based on the size of the current block, and the size of the current block is calculated by multiplying the width of the current block and the height of the current block.

According to the present invention, image compression efficiency can be improved.

According to the present invention, the intra prediction mode can be efficiently derived while reducing the computational complexity.

1 is a diagram schematically illustrating a video / image coding system to which the present invention may be applied.

2 is a diagram schematically illustrating a configuration of a video encoding apparatus to which the present invention may be applied.

3 is a diagram schematically illustrating a configuration of a video decoding apparatus to which the present invention may be applied.

4 is a diagram schematically illustrating some components of an encoding apparatus according to the present invention.

5 is a diagram schematically illustrating some components of a decoding apparatus according to the present invention.

6 schematically illustrates an intra prediction method according to the present invention.

7 shows an example of neighboring blocks for MPM candidate derivation.

8 schematically illustrates an example of a block size based MPM list construction method.

9 schematically illustrates another example of a block size based MPM list construction method.

10 schematically illustrates an image encoding method by an encoding apparatus according to the present invention.

11 schematically illustrates an image decoding method by a decoding apparatus according to the present invention.

12 schematically illustrates a content streaming system structure.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the invention to the specific embodiments. The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the spirit of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. The terms "comprise" or "having" herein are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and that one or more other features It is to be understood that the numbers, steps, operations, components, parts or figures do not exclude in advance the existence or the possibility of adding them.

On the other hand, each configuration in the drawings described in the present invention are shown independently for the convenience of description of the different characteristic functions, it does not mean that each configuration is implemented in separate hardware or separate software. For example, two or more of each configuration may be combined to form one configuration, or one configuration may be divided into a plurality of configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of the present invention without departing from the spirit of the present invention.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant description of the same components is omitted.

1 is a diagram schematically illustrating a video / image coding system to which the present invention may be applied.

Referring to FIG. 1, a video / image coding system may include a source device and a receiving device. The source device may transmit the encoded video / image information or data to the receiving device through a digital storage medium or a network in the form of a file or streaming.

The source device may include a video source, an encoding apparatus, and a transmitter. The receiving device may include a receiving unit, a decoding apparatus and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or an external component.

The video source may acquire the video / image through a process of capturing, synthesizing, or generating the video / image. The video source may comprise a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, video / image archives including previously captured video / images, and the like. Video / image generation devices may include, for example, computers, tablets and smartphones, and may (electronically) generate video / images. For example, a virtual video / image may be generated through a computer or the like. In this case, the video / image capturing process may be replaced by a process of generating related data.

The encoding device may encode the input video / image. The encoding apparatus may perform a series of procedures such as prediction, transform, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.

The transmitter may transmit the encoded video / video information or data output in the form of a bitstream to the receiver of the receiving device through a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmission unit may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast / communication network. The receiver may receive / extract the bitstream and transmit the received bitstream to the decoding apparatus.

The decoding apparatus may decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding apparatus.

The renderer may render the decoded video / image. The rendered video / image may be displayed through the display unit.

The present invention relates to video / picture coding. For example, the methods / embodiments disclosed in the present invention may include a versatile video coding (VVC) standard, an essential video coding (EVC) standard, an AOMedia Video 1 (AV1) standard, a second generation of audio video coding standard (AVS2) or It can be applied to the method disclosed in the image coding standard (ex. H.267 or H.268, etc.).

In the present invention, various embodiments related to video / image coding are presented, and unless otherwise stated, the above embodiments may be performed in combination with each other.

In the present invention, a video may refer to a set of images over time. A picture generally refers to a unit representing one image in a specific time zone, and a slice / tile is a unit constituting a part of a picture in coding. The slice / tile may comprise one or more coding tree units (CTUs). One picture may consist of one or more slices / tiles. One picture may consist of one or more tile groups. One tile group may include one or more tiles. The brick may represent a rectangular region of CTU rows within a tile in the picture.

In the present invention, tile groups and slices may be used interchangeably. For example, in this document, tile group / tile group header may be called slice / slice header.

A pixel or a pel may refer to a minimum unit constituting one picture (or image). Also, 'sample' may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a value of a pixel, and may represent only a pixel / pixel value of a luma component or only a pixel / pixel value of a chroma component.

A unit may represent a basic unit of image processing. The unit may include at least one of a specific region of the picture and information related to the region. One unit may include one luma block and two chroma (ex. Cb, cr) blocks. The unit may be used interchangeably with terms such as block or area in some cases. In a general case, an M × N block may include samples (or sample arrays) or a set (or array) of transform coefficients of M columns and N rows.

In the present invention, "/" and "," are interpreted as "and / or". For example, "A / B" is interpreted as "A and / or B" and "A, B" is interpreted as "A and / or B". In addition, "A / B / C" means "at least one of A, B and / or C". In addition, "A, B, C" also means "at least one of A, B, and / or C."

In addition, in this document "or" is interpreted as "and / or". For example, "A or B" may mean 1) "A" only, 2) "B" only, or 3) "A and B". In other words, "or" in this document may mean "additionally or alternatively".

2 is a diagram schematically illustrating a configuration of a video encoding apparatus to which the present invention may be applied.

Referring to FIG. 2, the encoding apparatus 100 may include an image splitter 110, a subtractor 115, a transformer 120, a quantizer 130, an inverse quantizer 140, an inverse transformer 150, The adder 155, the filter 160, the memory 170, the inter predictor 180, the intra predictor 185, and the entropy encoder 190 may be configured. The inter predictor 180 and the intra predictor 185 may be collectively called a predictor. That is, the predictor may include an inter predictor 180 and an intra predictor 185. The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in the residual processing unit. The residual processing unit may further include a subtracting unit 115. The image divider 110, the subtractor 115, the transformer 120, the quantizer 130, the inverse quantizer 140, the inverse transformer 150, the adder 155, and the filter 160 are described above. ), The inter predictor 180, the intra predictor 185, and the entropy encoder 190 may be configured by one hardware component (for example, an encoder chipset or a processor) according to an embodiment. In addition, the memory 170 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium. The hardware component may further include the memory 170 as an internal / external component.

The image divider 110 may divide an input image (or a picture or a frame) input to the encoding apparatus 100 into one or more processing units. For example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively divided according to a quad-tree binary-tree ternary-tree (QTBTTT) structure from a coding tree unit (CTU) or a largest coding unit (LCU). Can be. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad tree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, the quad tree structure may be applied first and the binary tree structure and / or ternary tree structure may be applied later. Alternatively, the binary tree structure may be applied first. The coding procedure according to the present invention may be performed based on the final coding unit that is no longer split. In this case, the maximum coding unit may be used as the final coding unit immediately based on coding efficiency according to the image characteristic, or if necessary, the coding unit is recursively divided into coding units of lower depths and optimized. A coding unit of size may be used as the final coding unit. Here, the coding procedure may include a procedure of prediction, transform, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be partitioned or partitioned from the last coding unit described above, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as block or area in some cases. In a general case, an M × N block may represent a set of samples or transform coefficients composed of M columns and N rows. A sample may generally represent a pixel or a value of a pixel, and may represent only a pixel / pixel value of a luma component or only a pixel / pixel value of a chroma component. A sample may be used as a term corresponding to one picture (or image) for a pixel or a pel.

The encoding apparatus 100 subtracts the prediction signal (predicted block, prediction sample array) output from the inter prediction unit 180 or the intra prediction unit 185 from the input image signal (original block, original sample array). A signal may be generated (residual signal, residual block, residual sample array), and the generated residual signal is transmitted to the converter 120. In this case, as shown, a unit for subtracting a prediction signal (prediction block, prediction sample array) from an input image signal (original block, original sample array) in the encoder 100 may be referred to as a subtraction unit 115. The prediction unit may perform a prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied on a current block or CU basis. As described later in the description of each prediction mode, the prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the generated information to the entropy encoding unit 190. The information about the prediction may be encoded in the entropy encoding unit 190 and output in the form of a bitstream.

The intra predictor 185 may predict the current block by referring to the samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located apart according to the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. Non-directional mode may include, for example, DC mode and planner mode (Planar mode). The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of detail of the prediction direction. However, as an example, more or less number of directional prediction modes may be used depending on the setting. The intra predictor 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

The inter predictor 180 may derive the predicted block with respect to the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of the motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), or the like, and a reference picture including the temporal neighboring block is called a collocated picture (colPic). It may be. For example, the inter prediction unit 180 constructs a motion information candidate list based on neighboring blocks, and provides information indicating which candidates are used to derive the motion vector and / or reference picture index of the current block. Can be generated. Inter prediction may be performed based on various prediction modes. For example, in the case of a skip mode and a merge mode, the inter prediction unit 180 may use motion information of a neighboring block as motion information of a current block. In the skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block is used as a motion vector predictor, and the motion vector of the current block is signaled by signaling a motion vector difference. Can be directed.

The prediction unit may generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction to predict one block, and may simultaneously apply intra prediction and inter prediction. This may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copy (IBC) to predict a block. The intra block copy may be used for content video / video coding of a game or the like, for example, screen content coding (SCC). The IBC basically performs prediction in the current picture but may be performed similarly to inter prediction in that a reference block is derived in the current picture. That is, the IBC can use at least one of the inter prediction techniques described in this document.

The prediction signal generated by the predictor (including the inter predictor 180 and / or the intra predictor 185) may be used to generate a reconstruction signal or to generate a residual signal. The transformer 120 may apply transform techniques to the residual signal to generate transform coefficients. For example, the transformation technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). It may include. Here, GBT means a conversion obtained from this graph when the relationship information between pixels is represented by a graph. The CNT refers to a transform that is generated based on and generates a prediction signal by using all previously reconstructed pixels. In addition, the conversion process may be applied to pixel blocks having the same size as the square, or may be applied to blocks of variable size rather than square.

The quantization unit 130 quantizes the transform coefficients and transmits them to the entropy encoding unit 190. The entropy encoding unit 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs the bitstream as a bitstream. have. The information about the quantized transform coefficients may be referred to as residual information. The quantization unit 130 may rearrange block quantized transform coefficients into a one-dimensional vector form based on a coefficient scan order, and quantize the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. Information about transform coefficients may be generated. The entropy encoding unit 190 may perform various encoding methods such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and the like. The entropy encoding unit 190 may encode information necessary for video / image reconstruction other than quantized transform coefficients (for example, values of syntax elements) together or separately. The encoded information (eg, encoded video / picture information) may be transmitted or stored in units of NALs (network abstraction layer) in the form of a bitstream. The video / image information may further include information about various parameter sets such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. Signaling / transmitted information and / or syntax elements described later in this document may be encoded and included in the bitstream through the above-described encoding procedure. The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The signal output from the entropy encoding unit 190 may include a transmitting unit (not shown) for transmitting and / or a storing unit (not shown) for storing as an internal / external element of the encoding apparatus 100, or the transmitting unit It may be included in the entropy encoding unit 190.

The quantized transform coefficients output from the quantization unit 130 may be used to generate a prediction signal. For example, the inverse quantization and inverse transform may be applied to the quantized transform coefficients through the inverse quantization unit 140 and the inverse transform unit 150 to restore the residual signal (residual block or residual samples). The adder 155 adds the reconstructed residual signal to the predicted signal output from the inter predictor 180 or the intra predictor 185 so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) is added. Can be generated. If there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be called a restoration unit or a restoration block generation unit. The generated reconstruction signal may be used for intra prediction of the next block to be processed in the current picture, and may be used for inter prediction of the next picture through filtering as described below.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied during picture encoding and / or reconstruction.

The filtering unit 160 may improve subjective / objective image quality by applying filtering to the reconstruction signal. For example, the filtering unit 160 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture is stored in the memory 170, specifically, the DPB of the memory 170. Can be stored in The various filtering methods may include, for example, deblocking filtering, a sample adaptive offset, an adaptive loop filter, a bilateral filter, and the like. The filtering unit 160 may generate various information related to the filtering and transmit the generated information to the entropy encoding unit 190 as described later in the description of each filtering method. The filtering information may be encoded in the entropy encoding unit 190 and output in the form of a bitstream.

The modified reconstructed picture transmitted to the memory 170 may be used as the reference picture in the inter predictor 180. When the inter prediction is applied through the encoding apparatus, the encoding apparatus may avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus, and may improve encoding efficiency.

The memory 170 DPB may store the modified reconstructed picture for use as a reference picture in the inter predictor 180. The memory 170 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the blocks in the picture that have already been reconstructed. The stored motion information may be transmitted to the inter predictor 180 to use the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 may store reconstructed samples of reconstructed blocks in the current picture, and transfer the reconstructed samples to the intra predictor 185.

3 is a diagram schematically illustrating a configuration of a video decoding apparatus to which the present invention may be applied.

Referring to FIG. 3, the decoding apparatus 200 includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, and an inter The prediction unit 260 and the intra prediction unit 265 may be configured. The inter predictor 260 and the intra predictor 265 may be collectively called a predictor. That is, the predictor may include an inter predictor 180 and an intra predictor 185. The inverse quantization unit 220 and the inverse transform unit 230 may be collectively called a residual processing unit. That is, the residual processor may include an inverse quantization unit 220 and an inverse transform unit 230. The entropy decoding unit 210, the inverse quantization unit 220, the inverse transformer 230, the adder 235, the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 are described above. It may be configured by one hardware component (for example, decoder chipset or processor). In addition, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium. The hardware component may further include the memory 250 as an internal / external component.

When a bitstream including video / image information is input, the decoding apparatus 200 may reconstruct an image corresponding to a process in which video / image information is processed in the encoding apparatus of FIG. 2. For example, the decoding apparatus 200 may derive units / blocks based on the block division related information obtained from the bitstream. The decoding apparatus 200 may perform decoding using a processing unit applied in the encoding apparatus. Thus, the processing unit of decoding may be a coding unit, for example, and the coding unit may be divided along the quad tree structure, binary tree structure and / or ternary tree structure from the coding tree unit or the largest coding unit. One or more transform units may be derived from the coding unit. The reconstructed video signal decoded and output through the decoding apparatus 200 may be reproduced through the reproducing apparatus.

The decoding apparatus 200 may receive a signal output from the encoding apparatus of FIG. 2 in the form of a bitstream, and the received signal may be decoded through the entropy decoding unit 210. For example, the entropy decoding unit 210 may parse the bitstream to derive information (eg, video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information about various parameter sets such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding apparatus may further decode the picture based on the information about the parameter set and / or the general restriction information. Signaling / received information and / or syntax elements described later in this document may be decoded through the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 210 decodes the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, quantized values of syntax elements required for image reconstruction, and transform coefficients for residuals. Can be output. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in a bitstream, and decodes syntax element information and decoding information of neighboring and decoding target blocks or information of symbols / bins decoded in a previous step. The context model may be determined using the context model, the probability of occurrence of a bin may be predicted according to the determined context model, and arithmetic decoding of the bin may be performed to generate a symbol corresponding to the value of each syntax element. have. In this case, the CABAC entropy decoding method may update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. The information related to the prediction among the information decoded by the entropy decoding unit 210 is provided to the predictor (the inter predictor 260 and the intra predictor 265), and the entropy decoding performed by the entropy decoder 210 is performed. Dual values, that is, quantized transform coefficients and related parameter information, may be input to the inverse quantizer 220. In addition, information on filtering among information decoded by the entropy decoding unit 210 may be provided to the filtering unit 240. Meanwhile, a receiver (not shown) that receives a signal output from the encoding apparatus may be further configured as an internal / external element of the decoding apparatus 200, or the receiver may be a component of the entropy decoding unit 210. Meanwhile, the decoding apparatus according to this document may be called a video / image / picture decoding apparatus, and the decoding apparatus may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). It may be. The information decoder may include the entropy decoding unit 210, and the sample decoder may include the inverse quantizer 220, an inverse transformer 230, an adder 235, a filter 240, and a memory 250. ), The inter predictor 260, and the intra predictor 265.

The inverse quantization unit 220 may dequantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the reordering may be performed based on the coefficient scan order performed in the encoding apparatus. The inverse quantization unit 220 may perform inverse quantization on quantized transform coefficients using a quantization parameter (for example, quantization step size information), and may obtain transform coefficients.

The inverse transformer 230 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

The prediction unit may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on the information about the prediction output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode.

The prediction unit may generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction to predict one block, and may simultaneously apply intra prediction and inter prediction. This may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copy (IBC) to predict a block. The intra block copy may be used for content video / video coding of a game or the like, for example, screen content coding (SCC). The IBC basically performs prediction in the current picture but may be performed similarly to inter prediction in that a reference block is derived in the current picture. That is, the IBC can use at least one of the inter prediction techniques described in this document.

The intra predictor 265 may predict the current block by referring to the samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located apart according to the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra predictor 265 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

The inter predictor 260 may derive the predicted block for the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of the motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block existing in the current picture and a temporal neighboring block present in the reference picture. For example, the inter prediction unit 260 may construct a motion information candidate list based on neighboring blocks and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information about the prediction may include information indicating a mode of inter prediction for the current block.

The adder 235 reconstructs the obtained residual signal by adding the obtained residual signal to a prediction signal (predicted block, prediction sample array) output from the prediction unit (including the inter prediction unit 260 and / or the intra prediction unit 265). A signal (restored picture, reconstructed block, reconstructed sample array) can be generated. If there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block may be used as the reconstructed block.

The adder 235 may be called a restoration unit or a restoration block generation unit. The generated reconstruction signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering as described below, or may be used for inter prediction of the next picture.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in the picture decoding process.

The filtering unit 240 may improve subjective / objective image quality by applying filtering to the reconstruction signal. For example, the filtering unit 240 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture is stored in the memory 250, specifically, the DPB of the memory 250. Can be sent to. The various filtering methods may include, for example, deblocking filtering, a sample adaptive offset, an adaptive loop filter, a bilateral filter, and the like.

The (modified) reconstructed picture stored in the DPB of the memory 250 may be used as the reference picture in the inter predictor 260. The memory 250 may store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the blocks in the picture that are already reconstructed. The stored motion information may be transmitted to the inter predictor 260 in order to use the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 250 may store reconstructed samples of reconstructed blocks in the current picture, and may transfer the reconstructed samples to the intra predictor 265.

In the present specification, the embodiments described by the filtering unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the encoding apparatus 100 are respectively the filtering unit 240 and the inter prediction of the decoding apparatus 200. The same may also apply to the unit 260 and the intra predictor 265.

Intra prediction may refer to prediction that generates prediction samples for the current block based on reference samples in a picture to which the current block belongs (hereinafter, referred to as a current picture). When intra prediction is applied to the current block, peripheral reference samples to be used for intra prediction of the current block may be derived. The peripheral reference samples of the current block are samples adjacent to the left border of the current block of size nWxnH and a total of 2xnH samples neighboring to the bottom-left, samples adjacent to the top border of the current block. And a total of 2 × nW samples neighboring to the top-right and one sample neighboring to the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include a plurality of upper peripheral samples and a plurality of left peripheral samples. In addition, the peripheral reference samples of the current block are a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block and the lower right side of the current block. It may include one sample neighboring (bottom-right).

However, some of the peripheral reference samples of the current block may not be decoded yet or available. In this case, the decoder can construct the surrounding reference samples to use for prediction by substituting the samples that are not available with the available samples. Alternatively, peripheral reference samples to be used for prediction may be configured through interpolation of the available samples.

When neighboring reference samples are derived, (i) the prediction sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the neighboring reference samples of the current block. The prediction sample may be derived based on a reference sample present in a specific (prediction) direction with respect to the prediction sample. In case of (i), it may be called non-directional mode or non-angle mode, and in case of (ii), it may be called directional mode or angular mode.

The interpolation between the second neighboring sample and the first neighboring sample located in a direction opposite to the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the neighboring reference samples may be performed. Prediction samples may be generated. The above case may be referred to as linear interpolation intra prediction (LIP). In addition, chroma prediction samples may be generated based on luma samples using a linear model. In this case, it may be called LM mode.

In addition, a temporary prediction sample of the current block is derived based on filtered neighbor reference samples, and at least one of the existing neighbor reference samples, that is, unfiltered neighbor reference samples, derived according to the intra prediction mode. A weighted sum of a reference sample and the temporary prediction sample may be used to derive the prediction sample of the current block. The above case may be referred to as position dependent intra prediction (PDPC).

In addition, a reference sample line having the highest prediction accuracy among the neighboring multi-reference sample lines of the current block is selected to derive the prediction sample by using the reference sample located in the prediction direction in the corresponding line, and then decode the used reference sample line. Intra-prediction encoding may be performed by instructing (signaling) the device. The above case may be referred to as multi-reference line (MRL) intra prediction or MRL based intra prediction.

In addition, intra prediction may be performed based on the same intra prediction mode by dividing the current block into vertical or horizontal subpartitions, and peripheral reference samples may be derived and used in units of the subpartition. That is, in this case, the intra prediction mode for the current block is equally applied to the subpartitions, and the intra prediction performance may be improved in some cases by deriving and using the peripheral reference samples in the subpartition unit. Such a prediction method may be called intra sub-partitions (ISP) or ISP based intra prediction.

The above-described intra prediction methods may be called an intra prediction type separately from the intra prediction mode. The intra prediction type may be called in various terms such as an intra prediction technique or an additional intra prediction mode. For example, the intra prediction type (or additional intra prediction mode) may include at least one of the above-described LIP, PDPC, MRL, and ISP. A general intra prediction method except for a specific intra prediction type such as LIP, PDPC, MRL, or ISP may be called a normal intra prediction type. The normal intra prediction type may be generally applied when the specific intra prediction type is not applied, and prediction may be performed based on the above-described intra prediction mode. Meanwhile, post-processing filtering may be performed on the predicted sample derived as needed.

In detail, the intra prediction procedure may include an intra prediction mode / type determination step, a peripheral reference sample derivation step, and an intra prediction mode / type based prediction sample derivation step. In addition, a post-filtering step may be performed on the predicted sample derived as needed.

Meanwhile, in addition to the above-described intra prediction types, affine linear weighted intra prediction (ALWIP) may be used. The ALWIP may be referred to as linear weighted intra prediction (LWIP) or matrix weighted intra prediction or matrix based intra prediction (MIP). If the MIP is applied to the current block, i) perform a matrix-vector-multiplication procedure using i) surrounding reference samples on which an averaging procedure has been performed, and iii) if necessary. Accordingly, a horizontal / vertical interpolation procedure may be further performed to derive prediction samples for the current block. The intra prediction modes used for the MIP may be configured differently from the intra prediction modes used in the LIP, PDPC, MRL, ISP intra prediction, or normal intra prediction. The intra prediction mode for the MIP may be called a MIP intra prediction mode, a MIP prediction mode or a MIP mode. For example, the matrix and offset used in the matrix vector multiplication may be set differently according to the intra prediction mode for the MIP. In this case, the matrix may be referred to as a (MIP) weighting matrix, and the offset may be referred to as a (MIP) offset vector or a (MIP) bias vector.

When intra prediction is applied, the intra prediction mode applied to the current block may be determined using the intra prediction mode of the neighboring block. For example, the decoding apparatus receives one of the MPM candidates in the most probable mode (MPM) list derived based on the intra prediction mode and additional candidate modes of the neighboring block (eg, left and / or upper neighboring block) of the current block. The selected MPM index may be selected or one of the remaining intra prediction modes not included in the MPM candidates (and planner mode) may be selected based on the remaining intra prediction mode information. The MPM list may be configured to include or not include a planner mode as a candidate. For example, when the MPM list includes a planner mode as a candidate, the MPM list may have six candidates. When the MPM list does not include a planner mode as a candidate, the MPM list may have three or five candidates. You can have a candidate. When the MPM list does not include a planner mode as a candidate, a not planner flag (eg, intra_luma_not_planar_flag) indicating whether an intra prediction mode of the current block is not a planner mode may be signaled. For example, the MPM flag is signaled first, and the MPM index and not planner flag may be signaled when the value of the MPM flag is one. In addition, the MPM index may be signaled when the value of the not planner flag is 1. Here, the configuration of the MPM list so as not to include a planner mode as a candidate does not mean that the planner mode is not an MPM, but because a planner mode is always considered as an MPM, a planner signal is signaled first. This is to check whether the mode is first.

For example, whether the intra prediction mode applied to the current block is in MPM candidates (and planner mode) or in remaining mode may be indicated based on the MPM flag (ex. Intra_luma_mpm_flag). A value of 1 of the MPM flag may indicate that the intra prediction mode for the current block is within MPM candidates (and planner mode), and a value of 0 of the MPM flag indicates that the intra prediction mode for the current block is MPM candidates (and planner mode). ) May be absent. The not planar flag (ex. Intra_luma_not_planar_flag) value 0 may indicate that the intra prediction mode for the current block is planner mode, and the not planar flag value 1 indicates that the intra prediction mode for the current block is not planner mode. Can be. The MPM index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information may be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information may index remaining intra prediction modes not included in the MPM candidates (and planner mode) among all intra prediction modes in order of prediction mode number to indicate one of them. The intra prediction mode may be an intra prediction mode for a luma component (sample). Hereinafter, intra prediction mode information includes the MPM flag (ex. Intra_luma_mpm_flag), the not planar flag (ex. Intra_luma_not_planar_flag), the MPM index (ex. Mpm_idx or intra_luma_mpm_idx), and the remaining intra prediction mode information (rem_intra_luma_intra_lum_mode). It may include at least one. In this document, the MPM list may be referred to in various terms such as MPM candidate list and candModeList. When the MIP is applied to the current block, a separate mpm flag (ex. Intra_mip_mpm_flag), mpm index (ex. Intra_mip_mpm_idx), remaining intra prediction mode information (ex. Intra_mip_mpm_remainder) for MIP may be signaled, and the not planar The flag is not signaled.

One embodiment of the present invention may propose a method of constructing an MPM list depending on a block size. One embodiment may be applied to encoding / decoding for still pictures and moving pictures, and the encoded / decoded still pictures and moving pictures may have a bitstream format or another type of file format. Here, the bitstreams and files may be stored in various kinds of storage of devices. Or it may be stored in a stream on various kinds of network environment, such as a cellular network, an Internet protocol. In one embodiment, when streaming to user terminal devices having a display, the bitstream or files can be decoded and played back. One embodiment of the present invention can be applied to all devices having an encoder and / or a decoder regardless of the display of the bitstream.

The method of constructing a block size dependent MPM list according to an embodiment may effectively reduce the overhead of signaling intra modes as the number of intra prediction modes increases. In addition, the intra mode may be efficiently coded through the method according to an embodiment, thereby improving coding efficiency in terms of reducing signaling overhead.

In other words, the method according to an embodiment may reduce the signaling overhead of the intra prediction mode, and according to the reduction of the overhead, the Bjntegaard Distortion rate (BD-rate) Peak Signal to Noise Ratio (PSNR) Improved coding efficiency can be achieved in terms of In addition, bitstream throughput at smaller block sizes can be improved such that the method according to one embodiment is more hardware friendly.

4 is a diagram schematically illustrating some components of an encoding apparatus according to the present invention.

5 is a diagram schematically illustrating some components of a decoding apparatus according to the present invention.

4 and 5, the intra prediction method may exploit correlation between samples and use the difference between the original block and the prediction of the original block. Transform and quantization can be applied to this residual to further remove spatial redundancy. Here, the residual may refer to the difference between the original block and the prediction of the original block, and the prediction of the original block may refer to the predicted block for the original block.

6 schematically illustrates an intra prediction method according to the present invention.

Referring to FIG. 6, an intra prediction method according to an embodiment of the present invention may be configured by three steps of reference sample construction, sample prediction, and post filtering.

For example, sample prediction may use known peripheral reference samples and intra prediction mode to predict unknown samples. That is, the prediction sample for the current sample may be generated using the decoded neighbor reference samples and the intra prediction mode. Multiple lines of reference samples can also be used for more accurate predictions.

The best intra mode on the encoding device side may be determined by jointly optimizing bit rate and distortion. The optimal intra mode can then be coded in the bitstream, and the decoding apparatus can perform intra prediction using the parsed optimal intra mode. In other words, the encoding apparatus may derive the optimal intra mode, encode the information on the derived optimal intra mode, and output the bitstream. The decoding apparatus may obtain information about an optimal intra mode by parsing the bitstream, and may perform intra prediction based on the information. However, an increased number of intra prediction modes may require efficient intra mode coding in order to minimize signaling overhead.

7 shows an example of neighboring blocks for MPM candidate derivation.

The Most Probable Mode (MPM) list may be constructed using blocks coded in the peripheral intra mode in the encoding apparatus and the decoding apparatus. In addition, the overhead may be minimized by signaling the MPM index when the best mode is one of the MPMs. Since the MPM is coded in truncated unary code, order may be important in terms of saving bits.

Referring to FIG. 7, for example, an order for configuring six MPM candidates may include the intra mode of the left peripheral block D of the current block, the intra mode of the upper peripheral block B of the current block, and the planar mode. mode), a DC mode, an intra mode of the lower left peripheral block E of the current block, an intra mode of the right upper peripheral block C of the current block, and an intra mode of the upper left peripheral block A of the current block. . If the optimal intra mode is not in the MPM list, a flag may be signaled to indicate an exception. Thus, the optimal intra mode can be coded using variable length coding or fixed length coding. The number of MPMs may depend on the number of intra modes. In general, the number of MPMs may increase with the number of intra modes, but may not increase. For example, when the number of intra modes is 35 and / or 67, the number of MPMs may have various numbers such as 3, 4, 5, 6, etc., which may be designed. If blocks coded in the peripheral intra mode have information about the intra mode, these intra modes can be used to construct the MPM list.

8 schematically illustrates an example of a block size based MPM list construction method.

In one embodiment, various MPM list construction algorithms may be used for bit reduction of this overhead. For example, a 6 MPM list construct may perform exhaustive search through the locations of the various perimeters, and may constantly perform pruning checks to exclude the same intra mode.

Here, 6 MPM list construction can be used for the intra mode coding method to accommodate the increased number of directional intra modes. Two main technical aspects may relate to derivation of six MPM candidates and entropy coding of six MPM candidates and non-MPM modes. Modes containing MPM lists may be classified into three groups. The three groups may include neighbor intra modes, derived intra modes, and default intra modes. Here, intra modes may be referred to as intra prediction modes.

Five peripheral intra prediction modes may be used to form the MPM list. The five neighboring intra prediction modes may refer to the intra prediction modes of the five neighboring blocks of the current block, and the positions of the five neighboring blocks may be the same as the positions used in the merge mode, A, B of FIG. And may be the same as the positions of C, D and E.

The initial MPM list may be formed by inserting five peripheral intra modes, a planner mode, and a DC mode into the MPM list, and may remove duplicated modes through a pruning process. In other words, the pruning process can be used to remove duplicated modes so that unique modes are included in the MPM list. That is, duplicate modes may be removed through pruning on the initial MPM list. The order of the modes in the initial mode or the initial MPM list is the intra mode of the left peripheral block D of the current block, the intra mode of the upper peripheral block B of the current block, the planar mode, the DC mode, of the current block. An intra mode of the lower left peripheral block E, an intra mode of the upper right peripheral block C of the current block, and an intra mode of the upper left peripheral block A of the current block.

If the MPM list is not full, such as when there are less than six MPM candidates in the MPM list, derived modes may be added. The derived modes may be derived as -1 or +1 modes for the angle modes included in the MPM list, and may not apply for non-angle modes such as DC or planner.

Finally, if the MPM list is still not complete, default modes can be added. The order in which the default intra mode is added may be a vertical intra prediction mode, a horizontal intra prediction mode, a second intra prediction mode, and a 3 (diagonal) intra prediction mode. Through this process, a unique list of 6 MPM modes can be generated.

Truncated unary binarization of MPM candidates may be used for entropy coding of 6 MPM candidates. The first three bins may be coded context according to the MPM mode associated with the bin currently being signaled. The MPM mode can be classified into one of three categories. The three categories may include whether the mode is a horizontal mode, a vertical mode, or a non-directional mode. The horizontal mode may refer to a mode in which the MPM mode is the same as or smaller than the diagonal direction or the horizontal directional mode. The vertical mode may refer to a mode in which the MPM mode is larger than the diagonal direction or the vertical directional mode. Non-directional mode may refer to DC mode and planner mode. Thus, three contexts can be used to signal the MPM index.

The coding of the remaining 61 non-MPM candidates (non-MPMs) may be as follows. 61 non-MPM modes can be divided into two sets first. The two sets may include a set of selected modes and a set of non-selected modes. The selected set of modes may include 16 modes, and the remaining 45 modes may be assigned to the set of unselected modes. The mode set to which the current mode belongs may be indicated in the bitstream with a flag. The mode from the selected set may then be signaled with a 4 bit fixed length code. In addition, the modes from the unselected set can be coded with truncated binary code. The selected set of modes may be generated by sub-sampling a total of 61 non-MPM modes with an index as in Equation (1).

In other words, in one embodiment, when the MPM list is not full, the adjacent angle mode followed by the predefined default mode may be considered with the pruning check. Here, exhaustive search and endless pruning may have advantages for bit rate savings. However, this may increase the number of hardware operation cycles for MPM list construction of each block. Considering the worst case scenario, if a 4k image of 3840x2160 is divided into 4x4 blocks for intra prediction, the increased hardware operating cycles for each 4x4 block may be important for throughput.

Accordingly, an embodiment may propose a method of constructing an MPM list depending on a block size in consideration of hardware throughput as well as saving bit rate as follows.

Referring to FIG. 8, an embodiment may determine whether a block size is less than a threshold. In other words, it may be checked whether the target block is a small block. Here, the target block may be referred to as a current block, and the MPM list depending on the above-described block size may mean an MPM list of the target block. For example, the number of pixels in the target block may be compared with a given threshold as shown in equation (2).

In Equation 2, TH _k may mean a threshold, and may have a value such as 32, 64, 128, or the like. For example, if TH _k is 64, blocks such as 4x4, 4x8 and 8x4 sizes may use the simple k MPM list construction method. In other words, TH _k may represent a threshold associated with the size of the block, compare the threshold with the number of samples in the block obtained by multiplying the block's width (block_width) and the block's height (block_height), If the number is smaller than the threshold, a simple k MPM list construction method may be used for the block. Here, k may represent the number of MPM candidates in the MPM list.

For example, a width threshold and a height threshold may be used as shown in Equation 3 to classify a target block between a small block and a large block.

In Equation 3, TH _w may mean a width threshold, TH _h may indicate a height threshold, and TH _w and TH _h may have values such as 4, 8, or 16, respectively. For example, if TH _w and TH _h are 4 respectively, 4 × 4 blocks may use a simple k MPM list construction method.

In other words, an embodiment may exclude blocks having a predetermined size or less from using a simple k MPM list construction method. Alternatively, a simple k MPM list construction method may be applied to blocks between the first size and the second size. To this end, one embodiment may compare each of the width and height of the block with a threshold. That is, a simple k MPM list construction method may be applied to blocks whose width is greater than or equal to TH _{w and} whose width is greater than or equal to TH _h .

In one embodiment the threshold values may be a tradeoff between bit rate savings and hardware throughput. That is, the bit rate saving aspect and the hardware throughput aspect may be tradeoff relations according to threshold values. If the threshold is large, throughput can be increased by applying a simpler k MPM than a sophisticated k MPM for relatively small blocks. However, if the threshold value is too large, the bit rate savings can be reduced. Thus, an appropriate threshold value can be defined to achieve proper balancing between bit rate savings and throughput. For example, in the case of an art video codec, the TH _k value may be 64 for balanced throughput, but is not limited thereto.

In one embodiment, when the target block satisfies the above conditions, such as belonging to a small block, a simple k MPM list construction method may be applied to reduce the worst case complexity. For example, a simple k MPM could be a HEVC 3 MPM list construction that only performs two position search and few pruning and condition checks. Or left and top modes (as HEVC), but a 6 MPM list may be generated. In this case, the complexity may be similar to the HEVC 3 MPM list construction, but six MPM candidates may be generated.

If one embodiment does not satisfy the above conditions, a sophisticated k MPM list construction method may be applied. For example, a 6 MPM list construction with five peripheral modes shown in FIG. 7 may be used to improve coding performance. Here, the peripheral modes may represent intra modes of neighboring blocks of the current block. Sophisticated MPM generation methods may require more operations (compare) to generate the MPM list because relatively more peripheral modes are compared with each other. However, sophisticated MPM lists can provide improved prediction for coding modes and lead to higher coding performance.

Here, the above-described parameter k is configurable and can be changed. For example, k may be 3 when the number of intra modes is 35 and 6 when k is the number of intra modes. However, k may be 5, but is not limited thereto.

For example, a simple k MPM list may be configured as shown in Table 1 when k is 6.

In Table 1, DiagonalDownIntraMode can represent a downward diagonal intra prediction mode, a left downward diagonal intra prediction mode, or an intra prediction mode 2, and DiagonalIntraMode can represent a diagonal intra prediction mode, a left upward diagonal intra prediction mode, or an intra 34 prediction mode. Can be. In addition, DiagonalUpIntraMode may indicate an upward diagonal intra prediction mode, a right upward diagonal intra prediction mode, or an intra prediction mode 66.

9 schematically illustrates another example of a block size based MPM list construction method.

The MPM list may include a simple MPM list and a sophisticated MPM list. For example, a simple MPM list scheme may share common parts with a sophisticated MPM list scheme.

Referring to FIG. 9, the simple k MPM list construct may be composed of "k MPM Part A" and "k MPM Part B", and the sophisticated k MPM list construct may be composed of "k MPM Part A" and "k MPM Part. C ". That is, the simple k MPM list construction method may have some common processes with the sophisticated k MPM list construction method.

For example, k MPM Part A may include a list structure based on left and top conditions, and k MPM Part B may also include a list structure based on left and top conditions. However, k MPM Part C may include a list structure based on other things such as neighboring blocks A, B, C, D, and E shown in FIG. 7.

10 schematically illustrates an image encoding method by an encoding apparatus according to the present invention.

The method disclosed in FIG. 10 may be performed by the encoding apparatus disclosed in FIG. 2. Specifically, for example, S1000 to S1030 of FIG. 10 may be performed by the prediction unit of the encoding apparatus, and S1040 may be performed by the entropy encoding unit of the encoding apparatus. In addition, although not shown, a process of deriving a residual sample for the current block based on an original sample and a prediction sample for the current block may be performed by a subtractor of the encoding apparatus. The generating of the information about the residual on the basis of the current block may be performed by a converter of the encoding apparatus. The encoding of the information on the residual and the prediction of the current block may be performed. It may be performed by the entropy encoding unit of the encoding device.

The encoding apparatus configures an MPM list based on neighboring blocks of the current block (S1000). The MPM list may be referred to in various terms such as MPM candidate list and candModeList. The MPM list may be configured to include or not include a planner mode as a candidate. For example, when the MPM list includes a planner mode as a candidate, the MPM list may have six candidates. When the MPM list does not include a planner mode as a candidate, the MPM list may have three or five candidates. You can have a candidate. When the MPM list does not include a planner mode as a candidate, a not planner flag (eg, intra_luma_not_planar_flag) indicating whether an intra prediction mode of the current block is not a planner mode may be signaled.

In an embodiment, the MPM list may be derived based on the size of the current block, and the size of the current block may be calculated as Equation 2 by multiplying the width of the current block and the height of the current block. .

For example, the MPM list may include a simple MPM list when the size of the current block is less than a first threshold TH _k . For example, if TH _k is 64, blocks such as 4x4, 4x8 and 8x4 sizes may use the simple k MPM list construction method. In other words, TH _k may represent a threshold associated with the size of the block, compare the threshold with the number of samples in the block obtained by multiplying the block's width (block_width) and the block's height (block_height), If the number is smaller than the threshold, a simple k MPM list construction method may be used for the block.

Alternatively, in the MPM list, as in Equation 3, the size of the current block is less than the first threshold TH _k , the width of the current block is greater than or equal to the second threshold TH _w , and the height of the current block is third. If it is greater than or equal to the threshold value TH _h , the simple MPM list may be included. For example, if TH _w and TH _h are 4 respectively, 4 × 4 blocks may use a simple k MPM list construction method. In other words, an embodiment may exclude blocks having a predetermined size or less from using a simple k MPM list construction method.

The simple MPM list is configured based on a neighbor intra prediction mode, a derived intra prediction mode, and a default intra prediction mode, wherein the peripheral intra prediction mode is a left neighboring block and an upper side of the current block. It may include an intra prediction mode of at least one of the neighboring blocks.

For example, the simple MPM list may be configured as shown in Table 1. In other words, the simple MPM list includes -1 intra prediction mode (leftIntraMode -1), the number based on the number of intra prediction modes of the left neighboring block of the current block, the number of intra prediction modes of the left neighboring block, and the number The intra prediction mode may include +1 intra prediction mode (leftIntraMode +1), the planar mode (PlanarIntraMode), the +3 intra prediction mode (leftIntraMode +3), and the DC mode (DCIntraMode) based on the number. Alternatively, the simple MPM list includes a planar mode (PlanarIntraMode), a vertical intra prediction mode (VerticalIntraMode), a left downward diagonal intra prediction mode (DiagonalDownIntraMode), a horizontal intra prediction mode (HorizontalIntraMode), and a left upward diagonal intra prediction mode. It may include a +3 intra prediction mode (DiagonalIntraMode +3) and a right upward diagonal intra prediction mode (DiagonalUpIntraMode). Alternatively, the simple MPM list may include an intra prediction mode (leftIntraMode) of the left neighboring block of the current block, an intra prediction mode of the upper neighboring block of the current block (aboveIntraMode), a planar mode (PlanarIntraMode), and a left neighboring block of the current block. -1 intra prediction mode (leftIntraMode -1) based on the number of intra prediction modes, +1 intra prediction mode (leftIntraMode +1) based on the number, the number of intra prediction modes of the number and the upper neighboring block It may include +2 intra prediction modes (leftIntraMode + aboveIntraMode +2) based on the sum of.

For example, the MPM list may include a sophisticated MPM list when the size of the current block is greater than or equal to the first threshold TH _k . Here, the sophisticated MPM list may include the simple MPM list and at least one MPM candidate in common. For example, the configuration of the simple MPM list may be configured based on the intra prediction mode of at least one of the left neighboring block and the upper neighboring block of the current block. The intra prediction mode may be configured based on at least one of a neighboring block, an upper left neighboring block, an upper neighboring block, and a right upper neighboring block. That is, candidates associated with the left neighboring block and the upper neighboring block may be commonly used.

The encoding apparatus derives the intra prediction mode of the current block based on the MPM list (S1010). That is, an intra prediction mode for prediction of the current block among candidates in the MPM list may be derived. Alternatively, a candidate for prediction of the current block among the candidates in the MPM list may be selected, and an intra prediction mode of the selected candidate may be derived.

The encoding apparatus derives the MPM index of the current block (S1020). Here, the MPM index may mean information indicating one of the MPM candidates included in the MPM list. That is, the MPM index may be information indicating a candidate having an intra prediction mode derived for prediction of the current block among candidates in the MPM list. The MPM index may be referred to as an index, and the MPM index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element.

The encoding apparatus generates a predicted block for the current block based on the intra prediction mode (S1030). Here, the derived intra prediction mode may be a directional mode or a non-directional mode. The encoding apparatus may generate a prediction sample based on the intra prediction mode, and may use the prediction sample directly as a reconstruction sample according to the prediction mode. In addition, the encoding apparatus may generate a residual sample based on the original sample and the generated prediction sample. The encoding apparatus may generate information about the residual based on the residual sample. The information about the residual may include transform coefficients related to the residual sample. The encoding apparatus may derive the reconstructed sample based on the prediction sample and the residual sample. That is, the encoding apparatus may derive the reconstructed sample by adding the prediction sample and the residual sample. Here, the encoding apparatus may generate a residual block based on the original block and the predicted block, and may generate information about the residual based on this. In addition, the encoding apparatus may output information in the form of a bitstream by encoding the information about the residual. The bitstream may be transmitted to a decoding apparatus via a network or a storage medium.

The encoding apparatus generates, encodes, and outputs prediction information about the current block including the MPM index (S1040). The encoding apparatus may encode and output image information including information on prediction of the current block in a bitstream form. For example, the encoding apparatus may determine the prediction mode of the current block, and generate information indicating the prediction mode. Also, information about the MPM index may be generated. In addition, it is possible to generate information about the residual. The above-described information on prediction of the current block may include all of the above-described information or may include only a part of the information. The bitstream may be transmitted to a decoding apparatus via a network or a storage medium.

11 schematically illustrates an image decoding method by a decoding apparatus according to the present invention.

The method disclosed in FIG. 11 may be performed by the decoding apparatus disclosed in FIG. 3. Specifically, for example, S1100 to S61130 of FIG. 11 may be performed by the prediction unit of the decoding apparatus. In addition, although not shown, a process of acquiring image information including information on prediction of a current block and information on residual through a bitstream may be performed by an entropy decoding unit of the decoding apparatus. The process of deriving the residual sample for the current block based on the dual information may be performed by an inverse transform unit of the decoding apparatus, and the process of generating a reconstructed picture based on the prediction sample and the residual sample may be performed. It may be performed by an adder of the decoding apparatus.

The decoding apparatus obtains an MPM index of the current block (S1100). Here, the MPM index may mean information indicating one of the MPM candidates included in the MPM list. That is, intra prediction may be performed based on the MPM candidate indicated by the MPM index. The MPM index may be referred to as an index, and the MPM index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element.

The decoding apparatus configures an MPM list based on neighboring blocks of the current block (S1110). The MPM list may be referred to in various terms such as MPM candidate list and candModeList. The MPM list may be configured to include or not include a planner mode as a candidate. For example, when the MPM list includes a planner mode as a candidate, the MPM list may have six candidates. When the MPM list does not include a planner mode as a candidate, the MPM list may have three or five candidates. You can have a candidate. When the MPM list does not include a planner mode as a candidate, a not planner flag (eg, intra_luma_not_planar_flag) indicating whether an intra prediction mode of the current block is not a planner mode may be signaled.

In an embodiment, the MPM list may be derived based on the size of the current block, and the size of the current block may be calculated as Equation 2 by multiplying the width of the current block and the height of the current block. .

For example, the MPM list may include a simple MPM list when the size of the current block is less than a first threshold TH _k . For example, if TH _k is 64, blocks such as 4x4, 4x8 and 8x4 sizes may use the simple k MPM list construction method. In other words, TH _k may represent a threshold associated with the size of the block, compare the threshold with the number of samples in the block obtained by multiplying the block's width (block_width) and the block's height (block_height), If the number is smaller than the threshold, a simple k MPM list construction method may be used for the block.

Alternatively, in the MPM list, as in Equation 3, the size of the current block is less than the first threshold TH _k , the width of the current block is greater than or equal to the second threshold TH _w , and the height of the current block is third. If it is greater than or equal to the threshold value TH _h , the simple MPM list may be included. For example, if TH _w and TH _h are 4 respectively, 4 × 4 blocks may use a simple k MPM list construction method. In other words, an embodiment may exclude blocks having a predetermined size or less from using a simple k MPM list construction method.

The simple MPM list is configured based on a neighbor intra prediction mode, a derived intra prediction mode, and a default intra prediction mode, wherein the peripheral intra prediction mode is a left neighboring block and an upper side of the current block. It may include an intra prediction mode of at least one of the neighboring blocks.

For example, the simple MPM list may be configured as shown in Table 1. In other words, the simple MPM list includes -1 intra prediction mode (leftIntraMode -1), the number based on the number of intra prediction modes of the left neighboring block of the current block, the number of intra prediction modes of the left neighboring block, and the number The intra prediction mode may include +1 intra prediction mode (leftIntraMode +1), the planar mode (PlanarIntraMode), the +3 intra prediction mode (leftIntraMode +3), and the DC mode (DCIntraMode) based on the number. Alternatively, the simple MPM list includes a planar mode (PlanarIntraMode), a vertical intra prediction mode (VerticalIntraMode), a left downward diagonal intra prediction mode (DiagonalDownIntraMode), a horizontal intra prediction mode (HorizontalIntraMode), and a left upward diagonal intra prediction mode. It may include a +3 intra prediction mode (DiagonalIntraMode +3) and a right upward diagonal intra prediction mode (DiagonalUpIntraMode). Alternatively, the simple MPM list may include an intra prediction mode (leftIntraMode) of the left neighboring block of the current block, an intra prediction mode of the upper neighboring block of the current block (aboveIntraMode), a planar mode (PlanarIntraMode), and a left neighboring block of the current block. -1 intra prediction mode (leftIntraMode -1) based on the number of intra prediction modes, +1 intra prediction mode (leftIntraMode +1) based on the number, the number of intra prediction modes of the number and the upper neighboring block It may include +2 intra prediction modes (leftIntraMode + aboveIntraMode +2) based on the sum of.

For example, the MPM list may include a sophisticated MPM list when the size of the current block is greater than or equal to the first threshold TH _k . Here, the sophisticated MPM list may include the simple MPM list and at least one MPM candidate in common. For example, the configuration of the simple MPM list may be configured based on the intra prediction mode of at least one of the left neighboring block and the upper neighboring block of the current block. The intra prediction mode may be configured based on at least one of a neighboring block, an upper left neighboring block, an upper neighboring block, and a right upper neighboring block. That is, candidates associated with the left neighboring block and the upper neighboring block may be commonly used.

The decoding apparatus derives an intra prediction mode of the current block based on the MPM index and the MPM list (S1120). That is, the intra prediction mode indicated by the MPM index among the candidates in the MPM list can be derived. Alternatively, a candidate indicated or indicated by the MPM index among the candidates in the MPM list may be selected, and an intra prediction mode of the selected candidate may be derived.

The decoding apparatus generates a predicted block for the current block based on the intra prediction mode (S1130). Here, the derived intra prediction mode may be a directional mode or a non-directional mode. The decoding apparatus may generate a prediction sample based on an intra prediction mode, and may directly use the prediction sample as a reconstruction sample according to the prediction mode, or generate a reconstruction sample by adding a residual sample to the prediction sample. . If there is a residual sample for the current block, the decoding apparatus may obtain information about the residual for the current block from the bitstream. The information about the residual may include transform coefficients regarding the residual sample. The decoding apparatus may derive the residual sample (or residual sample array) for the current block based on the residual information. The decoding apparatus may generate a reconstructed sample based on the prediction sample and the residual sample, and may derive a reconstructed block or a reconstructed picture based on the reconstructed sample. Thereafter, as described above, the decoding apparatus may apply an in-loop filtering procedure, such as a deblocking filtering and / or SAO procedure, to the reconstructed picture to improve subjective / objective picture quality as needed.

In the above-described embodiment, the methods are described based on a flowchart as a series of steps or blocks, but the present invention is not limited to the order of the steps, and any steps may occur in a different order or at the same time than the other steps described above. have. In addition, those skilled in the art will appreciate that the steps shown in the flowcharts are not exclusive and that other steps may be included or one or more steps in the flowcharts may be deleted without affecting the scope of the present invention.

The above-described method according to the present invention may be implemented in software, and the encoding device and / or the decoding device according to the present invention may perform image processing of, for example, a TV, a computer, a smartphone, a set-top box, a display device, and the like. It can be included in the device.

When the embodiments of the present invention are implemented in software, the above-described method may be implemented as a module (process, function, etc.) for performing the above-described function. The module may be stored in memory and executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor by a variety of known means. The processor may include application-specific integrated circuits (ASICs), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium and / or other storage device.

12 schematically illustrates a content streaming system structure.

That is, the embodiments described in the present invention may be implemented and performed on a processor, a microprocessor, a controller or a chip. For example, the functional units shown in each drawing may be implemented and performed on a computer, processor, microprocessor, controller, or chip.

In addition, the decoding apparatus and encoding apparatus to which the present invention is applied include a multimedia broadcasting transmitting and receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video chat device, a real time communication device such as video communication, and mobile streaming. Devices, storage media, camcorders, on-demand (VoD) service providing devices, OTT video (Over the top video) devices, Internet streaming service providing devices, three-dimensional (3D) video devices, video telephony video devices, and medical video devices. It may be included and used to process a video signal or a data signal. For example, the OTT video device may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

In addition, the processing method to which the present invention is applied can be produced in the form of a program executed by a computer, and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer readable recording medium includes all types of storage devices and distributed storage devices for storing computer readable data. The computer-readable recording medium may be, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical disc. It may include a data storage device. The computer-readable recording medium also includes media embodied in the form of a carrier wave (for example, transmission over the Internet). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network. In addition, embodiments of the present invention may be implemented as a computer program product by a program code, the program code may be performed on a computer by an embodiment of the present invention. The program code may be stored on a carrier readable by a computer.

In addition, the content streaming system to which the present invention is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

The encoding server compresses content input from multimedia input devices such as a smart phone, a camera, a camcorder, etc. into digital data to generate a bitstream and transmit the bitstream to the streaming server. As another example, when multimedia input devices such as smart phones, cameras, camcorders, etc. directly generate a bitstream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method to which the present invention is applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

The streaming server transmits multimedia data to the user device based on a user request through the web server, and the web server serves as an intermediary for informing the user of what service there is. When a user requests a desired service from the web server, the web server transmits it to a streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server plays a role of controlling a command / response between devices in the content streaming system.

The streaming server may receive content from a media store and / or an encoding server. For example, when the content is received from the encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined time.

Examples of the user device include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, a slate PC, Tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, glass glasses, head mounted displays), digital TVs, desktops Computer, digital signage, and the like. Each server in the content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

Claims (15)

In the image decoding method performed by the decoding apparatus, Obtaining a Most Probable Mode (MPM) index of the current block; Constructing an MPM list based on neighboring blocks of the current block; Deriving an intra prediction mode of the current block based on the MPM list and the MPM index; And Generating a predicted block for the current block based on the intra prediction mode, The MPM list is derived based on the size of the current block, And the size of the current block is calculated by multiplying the width of the current block by the height of the current block. The method of claim 1, And the MPM list comprises a simple MPM list if the size of the current block is less than a first threshold. The method of claim 1, The MPM list includes the simple MPM list when the size of the current block is less than a first threshold, the width of the current block is greater than or equal to a second threshold, and the height of the current block is greater than or equal to a third threshold. , Video decoding method. The method of claim 2, The simple MPM list, Configured based on a neighbor intra prediction mode, a derived intra prediction mode, and a default intra prediction mode, The neighbor intra prediction mode includes an intra prediction mode of at least one of a left neighbor block and an upper neighbor block of the current block. The method of claim 2, The simple MPM list, An intra prediction mode of the left neighboring block of the current block, an intra prediction mode of the number 1 of the intra prediction mode of the left neighboring block, an intra prediction mode of the +1 number of times based on the number, a planar mode, And an intra prediction mode and a DC mode of +3 times based on the number. The method of claim 2, The simple MPM list, +3 intra prediction modes and right upward diagonal intra prediction modes based on the number of planar modes, vertical intra prediction modes, left downward diagonal intra prediction modes, horizontal intra prediction modes, and left upward diagonal intra prediction modes. Image decoding method comprising a. The method of claim 2, The simple MPM list, -1 intra based on the number of the intra prediction mode of the left neighboring block of the current block, the intra prediction mode of the upper neighboring block of the current block, the planar mode, and the intra prediction mode of the left neighboring block of the current block And a prediction mode, an intra prediction mode +1 times based on the number, and an intra prediction mode +2 times based on the sum of the number and the number of intra prediction modes of the upper neighboring block. . The method of claim 2, And the MPM list comprises a sophisticated MPM list if the size of the current block is greater than or equal to the first threshold. The method of claim 8, And the elaborate MPM list comprises the simple MPM list and at least one MPM candidate in common. In the video encoding method performed by the encoding device, Constructing a Most Probable Mode (MPM) list based on neighboring blocks of the current block; Deriving an intra prediction mode of the current block based on the MPM list; Deriving an MPM index of the current block; Generating a predicted block for the current block based on the intra prediction mode; And Generating, encoding, and outputting prediction information about the current block including the MPM index, The MPM list is derived based on the size of the current block, And the size of the current block is calculated by multiplying the width of the current block by the height of the current block. The method of claim 10, And the MPM list comprises a simple MPM list if the size of the current block is less than a first threshold. The method of claim 10, The MPM list includes the simple MPM list when the size of the current block is less than a first threshold, the width of the current block is greater than or equal to a second threshold, and the height of the current block is greater than or equal to a third threshold. , Video encoding method. The method of claim 11, And the MPM list comprises a sophisticated MPM list if the size of the current block is greater than or equal to the first threshold. The method of claim 13, And the elaborate MPM list comprises the simple MPM list and at least one MPM candidate in common. A decoding apparatus for performing an image decoding method, An entropy decoding unit for obtaining prediction information about the current block; And Obtain a Most Probable Mode (MPM) index of the current block, construct an MPM list based on neighboring blocks of the current block, derive an intra prediction mode of the current block based on the MPM list and the MPM index, and A prediction unit generating a predicted block for the current block based on the intra prediction mode, The MPM list is derived based on the size of the current block, And the size of the current block is calculated by multiplying the width of the current block by the height of the current block.

Images