HK40105225A - Image encoding/decoding method, medium and method of transmitting bitstream - Google Patents

Description

Image encoding/decoding methods, media, and methods for transmitting bit streams

This application is a divisional application of the invention patent application filed on April 1, 2019, with application number 201980023756.9 and title "Method and Apparatus for Encoding/Decoding Images".

Technical Field

This disclosure relates to techniques for encoding and decoding images, and more specifically, to methods and apparatus for performing encoding/decoding in intra-frame prediction.

Background Technology

With the widespread use of the Internet and portable terminals, and the development of information and communication technologies, multimedia data is being used more and more. Therefore, in order to provide various services or perform various tasks through image prediction in various systems, there is an urgent need to improve the performance and efficiency of image processing systems. However, research and development achievements have not kept pace with this trend.

Therefore, existing methods and apparatuses for encoding/decoding images require performance improvements in image processing, particularly in image encoding or image decoding.

Summary of the Invention

Technical issues

The purpose of this disclosure is to provide a method and apparatus for performing intra-frame prediction. Another purpose of this disclosure is to provide a method and apparatus for performing sub-block-wise intra-frame prediction. A further purpose of this disclosure is to provide a method and apparatus for dividing the frame into sub-blocks and determining the coding order of the sub-blocks.

Technical solutions

In the method and apparatus for image encoding/decoding according to the present disclosure, candidate groups for partitioning types of the current block are configured, and partitioning types from the current block to sub-blocks are determined based on the candidate groups and candidate indices. Intra-prediction modes are derived on a per-block basis, and intra-prediction is performed on the current block based on the intra-prediction modes and partitioning types of the current block.

Beneficial effects

According to this disclosure, encoding/decoding performance can be improved through intra-frame prediction on a sub-block basis. Furthermore, according to this disclosure, prediction accuracy can be improved by effectively configuring candidate groups for sub-block partitioning types. Additionally, according to this disclosure, intra-frame prediction encoding/decoding efficiency can be improved by adjusting the encoding order on a sub-block basis.

Attached Figure Description

Figure 1 is a conceptual diagram illustrating an image encoding and decoding system according to an embodiment of the present disclosure.

Figure 2 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

Figure 3 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.

Figure 4 is an example diagram illustrating the various partitioning types that can be obtained in the block splitter of this disclosure.

Figure 5 is an example diagram illustrating an intra-prediction mode according to an embodiment of the present disclosure.

Figure 6 is an example diagram illustrating the configuration of reference pixels for intra-frame prediction according to an embodiment of the present disclosure.

Figure 7 is a conceptual diagram illustrating a target block for intra-frame prediction and a block adjacent to the target block according to an embodiment of the present disclosure.

Figure 8 is a diagram showing the various sub-block partitioning types that can be obtained from the coded block.

Figure 9 is an example diagram illustrating a reference pixel region for an intra-frame prediction mode according to an embodiment of the present disclosure.

Figure 10 is an example diagram illustrating the encoding order available in the upper right diagonal prediction mode according to an embodiment of the present disclosure.

Figure 11 is an example diagram illustrating the encoding order available in horizontal mode according to an embodiment of the present disclosure.

Figure 12 is an example diagram illustrating the coding order available in the lower right diagonal prediction mode according to an embodiment of the present disclosure.

Figure 13 is an example diagram illustrating the encoding order available in vertical mode according to an embodiment of the present disclosure.

Figure 14 is an example diagram illustrating the encoding order available in the lower left diagonal prediction mode according to an embodiment of the present disclosure.

Figure 15 is an example diagram illustrating the coding order based on intra-frame prediction mode and partitioning type according to an embodiment of the present disclosure.

Detailed Implementation

In the method and apparatus for image encoding/decoding according to the present disclosure, candidate groups of partition types available for the current block can be configured, partition types from the current block to sub-blocks can be determined based on candidate groups and candidate indices, intra-prediction modes can be derived on a per-block basis, and intra-prediction can be performed on the current block based on the intra-prediction mode of the current block and the sub-block partition type.

The mode of the present invention

This disclosure can be modified in various ways and has various implementations. Specific implementations of this disclosure will be described with reference to the accompanying drawings. However, these implementations are not intended to limit the technical scope of this disclosure, and it should be understood that this disclosure covers various modifications, equivalents, and alternatives within the scope and concept of this disclosure.

The terms first, second, A, and B used in this disclosure may be used to describe various components, rather than to limit them. These expressions are used only to distinguish one component from another. For example, without departing from the scope of this disclosure, a first component may be referred to as a second component, and a second component may be referred to as a first component. The term and/or covers a combination of or any one of several related terms.

When a component is said to be “connected to” or “coupled to” another component, it should be understood that a component is connected to the other component directly or through any other component. On the other hand, when a component is said to be “directly connected to” or “directly coupled to” another component, it should be understood that there are no other components between the components.

The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. Unless the context clearly specifies otherwise, the singular form includes the plural reference. In this disclosure, the terms “comprising” or “having” indicate the presence of a feature, number, step, operation, component, part, or combination thereof, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, terms including technical or scientific terms used in this disclosure may have the same meaning as commonly understood by those skilled in the art. Terms that are commonly defined in dictionaries may be interpreted as having the same or similar meaning as in the context of the relevant art. Unless otherwise defined, these terms should not be interpreted as having an ideal or overly formal meaning.

Typically, an image can include one or more color spaces depending on its color format. The image can include one or more images of the same size or one or more images of different sizes. For example, the YCbCr color configuration can support color formats such as 4:4:4, 4:2:2, 4:2:0, and monochrome (consisting only of Y). For example, YCbCr 4:2:0 can consist of one luminance component (Y in this example) and two chrominance components (Cb and Cr in this example). In this case, the configuration ratio of the chrominance and luminance components can be 1:2 width-to-height. For example, in the case of 4:4:4, it can have the same configuration ratio in terms of width and height. As in the examples above, when an image includes one or more color spaces, the image can be divided into color spaces.

Images can be classified into I, P, and B based on their image type (e.g., picture, slice, tile, etc.). I images can be images encoded/decoded without a reference image. P images can be images encoded/decoded using a reference image but only allowing forward prediction. B images can be images encoded/decoded using a reference image and allowing bidirectional prediction. However, depending on the encoding/decoding configuration, some types (P and B) can be combined, or image types with different compositions can be supported.

The various encoded/decoded information generated in this disclosure can be processed explicitly or implicitly. Explicit processing can be understood as the following: generating selection information from a group of candidates indicating one of several candidates related to the encoded information in a sequence, slice, tile, block, or sub-block via an encoder, including this selection information in the bitstream, and reconstructing the relevant information into decoded information by parsing the relevant information at the same unit level as in the encoder via a decoder. Implicit processing can be understood as processing the encoded/decoded information with the same procedures, rules, etc., at both the encoder and decoder.

Figure 1 is a conceptual diagram illustrating an image encoding and decoding system according to an embodiment of the present disclosure.

Referring to FIG1, each of the image encoding device 105 and the image decoding device 100 may be a user terminal, such as a personal computer (PC), laptop computer, personal digital assistant (PDA), portable multimedia player (PMP), portable game console (PSP), wireless communication terminal, smartphone, or television (TV), or a server terminal (e.g., an application server or service server). Each of the image encoding device 105 and the image decoding device 100 may be any of a variety of devices, each device including: a communication device such as a communication modem that communicates with various devices or wired/wireless communication networks; a memory 120 or 125 that stores various programs and data for inter-frame prediction or intra-frame prediction to encode or decode images; or a processor 110 or 115 that performs calculation and control operations by executing programs.

Furthermore, the image encoding device 105 can transmit images encoded as bitstreams to the image decoding device 100 in real time or non-real time via wired/wireless communication networks such as the Internet, short-range wireless communication networks, wireless local area networks (WLANs), wireless broadband (Wi-Bro) networks, or mobile communication networks, or via various communication interfaces such as cables or universal serial buses (USB). The image decoding device 100 can reconstruct the image from the received bitstream by decoding the bitstream and reproduce the image. Additionally, the image encoding device 105 can transmit images encoded as bitstreams to the image decoding device 100 via a computer-readable recording medium.

Although the aforementioned image encoding and image decoding devices can be separate devices, depending on the implementation, they can be incorporated into a single image encoding/decoding device. In this case, some components of the image encoding device can be substantially the same as their corresponding components of the image decoding device. Therefore, these components can be configured to include the same structure or perform at least the same function.

Therefore, in the following detailed description of the technical components and their operating principles, redundant descriptions of the corresponding technical components will be omitted. Furthermore, since the image decoding apparatus is a computing device that applies the image encoding method executed in the image encoding apparatus to decoding, the following description will focus on the image encoding apparatus.

The computing device may include: a memory storing programs or software modules that perform image encoding and/or image decoding methods; and a processor connected to the memory and executing the programs. The image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder.

Figure 2 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

Referring to FIG2, the image encoding device 20 may include a prediction unit 200, a subtraction unit 205, a transformation unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transformation unit 225, an addition unit 230, a filter unit 235, an encoded image buffer 240, and an entropy encoding unit 245.

The prediction unit 200 can be implemented using a prediction module as a software module, and generates prediction blocks by using intra-frame prediction or inter-frame prediction for the blocks to be encoded. The prediction unit 200 can generate prediction blocks by predicting the current block to be encoded in the image. In other words, the prediction unit 200 can generate prediction blocks with predicted pixel values for each pixel by predicting the pixel values of pixels in the current block based on inter-frame prediction or intra-frame prediction. Furthermore, the prediction unit 200 can provide the encoding unit with information required to generate prediction blocks (e.g., information about prediction modes such as intra-frame prediction modes or inter-frame prediction modes), so that the encoding unit can encode the information about the prediction modes. The processing unit undergoing prediction, the prediction method, and specific details about the processing unit can be configured according to the encoding/decoding configuration. For example, the prediction method and prediction mode can be determined based on the prediction unit, and prediction can be performed based on the transform unit.

Inter-frame prediction units can distinguish between translational motion models and non-translational motion models based on the motion prediction method. For translational motion models, prediction is performed only by considering parallel translation, while for non-translational motion models, prediction can be performed by considering motions such as rotation, perspective, and zoom in/out in addition to parallel translation. Under the assumption of unidirectional prediction, translational motion models may require one motion vector, while non-translational motion models may require one or more motion vectors. In the case of non-translational motion models, each motion vector can be information applied to a preset position in the current block (e.g., the top-left and top-right vertices of the current block), and the position of the region to be predicted in the current block can be obtained at the pixel level or sub-block level based on the corresponding motion vector. Inter-frame prediction units can apply some post-processing commonly to each motion model and apply other post-processing individually.

The inter-frame prediction unit may include a reference image construction unit, a motion estimation unit, a motion compensator, a motion information determination unit, and a motion information encoder. The reference image construction unit may include encoded images from a reference image list L0 or L1 that precede or follow the current image. Predictive blocks can be obtained from reference images included in the reference image list, and depending on the encoding configuration, the current image may also be configured as a reference image and included in at least one reference image list.

The reference image construction unit of the inter-frame prediction unit may include a reference image interpolator. The reference image interpolator can perform interpolation on fractional pixels according to the interpolation precision. For example, an 8-tap DCT-based interpolation filter can be applied to the luma component, and a 4-tap DCT-based interpolation filter can be applied to the chroma component.

The motion estimation unit of the inter-frame prediction unit can use a reference image to detect blocks that are highly correlated with the current block. Various methods can be used for this, such as full search-based block matching (FBMA) and three-step search (TSS). The motion compensator can obtain the predicted block during the motion estimation process.

The motion information determination unit of the inter-frame prediction unit can perform the process of selecting the best motion information for the current block. Motion information can be encoded using motion information encoding modes such as skip mode, merge mode, and contention mode. The motion information encoding mode can be configured according to the modes supported by the motion model combination. Examples of such modes include (translation) skip mode, (non-translation) skip mode, (translation) merge mode, (non-translation) merge mode, (translation) contention mode, and (non-translation) contention mode. Depending on the encoding configuration, a subset of the modes can be included in the candidate group.

In motion information encoding mode, predicted values (motion vectors, reference images, predicted directions, etc.) of motion information about the current block can be obtained from at least one candidate block. When two or more candidate blocks are supported, optimal candidate selection information can be generated. In skip mode (no residual signal) and merge mode (with residual signal), the predicted values can be used as motion information about the current block without any processing, while in contention mode, difference information between the motion information of the current block and the predicted values can be generated.

Candidate groups of predicted motion information for the current block can be adaptively configured in various ways based on the motion information encoding pattern. Motion information about spatially adjacent blocks (e.g., left block, top block, top-left block, top-right block, and bottom-left block, etc.) can be included in the candidate groups, motion information about temporally adjacent blocks can be included in the candidate groups, and mixed motion information about spatial and temporal candidates can be included in the candidate groups.

Temporally adjacent blocks can include blocks in other images that correspond to (match) the current block, and can refer to the left, right, top, bottom, top-left, top-right, bottom-left, and bottom-right blocks of this block. Hybrid motion information can refer to information obtained as the average, median, etc., of motion information about spatially adjacent blocks and motion information about temporally adjacent blocks.

Motion information can be prioritized to configure candidate groups for motion information predictions. The order in which motion information is included in the candidate groups can be set according to priority. A candidate group can be fully configured when the number of motion information entries in the candidate group is equal to the number of candidates in the candidate group (determined by the motion information encoding mode), based on priority. Motion information can be prioritized in the order of motion information about spatially adjacent blocks, motion information about temporally adjacent blocks, and a mixture of spatially and temporally adjacent blocks. However, the priority can also be modified.

For example, motion information about spatially adjacent blocks can be included in the candidate group in the order of left block, top block, top right block, bottom left block, and top left block, and motion information about temporally adjacent blocks can be included in the candidate group in the order of bottom right block, middle block, right block, and bottom block.

Subtraction unit 205 can generate residual blocks by subtracting the prediction block from the current block. In other words, subtraction unit 205 can calculate the difference between the pixel value of each pixel in the current block to be encoded and the predicted pixel value of the corresponding pixel in the prediction block generated by the prediction unit, to generate a residual signal in the form of a block, i.e., a residual block. Furthermore, subtraction unit 205 can generate residual blocks using units other than those obtained by the block segmenter described later.

Transformation unit 210 can transform a spatial signal into a frequency signal. The signal obtained through the transformation process is called the transform coefficient. For example, a residual block having a residual signal received from a subtraction unit can be transformed into a transform block with transform coefficients, and the input signal is determined according to the encoding configuration, which is not limited to the residual signal.

Transformation units can transform residual blocks using transformation schemes such as Hadamard transform, Discrete Sine Transform (DST) based transform, or DCT based transform. These transformation schemes can be changed and modified in various ways.

It can support at least one of the transformation schemes, and can support at least one sub-transformation scheme for each transformation scheme. A sub-transformation scheme can be obtained by modifying a portion of the fundamental vectors in the transformation scheme.

For example, in the case of DCT, one or more sub-transformation schemes DCT-1 to DCT-8 can be supported, and in the case of DST, one or more sub-transformation schemes DST-1 to DST-8 can be supported. A transformation scheme candidate group can be configured with a portion of the sub-transformation schemes. For example, DCT-2, DCT-8, and DST-7 can be grouped into a candidate group for transform.

Transformations can be performed in both the horizontal and vertical directions. For example, DCT-2 can perform a one-dimensional transformation in the horizontal direction, and DST-7 can perform a one-dimensional transformation in the vertical direction. Using two-dimensional transformations, pixel values can be transformed from the spatial domain to the frequency domain.

A fixed transform scheme can be used, or the transform scheme can be adaptively selected based on the encoding/decoding configuration. In the latter case, the transform scheme can be selected explicitly or implicitly. When the transform scheme is explicitly selected, information about the transform scheme or set of transform schemes applied in each of the horizontal and vertical directions can be generated, for example, at the block level. When the transform scheme is implicitly selected, the encoding configuration can be limited based on image type (I/P/B), color components, block size, block shape, intra-frame prediction mode, etc., and a predetermined transform scheme can be selected based on the encoding configuration.

Furthermore, some transformations can be skipped depending on the encoding configuration. That is, one or more horizontal and vertical units can be explicitly or implicitly omitted.

Furthermore, the transform unit can send the information needed to generate the transform block to the encoding unit, which encodes the information, includes the encoded information in the bitstream, and sends the bitstream to the decoder. Therefore, the decoder's decoding unit can parse the information from the bitstream for inverse transform.

The quantization unit 215 can quantize the input signal. The signal obtained from the quantization is called the quantization coefficient. For example, the quantization unit 215 can obtain a quantized block with quantization coefficients by quantizing a residual block having residual transform coefficients received from the transform unit, and can determine the input signal according to an encoding configuration, which is not limited to the residual transform coefficients.

The quantization unit can quantize the residual block of the transformation using quantization schemes such as, but not limited to, dead-zone uniform boundary value quantization and quantization weighted matrix. These quantization schemes can be changed and modified in various ways.

Quantization can be skipped based on the encoding configuration. For example, quantization (and inverse quantization) can be skipped based on the encoding configuration (e.g., a quantization parameter of 0, i.e., a lossless compression environment). In another example, the quantization process can be omitted when the characteristics of the image are taken into account and quantization-based compression performance is not performed. Quantization can be skipped in the entirety or a portion of a quantization block (M×N) (M/2×N/2, M×N/2, or M/2×N), and quantization skip selection information can be set explicitly or implicitly.

The quantization unit can send the information needed to generate the quantization block to the encoding unit, which then encodes the information, includes the encoded information in the bitstream, and sends the bitstream to the decoder. Therefore, the decoder's decoding unit can parse the information from the bitstream for inverse quantization.

Although the above example was described assuming that the residual block is transformed and quantized by a transform unit and a quantization unit, a residual block with transform coefficients can be generated by transforming the residual signal without quantization. The residual block can undergo quantization only without transformation. Alternatively, the residual block can undergo both transformation and quantization. These operations can be determined based on the coding configuration.

The inverse quantization unit 220 performs inverse quantization on the residual block quantized by the quantization unit 215. That is, the inverse quantization unit 220 generates a residual block with frequency coefficients by performing inverse quantization on the quantized frequency coefficient sequence.

The inverse transform unit 225 performs an inverse transform on the residual block dequantized by the inverse quantization unit 220. That is, the inverse transform unit 225 performs an inverse transform on the frequency coefficients of the dequantized residual block to generate a residual block with pixel values, i.e., a reconstructed residual block. The inverse transform unit 225 can perform the inverse transform by reversing the transform scheme used by the transform unit 210.

Addition unit 230 reconstructs the current block by adding the predicted block predicted by prediction unit 200 to the residual block recovered by inverse transform unit 225. The reconstructed current block is stored as a reference picture (or reference block) in encoded picture buffer 240 for use as a reference picture when encoding the next block, another block, or another picture of the current block.

Filter unit 235 may include one or more post-processing filters, such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF). The deblocking filter removes block distortion occurring at boundaries between blocks in the reconstructed image. The ALF performs filtering based on values obtained by comparing the reconstructed image after filtering the blocks with the deblocking filter with the original image. The SAO reconstructs the pixel-level offset difference between the original image and the residual blocks to which the deblocking filter was applied. These post-processing filters can be applied to the reconstructed image or blocks.

The encoded image buffer 240 can store blocks or images reconstructed by the filter unit 235. The reconstructed blocks or images stored in the encoded image buffer 240 can be provided to the prediction unit 200 that performs intra-frame prediction or inter-frame prediction.

Entropy coding unit 245 scans the generated quantization frequency coefficient sequence using various scanning methods to generate a quantization coefficient sequence, encodes the quantization coefficient sequence using entropy coding, and outputs the entropy-coded coefficient sequence. The scanning pattern can be configured as one of various patterns (e.g., zigzag, diagonal, and raster). Furthermore, encoded data including the encoded information received from each component can be generated and output in a bitstream.

Figure 3 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.

Referring to FIG3, the image decoding device 30 can be configured to include an entropy decoder 305, a prediction unit 310, an inverse quantization unit 315, an inverse transform unit 320, an addition/subtraction unit 325, a filter 330, and a decoded image buffer 335.

Furthermore, the prediction unit 310 can be configured to include an intra-frame prediction module and an inter-frame prediction module.

When an image bitstream is received from the image encoding device 20, it can be sent to the entropy decoder 305.

The entropy decoder 305 can decode the bitstream into decoded data including quantization coefficients and decoded information to be sent to each component.

Prediction unit 310 can generate prediction blocks based on data received from entropy decoder 305. A list of reference images can be generated using a default configuration scheme based on reference images stored in decoded image buffer 335.

The inter-frame prediction unit may include a reference image construction unit, a motion compensator, and a motion information decoder. Some of these components may perform the same processes as in the encoder, while others may perform the encoder's processes in reverse.

The inverse quantization unit 315 can inverse quantize the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 305.

The inverse transform unit 320 can generate residual blocks by applying inverse DCT, inverse integer transform, or similar inverse transform techniques to the transform coefficients.

The inverse quantization unit 315 and the inverse transform unit 320 can perform the processing of the transform unit 210 and the quantization unit 215 of the image encoding apparatus 20 in reverse, and can be implemented in various ways. For example, the inverse quantization unit 315 and the inverse transform unit 320 can use the same processing and inverse transform shared with the transform unit 210 and the quantization unit 215, and can perform the transform and quantization in reverse using information received from the image encoding apparatus 20 about the transform and quantization processing (e.g., transform size, transform shape, quantization type, etc.).

The residual block, which has already been inversely quantized and inversely transformed, can be added to the prediction block derived by prediction unit 310 to produce a reconstructed image block. The addition can be performed by addition/subtraction unit 325.

Regarding filter 330, a deblocking filter can be applied when needed to remove blocking artifacts from the reconstructed image blocks. Additional loop filters can be used to improve video quality before and after decoding.

The reconstructed and filtered image blocks can be stored in the decoded image buffer 335.

Although not shown in the accompanying drawings, the image encoding/decoding apparatus may also include an image segmenter and a block segmenter.

An image segmenter can segment (or divide) an image into at least one processing unit, such as a color space (e.g., YCbCr, RGB, or XYZ), a tile, a slice, or a basic coding unit (maximum coding unit or coding tree unit (CTU)). A block segmenter can further segment a basic coding unit into at least one processing unit (e.g., a coding unit, a prediction unit, a transform unit, a quantization unit, an entropy coding unit, and an in-loop filtering unit).

Basic coding units can be obtained by segmenting an image at regular intervals in both the horizontal and vertical directions. In this way, an image can be segmented into, but not limited to, tiles, slices, etc. While partitioning units such as tiles or slices can include integer multiples of the basic coding block, partitioning units located at the image edges may be an exception. In such cases, the size of the basic coding block can be adjusted.

For example, an image can be divided into basic coding units and then further subdivided into these units, or the image can be divided into these units and then further subdivided into basic coding units. This disclosure assumes the former order of division and subdivision into units, which should not be construed as limiting the scope of this disclosure. The latter is also possible depending on the encoding/decoding configuration. In the latter case, the size of the basic coding units can be adaptively changed based on the division units (e.g., tiles). That is, this means that basic coding blocks of different sizes can be supported within each division unit.

In this disclosure, the following description will be given by way of the understanding that the image is divided into basic coding units by default. The default setting may mean that the image is not divided into tiles or slices, or that the image is a single tile or slice. However, as stated above, even when the image is first divided into partitioning units (tiles, slices, etc.) and then divided into basic coding units based on those partitioning units (i.e., when the number of each partitioning unit is not an integer multiple of the number of basic coding units), it should be understood that the various implementations described below can be applied in the same manner or with some modifications.

In a partitioning unit, a slice can be a group of at least one consecutive blocks according to a scan pattern, and a tile can be a rectangular group of spatially adjacent blocks. Other partitioning units can be supported and constructed according to their constraints. Slices and tiles can be partitioning units supported for parallelization purposes. For this purpose, references between partitioning units can be restricted (i.e., references are not allowed).

Slicing can generate division information for each cell using information about the starting position of consecutive blocks, and in the case of tiles, tiles can generate information about horizontal and vertical dividing lines or generate position information of the tiles (e.g., top left, top right, bottom left, and bottom right positions).

Each slice and tile can be divided into multiple units based on the encoding/decoding configuration.

For example, unit A may include configuration information that affects the encoding/decoding process (i.e., tile header or slice header), while unit B may not include configuration information. Alternatively, unit A may be a unit that is not allowed to reference another unit during encoding/decoding, while unit B may be a unit that is allowed to reference another unit. Furthermore, unit A may include another unit B that is hierarchical to unit B, or may be peer to unit B.

Unit A and Unit B can be a slice and a tile (or a tile and a slice), respectively. Alternatively, each of Unit A and Unit B can be a slice or a tile. For example, Unit A can be slice/tile type 1, and Unit B can be slice/tile type 2.

Each of Type 1 and Type 2 can be a slice or a tile. Alternatively, Type 1 can be multiple slices or tiles (a group of slices or a group of tiles) (including Type 2), and Type 2 can be a single slice or tile.

As described above, this disclosure assumes that the image consists of a slice or tile. However, if two or more partitioning units are generated, the above description can be applied to and understood in the embodiments described below. Furthermore, unit A and unit B are examples of features that partitioning units can have, and examples of combining units A and unit B from various examples are also possible.

The block segmenter can obtain information about basic coding units from the image segmenter, and the basic coding units can refer to the basic (or starting) units used for prediction, transformation, quantization, etc., in image encoding/decoding processing. In this case, the basic coding unit can consist of one luma basic coding block (maximum coding block or CTB) and two basic chroma coding blocks, depending on the color format (YCbCr in this example), and the size of each block can be determined according to the color format. Coding blocks (CBs) can be obtained according to the segmentation process. A CB can be understood as a unit that is no longer further subdivided due to certain constraints, and can be set as the starting unit for dividing into subunits. In this disclosure, blocks conceptually include various shapes such as triangles, circles, etc., and are not limited to squares. For ease of description, it is assumed that the blocks are rectangular.

While the following description is given in the context of a single color component, it is also applicable in proportion to the ratios according to the color format, with some modifications (e.g., in the case of YCbCr 4:2:0, the width-to-height length ratio of the luminance component to the chrominance component is 2:1). Furthermore, although block partitioning depending on other color components (e.g., depending on the block partitioning result of Y in Cb/Cr) is possible, it should be understood that independent block partitioning for each color component is also possible. Moreover, although a common block partitioning configuration can be used (considering its proportionality to the length ratio), it is also necessary to consider and understand the use of separate block partitioning configurations based on the color components.

In the block splitter, a block can be represented as M×N, and the maximum and minimum values of each block can be obtained within this range. For example, if the maximum and minimum values of a block are 256×256 and 4×4 respectively, then blocks of size ^2m × ²ⁿ (where m and n are integers from 2 to 8 in this example), blocks of size ^2m × ²ⁿ (where m and n are integers from 2 to 128 in this example), or blocks of size m×m (where m and n are integers from 4 to 256 in this example) can be obtained. In this paper, m and n can be equal or different, and one or more ranges such as maximum and minimum values can be generated to support the blocks.

For example, information about the maximum and minimum size of a block can be generated, and in some partitioning configurations, information about the maximum and minimum size of a block can be generated. In the former case, the information can be about the range of maximum and minimum sizes that can be generated in the image, while in the latter case, the information can be about the maximum and minimum sizes that can be generated based on some partitioning configuration. The partitioning configuration can be defined by image type (I/P/B), color components (YCbCr, etc.), block type (encoding/prediction/transformation/quantization), partition type (index or type), and partitioning scheme (quadtree (QT), binary tree (BT), and ternary tree (TT) as tree methods, and SI2, SI3, and SI4 as type methods).

Furthermore, constraints can exist on the aspect ratio (block shape) that can be used for blocks, and boundary values can be set at this point. Only blocks less than or equal to/less than the boundary value k can be supported, where k can be limited by the aspect ratio A/B (A is the longer or equal value between the width and height, and B is another value). k can be a real number equal to or greater than 1, such as 1.5, 2, 3, 4, etc. As in the example above, constraints on the shape of a block in an image can be supported, or one or more constraints can be supported depending on the partitioning configuration.

In summary, whether a block partition is supported can be determined based on the aforementioned scope and constraints, as well as the partitioning configuration described later. For example, partitioning can be supported when a candidate (child block) to be partitioned from a block (parent block) meets the conditions for supporting a block; otherwise, partitioning may not be supported.

The block segmenter can be configured with respect to each component of the image encoding and decoding apparatus, and the size and shape of the blocks can be determined during the process. Different blocks can be configured according to the components. The blocks may include prediction blocks for prediction units, transform blocks for transform units, and quantization blocks for quantization units. However, this disclosure is not limited thereto, and block units may be defined separately for other components. Although the shape of each of the inputs and outputs in each component is described as a square in this disclosure, the inputs and outputs of some components may have any other shape (e.g., triangles).

The size and shape of the initial (or starting) block in the block segmenter can be determined from the higher units. The initial block can be segmented into smaller blocks. Once the optimal size and shape are determined based on the block partitioning, that block can be identified as the initial block for the lower units. The higher units can be coding blocks, and the lower units can be prediction blocks or transform blocks, and this disclosure is not limited thereto. Instead, various modifications are possible. Once the initial blocks of the lower units are determined as in the examples above, a partitioning process can be performed to detect blocks of optimal size and shape, similar to the higher units.

In summary, the block splitter can divide a basic coded block (or the largest coded block) into at least one coded block, and a coded block can be divided into at least one prediction block/transform block/quantization block. Furthermore, a prediction block can be divided into at least one transform block/quantization block, and a transform block can be divided into at least one quantization block. Some blocks can be dependent on other blocks (i.e., defined by higher and lower units) or independent of other blocks. For example, a prediction block can be a higher unit above a transform block, or it can be a unit independent of the transform block. Various relationships can be established based on the type of block.

Based on the encoding/decoding configuration, it can be determined whether higher and lower units are combined. Combining units means that blocks of higher units undergo encoding/decoding processing by lower units (e.g., in prediction units, transform units, inverse transform units, etc.) without being split into lower units. In other words, this may mean sharing the partitioning process among multiple units and generating partitioning information in one unit (e.g., a higher unit).

For example, (when a coded block is combined with a prediction block or a transform block), the coded block can undergo prediction, transform, and inverse transform.

For example, (when a coded block is combined with a prediction block), the coded block can undergo prediction, and a transform block that is equal to or smaller in size than the coded block can undergo both transformation and inverse transformation.

For example, (when a coded block is combined with a transform block), a prediction block that is equal to or smaller in size than the coded block can be predicted, and the coded block can be transformed and inverse transformed.

For example, (when a prediction block is combined with a transform block), a prediction block that is equal to or smaller in size than a coding block can undergo prediction, transform, and inverse transform.

For example, (when there is no block combination), a prediction block that is equal to or smaller than the coded block in size can be predicted, and a transform block that is equal to or smaller than the coded block in size can be transformed and inverse transformed.

Although various cases of coded blocks, prediction blocks, and transform blocks have been described in the examples above, this disclosure is not limited thereto.

For the combination of units, a fixed configuration can be supported in the image, or an adaptive configuration can be supported considering various encoding/decoding factors. Encoding/decoding factors include image type, color components, encoding mode (intra-frame/inter-frame), partitioning configuration, block size/shape/position, aspect ratio, prediction-related information (e.g., intra-frame prediction mode, inter-frame prediction mode, etc.), transform-related information (e.g., transform scheme selection information, etc.), and quantization-related information (e.g., quantization region selection information and quantization transform coefficient encoding information), etc.

Once a block of optimal size and shape has been detected as described above, pattern information (e.g., partitioning information) for that block can be generated. This pattern information, along with information generated from the components to which the block belongs (e.g., prediction-related information and transformation-related information), can be included in the bitstream and sent to the decoder, where it can be parsed at the same unit level for use in the video decoding process.

The partitioning scheme will now be described. Although for ease of description, it is assumed that the initial block is shaped as a square, this disclosure is not limited thereto, and the description can be applied in the same or similar manner to cases where the initial block is a rectangle.

The block splitter can support various types of partitioning. For example, it can support tree-based partitioning or index-based partitioning, and it can also support other methods. In tree-based partitioning, the partitioning type can be determined based on various types of information (e.g., information indicating whether to perform a partition, the type of tree, the partitioning direction, etc.), while in index-based partitioning, specific index information can be used to determine the partitioning type.

Figure 4 is an example diagram illustrating various partition types that can be obtained in the block divider of this disclosure. In this example, it is assumed that the partition types shown in Figure 4 are obtained through a single partition operation (or process), which should not be construed as limiting the scope of this disclosure. Partition types can also be obtained through multiple partition operations. Furthermore, other partition types not shown in Figure 4 are also available.

(Tree-based partitioning)

In the tree-based partitioning of this disclosure, QT, BT, and TT can be supported. If one tree method is supported, this can be called single-tree partitioning, while if two or more tree methods are supported, this can be called multi-tree partitioning.

In QT, a block is divided into two partitions (n) in each of the horizontal and vertical directions, while in BT, a block is divided into two partitions (b to g) in either the horizontal or vertical direction. In TT, a block is divided into three partitions (h to m) in either the horizontal or vertical direction.

In Qt, a block can be divided into four partitions (o and p) by restricting the partitioning direction to either the horizontal or vertical direction. Furthermore, in BT, it supports partitioning the block into only equal-sized partitions (b and c), partitioning the block into only partitions of different sizes (d to g), or both of these partitioning types. Additionally, in TT, it supports partitioning the block into partitions focused only on a specific direction (1:1:2 or 2:1:1 in the left-to-right or top-to-bottom direction) (h, j, k, and m), partitioning the block into partitions focused on the center (1:2:1) (i and l), or both of these partitioning types. Furthermore, it supports partitioning the block into four partitions in each of the horizontal and vertical directions (i.e., a total of 16 partitions) (q).

The tree method supports partitioning blocks into z partitions (b, d, e, h, i, j, o) only horizontally, z partitions (c, f, g, k, l, m, p) only vertically, or both. In this paper, z can be an integer equal to or greater than 2, such as 2, 3, or 4.

In this disclosure, it is assumed that partition type n is supported as QT, partition types b and c are supported as BT, and partition types i and l are supported as TT.

Depending on the encoding/decoding configuration, one or more tree partitioning schemes can be supported. For example, QT, QT/BT, or QT/BT/TT can be supported.

In the examples above, the basic tree partitioning scheme is QT, and BT and TT are included as additional partitioning schemes depending on whether other trees are supported. However, various modifications can be made. Information indicating whether other trees are supported (bt_enabled_flag, tt_enabled_flag, and bt_tt_enabled_flag, where 0 indicates no support and 1 indicates support) can be implicitly determined based on encoding/decoding settings or explicitly determined in units such as sequences, images, slices, or tiles.

The partitioning information can include indications of whether to perform a partition (tree_part_flag or qt_part_flag, bt_part_flag, tt_part_flag, and bt_tt_part_flag, which can have values of 0 or 1, where 0 indicates no partitioning and 1 indicates partitioning). Additionally, depending on the partitioning scheme (BT and TT), information about the partitioning direction can be added (dir_part_flag, or bt_dir_part_flag, tt_dir_part_flag, and bt_tt_dir_part_flag, which have values of 0 or 1, where 0 indicates <width/horizontal> and 1 indicates <height/vertical>). This information can be generated during the partitioning process.

When multi-tree partitioning is supported, various partitioning information can be configured. For ease of description, the following description provides an example of how to configure partitioning information at a depth level (that is, but recursive partitioning can be performed by setting one or more supported partitioning depths).

In Example 1, the information indicating whether to perform a partition is checked. If no partition is performed, the partitioning ends.

If a partition is performed, the selection information regarding the partition type is checked (e.g., tree_idx, 0 for QT, 1 for BT, and 2 for TT). The partition direction information is additionally checked based on the selected partition type, and the process proceeds to the next step (if additional partitioning is possible due to reasons such as the partition depth not yet reaching its maximum value, the process restarts from the beginning; if additional partitioning is not possible, the partitioning process ends).

In Example 2, the process checks whether the partitioning is performed using a specific tree scheme (QT), and proceeds to the next step. If the partitioning is not performed using a tree scheme (QT), the process checks whether the partitioning is performed using another tree scheme (BT). In this case, if the partitioning is not performed using that tree scheme, the process checks whether the partitioning is performed using a third tree scheme (TT). If the partitioning is not performed using a third tree scheme (TT), the partitioning process ends.

If partitioning is performed using a tree-based approach (QT), the process proceeds to the next step. Alternatively, if partitioning is performed using a second tree-based approach (BT), the partitioning direction information is checked, and the process proceeds to the next step. If partitioning is performed using a third tree-based approach (TT), the partitioning direction information is checked, and the process proceeds to the next step.

In Example 3, the system checks whether the partitioning should be performed using a tree scheme (QT). If not, it checks whether the partitioning should be performed using another tree scheme (BT and TT). If no partitioning is performed, the partitioning process ends.

If partitioning is performed using a tree scheme (QT), the process proceeds to the next step. Alternatively, if partitioning is performed using other tree schemes (BT and TT), the partitioning direction information is checked, and the process proceeds to the next step.

While the examples above either prioritize the tree partitioning schemes (Examples 2 and 3) or do not prioritize them (Example 1), various modifications are also possible. Furthermore, in the examples above, the partitioning in the current step is independent of the partitioning result of the previous step. However, it is also possible to configure the partitioning in the current step to depend on the partitioning result of the previous step.

In Examples 1 through 3, if some tree partitioning scheme (QT) was executed in a previous step, and the process proceeds to the current step, the same tree partitioning scheme (QT) can also be supported in the current step.

On the other hand, if a certain tree partitioning scheme (QT) was not executed in a previous step and therefore another tree partitioning scheme (BT or TT) was executed, and then the process proceeds to the current step, it can be configured to support other tree partitioning schemes (BT and TT) besides a certain tree partitioning scheme (QT) in the current step and subsequent steps.

In the above cases, the tree configuration supporting block partitioning can be adaptive, and therefore the partitioning information described above can also be configured differently. (Assuming the example described below is Example 3). That is, if partitioning was not performed using a certain tree scheme (QT) in a previous step, the partitioning process can be performed without considering the tree scheme (QT) in the current step. Furthermore, partitioning information associated with a particular tree scheme (e.g., information indicating whether partitioning should be performed, information about the partitioning direction, etc. In this example <QT>, information indicating whether partitioning should be performed) can be removed.

The above examples illustrate adaptive partitioning information configuration for cases where block partitioning is allowed (e.g., block size is within the range of maximum and minimum values, the partitioning depth of each tree scheme has not reached the maximum depth (allowed depth), etc.). Even when block partitioning is restricted (e.g., block size does not exist within the range of maximum and minimum values, the partitioning depth of each tree scheme has reached the maximum depth, etc.), partitioning information can still be configured adaptively.

As already mentioned, in this disclosure, tree-based partitioning can be performed recursively. For example, if the partition flag of a coded block with partition depth k is set to 0, then block encoding is performed in the coded block with partition depth k. If the partition flag of a coded block with partition depth k is set to 1, then block encoding is performed in N sub-coded blocks with partition depth k+1 according to the partitioning scheme (where N is an integer equal to or greater than 2, such as 2, 3, and 4).

In the above process, a sub-coded block can be set as a coded block (k+1) and divided into sub-coded blocks (k+2). This hierarchical partitioning scheme can be determined based on partitioning configurations such as the partitioning range and the allowed partitioning depth.

In this case, the bitstream structure representing the partitioning information can be selected from one or more scanning methods. For example, the bitstream of partitioning information can be configured based on the order of partitioning depth or based on whether partitioning is performed.

For example, in the case of partition depth order, partition information is obtained at the current depth level based on the initial block, and then at the next depth level. In the case of whether partitioning is performed, additional partition information is first obtained from the blocks split from the initial block, and other additional scanning methods can be considered.

(Index-based partitioning)

In the index-based partitioning of this disclosure, both Constant Partition Index (CSI) and Variable Partition Index (VSI) schemes can be supported.

In the CSI scheme, k sub-blocks can be obtained by dividing the blocks in a predetermined direction, where k can be an integer equal to or greater than 2, such as 2, 3, or 4. Specifically, the size and shape of the sub-blocks can be determined based on k, regardless of the size and shape of the blocks. The predetermined direction can be one or a combination of two or more of the following: horizontal, vertical, and diagonal (top left to bottom right or bottom left to top right).

In the index-based CSI partitioning scheme of this disclosure, z candidates can be obtained by partitioning in the horizontal or vertical direction. In this case, z can be an integer equal to or greater than 2, such as 2, 3, or 4, and the sub-blocks can be equal in width and height, and equal or different in the other of width and height. The width or height ratio of the sub-blocks is _A1 : _A2 : ... : _AZ , and each of _A1 to _AZ can be an integer equal to or greater than 1, such as 1, 2, or 3.

Furthermore, candidates can be obtained by dividing the blocks into x and y partitions along the horizontal and vertical directions, respectively. Each of x and y can be an integer equal to or greater than 1, such as 1, 2, 3, or 4. However, candidates where both x and y are 1 may be restricted (because a already exists). Although Figure 4 shows a case where the sub-blocks have the same width or height ratio, candidates with different width or height ratios can also be included.

Furthermore, candidates can be divided into w partitions along one of the diagonal directions—top left to bottom right and bottom left to top right. In this paper, w can be an integer equal to or greater than 2, such as 2 or 3.

Referring to Figure 4, based on the length ratio of each sub-block, the partitioning type can be classified into symmetric partitioning type (b) and asymmetric partitioning type (d and e). Furthermore, the partitioning type can be classified into partitioning types concentrated in a specific direction (k and m) and center partitioning type (k). The partitioning type can be defined by various encoding/decoding factors, including sub-block shape and sub-block length ratio, and the supported partitioning types can be implicitly or explicitly determined based on the encoding/decoding configuration. Therefore, candidate groups can be determined based on the partitioning types supported in the index-based partitioning scheme.

In the VSI scheme, with a fixed width w or height h for each sub-block, one or more sub-blocks can be obtained by dividing in a predetermined direction. In this document, each of w and h can be an integer equal to or greater than 1, such as 1, 2, 4, or 8. Specifically, the number of sub-blocks can be determined based on the size and shape of the block and the value of w or h.

In the index-based VSI partitioning scheme of this disclosure, candidates can be divided into sub-blocks, each of which has a fixed width and length. Alternatively, candidates can be divided into sub-blocks, each of which has a fixed width and length. Since the width or height of the sub-blocks is fixed, equal division in the horizontal or vertical direction is allowed. However, this disclosure is not limited thereto.

If the size of the first block is M×N, and the width w of each sub-block is fixed, the height h of each sub-block is fixed, or both the width w and the height h of each sub-block are fixed, then the number of sub-blocks obtained can be (M*N)/w, (M*N)/h, or (M*N)/w/h.

Depending on the encoding/decoding configuration, only one or both of the CSI and VSI schemes can be supported, and information about the supported schemes can be determined implicitly or explicitly.

This disclosure will be described in the context of the supported CSI schemes.

Candidate groups can be configured to include two or more candidates from an index-based partitioning scheme, depending on the encoding/decoding settings.

For example, candidate groups such as {a, b, c}, {a, b, c, n}, or {a to g and n} can be formed. Candidate groups can include examples of block types predicted to occur multiple times based on general statistical characteristics, such as blocks that are divided into two partitions in the horizontal or vertical direction, or in each of the horizontal and vertical directions.

Alternatively, candidate groups can be configured such as {a, b}, {a, o}, or {a, b, o}, or candidate groups such as {a, c}, {a, p}, or {a, c, p}. A candidate group can be an example that includes candidates divided into two partitions horizontally and four partitions vertically. This could be an example of configuring a candidate group with block types predicted to be primarily partitioned in a specific direction.

Alternatively, candidate groups such as {a, o, p} or {a, n, q} can be configured. This could be an example of configuring candidate groups to include block types that are predicted to be divided into many partitions smaller than the blocks before the partition.

Alternatively, candidate groups such as {a, r, s} can be configured, and can be, for example, determining the best partitioning result of rectangular shapes that can be obtained from the blocks before partitioning by other methods (tree methods), and configuring non-rectangular shapes as candidate groups.

As noted in the examples above, various candidate group configurations can be available, and one or more candidate group configurations can be supported, taking into account various encoding/decoding factors.

Once the candidate groups are fully configured, various partitioning information configurations can be made available.

For example, index selection information can be generated for candidate groups including undivided candidate a and divided candidates b to s.

Alternatively, information indicating whether to perform a partition (information indicating whether the partition type is 'a') can be generated. If a partition is performed (if the partition type is not 'a'), index selection information about the candidate groups including the partitioned candidates b to s can be generated.

The partitioning information can be configured in many other ways than those described above. In addition to information indicating whether to perform a partition, binary bits can be assigned to the index of each candidate in the candidate group in various ways, such as fixed-length binarization, variable-length binarization, etc. If the number of candidates is 2, 1 bit can be assigned to the index selection information, and if the number of candidates is 3, one or more bits can be assigned to the index selection information.

Compared to tree-based partitioning schemes, partition types that are predicted to occur multiple times can be included in the candidate groups of index-based partitioning schemes.

Since the number of bits used to represent index information can increase based on the number of supported candidate groups, this scheme is suitable for single-level partitioning (e.g., partition depth is limited to 0) rather than tree-based hierarchical partitioning (recursive partitioning). That is, a single partitioning operation can be supported without further subdividing the sub-blocks obtained through index-based partitioning.

This may mean that further subdividing into smaller blocks of the same type is impossible (e.g., a coded block obtained through index-based partitioning cannot be further subdivided into coded blocks), and it also means that further subdividing into blocks of different types is also impossible (e.g., it is impossible to partition a coded block into prediction blocks and coded blocks). Clearly, this disclosure is not limited to the above examples, and other modifications are also possible.

Now, a description will be given of determining the block partitioning configuration primarily based on the block type in the encoding/decoding factors.

First, encoded blocks can be obtained during the partitioning process. A tree-based partitioning scheme can be used for the partitioning process, and partitioning types such as a (no partitioning), n (QT), b, c (BT), i, or l (TT) as shown in Figure 4 can be generated based on the tree structure. Depending on the encoding/decoding configuration, various combinations of tree types (e.g., QT/QT+BT/QT+BT+TT) may be available.

The following example illustrates the process of ultimately splitting the coded block obtained in the above steps into prediction blocks and transform blocks. It assumes that prediction, transform, and inverse transform are performed based on the size of each partition.

In Example 1, prediction can be performed by setting the size of the prediction block to be equal to the size of the coding block, and transformation and inverse transformation can be performed by setting the size of the transform block to be equal to the size of the coding block (or prediction block).

In Example 2, prediction can be performed by setting the size of the prediction block to be equal to the size of the coding block. Transform blocks can be obtained by dividing the coding block (or prediction block), and transform and inverse transform can be performed based on the size of the obtained transform blocks.

Here, a tree-based partitioning scheme can be used for the partitioning process, and partitioning types such as a (no partitioning), n (QT), b, c (BT), i, or l (TT) as shown in Figure 4 can be obtained depending on the tree type. Various combinations of tree types (e.g., QT/QT+BT/QT+BT+TT) may be available depending on the encoding/decoding configuration.

Here, the partitioning process can be an index-based partitioning scheme. The partition type can be obtained based on the index type, for example, a (no partition), b, c, or d in Figure 4. Various candidate groups such as {a, b, c} and {a, b, c, d} can be configured according to the encoding/decoding configuration.

In Example 3, a prediction block can be obtained by dividing the coding block, and this prediction block undergoes prediction based on the size of the obtained prediction block. For the transform block, its size is set to the size of the coding block, and both transform and inverse transform can be performed on the transform block. In this example, the prediction block and the transform block can be independent.

Index-based partitioning schemes can be used in the partitioning process, and the partition type can be obtained based on the index type (e.g., a (no partition), b through g, n, r, or s in Figure 4). Various candidate groups can be configured based on the encoding/decoding configuration, such as {a, b, c, n}, {a through g, n}, and {a, r, s}.

In Example 4, a prediction block can be obtained by dividing the coded block, and this prediction block undergoes prediction based on the size of the obtained prediction block. For the transform block, its size is set to the size of the prediction block, and a transform and inverse transform can be performed on the transform block. In this example, the transform block can have a size equal to the size of the obtained prediction block, or the obtained prediction block can have a size equal to the size of the transform block (setting the size of the transform block to the size of the prediction block).

Tree-based partitioning schemes can be used in the partitioning process, and partition types can be generated based on the tree type (e.g., a (no partition), b, c (BT), i, l (TT), or n (QT) in Figure 4). Various combinations of tree types (e.g., QT/BT/QT+BT) may be available depending on the encoding/decoding configuration.

Here, an index-based partitioning scheme can be used in the partitioning process, and the partitioning type can be obtained based on the index type (e.g., a (no partition), b, c, n, o, or p in Figure 4). Various candidate groups can be configured based on the encoding/decryption configuration, such as {a, b}, {a, c}, {a, n}, {a, o}, {a, p}, {a, b, c}, {a, o, p}, {a, b, c, n}, and {a, b, c, n, p}. Furthermore, candidate groups can be configured in a standalone VSI scheme or in a combination of CSI and VSI schemes as an index-based partitioning scheme.

In Example 5, a prediction block can be obtained by partitioning the coded block, and this prediction block undergoes prediction based on the size of the obtained prediction block. Alternatively, a transform block can be obtained by partitioning the coded block, and this transform block undergoes transform and inverse transform based on the size of the obtained transform block. In this example, each of the prediction block and the transform block can be obtained by partitioning the coded block.

Here, tree-based partitioning schemes and index-based partitioning schemes can be used in the partitioning process, and candidate groups can be configured in the same or similar way as in Example 4.

In this context, the above examples represent possible scenarios depending on whether the process for partitioning each block type is shared, and should not be construed as limiting the scope of this disclosure. Various modifications are also possible. Furthermore, various encoding/decoding factors and block types can be considered to determine the block partitioning configuration.

Encoding/decoding factors may include image type (I/P/B), color components (YCbCr), block size/shape/position, block aspect ratio, block type (coded block, prediction block, transform block, or quantization block), partitioning state, coding mode (intra-frame/inter-frame), prediction-related information (intra-frame prediction mode or inter-frame prediction mode), transform-related information (transform scheme selection information), and quantization-related information (quantization region selection information and quantization transform coefficient coding information).

In the image coding method according to embodiments of the present disclosure, intra-frame prediction can be configured as follows. The intra-frame prediction of the prediction unit may include a reference pixel configuration step, a prediction block generation step, a prediction mode determination step, and a prediction mode encoding step. Furthermore, the image coding apparatus may be configured to include a reference pixel configuration unit, a prediction block generator, and a prediction mode encoder to perform the reference pixel configuration step, the prediction block generation step, the prediction mode determination step, and the prediction mode encoding step. Some of the above steps may be omitted, or other steps may be added. These steps may be performed in an order different from the order described above.

Figure 5 is an example diagram illustrating an intra-prediction mode according to an embodiment of the present disclosure.

Referring to Figure 5, the 67 prediction modes are grouped into prediction mode candidate groups for intra-frame prediction. Although the following description is given with 65 prediction modes being directional modes and 2 being non-directional modes (DC and planar modes) out of the 67 prediction modes, this disclosure is not limited thereto, and various configurations are available. Directional modes can be distinguished from each other by slope information (e.g., dy/dx) or angle information (degrees). All or part of a prediction mode can be included in the prediction mode candidate group for the luma component or chroma component, and other additional modes can be included in the prediction mode candidate group.

In this disclosure, a straight line can be used to guide the orientation mode, and a curved orientation mode can be additionally configured as the prediction mode. Furthermore, the non-orientation mode can include a DC mode and a planar mode. In the DC mode, the prediction block is obtained by averaging (or weighted averaging) the pixels of blocks adjacent to the current block (e.g., the top block, left block, top-left block, top-right block, and bottom-right block). In the planar mode, the prediction block is obtained by linearly interpolating the pixels of adjacent blocks.

In DC mode, reference pixels for generating prediction blocks can be obtained from any combination of blocks such as left, top, left + top, left + bottom left, top + top right, left + top + bottom left + top left + top right, etc. The location of the block from which the reference pixels are obtained can be determined based on the encoding/decoding configuration, which is defined by image type, color components, block size/type/position, etc.

In planar mode, pixels for generating prediction blocks can be obtained from regions with reference pixels (e.g., left region, top region, top-left region, top-right region, and bottom-left region) and regions without reference pixels (e.g., right region, bottom region, and bottom-right region). Regions without reference pixels (i.e., uncoded) can be implicitly obtained by using one or more pixels from regions with reference pixels (e.g., by duplication or weighted averaging), or information about at least one pixel in regions not composed of reference pixels can be explicitly generated. Therefore, as described above, prediction blocks can be generated using regions with and without reference pixels.

Other non-directional modes besides those described above may also be included. This disclosure primarily describes linear directional and non-directional modes, DC, and plane modes. However, modifications to the modes are possible.

The prediction patterns shown in Figure 5 can be supported consistently regardless of the block size. Furthermore, the prediction patterns supported based on the block size can differ from those in Figure 5.

For example, the number of candidate groups for prediction patterns can be adaptive (e.g., although the angle between every two adjacent prediction patterns is equal, the angle can be set differently, such as 9, 17, 33, 65, or 129 directional patterns). Alternatively, the number of candidate groups for prediction patterns can be fixed but with different configurations (e.g., directional pattern angles and non-directional pattern types).

Furthermore, the prediction patterns shown in Figure 5 can be consistently supported regardless of the block type. Additionally, the prediction patterns supported may differ from those shown in Figure 5, depending on the block type.

For example, the number of prediction mode candidates can be adaptive (e.g., more or fewer prediction modes can be derived in the horizontal or vertical direction based on the width-to-height ratio of the block). Alternatively, the number of prediction mode candidates can be fixed but with different configurations (e.g., prediction modes can be derived more finely along the horizontal or vertical direction based on the width-to-height ratio of the block).

Alternatively, a larger number of prediction modes can be supported for the longer side of the block, while a smaller number of prediction modes can be supported for the shorter side. Regarding the prediction mode spacing on the longer side of the block, modes located to the right of mode 66 (e.g., modes at +45 degrees or greater angle to mode 50, such as modes 67 to 80) or modes located to the left of mode 2 (e.g., modes at -45 degrees or less angle to mode 18, such as modes -1 to -14) can be supported. This can be determined based on the block's aspect ratio, and vice versa.

Although this disclosure primarily describes fixed-support prediction modes for patterns such as those in Figure 5 (regardless of any encoding/decoding factors), it is also possible to configure adaptive-support prediction modes based on the encoding configuration.

Predicted patterns can be classified based on horizontal and vertical patterns (patterns 18 and 50) and some diagonal patterns (diagonal upper right pattern 2, diagonal lower right pattern 34, diagonal lower left pattern 66, etc.). This classification can be based on some directionality (or angles such as 45 degrees, 90 degrees, etc.).

The modes located at both ends of the directional mode (mode 2 and mode 66) can be used as reference modes for classification of the prediction mode, which is possible when the intra-prediction mode is configured as shown in Figure 5. That is, the reference mode can be changed when the prediction mode is configured adaptively. For example, mode 2 can be replaced by a mode with an exponent less than or greater than 2 (e.g., -2, -1, 3, 4, etc.), or mode 66 can be replaced by a mode with an exponent less than or greater than 66 (e.g., 64, 66, 67, 68, etc.).

In addition, additional prediction modes related to color components (color replication mode and color mode) can be included in the prediction mode candidate group. Color replication mode can refer to a prediction mode related to the method of obtaining data for generating prediction blocks from a region located in another color space, and color mode can refer to a prediction mode related to the method of obtaining prediction modes from a region located in another color space.

Figure 6 is an example diagram illustrating a reference pixel configuration for intra-frame prediction according to an embodiment of the present disclosure. The size and shape M×N of the prediction block can be obtained through a block segmenter.

Although intra-frame prediction can typically be performed based on prediction blocks, it can also be performed based on coded blocks or transform blocks, depending on the block segmenter configuration. After examining the block information, the reference pixel configuration unit can configure reference pixels for predicting the current block. Reference pixels can be managed in temporary memory (e.g., an array, a primary array, a secondary array, etc.). Reference pixels can be generated and removed in each intra-frame prediction process of the block, and the size of the temporary memory can be determined based on the reference pixel configuration.

In this example, it is assumed that the prediction for the current block uses the left block, top block, top-left block, top-right block, and bottom-left block of the current block. However, this disclosure is not limited to this, and the prediction for the current block can use block candidate groups with different configurations. For example, candidate groups of adjacent blocks for a reference pixel can be determined based on raster or Z-scan, and some candidates can be removed from the candidate groups according to the scan order, or another block candidate group (e.g., right block, bottom block, and bottom-right block) can be further included.

Furthermore, if prediction modes such as color replication are supported, some regions of different color spaces can be used for the prediction of the current block. Therefore, this region can also be considered for reference pixels.

Figure 7 is a conceptual diagram illustrating blocks adjacent to a target block used for intra-frame prediction according to an embodiment of the present disclosure. Specifically, the left image of Figure 7 shows blocks adjacent to the current block in the current color space, and the right image of Figure 7 shows corresponding blocks in another color space. For ease of description, the following description will be given assuming that the blocks adjacent to the current block in the current color space are a basic reference pixel configuration.

As shown in Figure 6, neighboring pixels in the left block, top block, top-left block, top-right block, and bottom-left block (Ref_L, Ref_T, Ref_TL, Ref_TR, and Ref_BL in Figure 6) can be configured as reference pixels for prediction of the current block. Although the reference pixel is usually the pixel closest to the current block among the neighboring blocks, as indicated by reference symbol a (called the reference pixel line) in Figure 6, other pixels (pixel b in Figure 6 and the pixels in the other outer lines) can also be used as reference pixels.

Pixels adjacent to the current block can be classified into at least one reference pixel line. The pixel closest to the current block can be represented by ref_0 (e.g., pixels at a distance of 1 from the boundary pixel of the current block, p(-1, -1) to p(2m-1, -1) and p(-1, 0) to p(-1, 2n-1)), the second closest pixel can be represented by ref_1 (e.g., pixels at a distance of 2 from the boundary pixel of the current block, p(-2, -2) to p(2m, -2) and p(-2, -1) to p(-2, 2n)), and the third closest pixel is represented by ref_2 (e.g., pixels at a distance of 3 from the boundary pixel of the current block, p(-3, -3) to p(2m+1, -3) and p(-3, -2) to p(-3, 2n+1)). That is, reference pixel lines can be defined based on the distance between the boundary pixels of the current block and their adjacent pixels.

It can support N or more reference pixel lines, and N can be an integer equal to or greater than 1, such as 1 to 5. Typically, reference pixel lines are included in the candidate group of reference pixel lines in ascending order of distance. However, this disclosure is not limited to this. For example, when N is 3, the candidate group can include reference pixel lines sequentially, such as <ref_0, ref_1, ref_2>. Candidate groups can also be configured non-sequentially, such as <ref_0, ref_1, ref_3> or <ref_0, ref_2, ref_3>, or a candidate group without a closest reference pixel line can be configured.

Prediction can be performed using all or part (one or more) of the reference pixel lines in the candidate group.

For example, one of a plurality of reference pixel lines can be selected based on the encoding/decoding configuration, and the reference pixel line can be used to perform intra-frame prediction. Alternatively, two or more of the plurality of reference pixel lines can be selected, and the selected reference pixel lines can be used to perform intra-frame prediction (e.g., by weighted averaging of the data from the reference pixel lines).

Reference pixel lines can be selected implicitly or explicitly. For example, implicit selection means selecting reference pixel lines based on an encoding/decoding configuration defined by a combination of one or more factors, such as image type, color components, and block size/shape/position. Explicit selection means that reference pixel line selection information can be generated at the block level.

Although this disclosure is described in the context of performing intra-frame prediction using the nearest reference pixel line, it should be understood that when multiple reference pixel lines are used, various implementations described later can be implemented in the same or similar manner.

For example, configurations that implicitly determine information by considering the use of the nearest reference pixel line individually can be supported for intra-block prediction as described later. That is, intra-block prediction can be performed using preset reference pixel lines, and the reference pixel lines can be implicitly selected. The reference pixel line can be, but is not limited to, the nearest reference pixel line.

Alternatively, reference pixel lines can be adaptively selected for intra-block-based prediction, and various reference pixel lines, including the closest reference pixel line, can be selected to perform intra-block-based prediction. In other words, intra-block-based prediction can be performed using reference pixel lines determined considering various encoding/decoding factors, and the reference pixel lines can be selected implicitly or explicitly.

According to this disclosure, a reference pixel configuration unit for intra-frame prediction may include a reference pixel generator, a reference pixel interpolator, and a reference pixel filter unit. The reference pixel configuration unit may include all or some of the above components.

The reference pixel configuration unit can distinguish between available and unavailable pixels by checking the availability of its reference pixels. A reference pixel is determined to be unavailable when it meets at least one of the following conditions.

For example, a reference pixel can be determined to be unavailable if at least one of the following conditions is met: the reference pixel is located outside the image boundary; the reference pixel does not belong to the same partitioning unit as the current block (e.g., units that do not allow mutual reference, such as slices or tiles. However, units that allow mutual reference are an exception, although slices or tiles are exceptions); and the reference pixel has not yet been fully encoded/decoded. That is, if none of the above conditions are met, the reference pixel can be determined to be available.

The use of reference pixels may be limited by the encoding/decoding configuration. For example, even if a reference pixel is determined to be available based on the above conditions, its use may be limited depending on whether constrained intra-prediction is performed (e.g., indicated by constrained_intra_pred_flag). Constrained intra-prediction can be performed when blocks reconstructed by referencing another image are prohibited for the purpose of preventing robust encoding/decoding errors due to external factors such as the communication environment.

When constrained intra-pred prediction is disabled (e.g., constrained_intra_pred_flag = 0 for I-type or P- or B-type images), all reference pixel candidate blocks can be available.

Conversely, when constrained intra-pred prediction is activated (e.g., constrained_intra_pred_flag = 1 for P or B image types), it can be determined whether to use a reference pixel candidate block based on the coding mode (intra-frame or inter-frame). However, this condition can be set based on various other coding/decoding factors.

Since the reference pixel is in one or more blocks, it can be classified into three types based on its availability: fully available, partially available, and completely unavailable. In cases other than fully available, reference pixels can be filled or generated at the locations of unavailable candidate blocks.

When a reference pixel candidate block is available, the pixel at the corresponding position can be included in the reference pixel memory for the current block. The pixel data of a pixel can be copied as is, or it can be included in the reference pixel memory after processes such as reference pixel filtering and reference pixel interpolation. Conversely, when a reference pixel candidate block is unavailable, the pixel obtained through the reference pixel generation process can be included in the reference pixel memory for the current block.

The following describes examples of generating reference pixels at the location of unavailable blocks using various methods.

For example, a reference pixel can be generated using any pixel value. This arbitrary pixel value can be one of a range of pixel values (e.g., the minimum, maximum, or median of the range) within a range of pixel values (e.g., a range of pixel values based on bit depth or a range of pixel values based on the pixel distribution of the corresponding image). Specifically, this example may be applicable when the entire block of candidate reference pixels is unavailable.

Alternatively, reference pixels can be generated from regions where image encoding/decoding has been completed. Specifically, reference pixels can be generated from at least one available block adjacent to an unavailable block. At least one of extrapolation, interpolation, or duplication can be used to generate reference pixels.

After the reference pixel interpolator completes the configuration of the reference pixels, it can generate fractional reference pixels through linear interpolation between the reference pixels. Alternatively, the reference pixel interpolation process can be performed after the reference pixel filtering process described later.

Interpolation is not performed in horizontal mode, vertical mode, and some diagonal modes (e.g., modes at 45 degrees to the vertical/horizontal lines, such as the upper right diagonal, lower right diagonal, and lower left diagonal modes corresponding to modes 2, 34, and 66 in Figure 5), non-directional mode, color copy mode, etc. Interpolation can be performed in other modes (other diagonal modes).

Based on the prediction pattern (e.g., the directionality of the prediction pattern, dy/dx, etc.) and the positions of the reference and predicted pixels, the position of the pixel to be interpolated (i.e., the fractional unit to be interpolated, ranging from 1/2 to 1/64) can be determined. In this case, a filter can be applied (e.g., assuming the same filter in the equation used to determine the filter coefficients or filter tap length, but assuming a filter whose coefficients are adjusted only according to fractional precision such as 1/32, 7/32, or 19/32), or one of multiple filters can be selected and applied based on the fractional unit (e.g., assuming filters using different equations to determine the filter coefficients or filter tap length).

In the former case, integer pixels can be used as input for fractional pixel interpolation, while in the latter case, the input pixels are different in each step (e.g., integer pixels for 1/2 units, and integer pixels and 1/2 unit pixels for 1/4 units). However, this disclosure is not limited thereto and will be described in the context of the former case.

Fixed or adaptive filtering can be performed on the reference pixels during interpolation. The choice between fixed or adaptive filtering can be determined based on the encoding/decoding configuration (e.g., image type, color components, block position/size/shape, block aspect ratio, and prediction mode, one or more).

In fixed filtering, a single filter can be used to perform reference pixel interpolation, while in adaptive filtering, one of multiple filters can be used to perform reference pixel interpolation.

In adaptive filtering, one of several filters can be implicitly or explicitly determined based on the encoding/decoding configuration. Filters can include 4-tap DCT-IF filters, 4-tap cubic filters, 4-tap Gaussian filters, 6-tap Wiener filters, and 8-tap Kalman filters. Supported filter candidate groups can be defined differently based on the color components (e.g., filters of the same or different types, and filter tap lengths that are short or long).

Since the reference pixel filter unit reduces residual degradation in the encoding/decoding process, filtering can be performed on the reference pixel to improve prediction accuracy. The filter used can be, but is not limited to, a low-pass filter. It can be determined whether filtering is applied based on the encoding/decoding configuration (derived from the preceding description). Furthermore, when applying filtering, fixed filtering or adaptive filtering can be used.

Fixed filtering means either no reference pixel filtering or using a single filter to apply reference pixel filtering. Adaptive filtering means determining whether to apply filtering based on the encoding/decoding configuration, and when two or more filter types are supported, one of the filter types can be selected.

For filter types, multiple filters that can be distinguished from each other by filter coefficients, filter tap lengths, etc., can be supported, such as a 3-tap filter of [1, 2, 1]/4 and a 5-tap filter of [2, 3, 6, 3, 2]/16.

The reference pixel interpolator and reference pixel filter unit described in the reference pixel configuration steps can be components needed to improve prediction accuracy. These two processes can be performed independently or in combination (i.e., within a single filter).

The prediction block generator can generate prediction blocks in at least one prediction mode and can use reference pixels based on the prediction mode. Reference pixels can be used in methods such as extrapolation (directional mode) or methods such as interpolation, averaging (DC), or replication (non-directional mode).

The prediction mode determination unit performs the process of selecting the best mode from a group of multiple prediction mode candidates. Typically, a rate-distortion scheme that takes into account block distortion (e.g., distortion of the current block and the reconstructed block, sum of absolute differences (SAD), and sum of squared differences (SSD)) and the number of bits generated according to the mode can be used to determine the best mode based on the coding cost. The predicted block generated based on the prediction mode determined in the above process can be sent to the subtraction unit and the addition unit.

All prediction modes in the candidate prediction mode group can be searched to determine the optimal prediction mode, or the optimal prediction mode can be selected in different decision processes to reduce computational cost/complexity. For example, in the first step, some modes with good performance in terms of image quality degradation can be selected from all intra-frame prediction mode candidates, and in the second step, the number of generated bits and the image quality degradation of the mode selected in the first step can be considered to select the optimal prediction mode. Besides this approach, various other methods can be applied to determine the optimal prediction mode with reduced computational cost/complexity.

Furthermore, although the prediction mode determination unit can typically be included only in the encoder, it can also be included in the decoder, depending on the encoding/decoding configuration, for example, when template matching is included as the prediction method or when the intra-prediction mode is derived from a region adjacent to the current block. In the latter case, it can be understood that a method of implicitly obtaining the prediction mode in the decoder is used.

The predictive mode encoder can encode the predictive mode selected by the predictive mode determination unit. It can encode information about the index of the predictive mode in the predictive mode candidate group, or it can predict the predictive mode and encode information about the predictive mode. The former method can be applied to, but is not limited to, the luma component, while the latter method can be applied to, but is not limited to, the chromaticity component.

When a prediction pattern is predicted and then encoded, the predicted value (or prediction information) of the prediction pattern can be called the most probable pattern (MPM). An MPM can include one or more prediction patterns. The number of MPMs, k, can be determined based on the number of prediction pattern candidate groups (k is an integer equal to or greater than 1, such as 1, 2, 3, or 6). When multiple prediction patterns exist as MPMs, they can be called MPM candidate groups.

MPM candidate groups can be supported in a fixed configuration or in an adaptive configuration based on various encoding/decoding factors. In the example of the adaptive configuration, the candidate group configuration can be determined based on which reference pixel layer is used among multiple reference pixel layers and whether intra-prediction is performed at the block level or sub-block level. For ease of description, it is assumed that MPM candidate groups are configured in one configuration, and it should be understood that not only MPM candidate groups but also other intra-prediction candidate groups can be adaptively configured.

MPM is a concept that supports the efficient encoding of prediction patterns, and candidate groups can be configured with prediction patterns that have a high probability of being used as the prediction pattern for the current block.

For example, MPM candidate groups may include preset prediction patterns (or statistically frequent prediction patterns, DC patterns, planar patterns, vertical patterns, horizontal patterns, or some diagonal patterns) and prediction patterns of adjacent blocks (left block, top block, top-left block, top-right block, bottom-left block, etc.). The prediction patterns of adjacent blocks can be obtained from L0 to L3 (left block), T0 to T3 (top block), TL (top-left block), R0 to R3 (top-right block), and B0 to B3 (bottom-left block) in Figure 7.

If an MPM candidate group can be formed from two or more sub-block positions (e.g., L0 and L2) in adjacent blocks (e.g., the left block), then the prediction modes of the corresponding blocks can be configured in the candidate group according to a predefined priority (e.g., L0-L1-L2). Alternatively, when an MPM candidate group cannot be configured from two or more sub-block positions, the prediction mode of the sub-block at a predefined position (e.g., L0) can be configured in the candidate group. Specifically, the prediction modes at positions L3, T3, TL, R0, and B0 in adjacent blocks can be selected as the prediction modes of adjacent blocks and included in the MPM candidate group. The above description pertains to the case where the prediction modes of adjacent blocks are configured in the candidate group and should not be construed as limiting the scope of this disclosure. In the following examples, it is assumed that the prediction modes at predetermined positions are configured in the candidate group.

When one or more prediction patterns are included in the MPM candidate group, patterns derived from the previously included prediction patterns can be additionally included in the MPM candidate group. Specifically, when the k-th pattern (directional pattern) is included in the MPM candidate group, patterns derived from that pattern (patterns at a distance of +a or -b from the k-th pattern, where each of a and b is an integer equal to or greater than 1, such as 1, 2, or 3) can be additionally included in the MPM candidate group.

Patterns can be prioritized for configuring MPM candidate groups. MPM candidate groups can be configured to include prediction patterns in the order of adjacent block prediction patterns, preset prediction patterns, and derived prediction patterns. Configuring MPM candidate groups can be accomplished by filling the maximum number of MPM candidates according to priority. In the above process, if a prediction pattern is the same as a previously included prediction pattern, that prediction pattern may not be included in the MPM candidate group, and a next-priority candidate can be selected and subjected to redundancy checks.

The following description is given under the assumption that the MPM candidate group includes 6 prediction modes.

For example, MPM candidate groups can be formed in the order of L-T-TL-TR-BL-planar-DC-vertical-horizontal-diagonal. Prediction patterns of adjacent blocks can be included in the MPM candidate groups according to priority, and then preset prediction patterns can be configured separately in this case.

Alternatively, MPM candidate groups can be formed in the order of L-T-plane-DC-<L+1>-<L-1>-<T+1>-<T-1>-vertical-horizontal-diagonal. In this case, the prediction patterns of some neighboring blocks and some preset prediction patterns can be included in priority, and additionally, patterns derived based on the assumption that prediction patterns in directions similar to those of neighboring blocks will be generated, as well as some preset prediction patterns, can be included.

The examples above are only a part of the MPM candidate group configuration. This disclosure is not limited to this, and various modifications may be available.

MPM candidate groups can be represented based on indices within the candidate group using binarization such as unary binarization or truncated Ricean binarization. That is, short bits can be assigned to candidates with small indices, and long bits can be assigned to candidates with large indices to represent the pattern bits.

Patterns not included in the MPM candidate group can be classified as non-MPM candidate groups. Two or more non-MPM candidate groups can be defined based on the encoding/decoding configuration.

The following description is given under the following assumptions: the prediction mode candidate group includes 67 modes, including directed and non-directed modes, supporting 6 MPM candidates, and therefore the non-MPM candidate group includes 61 prediction modes.

If a non-MPM candidate group is configured, it means that there are other predicted patterns that are not included in the MPM candidate group, and therefore no additional candidate group configuration processing is required. Therefore, binarization such as fixed-length binarization and truncated univariate binarization can be used based on the index within the non-MPM candidate group.

Assuming two or more non-MPM candidate groups are configured, these non-MPM candidate groups are classified as non-MPM_A (candidate group A) and non-MPM_B (candidate group B). Assume that candidate group A (p candidates, where p equals or greater than the number of MPM candidates) includes prediction patterns with a higher probability of becoming the prediction pattern for the current block than candidate group B (q candidates, where q equals or greater than the number of candidates in candidate group A). In this paper, processing for configuring candidate group A can be added.

For example, some isometric prediction patterns in the directional pattern (e.g., patterns 2, 4, and 6) can be included in candidate group A, or preset prediction patterns (e.g., patterns derived from prediction patterns included in the MPM candidate group) can be included in candidate group A. The remaining prediction patterns after the MPM candidate group configuration and candidate group A configuration can form candidate group B without additional candidate group configuration processing. Binarization such as fixed-length binarization and truncated univariate binarization can be used based on the indices in candidate group A and candidate group B.

The above example is one example of a scenario where two or more non-MPM candidate groups are formed. This disclosure is not limited to this, and various modifications may be available.

The following describes the process of predicting and encoding the prediction pattern.

You can check information (mpm_flag) indicating whether the prediction mode of the current block is consistent with the MPM (or the mode in the MPM candidate group).

When the prediction mode of the current block matches the MPM, the MPM index information (mpm_idx) can be additionally checked according to the MPM configuration (one or more configurations). Then, the encoding process of the current block is completed.

If the prediction pattern of the current block does not match any MPM, the non-MPM index information (remaining_idx) can be checked if a configured non-MPM candidate group exists. Then, the encoding process of the current block is completed.

If multiple non-MPM candidate groups are configured (two in this example), you can check information indicating whether the prediction mode of the current block is consistent with any prediction mode in candidate group A (non_mpm_flag).

If the prediction pattern of the current block matches any candidate in candidate group A, the index information about candidate group A (non_mpm_A_idx) can be checked; if the prediction pattern of the current block does not match any candidate in candidate group A, the index information of candidate B (remaining_idx) can be checked. Then, the encoding process of the current block is completed.

When the candidate group configuration for prediction modes is fixed, the prediction modes supported by the current block, the prediction modes supported by adjacent blocks, and the preset prediction modes can use the same prediction number index.

When the prediction pattern candidate group configuration is adaptive, the prediction pattern supported by the current block, the prediction pattern supported by adjacent blocks, and the preset prediction pattern can use the same prediction number index or different prediction number indices. Referring to Figure 5, the following description is given.

In prediction mode coding processing, processes such as unifying (or adjusting) prediction mode candidate groups to configure MPM candidate groups can be performed. For example, the prediction mode of the current block can be one of the prediction mode candidate groups with modes -5 to 61, and the prediction modes of adjacent blocks can be one of the prediction modes with modes 2 to 66. In this case, since it may not be supported to use a portion of the prediction modes of adjacent blocks (mode 66) as the prediction mode of the current block, the process of unifying the prediction modes can be performed in the prediction mode coding process. That is, this process may not be needed when a fixed intra-frame prediction mode candidate group configuration is supported, but it may be needed when an adaptive intra-frame prediction mode candidate group configuration is supported, which will not be described in detail here.

Unlike MPM-based methods, encoding can be performed by assigning an index to a prediction pattern candidate group.

For example, prediction patterns are indexed according to a predefined priority. When a prediction pattern is selected as the prediction pattern for the current block, the index of the selected prediction pattern is encoded. This refers to configuring a fixed group of prediction pattern candidates and assigning fixed indexes to prediction patterns.

Alternatively, a fixed index allocation method may be unsuitable when adaptively configuring prediction mode candidate groups. Therefore, prediction modes can be indexed according to adaptive priority. When a prediction mode is selected as the prediction mode for the current block, the selected prediction mode can be encoded. This method enables efficient encoding of prediction modes because it changes the index assigned to the prediction mode due to the adaptive configuration of the prediction mode candidate group. In other words, adaptive priority can aim to assign candidates with a high probability of being selected as the prediction mode for the current block to the index in which short mode bits are generated.

The following description is based on the following assumption: In the prediction mode candidate group (in the case of chromaticity component), eight prediction modes are supported, including preset prediction modes (directional and non-directional modes), color replication mode, and color mode.

For example, suppose we support four preset modes: planar mode, DC mode, horizontal mode, vertical mode, and diagonal mode (bottom left diagonal in this example), one color mode C, and three color replication modes CP1, CP2, and CP3. Prediction modes can be indexed in the order of preset prediction mode, color replication mode, and color mode.

In this case, the preset prediction modes, which are directional and non-directional modes, and the color replication mode are prediction modes distinguished by the prediction method and are therefore easily identifiable. However, the color mode can be either a directional or non-directional mode, which may overlap with the preset prediction modes. For example, when the color mode is a vertical mode, the color mode may overlap with the vertical mode, which is one of the preset prediction modes.

When the number of prediction mode candidates is adaptively adjusted according to the encoding/decoding configuration, the number of candidates can be adjusted (8 -> 7) when redundancy exists. Alternatively, when the number of prediction mode candidates remains fixed, the index can be allocated by adding and considering another candidate when redundant prediction modes exist. Furthermore, adaptive prediction mode candidate groups can be supported even when variable modes such as color modes are included. Therefore, the adaptive index allocation case can be regarded as an example of configuring adaptive prediction mode candidate groups.

Now, a description of adaptive index allocation based on color mode will be given. It is assumed that the indexes are allocated in the order of plane (0)-vertical (1)-horizontal (2)-DC (3)-CP1 (4)-CP2 (5)-CP3 (6)-C (7). Furthermore, if the color mode does not match any preset prediction mode, it is assumed that the index allocation is performed in the above order.

For example, if the color mode matches one of the preset prediction modes (planar mode, vertical mode, horizontal mode, and DC mode), the prediction mode that matches index 7 of the color mode is filled. The preset prediction mode (bottom left diagonal) is filled at the index (one of 0 to 3) that matches the prediction mode. Specifically, when the color mode is horizontal, the indices can be assigned in the order of planar (0) - vertical (1) - bottom left diagonal (2) - DC (3) - CP1 (4) - CP2 (5) - CP3 (6) - horizontal (7).

Alternatively, when the color mode matches one of the preset prediction modes, the matching prediction mode is filled at index 0, and the preset prediction mode (lower left diagonal) is filled at index 7 of the color mode. In this case, if the filled prediction mode is not the existing index 0 (i.e., it is not a planar mode), the existing index configuration can be adjusted. Specifically, when the color mode is a DC mode, the indices can be assigned in the order of DC(0)-planar(1)-vertical(2)-horizontal(3)-CP1(4)-CP2(5)-CP3(6)-lower left diagonal(7).

The examples above are only a part of adaptive index allocation. This disclosure is not limited to this, and various modifications may be available. Furthermore, binarization such as fixed-length binarization, univariate binarization, truncated univariate binarization, and truncated Ricean binarization can be used based on the indexes in the candidate group.

Another example of performing encoding by assigning an index to a prediction pattern that belongs to a prediction pattern candidate group will be described.

For example, prediction modes and prediction methods are classified into multiple candidate groups, and indices are assigned to prediction modes belonging to the corresponding candidate groups, followed by encoding. In this case, the encoding of candidate group selection information can precede index encoding. For example, directional modes, non-directional modes, and color modes, which perform predictions in the same color space, can belong to one candidate group (called candidate group S), and color copy modes, which perform predictions in different color spaces, can belong to another candidate group (called candidate group D).

The following description is based on the following assumption: In the prediction mode candidate group (in the case of chromaticity component), nine prediction modes are supported, including the preset prediction mode, color replication mode, and color mode.

For example, suppose four preset prediction modes are supported in planar mode, DC mode, horizontal mode, vertical mode, and diagonal mode, one color mode C, and four color replication modes CP1, CP2, CP3, and CP4. Candidate group S may include five candidates as preset prediction modes and color modes, and candidate group D may include four candidates as color replication modes.

Candidate group S is an example of an adaptive configuration prediction pattern candidate group. An example of adaptive index allocation has already been described above, so it will not be described in detail here. Since candidate group D is an example of a fixed prediction pattern candidate group, a fixed index allocation method can be used. For example, the indexes can be allocated in the order CP1(0)-CP2(1)-CP3(2)-CP4(3).

Binarization techniques such as fixed-length binarization, univariate binarization, truncated univariate binarization, and truncated Ricean binarization can be used based on indices within candidate groups. This disclosure is not limited to the examples above, and various modifications may also be available.

Candidate groups for predictive pattern encoding, such as MPM candidate groups, can be configured at the block level. Alternatively, the process of configuring candidate groups can be omitted, and predefined candidate groups or candidate groups obtained by various methods can be used. This can be a configuration supported for the purpose of reducing complexity.

In Example 1, a predefined candidate group can be used, or one of multiple predefined candidate groups can be used depending on the encoding/decoding configuration. For example, in the case of MPM candidate groups, a predetermined candidate group such as {planar-DC-vertical-horizontal-lower left diagonal (66 in Figure 5)-lower right diagonal (34 in Figure 5)} can be used. Alternatively, a candidate group configured in the MPM candidate group configuration when adjacent blocks are unavailable can be applied.

In Example 2, a candidate group for blocks that have already been fully encoded can be used. Encoding blocks can be selected based on the encoding order (a predetermined scanning scheme, e.g., z-scan, vertical scan, horizontal scan, etc.) or from blocks adjacent to the current block (e.g., left block, top block, top-left block, top-right block, and bottom-left block). However, adjacent blocks can be limited to blocks located in partition units that can be referenced to the current block (e.g., partition units with attributes that can be referenced even if these blocks belong to different slices or tiles, such as different tiles belonging to the same tile group). If adjacent blocks belong to partition units that do not allow reference (e.g., when each block belongs to a different slice or tile and has attributes that may prevent mutual reference, e.g., when each block belongs to a different block group), the block at that location may be excluded from the candidates.

In this scenario, adjacent blocks can be determined based on the state of the current block. For example, when the current block is a square, a candidate group of available blocks can be borrowed (or shared) from blocks positioned according to a predetermined first priority. Alternatively, when the current block is a rectangle, a candidate group of available blocks can be borrowed from blocks positioned according to a predetermined second priority. A second or third priority can be supported based on the block's aspect ratio. To select candidate blocks to borrow, priorities can be set in various configurations, such as left-top-top-right-bottom-left-top, or top-left-top-left-top-right-bottom-left. In this case, all priorities from the first to the third priority can have the same or different configurations, or a subset of priorities can have the same configuration.

Candidate groups of the current block can be borrowed from adjacent blocks only if the candidate group of the current block is at or greater than/greater than a predetermined boundary value, or only if it is at or less than/less than a predetermined boundary value. The boundary value can be defined as the minimum or maximum block size that allows candidate group borrowing. The boundary value can be represented as the block width (W), block height (H), W×H, W*H, etc., where each of W and H can be an integer equal to or greater than 4, 8, 16, or 32.

In Example 3, a common candidate group can be formed from higher blocks that are part of a predetermined block group. The common candidate group can be used for lower blocks that belong to higher blocks. The number of lower blocks can be an integer equal to or greater than 1, such as 1, 2, 3, or 4.

In this case, the higher block can be a predecessor block (including the parent block) of the lower block, or it can be any group of blocks. A predecessor block can refer to a pre-segmented block (with a segmentation depth difference of 1 or greater) in a previous step during the partitioning process used to obtain the lower block. For example, the parent block of the 4N×2N child blocks 0 and 1 in candidate b of Figure 4 can be the 4N×4N child block in candidate a of Figure 4.

Candidate groups of higher blocks may be borrowed (or shared) only when the candidate group of a higher block is at or greater than/greater than a predetermined first boundary value, or only when it is at or less than/less than a predetermined second boundary value.

Boundary values can be defined as the minimum or maximum size of the block that a candidate group is allowed to borrow. Only one or two boundary values can be supported, and boundary values can be represented as the width W, height H, W×H, W*H, etc. of the block, where each of W and H is an integer of 8, 16, 32, 64 or higher.

On the other hand, a candidate group of a lower block can be borrowed from a higher block only if the candidate group of the lower block is at or greater than/greater than a predetermined third boundary value. Alternatively, a candidate group of a lower block can be borrowed from a higher block only if the candidate group of the lower block is at or less than/less than a predetermined fourth boundary value.

In this case, the boundary value can be defined as the minimum or maximum size of the block that the candidate group is allowed to borrow. Only one or two boundary values can be supported, and the boundary value can be represented as the width W, height H, W×H, W*H, etc. of the block, where each of W and H is an integer of 4, 8, 16, 32 or higher.

In this case, the first boundary value (or the second boundary value) can be equal to or greater than the third boundary value (or the fourth boundary value).

Candidate borrowing (or sharing) can be selectively used based on any of the above embodiments, and candidate group borrowing can be selectively used based on a combination of at least two of the first to third embodiments. Furthermore, candidate group borrowing can be selectively used based on any detailed configuration of the embodiments, and can be selectively used based on a combination of one or more detailed configurations.

Furthermore, information indicating whether to borrow candidate groups, information about block attributes (size/type/location/aspect ratio) involved in candidate group borrowing, and information about the partitioning status (partitioning scheme, partitioning type, partitioning depth, etc.) can be explicitly processed. Additionally, encoding factors such as image type and color components can serve as input variables in the candidate group borrowing configuration. Candidate group borrowing can be performed based on this information and the encoding/decoding configuration.

The prediction-related information generated by the prediction mode encoder can be sent to the encoding unit and included in the bitstream.

In the video decoding method according to embodiments of this disclosure, intra-frame prediction can be configured as follows. The intra-frame prediction of the prediction unit may include a prediction mode decoding step, a reference pixel configuration step, and a prediction block generation step. Additionally, the image decoding apparatus may be configured to include a prediction mode decoder, a reference pixel configuration unit, and a prediction block generator that perform the prediction mode decoding step, the reference pixel configuration step, and the prediction block generation step. Some of the above steps may be omitted, or other steps may be added. These steps may be performed in a different order than described above.

Since the reference pixel configuration unit and prediction block generator of the image decoding apparatus function identically to their counterparts in the image encoding apparatus, detailed descriptions of the reference pixel configuration unit and prediction block generator are not provided herein. The prediction mode decoder can reverse-engineer the methods used in the prediction mode encoder.

(Intra-frame prediction on a sub-block basis)

Figure 8 illustrates the various sub-block partitioning types that can be obtained from a coding block. A coding block can be called a parent block, and sub-blocks can be child blocks. Sub-blocks can be prediction units or transform units. In this example, it is assumed that a piece of prediction information is shared among sub-blocks. In other words, a prediction pattern can be generated and used in each sub-block.

Referring to Figure 8, the encoding order of the sub-blocks can be determined based on various combinations of sub-blocks a to p in Figure 8. For example, one of the following can be used: z-scan (left->right, top->bottom), vertical scan (top->bottom), horizontal scan (left->right), reverse vertical scan (bottom->top), and reverse horizontal scan (right->left).

The encoding order can be pre-agreed between the image encoder and image decoder. Alternatively, the encoding order of child blocks can be determined by considering the division direction of the parent block. For example, when the parent block is divided horizontally, the encoding order of the child blocks can be determined as vertical scanning. When the parent block is divided vertically, the encoding order of the child blocks can be determined as horizontal scanning.

When encoding is performed on a sub-block basis, reference data for prediction can be obtained at a closer location. Since only one prediction pattern is generated and this prediction pattern is shared among sub-blocks, encoding on a sub-block basis can be efficient.

For example, referring to Figure 8(b), when encoding is performed at the parent block level, pixels adjacent to the parent block can be used to predict the lower right sub-block. On the other hand, when encoding is performed at the sub-block level, pixels closer to the parent block can be used to predict the lower right sub-block because there are upper left, upper right, and lower left sub-blocks reconstructed in a predetermined encoding order (Z-scan in this example).

For intra-frame prediction based on optimal segmentation of image features, candidate groups can be configured based on one or more segmentation types with high occurrence probabilities.

In this example, we assume that an index-based partitioning scheme is used to generate partitioning information. Candidate groups can be formed using various partition types as follows.

Specifically, the candidate group can be configured to include N partition types. N can be an integer equal to or greater than 2, and the candidate group can include a combination of at least two of the seven partition types shown in Figure 8.

A parent block can be segmented into predetermined sub-blocks by selectively using any of a plurality of segmentation types from a candidate group. Selection can be based on an index signaled by the image coding device. The index may refer to information specifying the segmentation type of the parent block. Alternatively, selection can be made by considering attributes of the parent block in the image decoding device. Attributes may include position, size, shape, width, aspect ratio, one of width and height, segmentation depth, image type (I/P/B), color components (e.g., luminance or chrominance), intra-prediction mode value, whether the intra-prediction mode is non-directional, the angle of the intra-prediction mode, the position of a reference pixel, etc. A block may be a coded block or a prediction block and/or transform block corresponding to a coded block. The position of a block may refer to whether the block is located at the boundary of a predetermined image (or fragment image) of the parent block. The image (or fragment image) may be at least one of the picture, slice group, tile group, slice, tile, CTU row, or CTU to which the parent block belongs.

In Example 1, candidate groups such as {a to d} or {a to g} as shown in Figure 8 can be configured. Various partitioning types can be considered to obtain candidate groups. In the case of candidate group {a to d}, various binary bits can be assigned to each index (assuming the indices are assigned alphabetically. The following examples also adopt the same assumption).

[Table 1]

In Table 1 above, binary type 1 can be an example of binarization that considers all possible partition types, and binary type 2 is an example of binarization that first assigns a bit (the first bit) indicating whether partitioning is performed and then excludes only unsplit candidate partitions from the available partition types when partitioning is performed (the first bit is set to 1).

In Example 2, candidate groups such as {a, c, d} or {a, f, g} as shown in Figure 8 can be formed, which can be obtained by considering partitions in a specific direction (horizontal or vertical). In the case of candidate group {a, f, g}, various binary bits can be assigned to each index.

[Table 2]

index Binary type 1 Binary type 2 Binary type 3 0 0 10 10 1 10 0 11 2 11 11 0

Table 2 above is an example of binarization based on block attribute allocation, specifically, an example of binarization performed considering block shape. In binary type 1, 1 bit is allocated when the parent block is a square and when the parent block is not divided, and 2 bits are allocated when the parent block is divided in the horizontal or vertical direction.

In binary type 2, when the parent block is shaped into a horizontally elongated rectangle, 1 bit is allocated when the parent block is divided horizontally, and 2 bits are allocated for the rest. Binary type 3 could be an example of allocating 1 bit when the parent block is shaped into a vertically elongated rectangle. In binary type 3, 1 bit can be allocated when the parent block is divided vertically, and 2 bits can be allocated for the rest. This could be an example of allocating shorter bits when it is determined that further division may occur in the shape of the parent block. However, this disclosure is not limited to this, and modifications including the opposite case may also be available.

In Example 3, candidate groups such as {a, c, d, f, g} in Figure 8 can be configured, which can be another example of a candidate group configuration that considers partitioning in a particular direction.

[Table 3]

In Table 3, a flag indicating whether to perform a partition (the first bit) is first assigned. Then, in binary type 1, a flag indicating the number of partitions is assigned. If the flag indicating the number of partitions (the second bit) is set to 0, the number of partitions can be 2; and if the flag indicating the number of partitions (the second bit) is set to 1, the number of partitions can be 4. Furthermore, subsequent flags can indicate the partition direction. If a subsequent flag is 0, this indicates a horizontal partition; and if a subsequent flag is 1, this indicates a vertical partition. The above description is merely an example, and this disclosure is not limited thereto. Therefore, various modifications, including the opposite configuration, can be available.

Under normal circumstances, partitioning information can be supported, but the partitioning information can be modified to different configurations depending on the encoding/decoding environment. In other words, exceptions to the partitioning information can be supported, or the partitioning type represented by the partitioning information can be replaced with another partitioning type.

For example, depending on the block's attributes, there may be unsupported partition types. Since the characteristics of blocks have already been mentioned in previous examples, their detailed descriptions are not provided in this document. The block can be at least one of a parent block or a child block.

For example, suppose the supported partitioning type is {a, f, g} in Figure 8, and the size of the parent block is 4M × 4N. If some partitioning types are not supported due to minimum block size conditions in the image and the presence of parent blocks at the boundaries of a predetermined image (or segment image), various processing methods can be performed. For the following example, suppose the minimum width of the block in the image is 2M, and the minimum area of the block is 4*M*N.

First, candidate groups can be reconfigured by excluding unavailable partition types. Available candidates in the existing candidate groups can be 4M×4N, 4M×N, and M×4N, and the reconfigured candidate groups, without unavailable candidates (4M×N), can include partition types 4M×4N and 4M×N. In this case, binarization can be performed again on the candidates in the reconfigured candidate groups. In this example, 4M×4N or 4M×N can be selected using a flag (1 bit).

In another example, candidate groups can be reconfigured by replacing unavailable partition types with different partition types. Unavailable partition types could be vertical partition types (4 partitions). However, candidate groups can be reconfigured by replacing them with another partition shape that maintains the vertical partition type (e.g., 2M×4N). This allows the flag configuration to be maintained based on existing partition information.

As shown in the example above, the candidate group can be reconfigured by adjusting the number of candidates or replacing existing candidates. This example includes descriptions of several scenarios, and various modifications may be available.

(Sub-block encoding order)

The encoding order for various sub-blocks can be set as shown in Figure 8. The encoding order can be implicitly determined based on the encoding/decoding configuration. Partition type, image type, color components, size/shape/position of the parent block, block aspect ratio, information related to the prediction mode (e.g., intra-frame prediction mode, position of the reference pixel used, etc.), partition state, etc., can be included in the encoding/decoding factors.

Alternatively, the encoding order of sub-blocks can be explicitly handled. Specifically, candidate groups can be configured to include candidates with high occurrence probabilities based on the partition type, and selection information can be generated for one of the candidates. Therefore, the encoding order of candidates can be adaptively configured according to the partition type.

As in the example above, sub-blocks can be encoded in a fixed encoding order. However, an adaptive encoding order can also be applied.

Referring to Figure 8, many encoding orders can be obtained by sorting a to p in various ways. Since the positions and number of sub-blocks obtained can vary depending on the partition type, it may be important to set an encoding order specific to each partition type. Furthermore, for the partition type shown in Figure 8(a), where sub-block partitioning is not performed, no encoding order is needed. Therefore, when explicitly processing information about the encoding order, the partition information can be examined first, and then information about the encoding order of the sub-blocks can be generated based on the selected partition type information.

Referring to Figure 8(c), it is possible to support vertical scans with 0 and 1 applied to a and b respectively and reverse scans with 1 and 0 applied to a and b as candidates, and a 1-bit flag can be generated for selecting one of the scans.

Alternatively, referring to Figure 8(b), it can support z-scans with 0 to 3 applied to a to d, z-scans rotated 90 degrees to the left, z-scans with 1, 3, 0 and 2 applied to a to d, z-scans with 3, 2, 1 and 0 applied to a to d, z-scans rotated 90 degrees to the right, z-scans with 2, 0, 3 and 1 applied to a to d, and can generate one or more flags to select one of the scans.

The following describes the case where the encoding order of sub-blocks is implicitly determined. For ease of description, the encoding order is described when the partition types (c){or (f)} and (d){or (g)} of Figure 8 are obtained (refer to the prediction mode in Figure 5).

For example, when the intra-frame prediction mode is given as vertical mode, horizontal mode, bottom right diagonal mode (modes 19 to 49), bottom left diagonal mode (mode 51 or higher), and top right diagonal mode (mode 17 or lower), vertical scan and horizontal scan can be set.

Alternatively, when the intra-frame prediction mode is the bottom-left diagonal mode (mode 51 or higher), vertical scan and reverse horizontal scan can be set.

Alternatively, when the intra-frame prediction mode is the top-right diagonal mode (mode 17 or lower), you can set reverse vertical scan and horizontal scan.

In the examples above, the encoding order can be based on a predetermined scan order. In this case, the predetermined scan order can be one of Z-scan, vertical scan, and horizontal scan. Alternatively, the scan order can be determined based on the position and distance of pixels referenced during intra-frame prediction. For this purpose, inverse scanning can also be considered.

Figure 9 is an example diagram illustrating a reference pixel region for an intra-frame prediction mode according to an embodiment of the present disclosure. Referring to Figure 9, it can be noted that the reference region is shaded according to the direction of the prediction mode.

Figure 9(a) shows an example of dividing the region adjacent to the parent block into a left region, a top region, a top-left region, a top-right region, and a bottom-left region. Figure 9(b) shows the left and bottom-left regions referenced in the top-right diagonal mode, Figure 9(c) shows the left region referenced in the horizontal mode, and Figure 9(d) shows the left, top, and top-left regions referenced in the bottom-right diagonal mode. Figure 9(e) shows the top region referenced in the vertical mode, and Figure 9(f) shows the top and top-right regions referenced in the bottom-left diagonal mode.

When performing prediction using adjacent reference pixels (or reconstructed pixels of sub-blocks), a predefined encoding order for sub-blocks can advantageously avoid the need to send relevant information separately with signals. Various partition types are available, and therefore the following examples can be provided based on the reference region (or prediction mode) based on the partition type.

Figure 10 illustrates an example of the encoding order available in the upper-right diagonal pattern according to an embodiment of the present disclosure. Examples in Figures 10(a) through 10(g) in which higher priority is assigned to the lower-left adjacent sub-block can be provided.

Figure 11 illustrates an example of the encoding order available in horizontal mode according to an embodiment of the present disclosure. Examples in Figures 11(a) through 11(g) can be provided in which a higher priority is assigned to the left adjacent sub-block.

Figure 12 illustrates an example of the encoding order available in the bottom-right diagonal pattern according to an embodiment of this disclosure. Examples in Figures 12(a) through 12(g) in which higher priority is assigned to the top-left adjacent sub-block can be provided.

Figure 13 illustrates an example of the encoding order available in vertical mode according to an embodiment of the present disclosure. Examples in Figures 13(a) through 13(g) can be provided in which higher priority is assigned to the upper adjacent sub-block.

Figure 14 illustrates an example of the encoding order available in the lower left diagonal pattern according to an embodiment of the present disclosure. Examples in Figures 14(a) through 14(g) can be provided where a higher priority is assigned to the upper right adjacent sub-block.

In the example above, the encoding order is defined based on adjacent regions of the fully encoded/decoded code, and other modified examples are also available. Furthermore, various configurations that define the encoding order based on other encoding/decoding factors are possible.

Figure 15 is an example diagram illustrating the coding order considering intra-prediction mode and partition type according to an embodiment of the present disclosure. Specifically, the coding order of sub-blocks can be implicitly determined for each partition type based on the intra-prediction mode. Additionally, an example is described where, in the case of an unavailable partition type, the candidate group is reconfigured by replacing the unavailable partition type with another partition type. For ease of description, it is assumed that the parent block size is 4M×4N.

Referring to Figure 15(a), for intra-frame prediction in sub-block units, the parent block can be partitioned into a 4M×N form. In this example, a reverse vertical scan order can be used. If the 4M×N partition types are unavailable, the parent block can be partitioned into 2M×2N partitions according to a predetermined priority. Since the encoding order in this case is shown in Figure 15(a), it is not described in detail herein. If the 2M×2N partition type is unavailable, the parent block can be partitioned into 4M×2N partitions of the next priority. This supports partition types based on predetermined priorities, and the parent block can be partitioned into sub-blocks accordingly. If all predefined partition types are unavailable, it can be noted that it is impossible to partition the parent block into sub-blocks, and therefore the parent block is encoded.

Referring to Figure 15(b), for intra-frame prediction in sub-block units, the parent block can be divided into an M×4N form. As shown in Figure 15(a), assume a predetermined coding order based on each partition type. This example can be understood as the case of replacing the order with 2M×2N and 2M×4N.

Referring to Figure 15(c), for intra-frame prediction in units of sub-blocks, the parent block can be divided into 2M×2N partitions. This example can be understood as the case of replacement in the order of 4M×2N and 2M×4N.

Referring to Figure 15(d), for intra-frame prediction in units of sub-blocks, the parent block can be divided into 4M×N partitions. This example can be understood as the case of replacement in the order of 2M×2N and 4M×2N.

Referring to Figure 15(e), for intra-frame prediction in units of sub-blocks, the parent block can be divided into M×4N partitions. This example can be understood as the case of replacement in the order of 2M×2N and 2M×4N.

In the examples above, when a partitioning type is available, it is supported to partition into another partitioning type in a predetermined order. Various modified examples are available.

For example, when the partitioning type {4M×4N, M×4N, 4M×N} is supported, if the M×4N or 4M×N partitioning type is unavailable, it can be replaced with 2M×4N or 4M×2N.

As in the example above, the sub-block partitioning configuration can be determined based on various encoding/decoding factors. Since the encoding/decoding factors can be derived from the foregoing description of sub-block partitioning, they are not described in detail in this paper.

Furthermore, when determining sub-block partitioning, prediction and transformation can be performed as is. As mentioned above, intra-prediction modes can be determined at the parent block level, and predictions can be performed accordingly.

Furthermore, transformations and inverse transformations can be configured based on the parent block, and these transformations and inverse transformations can be performed at the child block level based on the parent block configuration. Alternatively, the relevant configuration can be determined based on the child block, and transformations and inverse transformations can be performed at the child block level based on that configuration. Transformations and inverse transformations can be performed based on one of the above configurations.

The methods of this disclosure can be implemented as program instructions executable by various computer devices and stored in a computer-readable medium. The computer-readable medium may include program instructions, data files, and data structures, individually or in combination. The program instructions recorded on the computer-readable medium may be specifically designed for this disclosure or may be known to those skilled in the art of computer software and therefore usable.

The computer-readable medium may include hardware devices particularly suitable for storing and executing program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, etc. The program instructions may include machine language code generated by a compiler or high-level language code that can be executed by an interpreter in a computer. The aforementioned hardware devices may be configured to operate as one or more software modules to perform operations according to this disclosure, and vice versa.

Furthermore, the above-described method or apparatus can be implemented by combining or separating all or part of its configuration or functions.

Although the present disclosure has been described above with reference to preferred embodiments thereof, those skilled in the art will understand that various modifications and variations may be made to the present disclosure without departing from the scope and spirit of the present disclosure.

[Industrial Applications]

This disclosure can be used for image encoding/decoding.

Claims (5)

1. An image decoding method performed by an image decoding device, the method comprising: Predicted blocks for decoding are generated by performing intra-frame prediction; The residual block for generating the decoded block; and The decoded block is reconstructed based on the predicted block and the residual block. The decoded block is reconstructed by adding the prediction block and the residual block. Intra-frame prediction includes interpolating reference samples. Depending on the intra-frame prediction mode of the decoded block used for interpolating reference samples, one of fixed filtering or adaptive filtering is selectively performed, and The fixed filtering uses a predefined interpolation filter in the image decoding device, while the adaptive filtering selectively uses one of a plurality of predefined interpolation filters in the image decoding device. 2. The method according to claim 1, The plurality of interpolation filters include at least one of a 4-tap cubic filter or a 4-tap Gaussian filter. 3. An image encoding method performed by an image encoding device, the method comprising: Predicted blocks for decoding are generated by performing intra-frame prediction; The residual block of the decoded block is generated based on the predicted block; and The decoded block is encoded by encoding the residual block. The residual block is generated by subtracting the prediction block from the decoded block. Intra-frame prediction includes interpolating reference samples. Depending on the intra-frame prediction mode of the decoded block used for interpolating reference samples, one of fixed filtering or adaptive filtering is selectively performed, and The fixed filtering uses an interpolation filter predefined in the image encoding apparatus, while the adaptive filtering selectively uses one of a plurality of interpolation filters predefined in the image encoding apparatus. 4. A computer-readable recording medium storing a bitstream generated by an image encoding method performed by an image encoding apparatus, the image encoding method comprising: Predicted blocks for decoding are generated by performing intra-frame prediction; The residual block of the decoded block is generated based on the predicted block; and The decoded block is encoded into the bit stream by encoding the residual block. The residual block is generated by subtracting the prediction block from the decoded block. Intra-frame prediction includes interpolating reference samples. Depending on the intra-frame prediction mode of the decoded block used for interpolating reference samples, one of fixed filtering or adaptive filtering is selectively performed, and The fixed filtering uses an interpolation filter predefined in the image encoding apparatus, while the adaptive filtering selectively uses one of a plurality of interpolation filters predefined in the image encoding apparatus. 5. A method for transmitting a bitstream generated by an image encoding method performed by an image encoding apparatus, the image encoding method comprising: Predicted blocks for decoding are generated by performing intra-frame prediction; The residual block of the decoded block is generated based on the predicted block; and The decoded block is encoded into the bit stream by encoding the residual block. The residual block is generated by subtracting the prediction block from the decoded block. Intra-frame prediction includes interpolating reference samples. Depending on the intra-frame prediction mode of the decoded block used for interpolating reference samples, one of fixed filtering or adaptive filtering is selectively performed, and The fixed filtering uses an interpolation filter predefined in the image encoding apparatus, while the adaptive filtering selectively uses one of a plurality of interpolation filters predefined in the image encoding apparatus.