WO2019185815A1 - Partitioning concepts for block-based picture coding - Google Patents
Partitioning concepts for block-based picture coding Download PDFInfo
- Publication number
- WO2019185815A1 WO2019185815A1 PCT/EP2019/057891 EP2019057891W WO2019185815A1 WO 2019185815 A1 WO2019185815 A1 WO 2019185815A1 EP 2019057891 W EP2019057891 W EP 2019057891W WO 2019185815 A1 WO2019185815 A1 WO 2019185815A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- tree
- partitioning
- split
- block
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
- H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
Definitions
- the present invention concerns partitioning concepts for partitioning blocks in block-based picture coding.
- the present invention is concerned with concepts for the block- based picture coding which concepts provide for a solution to the trade-off between signaling efficiency and coding flexibility and the trade-off between coding efficiency and coding complexity.
- the pictures of a video sequence are coded in a certain order, which is referred to as coding order.
- the coding order of pictures can differ from the capture and display order.
- each video picture is partitioned into blocks.
- a block comprises the samples of a rectangular area of a particular color component.
- the entity of the blocks of all color components that correspond to the same rectangular picture area, together with the associated syntax, is often referred to as unit.
- CTUs coding tree units
- CUs coding units
- PUs prediction units
- TUs transform units
- the associated sample blocks are referred to as coding tree blocks (CTBs), coding blocks (CBs), prediction blocks (PBs), and transform blocks (TBs), respectively.
- a video picture is initially partitioned into fixed sized units (i.e. , aligned fixed sized blocks for all color components).
- these units are referred to as coding tree units (CTUs).
- Their size (in luma samples) is transmitted in the sequence parameter set (SPS), which contains high-level syntax elements that are valid for an entire coding video sequence (which consists of an instantaneous decoding refresh (IDR) picture and all following pictures up to (but excluding) the next IDR picture).
- SPS sequence parameter set
- the CTU size can be chosen between 64x64, 32x32, and 16x16 luma samples.
- CTUs coding tree blocks
- the fixed sized CTUs can be adaptively split into smaller units and blocks.
- a picture can be comprised of one or multiple slices.
- a slice represents a collection of CTUs.
- a slice is associated with a slice type (e.g., I, P, or B slice) which determines the set of supported coding modes.
- a slice type e.g., I, P, or B slice
- several types of prediction such as motion vector prediction and intra-picture prediction are only supported between blocks inside a slice.
- a CTU can be split into multiple coding units (CUs).
- a coding unit is the entity for which a coding mode (e.g., intra- or inter-picture coding) is selected.
- the decomposition of a CTU into one or more CUs is specified by a quadtree (QT) and transmitted as part of the bitstream.
- QT quadtree
- a first binary decision (bin) called split_cu_flag is transmitted, which signals whether the entire CTU forms one single CU (i.e., the CTU is not decomposed into multiple CUs) or whether the CTU is partitioned into four equally sized square units.
- a further split_cu_flag is transmitted, which indicates whether the corresponding unit is further partitioned into four equally sized square units. This multi-level splitting is continued until for each of the units it is either signaled that the unit is not further split or a minimum CU size is reached. If a unit has the minimum CU size, no split_cu_flag is transmitted, but it is inferred that the unit is not further split.
- the minimum allowed CU size is signaled in the sequence parameter set; it can range from the size of the CTU to 8x8 luma samples (and always presents a square with an integer power of two for both the width and height).
- the resulting units of the described splitting are the coding units (CUs), for which a coding mode and associated parameters are selected and transmitted. Note that the partitioning represents a quadtree (QT) structure.
- Each of the resulting CUs is associated with a coding mode, which is selected at the encoder side and transmitted as part of the bitstream.
- the binary decisions split_cu_flag which specify the partitioning of a CTU into one or multiple CUs are coded using context-based adaptive binary arithmetic coding (CABAC).
- CABAC context-based adaptive binary arithmetic coding
- the CUs of a CTU are processed in the so-called z-scan order. That means, the four blocks that result from a split are processed in raster-scan order; and if any of the blocks is further partitioned, the corresponding four blocks (including the included smaller blocks) are processed before the next block of the higher splitting level is processed.
- a CU is coded in an intra-coding mode, an intra prediction mode for the luma signal and, if the video signal includes chroma components, another intra prediction mode for the chroma signals is transmitted.
- the luma coding block can also be split into four equally sized blocks, in which case, for each of these blocks, a separate luma intra prediction mode is transmitted. This splitting is indicated by a binary syntax element partjmode. The actual intra prediction and coding is done on the basis of transform blocks.
- a CU can be partitioned into multiple transform blocks.
- the subdivision into transform blocks is transmitted using a quadtree syntax.
- the root of the quadtree is formed by the coding unit.
- the syntax of the coding unit includes a binary decision called split_transform_f lag , which specifies whether the luma component of the CU consists of a single luma transform block or whether it is split into four equally sized square blocks. In the latter case, for each of the resulting luma blocks, another split_transform_flag is transmitted specifying whether the corresponding block is further split.
- This quadtree splitting is continued until for each of the resulting luma blocks, it is signaled that the block is not further split or the block reaches a minimum transform block size (indicated in the sequence parameter set).
- the first split_transform_flag (at the CU level) is not transmitted, but it is inferred that the luma coding block is split into four luma blocks.
- the splitting of the chroma blocks basically follows the splitting of the colocated luma block, i.e. , for each luma block the associated chroma blocks cover the same picture area.
- the luma block and the associated chroma blocks that cover the same picture area form a transform unit.
- ITU-T H.265 MPEG-H HEVC standard: If the resulting chroma blocks would be smaller than the minimum transform block size (as specified in the sequence parameter set), only the luma block is split.
- the binary syntax elements split_transform_flag are coded using CABAC.
- a transform block (TB) is the entity to which a single 2d transform is applied for coding the prediction residual signal. Note that all transform blocks in ITU-T H.265
- the TUs of an intra-coded CU are processed in z-scan order (see above). For each transform block (luma or chroma), first a prediction signal is generated using already reconstructed neighboring samples of the same color component. The algorithm that is used for generating the prediction signal is determined by the intra prediction mode. The resulting prediction error / residual signal is transmitted using transform coding. A two- dimensional transform is applied to the corresponding block of residual samples, the resulting transform coefficients are quantized (typically using uniform reconstruction quantizers), and the resulting quantization indexes, which are also referred to as transform coefficient levels, are entropy-coded.
- the entropy coding in ITU-T H.265 j MPEG-H HEVC uses the concept of context-based adaptive binary arithmetic coding (CABAC).
- CABAC context-based adaptive binary arithmetic coding
- Inter-picture coded CUs can be split into multiple prediction units.
- a prediction unit is the entity of a luma and, for color video, two associated chroma blocks (covering the same picture area), for which a single set of prediction parameters is used.
- a CU can be coded as a single prediction unit, or it can be split into two non-square or four square prediction units. The latter (four square PUs) is only supported if the CU has the minimum CU size specified in the sequence parameter set. The chosen splitting is indicated by the syntax element partjnode.
- ITU-T H.265 j MPEG-H HE VC supports in total 8 modes for partitioning a CU into PUs.
- An NxN CU can be partitioned into:
- the asymmetric splittings are only supported for CUs larger than or equal to 16x16 luma samples.
- MPEG-H HE VC are 8x4 and 4x8.
- Each set of motion parameters includes the number of motion hypotheses (one or two in ITU-T H.265
- a merge_flag is transmitted, which indicates whether the motion parameters are completely derived based on already coded motion parameters or whether the motion parameters are transmitted. If the merge_flag is equal to 1 , a list of motion parameter set candidates is derived, which typically includes motion parameters sets of already coded spatially neighboring blocks as well as one or more motion parameter sets that are derived from temporally co-located blocks.
- the used motion parameter set is indicated by a syntax element mergejdx, which specifies an index into the candidate list of motion parameter sets. If merge_flag is equal to 0, the motion parameters are transmitted as part of the bitstream.
- a first syntax element (inter_pred_idc in ITU-T H.265
- the reference picture lists are constructed based on slice header syntax elements.
- Each reference picture list contains one or more already coded pictures (which are stored in a decoded picture buffer) in a specified order.
- the syntax element inter_predjdc does not only indicate single-hypothesis prediction or bi-prediction, but for single-hypothesis prediction it additionally indicates whether list 0 or list 1 is used.
- the first motion hypothesis refers to list 0 and the second motion hypothesis refers to list 1.
- an associated motion vector is transmitted as part of the bitstream.
- the motion vectors are differentially coded.
- a list of motion vector predictor candidates is derived. It can include (potentially scaled) motion vectors of neighboring PUs as well as scaled motion vectors of temporally co-located blocks.
- the actually chosen motion vector predictor is signaled via an index into the motion vector predictor list.
- the difference between the actual motion vector for the PU and the selected motion vector predictor is transmitted.
- the prediction signal for a PU is formed by a displaced block in the selected reconstructed reference picture, where the displacement relative to the current PU is specified by the motion vector.
- the displacement is typically specified with sub-sample accuracy (in ITU-T H.265
- the prediction signal is generated by interpolating the reconstructed reference picture (typically, using separable FIR filters).
- the final prediction signal of PUs with multi-hypothesis prediction is formed by a weighted sum of the prediction signals for the individual motion hypothesis.
- luma and chroma blocks of a PU share the same set of motion parameters.
- the prediction signal for an inter-picture coded CU is composed of the prediction signals for the contained PUs.
- the prediction error signal (also called residual signal) for the luma and chroma blocks of a coding unit is transmitted via transform coding.
- the block of luma residual samples of a CU as well as the blocks of chroma residual samples (if present) are partitioned into transform blocks.
- the partitioning into transform blocks is the same as for intra-picture coded CUs described above.
- the partitioning of a CU into transform block is indicated by a quadtree syntax, which is also referred to as residual quadtree (RQT).
- the resulting transform blocks are coded using transform coding: A 2d transform is applied to the block of residual samples, the resulting transform coefficients are quantized, and the resulting transform coefficient levels (quantization indexes) are entropy coded.
- a skip_flag is transmitted. If this flag is equal to 1 , it indicates that the corresponding CU consists of a single prediction unit coded in merge mode (i.e., merge_flag is inferred to be equal to 1) and that all transform coefficients are equal to zero (i.e., the reconstruction signal is equal to the prediction signal). In that case, only the mergejdx is transmitted in addition to the skip_flag. If skip_flag is equal to 0, the prediction mode (inter or intra) is signaled, followed by the syntax features described above.
- CABAC context- based adaptive binary arithmetic coding
- Non-binary syntax elements are first binarized, i.e., they are mapped onto a series of binary decisions, which are also referred to as bins. Then, each bin (including binary syntax elements) is coded using a binary arithmetic coder.
- the so-called regular mode one adaptive binary probability model is chosen per bin; after coding the corresponding bin, the binary probability model is updated using the actual value of the bin.
- the entropy coder maintains a certain number of binary probability models.
- the selected probability model depends on the syntax element and the bin number (for non-binary syntax elements). It can additionally depend on already coded syntax elements or bins.
- the selection of binary probability models is also referred to as context modelling.
- MPEG-H HEVC provides a multitude of possibilities for coding a block of samples (such as a coding tree unit).
- the encoder has to select the partitionings (into CUs, PUs, and TUs), the coding modes and associated parameters (such as intra prediction modes and motion parameters), and the transform coefficient levels (quantization indexes).
- the selected coding parameters determine both the reconstruction quality as well as the number of bits required for representing the coding parameter inside the bitstream.
- Modern encoding algorithms are typically based on Lagrangian bit allocation.
- the maximum achievable coding efficiency of a codec is determined by the supported syntax features, such as the set of coding modes, the partitioning options, the supported intra prediction modes, etc.
- One of the most important syntax features are the supported partitioning options.
- a flexible partitioning allows an adaptation of the block sizes used for prediction and transform coding to the video signal.
- the partitioning has to be transmitted as part of the bitstream.
- the gain in prediction and transform coding for providing additional partitioning options has to outweigh the additional signaling overhead. Due to that reason, state-of-the-art video coding standards support only very simple partitioning concepts, such as quadtree partitionings.
- a partitioning concept based on generalized binary trees provides more flexibility than conventional partitioning concepts (such as the quadtree approach) for adapting the blocks used for prediction and transform coding to the video signal to be coded. And since the syntax for transmitting the selected partitioning is still very simple, the gain in prediction and transform coding typically outweighs the additional signaling overhead.
- the generalized partitioning concept is capable of increasing the achievable coding efficiency of a video codec.
- the described partitioning concept can be used for partitioning a block for coding mode signaling, intra-picture prediction, inter-picture prediction, transform coding, or any combination thereof. It can completely replace conventional partitioning concepts, or it can be combined with other partitioning concepts, such as a quadtree partitioning.
- the quadtree splitting is very simple, it has the disadvantage that often an object or motion boundary can only be reasonably approximated by splitting a large block into many smaller blocks. This potentially increases the bit rate for signaling the prediction parameters (such as motion parameters) as well as the bit rate for the transform coding of the prediction error signals.
- the Joint Exploration Test Model [3] specifies a combined quadtree/binary tree (QTBT) approach.
- QTBT quadtree/binary tree
- prediction units PUs
- TUs transform units
- a CTU is first split using the conventional quadtree approach.
- the quadtree leaf nodes can be further partitioned by a binary tree (BT) structure.
- a horizontal split which horizontally splits a NxM block into two Nx(M/2) blocks
- a vertical split which vertically splits a NxM block into two (N/2)xM blocks.
- the binary splitting can be continued until a minimum block width/block height is reached or a maximum number of (binary) splits were executed.
- the resulting blocks are used for coding mode signaling, intra-picture prediction, inter-picture prediction, and transform coding.
- an NxM block is split into three blocks: an Nx(M/4), an Nx(M/2), and an Nx(M/4) block (from top to bottom).
- the vertical triple split vertically partitions an NxM block into an (N/4)xM, an (N/2)xM, and an (N/4)xM block (from left to right).
- the present invention solves the above mentioned problems by providing encoders, decoders, methods for encoding, methods for decoding and corresponding computer programs according to the independent claims.
- the generalized binary splitting that is described herein represents a superset of all mentioned splitting approaches QT, QTBT, QTBT with additional triple splits, and QTBT with additional asymmetric splits into subblocks of sizes 1/4 and 3/4. It is characterized by a simple syntax and is capable of better adapting the block sizes to object and motion boundaries, which results in an improved coding efficiency.
- the partitioning concept as described herein may also be referred to as Generalized Binary Splitting (GBS).
- GBS Generalized Binary Splitting
- the inventive aspects of the present invention may, at least partially, be based on said GBS partitioning.
- the generalized binary splitting can be combined with any other splitting.
- the quadtree splitting (or any other splitting concept) can be used on a first level and the resulting blocks can be further partitioned using the generalized binary splitting.
- blocks obtained by the generalized binary splitting can be further partitioned using any other splitting concept.
- a first aspect is concerned with providing implicit splits and consecutive implicit splits wherein one or more implicit splits may be derived at the decoder side.
- a block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, signal partitioning information of the binary splitting tree in the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, signal the split option or the no-split option in the data stream, and if not, refrain from signaling the split option in the data stream.
- a block-based decoder for decoding a picture from a data stream, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, obtain partitioning information of the binary splitting tree from the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, obtain the split option or the no-split option from the data stream, and if not, implicitly infer the split option without an explicit split-option-signal in the data stream.
- a method for block-based encoding block-based encoding a picture into a data stream comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, signaling partitioning information of the binary splitting tree in the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, the method comprises a step of signaling the split option or the no-split option in the data stream, and if not, the method comprises a step of refraining from signaling the split option in the data stream.
- a method for block-based decoding a picture from a data stream comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, obtaining partitioning information of the binary splitting tree from the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, the method comprises a step of obtaining the split option or the no-split option from the data stream, and if not, the method comprises a step of implicitly inferring the split option without an explicit split-option-signal in the data stream.
- a computer readable digital storage medium is provided, having stored thereon a computer program having a program code for
- a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
- a second concept is concerned with differentiating nodes in a tree depth evaluation, wherein tree nodes according to this aspect may be handled differently than other tree nodes which do not fulfil the requirements of this aspect.
- a block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including at least one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size
- a corresponding block-based decoder is provided for decoding a picture from a data stream, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including at least one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root no
- a method for block-based encoding a picture into a data stream comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more proceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to
- a method for block-based decoding a picture from a data stream comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more proceeding tree nodes, via which the predetermined tree node is connected to the root node, are available in the data stream or not, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate
- a computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding. Furthermore, a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
- a third concept is concerned with providing tree type sets comprising one or more tree types, wherein different tree types for partitioning a block may be selected at one or more different partitioning levels.
- a block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to partition the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, select, for performing the multi-tree partitioning, a tree type set comprising one or more tree types, and signal the tree type set in the data stream, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
- a corresponding block-based decoder for decoding a picture from a data stream, wherein the decoder is configured to partition the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, perform the multi-tree partitioning using a tree type set comprising one or more tree types based on a signaling of the tree type set in the data stream, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
- a method for block-based encoding a picture into a data stream comprises the steps of partitioning the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, selecting, for performing the multi-tree partitioning, a tree type set comprising one or more tree types, and signaling the tree type set in the data stream, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
- a corresponding method for block-based decoding a picture from a data stream comprises the steps of partitioning the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, performing the multi-tree partitioning using a tree type set comprising one or more tree types, wherein the step of performing the multi-tree partitioning is based on a signaling of the tree type set in the data stream, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
- a computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding.
- a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
- a fourth aspect is concerned with providing asymmetric quad splits having a splitting ratio different from 1 ⁇ 2, wherein one or more implicit splits may be derived at the decoder side.
- a block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and signal partitioning information of the splitting tree in the data stream, wherein the encoder is further configured to perform for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and to signal in the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
- a corresponding decoder for decoding a picture from a data stream wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and obtain a partitioning information of the splitting tree from the data stream, wherein the decoder is further configured to perform for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and to obtain from the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
- a method for block-based encoding a picture into a data stream comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and signaling partitioning information of the splitting tree in the data stream, wherein the method further comprises steps of performing for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and signaling in the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
- a corresponding method for block-based decoding a picture from a data stream comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and obtaining a partitioning information of the splitting tree from the data stream, wherein the method further comprises steps of performing for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and obtaining from the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
- a computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding.
- a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
- a fifth aspect is concerned with an inheritance of partitioning parameters, wherein a currently coded block may inherit one or more partitioning parameters from a further block of a spatial or temporal vicinity.
- a block-based encoder for encoding a picture into a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding
- the encoder is configured to partition the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is encoded by means of spatial intra-prediction, the encoder is configured to inherit one or more partitioning parameters from a further block in the spatial vicinity of the at least one block, and to use said inherited one or more partitioning parameters for partitioning said at least one block in the recursive multi-tree partitioning, and if the at least one block is encoded by means of temporal inter-prediction, the encoder is configured to inherit one or more partitioning parameters from a further block in the temporal vicinity of the at least one block, and to use said inherited one
- a corresponding block-based decoder for decoding a picture from a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding
- the decoder is configured to partition the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters
- the decoder is configured to use, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one block, which inherited one or more partitioning parameters are inherited from a further block in the spatial vicinity of the at least one block, and if the at least one block is decoded by means of temporal inter- prediction, the decoder is configured to use, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the
- a method for block-based encoding a picture into a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding comprises the steps of partitioning the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is encoded by means of spatial intra-prediction, the method comprises further steps of inheriting one or more partitioning parameters from a further block in the spatial vicinity of the at least one block, and using said inherited one or more partitioning parameters for partitioning said at least one block in the recursive multi-tree partitioning, and if the at least one block is encoded by means of temporal inter-prediction, the method comprises further steps of inheriting one or more partitioning parameters from a further block in the temporal vicinity of the at least one block, and using said inherited partitioning parameters for partition
- a corresponding method for block-based decoding a picture from a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding comprises the steps of partitioning the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is decoded by means of spatial intraprediction, the method comprises a further step of using, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one block, which inherited partitioning parameters are inherited from a further block in the spatial vicinity of the at least one block, and if the at least one block is decoded by means of temporal inter-prediction, the method comprises a further step of using, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one
- a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
- Fig. 1 shows an exemplary partitioning of a 128x128 CTU
- Fig. 2 shows the partitioning tree describing the topology in fig. 1
- Fig. 3. shows a description of non-leaf nodes of the partitioning tree in fig. 2
- Fig. 4 shows an example of signaling of a split direction
- Fig. 5 shows an exemplary signaling of the split ratio for a particular block size
- Fig. 6 shows a reduced signaling from fig. 5 (for cases where not all split ratios are available)
- Fig. 7 shows three template layouts used to define prohibited partitioning tree
- Fig. 8 shows exemplary context modeling for a simple binary decision tree for
- Fig. 9 shows exemplary context modeling for another binary decision tree for
- Fig. 10 shows exemplary locations of neighboring blocks
- Fig. 1 1 shows an example for redundant parallel splits
- Fig. 12 shows an example for redundant perpendicular splits
- Fig. 13 shows an example for higher-order redundancies
- Fig. 14 shows an embodiment of an encoder according to an aspect
- Fig. 15 shows an embodiment of a decoder according to an aspect
- Fig. 16A shows a triple tree split according to the prior art
- Fig. 16B shows a triple tree like split according to an embodiment
- Fig. 17 shows a block diagram of an encoding method according to an embodiment
- Fig. 18 shows a block diagram of a decoding method according to an embodiment
- Fig. 19 shows a block diagram of an encoding and a decoding method according to an embodiment
- Fig. 20 shows a selection of tree type sets according to an embodiment
- Fig. 21 shows a further selection of tree type sets according to an embodiment
- Fig. 22 shows a further selection of tree type sets according to an embodiment
- Fig. 23 shows a block diagram of an encoding method according to an embodiment
- Fig. 24 shows a block diagram of a decoding method according to an embodiment
- Fig. 25 shows an asymmetric quad split based on a reduced GBS partitioning
- Fig. 26 shows a block diagram of an encoding method according to an embodiment
- Fig. 27 shows a block diagram of a decoding method according to an embodiment
- Fig. 28 shows picture partitioning according to an embodiment, in which one or more partitioning parameters for partitioning a current block may be inherited by a further previously coded block,
- Fig. 29 shows a schematic block diagram of a method for encoding a picture using the concept of Figure 28 in which one or more partitioning parameters for partitioning a current block may be inherited by a further previously coded block, according to an embodiment
- Fig. 30 shows a schematic block diagram of a method for decoding a picture using the concept of Figure 28 in which one or more partitioning parameters for partitioning a current block may be inherited by a further previously coded block, according to an embodiment.
- GBS generalized binary splitting
- the presented generalized binary splitting can be applied for all block partitionings required in video coding.
- it can be applied for partitioning a block into coding units (for signaling the coding mode), prediction units (for signaling prediction parameters such as motion parameters), or transform units (for transform coding of the residual signal).
- the signaling of the coding mode, the signaling of prediction parameters and the transform coding is applied to the same blocks (i.e., such a block represents a CU, PU, and TU) and the generalized binary splitting is used for specifying the corresponding partitioning of a fixed sized block into coding units (which also represent PUs and TUs).
- the generalized binary splitting can be applied individually to color components and auxiliary components, such as alpha channel or depth masks, respectively.
- generalized binary splitting can be applied to groups of those components.
- two different splitting trees can be used: one for partitioning the luma component and other for partitioning the chroma components.
- the generalized binary splitting can be combined with any other splitting.
- the quadtree splitting (or any other splitting concept) can be used on a first level and the resulting blocks can be further partitioned using the generalized binary splitting.
- blocks obtained by the generalized binary splitting can be further partitioned using any other splitting concept.
- the splitting structure can be represented as a binary splitting tree (also referred to as “partitioning tree” in the following).
- Each node represents a binary split with a split direction (horizontal or vertical) and a split ratio (which is the ratio of the width (or height) of the first resulting block and the width (or height) of the original block).
- the split direction is coded relative to the previous split direction (i.e. , the corresponding split on the next higher level). That means a split is coded as“parallel” or as "perpendicular” split, which always translates to either a“vertical” (splitting the width) or a“horizontal” split (splitting the height). If we assume that the current block was obtained by a horizontal split (at the last split level), then, if a“parallel split” is signaled, the current block is again horizontally split into two subblocks, and if a "perpendicular split” is signaled, the current block is vertically split into two subblocks.
- the“parallel split” specifies a vertical split and the“perpendicular split” specified a horizontal split. Since on the highest level (i.e., for the root level block) no previous split was signaled, it has to be defined by convention whether the last split level is inferred to represent a horizontal or a vertical split.
- a binary split is characterized by the split ratio.
- the split ratio specifies the ratio of the sizes (in number of samples) of the first resulting subblock (from top/left) and the block being split. Note that, since a block is always split into two subblocks, the split ratio also specifies ratio of the block widths (for vertical splits) or the ratio of the block heights (for horizontal blocks).
- the leaf nodes of the partitioning tree represent the actual units (consisting of a luma sample block and, for non-monochrome video, the co-located chroma sample blocks) being coded. These units, which shall be referred to as coding units in the following, are characterized by their positions and sizes, which can be inferred from the partitioning tree.
- Coding, prediction and transform coding can be applied at the leaf level of the splitting structure. It is also possible to apply the generalized binary splitting to a single color component or a subset of color components (e.g., both chroma components).
- a perpendicular split with a split ratio of 1/2 is the split that can be signaled using the minimum number of bits (or bins) in most configurations and will referred to as“default split” in the following.
- the partitioning is signaled as follows. Without loss of generality, we define that the last split for the root block (which did not result from an actual split) is considered as vertical split. But note that it could also be defined as horizontal split. First a binary decision perpend_split_flag is coded, which indicates whether a perpendicular split is applied. If perpend_split_flag is equal to 1 and the last split (i.e., the split from which a current block resulted) was a horizontal split, a vertical split is applied to the current block. If perpend_split_flag is equal to 1 and the last split was a vertical split (with our above defined convention this includes the case that our current block is the root blocks), this indicates that a horizontal split is applied.
- perpend_sp!it_flag is equal to 0
- another binary decision parallel_split_flag is transmitted. If parallel_split_flag is equal to 1 and the last split was a horizontal split, the current block is again horizontally split. If parallel_split_flag is equal to 1 and the last split was a vertical split (or was inferred as vertical split by our convention), the current block is vertically split. If parallel_split_flag is equal to 0, the current block is not further split, but represents a leaf block (equivalent to a“no-split” signaling).
- the corresponding binary decisions are not transmitted, but can be inferred at the decoder side. For example, if the last split was a vertical split and the width of the current block is equal to the minimum supported block width, the binary decision parallel_split_flag (which indicates whether the current block is again vertically split) is not transmitted, but inferred to be equal to 0.
- Either or both of the binary decisions perpend_split_flag and parallel_split_flag can be inferred if this is indicated by high level parameters, which may be signaled in the PPS (picture parameter set), SPS (sequence parameter set) or slice header.
- PPS picture parameter set
- SPS sequence parameter set
- slice header A binary decision tree for signaling the split direction is shown in fig. 4. If a horizontal split is to be applied, the split size M is defined as the length of the vertical edge of the block to be split (as the split will introduce a new horizontal edge). For a vertical split, the split size M is the length of the horizontal edge of the block to be split.
- the split pattern consists of alternating horizontal and vertical splits. Given a particular block, the block is either not further split or it is split using a split direction that is perpendicular to the previous split direction (at the higher level). An exception could be made for the root block (where no previous split exists); for the root block it could be explicitly signaled whether a horizontal or a vertical split is applied (or whether the block is not split at all).
- the split direction is coded using binary decisions h o r__s pi it_f I a g (indicating whether or not the block is horizontally split) and ver_split_flag (indicating whether or not the block is vertically split). These flags could be coded in any order; the second flag is only transmitted if the first flag indicates no splitting in the corresponding direction.
- the split and the split direction are coded using a split_flag and a sp!it_direction_flag, where the splitj!ag indicates whether the current block is split at all and the split_direction_flag indicates whether the split is applied in horizontal or vertical direction.
- the split_direction_flag indicates whether the split is applied in perpendicular direction to the last split or in the same direction as the last split.
- the split flags are coded using context-based adaptive arithmetic coding (as CABAC in H.265
- a split ratio is coded in addition to the split direction.
- the set of possible split ratios can be arbitrarily chosen for every block size provided the resulting subblocks can be encoded using the video encoding framework (e.g., an appropriate transformation size has to be available).
- the split ratio may be encoded as an element of a set as well as by binarization of the nominator and the denominator (in the former case, the set has to be predefined).
- the split ratio is coded based on a binary decision tree (with adaptive tree pruning according to the available split ratios) as will be described below in more detail.
- the split ratio can be coded as an index into an ordered list of available split ratios. The index can be mapped onto a series of binary decisions using any binarization scheme.
- the overall set of split ratios is defined as: ⁇ 1/2, 1/4, 3/4, 3/8, 5/8, 1/3, 2/3, 1/5, 2/5, 3/5, 4/5 ⁇ .
- the available set of split ratios represents a subset of this overall set.
- the subset for a particular block and a particular split direction can be determined by the size of the current block and/or the series of previous splits. If some of the ratios are not available for a particular split size (e.g., because restrictions on transform or prediction sizes or minimum block sizes prevent these split ratios or because the splits would result in redundant partitionings), the corresponding signaling flags are not transmitted in order to reduce the bit rate.
- all bins of the ratio signaling can be inferred and are not transmitted (e.g., for block sizes larger than the maximum block size for a non-default split). If, for a split direction, none of the split ratios can be applied (e.g., due to prohibitions enforced by high level parameters), the signaling of this split direction can be inferred as false.
- the binary splitting will be applied recursively until no further splitting is inferred or signaled. I.e., if a block is split into two subblocks, for each of the resulting subblocks another split consisting of a split direction and a split ratio is coded (where some or all of the corresponding binary decisions can be inferred based on syntax constraints). This process is continued until it is signaled (or inferred) that no block is further split.
- the generalized binary splitting concept is applied for partitioning fixed sized blocks (such as CTUs) into coding units, which also represent prediction units and transform units. For each of the resulting blocks, a coding mode and associated prediction parameters (such as intra prediction modes or motion parameters) are transmitted and the blocks are also used for transform coding of the residual signal.
- the resulting blocks are used as coding blocks (i.e., for signaling the coding mode), but the coding blocks can be further partitioned into prediction and/or transform blocks (either using the generalized binary splitting concept or any other splitting method).
- the high-level parameters controlling the behavior of generalized binary splitting can include a minimum block size (e.g., specified by a minimum block width and a minimum block height, or specified by a minimum number of samples), a maximum partition tree depth, a maximum partition tree depth following a first non-default split, a maximum block size for a non-default split, maximum and minimum allowed aspect ratios for the resulting blocks as well as constraints regarding the applicable split ratios.
- the high-level parameters may be signaled in a picture parameter set (PPS), sequence parameter set (SPS), slice header or any other high-level syntax structure. To reduce the number of bits required for transmitting the high-level parameters, boundaries and specific domains may be defined for the high-level parameters.
- the maximum aspect ratio may be defined as a power-of-2 bigger than 4 thus significantly reducing the number of possible options.
- the minimum aspect ratio may be defined as the reciprocal of the maximum aspect ratio or vice versa, in which case only the minimum or maximum aspect ratio needs to be transmitted.
- both the split direction and the split ratio are coded using context- based adaptive arithmetic coding (CABAC).
- CABAC context- based adaptive arithmetic coding
- these parameters are coded as a series of binary decisions with adaptive binary probability models (a subset of the bins can also be coded with non-adaptive probability models).
- the used binary probability models for the split decision may be selected based on the size of neighboring blocks.
- the underlying assumption is that small blocks often occur in the neighborhood of other small blocks (i.e., in finely partitioned areas of a video picture). This or a similar assumption may also be used to model the contexts for the split ratio coding. Note that non-symmetric splits result in a smaller and a larger block and smaller blocks may tend to be located in the neighborhood of other small blocks.
- some final partitioning patterns could be generated using more than one partitioning tree.
- some combinations of splits are prohibited by the syntax. This can be achieved by prohibiting some tree patterns from being introduced into the partitioning tree.
- a tree pattern is defined using either the actual“verticar’fhorizontai” or the directional '‘parallel7"perpendicular” split direction semantics, possibly mixed.
- a tree pattern also includes split ratios.
- FIG. 1 A specific partial partitioning of a 128 x 128 block (e.g., a CTU of size 128 x 128) using the means described in this example is shown in Figure 1.
- the partitioning tree describing the shown topology is shown in Figure 2, and Figure 3 explains which split directions and ratios were used at specific nodes.
- the overall set of split ratios is given by
- split ratios a/b are given as irreducible fractions (numerator and denominator are coprime).
- symmetric split with split ratio 1/2 several non-symmetric split ratios are included in the overall set of split ratios.
- split ratio (b- a)/b is also included. Both splits yield the same subblock sizes, but for one of the splits, the larger subblock represents the left/top part, and for the other split, the larger subblock represents the right/bottom part of the block to be split.
- the symmetric split is included in the overall set of split ratios, in which case we have
- Criterion 1 Granularity of block sizes
- a split with split ratio a/b is only supported if the split size M (i.e., the width (for vertical splits) or height (for horizontal splits) of the block to be split) satisfies the following equations
- Criterion 2 Block sizes, split depth and aspect ratios
- the set of available splits can be constraint by a minimum block size. That means, if a split would results in a block size that is smaller than a minimum block size, it is prohibited and removed from the list of available splits.
- the minimum and maximum aspect ratio Another possible constraint is the minimum and maximum aspect ratio. If a split would result in any subblock with an aspect ratio (block width)/(block height) greater than a maximum aspect ratio or less than a minimum aspect ratio, the corresponding split is removed from the set of available splits.
- the minimum aspect ratio could be set equal to the reciprocal of the maximum aspect ratio.
- the maximum aspect ratio could be set equal to 8.
- the split depth represents the depth of the binary split tree structure. Or in other words, for a particular block, the split depth represents the number of splits required to reach the current block from the root block. If the split depth of a current block is already equal to the maximum split depth, no further split is allowed. As an example, the maximum split depth could be set equal to 3.
- Parameters which control the minimum block size, the minimum and maximum aspect ratio, the maximum split depth, and/or values restricting the use of non-symmetric splits are preferably transmitted in a high-level syntax structure, similar to the sequence parameter set, the picture parameter set, or the slice header of H.265
- Each restriction corresponds to one of three layouts (a, b, c), i.e., subsets of a binary splitting tree; these layouts are depicted in Fig. 7.
- the first part of Table 1 specifies an example for prohibited parallel splits.
- the parallel splits that are prohibited for the first subblock are labeled by“layout a)” and the parallel splits that are prohibited for the second subblock are labeled by“layout b)”.
- the redundancy for consecutive parallel splits could also be removed by only prohibiting splits for the first subblock, or only prohibiting splits for the second subblock, or by using another combination of prohibiting splits for the first and second subblock.
- the coding order is defined in a way that for each split, the first subblock (top or left subblock) or all blocks contained in the first subblock i s/a re coded before the second subblock (bottom or right subblock) or all block contained in the second subblock.
- the four resulting blocks would be transmitted in the following order: top-left, bottom-left, top-right, bottom-right (see Fig. 12b).
- the bitstream contains a syntax element, which indicates whether redundant perpendicular splits are allowed or prohibited by the syntax.
- the actually used split ratio syntax is depending on the value of a corresponding syntax element.
- the syntax element that indicates whether redundant perpendicular splits are allowed or prohibited could be transmitted as part of any high-level syntax structure, such as the SPS, PPS, or slice header. It is also possible that this syntax element is transmitted as part of the low-level syntax, as for example, the CTU syntax or the CU syntax.
- redundant perpendicular splits are prohibited (either by convention or indicated by a high- level syntax element), it has to be defined which of the two variants is prohibited and which of the variants is supported. This decision could depend on which variant requires less binary decisions for coding or which variant most likely provides better intra-prediction signals or motion vector predictors, etc.
- Table 1 gives an example for specifying prohibited redundant splits. The prohibited perpendicular splits are marked with“layout c)”. They have to be interpreted in the following way.
- Figure 7 shows three different partitioning tree patterns that are prohibited if specific split directions and ratios for the nodes A, B, and C are met.
- Table 1 all configurations prohibited in the described example are listed. The local topology produced by those patterns can be exactly replicated using a different partitioning tree pattern. Note that the prohibitions are chosen in correspondence to the least expensive splits.
- Layout a) with A-Ratio 3/4 and B-Ratio 2/3 yields the same pattern as Layout b) with A-Ratio 1/2 and B-Ratio 1/2.
- the former is prohibited whereas the latter is allowed due to the fact that 1/2 is less expensive in signaling (when using the signaling scheme described below).
- Method A (1) Vertical split with ratio 1/4 applied to 16x16 block at location (0,0)
- Method B (1) Vertical split with ratio 3/4 applied to 16x16 block at location (0,0)
- both methods yield the following partitioning into 5 subblocks: 4x8 block at location (0,0), 8x8 block at location (4,0), 4x8 block at location (0,8), 8x8 block at location (4,8), and 4x16 block at location (12,0).
- the coding order for both methods is usually different (in Fig. 13, the coding order is indicated by the numbers inside the resulting blocks), so that both ways of splitting actually represent different coding modes. Such higher-order redundancies could be prohibited, or they could be allowed by the syntax.
- the symmetric split 1/2 is always allowed, as long as other conditions (e.g., a minimum block size) are not violated. But the split 3/4 is only allowed if it is followed by a 1/3 split in the same direction for the first subblock. The split 1/3 is only allowed if the current block represents the first subblock of a previous 3/4 split in the same direction.
- Such a configuration/syntax would actually lead to the same partitioning scheme as QTBT with additional triple splits (triple split are splits that partition a block into three subblocks with sizes 1/4, 1/2, 1/4, see state-of-the-art [4]).
- a tree adaption set could define a specific set of leaf nodes or branches, or a set of rules to select leaf nodes or branches added and/or removed from the default tree.
- the aspect of an explicitly signaled tree adaption set might include the signaling of the use of a tree adaption set and/or the signaling of an index, selecting a tree adaption set out of a predefined record of tree adaption sets, using variable or fixed length codes embedded in any high level syntax structure.
- Implicit signaling might range from deriving the use of additional tree adaption sets as well as selecting a tree adaption set out of a record by evaluation of syntax elements coded in any high level syntax structure and/or on CTU-level or below.
- One aspect could link specific profile and level syntax elements to specific tree adaption sets, in order to limit complexity or eliminate unsupported block width, block height or block aspect ratios, which might not be compatible with certain profile-level combinations.
- a tree adaption set might be derived from syntax elements of the temporal or spatial neighborhood, restricting or expanding the default tree, according to assumed local requirements.
- an implicit tree adaption set could be activated by the extraordinary shape of a block at a picture, tile or shape boundary. This might also include the overriding of generally applied restrictions like aspect-ratio restrictions. Note that at picture or tile boundaries, a coding tree unit often has a different block size. For example, assume an HD picture of 1920x1080 luma samples and a CTU size of 128x128 luma samples. Then, the CTUs at the bottom of the picture only have a size of 128x56 luma samples. Here, while the CTUs are initially still considered to consist of 128x128 luma samples, an implicit splitting could be inferred.
- a horizontal split with split ratio 1/2 and a following horizontal split with split ratio 7/8 for the first subblock could be inferred, which results in a 128x56 block; the remaining blocks that only include samples outside the picture are not coded.
- the binary decision tree for transmitting the split ratio could also be adapted on a picture or tile boundary. For example, splits that would introduce a block border outside the picture could be eliminated for the blocks at a boundary. This concept could also be used in connection with implicit splits.
- the block is vertically (at right boundary) or horizontally (at bottom boundary) split, but for transmitting the split ratio, all splits that would introduce a block border outside the picture/tile region are eliminated from the corresponding binary decision tree (i.e., the corresponding binary decision in the tree used for signaling the split ratio are inferred to be equal to 0 or 1 ).
- the high-level syntax includes syntax elements by which the encoder tells the decoder the actual configuration for the binary splitting tree.
- the encoder complexity only allows to evaluate a subset of the splitting options that are supported by the syntax.
- the encoder signals the decoder, which subset of the splitting options are used.
- the high-level syntax elements are transmitted in one or more high-level syntax structures, such as the sequence parameter set (SPS: valid for all pictures inside a coded video sequence), the picture parameter set (PPS: valid for one or more pictures), the slice header (valid for a slice), etc.
- the high-level syntax elements that restrict the available splitting option can include one or more of the following parameters:
- a minimum block sizes e.g., specified by a minimum block width and a minimum block height, or specified by a minimum number of samples
- a maximum partition tree depth following a first non-default split i.e., the tree depth after the first split with a split ratio not equal to 1 ⁇ 2 is limited
- a maximum block size for a non-default split i.e., splits with a ratio not equal to 1/2 are only supported if the block size is greater than or equal to a given size
- a minimum block size for a non-default split i.e., splits with a ratio not equal to 1/2 are only supported if the block size is less than or equal to a given size
- Fig. 5 depicts signaling diagrams for an exemplary split size of 64 (assuming no further restrictions).
- a special algorithm for pruning the binary decision tree can be employed.
- a non-limiting example uses a special algorithm to prune the signaling tree (an example for a full decision tree is shown in fig. 5).
- the concept uses basically the same tree of binary decisions for signaling all split ratios, but before any binary decision (bin) is actually coded, the tree is pruned based on the availability of the split ratios. Or in other words, binary decisions that can be inferred to be equal to 0 or 1 based on the availability of the split ratios are not transmitted, but inferred at the decoder side.
- the outputs of the signaling tree (leaf nodes of the tree) are marked according to their availability, where the availability can be derived according to any combination of the rules described above (including, minimum block size, block granularity, minimum and maximum aspect ratio, redundant parallel splits, redundant perpendicular splits, etc.).
- Each node contains an information about the context model that is used to encode the specific bin using the CABAC engine.
- the reduction (or pruning) algorithm reduces the decision tree as follows. Initially, it is assumed that all nodes of the binary decision tree for signaling the split ratio are available. Then, the leaf nodes of the tree are marked as available or unavailable according to the availability of the corresponding split ratios (which is derived according to any rules, for example, the rules or a subset of the rules specified above). Afterwards, the decision tree is pruned from the bottom to the top according to the following two rules: (1 ) If a node has no available child node (i.e. , both child nodes are marked as unavailable), the node is marked as unavailable (i.e., it is finally removed from the decision tree).
- the assignment of probability model sets (also referred to as context model sets) to binary decision nodes is not modified for reduced decision trees. That means, for each binary decision, the same context model set (among which a probability model is selected based on additional parameters) is used, regardless of whether the binary decision is coded as part of the full decision tree or any form of a reduced decision tree.
- the depth of a tree following a non-default split is restricted.
- a first non-default split would mean that scarce partitioning is done and the last few available depths will be used to fine-adjust the block edges.
- the probability model is selected based on the current depth of the splitting tree and the sizes of the neighboring blocks. If a subdivided block has small neighboring blocks on one end and large neighboring block on the other, it is assumed that the smaller of the blocks resulting from current subdivision will be placed at the side neighboring with other small blocks.
- the probability model for the binary decision labelled with“ctx6” is selected based on how much do the sizes of neighboring blocks vary (if they vary much, it is assumed that the 1/5 or 4/5 ratio could be more probable as they result in subdivisions with higher difference of areas).
- a specific example could define the context models as follows. A split area is shown in fig. 10.
- the anchors for neighbor access are named A1 , A2, B1 , and B2.
- the neighbors are inspected (the neighbors are defined as the units anchoring at the split block according to fig. 10). If the split direction is horizontal, is the neighbor at anchor B1 and N 2 is the neighbor at anchor B2. If the split direction is vertical Ni is the neighbor at anchor A1 and /V 2 is the neighbor at anchor A2.
- the neighboring blocks have a specific areas area ⁇ and area(/V 2 ).
- the context offset ctx 0 f f is defined as 1 if area (Ay equals area(iV 2 ). If area(JV t ) is greater than area(JV 2 ) the ctx off is defined as 0. In all other cases it is defined as 2.
- the selection of the binary probability models used for coding the binary decision is based on the assumption that a partition tree of a specific CTU in a picture is similar structured compared to a corresponding partition tree of a so called collocated CTU in an already decoded picture.
- a collocated CTU might be defined as CTU that is obtained when accessing a CTU in a given reference picture using the coordinates derived from the current CTU’s position and size, for example the CTU’s center position.
- From the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element of the collocated CU might be used to select probability models for signaling split decisions (in fig.4).
- syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element from the obtained collocated CU might be used to select probability models for signaling split ratio decisions (in fig.5-fig.9).
- syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element might be used to infer one or all binary split decisions (in fig.4) without additional signaling.
- syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element from the obtained collocated CU might be used to infer one, any or all split ratio decisions (in fig.5-fig.9).
- This Lagrangian approach can also be used for deciding between different split modes. For obtaining optimal decisions, it is important to proceed the encoding using a depth-first approach.
- the cost for a certain split mode is calculated as follows. First the cost of the first subblock is determined. At this point, the best coding mode (in rate-distortion sense) for the first subblock including all possible split modes is determined. Given the best coding mode for the first subblock, the best coding mode for the second subblock is determined (again, potential splits have to be taken into account).
- the rate-distortion cost J for a split mode is the sum of the rate-distortion costs for the subblocks. This concept can be implemented using a recursive function, similarto the following (simplified) pseudo-code: function rd_cost( block ) ⁇
- an encoder does not need to evaluate all coding options supported in the syntax. For decreasing the encoder complexity, it is typically beneficial to only test the most likely splitting modes and skip the evaluation of unlikely modes. Most notably, the maximum depth of a tree after the first non-default split is a very important parameter to balance the expected rd-performance and runtime. The maximum block size at which a non-default split is allowed can be used to restrict the testing of partitionings with large blocks, when much smaller blocks are more probable (e.g. for intra slices). Furthermore, the maximum aspect ratio tested in an encoder may be smaller than that supported by the syntax (correspondingly, the minimum aspect ratio tested may be larger than the minimum aspect ratio supported by the syntax).
- the generalized binary splitting framework for partitioning blocks into coding units, prediction units, transform units, etc. may affect other parts of a video codec.
- Another aspect is related to a coding of the last non-zero transform coefficient in a transform block.
- the position of the last non-zero transform coefficient is indicated by transmitting the x-coordinate and the y-coordinate of the last non-zero transform coefficient.
- the corresponding entropy codes were designed for blocks for which both the width and height represent integer powers of two.
- the binarization for the syntax elements last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix is modified as follows:
- the binarization codes for TrafoSize equal to an integer power of 2 are not changed. Note that version 1 includes redundancies (unnecessary bins for certain codewords), while the codeword tables for version 2 do not include redundant bins.
- the new partition structure requires various new transforms among them Hadamard transform matrices for prediction residuals.
- the Hadamard transform is used as a very simple approximation of a DOT.
- the sum of absolute Hadamard coefficients is, for example, used as rate- distortion measure in the sub-sample motion search or the pre-selection of intra prediction modes.
- the new luma block dimensions 96, 80, 48, 40, 24, 20, and 12 appear if the maximal CTU size is 128.
- the corresponding Hadamard matrices are not all explicitly constructed. In fact it is sufficient to construct Hadamard matrices of size 20 and 12 using Paley’s construction via finite fields.
- the other sizes are then constructed by Kronecker products since Kronecker products of Hadamard matrices are Hadamard matrices again.
- the construction of the other matrices via Kronecker products have the advantage that a full large sized Hadamard matrix never has to be constructed entirely.
- the number and topology of neighbors of a particular block may vary. Coding parameters of neighboring block are, for example, used for deriving merge candidates, motion vector prediction candidates, or most probable intra prediction modes.
- an adaption of the state-of-the-art strategy for selecting candidates among the neighboring blocks could led to an improved coding efficiency.
- the state-of-the-art strategies often implicitly assume that blocks have a square shape. But with the generalized splitting concept, the block width can be much larger than the block height or vice versa.
- the long block edge it can be preferable to consider more candidate block along the long block edge than along the short block edge.
- This may include but is not limited to: Scanning for all neighbors along the edges of a block (e.g. to find an extremum or a mean of a measure derived from the neighboring blocks); defining additional neighbor anchor points (e.g. in the case of scanning for merge candidates, adding anchors in the middle of the left and upper edge of a block).
- encoder side operation may be restricted in a trade-off between complexity and coding efficiency, e.g. through reduction of considered modes.
- restrictions can be facilitated in the signaling to reduce necessary bits, e.g. through implicit derivation of syntax elements on decoder side.
- the following aspects introduce embodiments and examples for further restrictions and extensions.
- a size of a block may be restricted to powers of 2, i.e. the binary logarithm of height and width in number of samples must be an integer number, which blocks may then also be referred to as log2 partitions.
- such blocks may comprise (in number of samples) a height and/or a width of, for example, 2°, 2 ⁇ 2 2 , .... 2" samples.
- a block comprising 64 x 128 samples (height x width) may be referred to as a log2-partition
- a block comprising 64 x 72 samples may be referred to as a non-log2-partition.
- the signaling of specific split decisions on encoder side may be omitted in favor of implicit derivation on decoder side.
- a node of a splitting tree is a non-log2 partition with respect to either its height or width
- further splitting is inevitable to progress towards log2 partitions that can host a CU as per the given constraint.
- the encoder may perform an implicit split of a non-log2-partition in order to create two or more log2-partitions.
- Split indications for such implicit splits of non-log2 partitions can hence be inferred at decoder side and explicitly signaling is necessary only for log2 partitions.
- Figure 14 shows an embodiment of an inventive encoder 10 for performing such implicit splits.
- the encoder 10 is a block-based encoder for block-based encoding a picture 12 into a data stream 14.
- the block-based encoder 10 is configured to partition the picture 12 into one or more leaf blocks 15 using recursive multi-tree partitioning along a binary splitting tree 13, wherein a height and a width of the leaf blocks 133, 13 4 , 13s, 13e, 13s, 13g in number of samples is a power of two.
- the blocks in which the picture 12 is partitioned may also be referred to as partitions.
- Each of the nodes 13i, 13 2 , .... 13g of the splitting tree 13 may belong to one partition.
- the partitioning of the picture 12 may comprise four root partitions, e.g. four CTUs.
- the partitioning of the picture 12 may comprise four root partitions, e.g. four CTUs.
- Figure 14 only the CTU in the upper right corner of the picture 12 will be exemplarily described in more detail now.
- the splitting tree 13 may comprise a root node 13i at a first partitioning level, or root block level, which root node 13i belongs to the aforementioned CTU.
- This root block may be partitioned along the splitting tree 13 to one or more leaf blocks 13 3 , 13 4 , 13 5 , 13 e , 13 e ,
- the aforementioned CTU may first be split, at a first partitioning level, by a quad split (QT) such that four subblocks result.
- QT quad split
- the reference numbers of the respective tree nodes of the splitting tree 13 are indicated inside each block of the picture 12 to which the respective tree node belongs.
- the blocks to which the tree nodes 13 3 , 13 4 , 13s belong may already be leaf blocks, wherein no further partitioning takes part.
- these blocks to which the tree nodes 13 3 , 13 4 , 13s belong may be Iog2-partitions.
- the block to which the tree node 13 2 belongs may not yet be a leaf block.
- this block is further partitioned, at the second partitioning level, using a bi-split, for instance a BT or a GBS split. Accordingly, the tree node 13 2 may be partitioned at the second partitioning level resulting into two further nodes 13e, 13 7 .
- the block to which the tree node 13e belongs may be a leaf block, wherein no further partitioning takes part.
- the block to which the tree node 13 6 belongs may be a log2-partition.
- the block to which the tree node 13 7 belongs may not yet be a leaf block.
- this block may be a non-log2 partition. Since it is not yet a leaf block, it will therefore be subjected to further partitioning.
- this block is further partitioned using a bi-split, for instance a BT or a GBS split. Accordingly, the tree node 13e may be partitioned at the third partitioning level resulting into two further nodes 13 e , 13 9 .
- the blocks to which the tree nodes 13 0 , 13g belong may be leaf blocks, wherein no further partitioning takes part. According to the convention of this aspect, these blocks to which the tree nodes 13s, 13g belong may be log2-partitions.
- the encoder 10 may explicitly signal log2-partitions in the data stream 14. However, the encoder 10 may implicitly split non-log2-partitions further in order to create two or more log2-partitions at subsequent levels. The one or more splits are done implicitly. That is, for said one or more implicit splits the encoder 10 refrains from signaling such implicit splits in the data stream 14. Therefore, the encoder 10 may have to check whether a partition is a log2-partition or a non-log2-partition.
- the encoder 10 of this aspect is configured to signal partitioning
- the binary splitting tree 13 in the data stream 14 by checking, for a tree node 13i, 13 2 , ... , 13 9 of the binary splitting tree 13 comprising a set of split options including a no-split option and a split option, whether a partition to which the respective tree node 13i, 13 2 , ... , 13 0 belongs, has a height and a width which are in number of samples a power of two.
- the encoder 10 is configured to signal the split option or the no-split option in the data stream 14.
- the encoder 10 is configured to refrain from signaling the split option in the data stream 14.
- Figure 15 shows the corresponding decoder 110 of this aspect.
- the decoder 1 10 is a block-based decoder for block-based decoding a picture 12 from a data stream 14.
- the decoder 110 reconstructs the original picture, except for possible quantization losses, such that a reconstructed picture 12' may be output by the decoder 1 10.
- the encoder 10 may split the log2-partition and signal the split option in the data stream 14. Alternatively, the encoder may 10 not split the log2-partition (e.g. if it is already a leaf block) and signal the no-split option in the data stream 14.
- the encoder 10 may implicitly split the non-log2-partition so as to create two or more subsequent log2-partitions at a subsequent partition level. Since this split is implicit, the encoder 10 may not signal any split option in the data stream 10. Thus, the decoder 110 may infer the split option itself.
- the encoder 10 may be configured to split the partition and to signal, in the data stream 14, the applied split option, or not split the partition and to signal the no-split option in the data stream 14.
- the encoder 10 is configured to split the partition such that a resulting sub-partition belonging to a tree node 13 2 , ... , 13 9 at a subsequent tree level has a height and a width which are in number of samples a power of two, and refrain from signaling the applied split option in the data stream 14.
- the decoder 110 is configured to partition the picture 12 into one or more leaf blocks 13 3 , 13 4 , 13s, 13 e , 13 8 , 13g using recursive multi-tree partitioning along a binary splitting tree 13, wherein a height and a width of the leaf blocks 13 3 , 13 4 , 13 5 , 13 6 , 13 8 , 13 9 in number of samples is a power of two.
- the decoder 1 10 may check whether a partition to which a tree node 13i, 13 2 , .... 13 9 of the splitting tree 13 belongs is a log2-partition or a non-log2- partition. If it is a log2-partition, then the decoder 110 may read from the data stream 14 an indication of the split option, e.g. which split mode and/or which split ratio to apply to said partition. In other words, the decoder 110 may obtain a split-option-signal from the data stream 14.
- the decoder 1 10 may implicitly infer, without obtaining an explicit split-option-signal from the data stream 14, the split option to be applied to the current partition.
- the decoder 1 10 is generally configured to obtain partitioning information of the binary splitting tree 13 from the data stream 14 for a tree node 1 3i , 13 2 , . . . . 1 3g of the binary splitting tree 13, wherein the tree node 13 comprises a set of split options including a no-split option and a split option.
- the decoder 110 may do so by checking whether a partition to which the tree node 13i, 13 2 , .... 13g belongs, has a height and a width which are in number of samples a power of two.
- the decoder 110 is configured to obtain, from the data stream 14, the split option or the no-split option for splitting the current partition.
- the decoder 110 is configured to implicitly infer the split option without obtaining an explicit split-option-signal from the data stream 14.
- the decoder 110 may split said log2-partition based on a split signal in the data stream 14. Alternatively, the decoder 110 may not split said partition if a no-split signal is signaled in the data stream 14.
- the decoder 110 may split said non-log2-partition into two or more log2-partitions on a next partitioning level. For this splitting, the decoder 1 10 may use a split option is implicitly inferred. That is, the decoder 110 may implicitly infer a split option and the decoder 110 may not obtain an explicit split-option-signal from the data stream 14 for the current non-log2- partition.
- the decoder 110 is configured to split the partition based on a split signal in the data stream 14, or not split the partition based on a no-split signal in the data stream 14. If the partition to which the tree node 13i , 13 2 , ... , 13g belongs does not have a height and/or a width which are in number of samples a power of two, i.e.
- the decoder 110 is configured to implicitly infer, without an explicit split-option- signal in the data stream 14, a split option that results in a sub-partition belonging to a tree node 13 2 , .... 13 9 at a subsequent tree level, which sub-partition has a height and a width which are in number of samples a power of two, and split the partition into the sub-partition according to said inferred split option.
- a conventional GBS partitioning would require indication of each split individually, e.g. starting with a N/4, 3N/4 - split indicated in the left side of the 64 x 64 block in Figure 16A and followed by a 2N/3, N/3 - split indicated in the right side of the 64 x 64 block in Figure 16A.
- a non-log2-partition such as the 3 N/4 partition resulting from the first split, may mandatorily be split by a parallel split into two log2-partitions with the split (and possibly the splitting ratio indication) being implicitly derived at decoder side.
- the described mandatory consecutive split could be restricted to certain node sizes, e.g. towards leaf level of the split tree.
- Figure 16B shows an example according to this aspect.
- the left block 164 in Figure 16B represents a first partition.
- Said first block 164 is again a 64 x 64 block that is split by a first vertical split having a split ratio of N/4, 3 N/4 into a first sub-partition 161 having a size of N/4, i.e. 64 x 16, and into a second sub-partition 162 having a size of 3N/4, i.e. 64 x 48.
- the first sub-partition 161 is a log2-partition
- the second sub-partition 162 is a non- log2-partition.
- the second sub-partition 162 is implicitly split into two further sub-partitions 162a, 162b such that these two resulting further sub-partitions 162a, 162b are log2-partitions.
- the second sub-partition 162 may be implicitly split by an implicit split having a split ratio of 2 N/3, N/3.
- the block 165 as depicted on the right side of Figure 16B may result therefrom. That is, the block 165 may comprise, after said implicit split, three partitions 161 , 162a, 162b, wherein each partition 161 , 162a, 162b is a log2-partition.
- the encoder of this aspect may refrain from signaling a split indication (e.g. split mode and split ratio) of the implicit second split in the data stream 14.
- the decoder 1 may infer a split indication (e.g. split mode and split ratio) for the implicit second split without obtaining an explicit split-indication-signal from the data stream 14, since the second split is an implicit split for splitting the non-log2-partition 162 into two or more log2-partitions 162a, 162b.
- the encoder 10 may be configured to split a predetermined partition 164 into at least three sub-partitions 161 , 162a, 162b, each of which having a height and a width which are in number of samples a power of two.
- the encoder 10 may do so by applying a first split to the predetermined partition 164, wherein said first split creates a first sub-partition 162 which does not have a height and a width which are in number of samples a power of two.
- the encoder 10 may signal the applied split option of the first split in the data stream 14.
- the encoder 10 is configured to subsequently apply a consecutive second split to the first sub-partition 162, wherein said consecutive second split creates a second and a third sub-partition 162a, 162b, each of which having a height and a width which are in number of samples a power of two. In this case, the encoder 10 may refrain from signaling the applied split option of the consecutive second split in the data stream 14.
- the decoder 1 10 may be configured to split a predetermined partition 164 into at least three sub-partitions 161 , 162a, 162b, each of which having a height and a width which are in number of samples a power of two.
- the decoder 1 10 may do so by obtaining a first split option from the data stream 14 and applying a first split according to said first split option to the predetermined partition 164, wherein said first split option creates a first sub-partition 162 which does not have a height and a width which are in number of samples a power of two.
- the decoder 110 is configured to implicitly infer, without an explicit split- option-signal in the data stream 14, a split option for a subsequent second split and to subsequently apply said second split according to the split option to the first sub-partition 162, wherein said second split creates a second and a third sub-partition 162a, 162b, each of which having a height and a width which are in number of samples a power of two.
- the implicit second split is a parallel split relative to the first split.
- the encoder 10 may be configured to choose a parallel split as the split option for the consecutive second split, wherein the consecutive second split is parallel to the first split.
- the decoder 110 may be configured to choose a parallel split as the split option for the consecutive second split, wherein the consecutive second split is parallel to the first split. Again, the consecutive second split is an implicit split, wherein the split option may be inferred from the decoder 110 without an explicit split-option-signaling in the data stream 14.
- the split option may represent an option whether a split shall be applied or not. Accordingly, the split option may either indicate whether a split or a no split shall be performed. Therefore, the split option may comprise a split indication or a no-split indication. The split option may be contained in the partitioning information.
- the partitioning information may also comprise information which kind of split to apply, e.g. a quad split, a binary split, a generalized binary split, a triple split, or the like. This may also be referred to as the split mode.
- the partitioning information may also comprise information about the split ratio.
- the encoder may refrain from encoding a split mode and/or a split ratio indication into the data stream. Accordingly, for implicit splits, the decoder may infer the split mode and/or the split ratio without any explicit split mode signal and/or split ratio signal in the data stream.
- the partitioning information may comprise at least one of a split mode and a split ratio
- the encoder 10 may be configured to omit signaling in the data stream 14 at least one of the split mode and the split ratio for the partition to which the respective tree node 13i, 13 2 . 13 9 belongs, if said partition does not have a height and a width which are in number of samples a power of two.
- the decoder 1 10 may accordingly be configured to infer from the data stream 14, without an explicit signaling in the data stream 14, at least one of the split mode and the split ratio for the partition to which the respective tree node 13i, 13 2 , ... , 13 9 belongs, if said partition does not have a height and a width which are in number of samples a power of two.
- implicit splits shall only be applied to tree nodes at a certain tree depth level, or in terms of blocks, the implicit splits shall only be applied to partitions having a predetermined size, for example a size that lies above a minimum block size to be coded. This minimum block size may be pre-defined.
- the encoder 10 may be configured to refrain from signaling the split option in the data stream 14 only if the partition or sub-partition, which shall be split according to said split option, comprises a predetermined size or if the corresponding tree node 13i, 13 2 , .... 13g of said partition or sub-partition is at a predetermined tree-depth level of the binary tree 13.
- the decoder 1 10 may be configured to implicitly infer the split option only if the partition or sub-partition, which shall be split according to the split option, comprises a predetermined size or if the corresponding tree node 13i, 13 2 , .... 13 9 of said partition or sub-partition is at a predetermined tree-depth level of the binary tree 13.
- the implicit splits may be performed towards a leaf block level. That is, the closer the current partitioning level is to the leaf block level, the more probable an implicit split will be.
- the above mentioned predetermined tree-depth level may be the level of the last tree node before a tree node belonging to a leaf block. That is, the implicit splits may only be performed for the last block that, if split, may lead to a leaf block.
- the above mentioned predetermined tree-depth level may be one level or two levels above the last tree node before a leaf block.
- the implicit splits may only be performed for the penultimate, i.e. the second to last, or even for the third to last level before the tree node belonging to a leaf block.
- Figure 17 shows a block diagram of a method for encoding a picture 12 into the data stream 14 according to the above described concept.
- the picture 12 is partitioned into one or more leaf blocks using recursive multitree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two. That is, the picture 12 is partitioned into blocks, wherein a leaf block is a log2-partition.
- partitioning information of the binary splitting tree 13 is signaled in the data stream 14 by checking, for a tree node 13i, 13 2 , .... 13 9 of the binary splitting tree 13 comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node 13i, 13 2 , .... 13 9 belongs, has a height and a width which are in number of samples a power of two, i.e. whether the respective partition is a log2- partition.
- the block diagram proceeds to a selection node 1703. If yes, i.e. if the current partition is a log2-partition, the method proceeds to block 1704 according to which the split option or the no-split option for said partition is signaled in the data stream 14.
- the method proceeds to block 1705 according to which it is refrained from signaling the split option in the data stream 14, i.e. the split option is not signaled in the data stream 14.
- Figure 18 shows a block diagram of a method for decoding a picture 12 from the data stream 14 according to the above described concept.
- the picture 12 is partitioned into one or more leaf blocks using recursive multitree partitioning along a binary splitting tree 13, wherein a height and a width of the leaf blocks in number of samples is a power of two. That is, the picture 12 is partitioned into blocks, wherein a leaf block is a log2-partition.
- partitioning information of the binary splitting tree 13 is obtained from the data stream 14 by checking, for a tree node 13i , 13 2 , .... 13g of the binary splitting tree 13 comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node 13i, 13 2 , ... , 13 9 belongs, has a height and a width which are in number of samples a power of two, i.e. by checking whether the current partition is a log2-partition.
- the block diagram proceeds to a selection node 1803. If yes, i.e. if the current partition is a log2-partition, the method proceeds to block 1804 according to which the split option or the no-split option for said partition is obtained from the data stream 14. If not, i.e. if the partition is a non-log2-partition, the method proceeds to block 1805 according to which the split option is implicitly inferred from the data stream 14 without an explicit split-option-signal in the data stream 14. That is, the split option is not obtained from the data stream 14.
- Another aspect of the invention targets to constrain a GBS based codec towards a better trade-off between coding efficiency and complexity.
- a state-of-the-art approach to control this trade-off is to restrict min/max depth of trees when searching and signaling, where tree depth is the maximum distance from a leaf node 13g to the root node 13i.
- a splitting indication can be inferred when a block does not yet fulfill the minimum tree depth and the splitting indication can be ruled out once the maximum tree depth is reached.
- the invention in this aspect is to differentiate tree nodes 1 3i , 13 2 , . . . . 13 9 when evaluating a depth of a tree 13,
- the characteristics by which tree nodes 13i, 13 2 , .... 13g could be handled differently could encompass but are not limited to: a) whether split signaling is implicit for a node 13i , 13 2 , .... 13g b) whether node size belongs to a given set of node sizes, the set being either predefined (e.g. sizes below 16 x 16) or dynamic (e.g. node sizes which presence in previously coded blocks lies below a threshold) c) whether a node 13i, 13 2 , .... 13g is disallowed to contain a CU
- the depth of a given node may be calculated differently, e.g. by omitting or weighting the count of nodes that adhere to one of the above characteristics.
- search space is altered in an adaptive fashion adaptive to the given tree characteristics.
- an encoder 10 for encoding a picture 12 into a data stream 14 is presented, wherein the encoder 10 is configured to partition the picture 12 into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree 13, as it was previously explained above with reference to Figure 14.
- the splitting tree 13 comprises a set of split options including at least one of a no-split option and a split option.
- a decoder 1 10 for decoding a picture 12 from the data stream 14 is presented, wherein the decoder 110 is configured to partition the picture 12 into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree 13, as it was previously explained above with reference to Figure 15.
- the splitting tree 13 comprises a set of split options including at least one of a no-split option and a split option.
- the encoder 12 and the decoder 100 may be configured to check, for a predetermined tree node 13i, 13 2 , .... 13g of the splitting tree 13, whether said predetermined tree node 13i, 13 2 , .... 13 9 meets a predetermined tree depth constraint.
- This check of the encoder 10 may depend on a distance of said predetermined tree node 13i, 13 2 , ... , 13g from a root node 13i of the splitting tree 13, or a number of splits of a partition to which said predetermined tree node 13i, 13 2 , .... 13 9 belongs along the splitting tree 13.
- the encoder 10 and the decoder 110 may evaluate said characteristics, e.g.: a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node 13 2 , .... 13 9 is connected to the root node 13i, are implicitly signaled or explicitly signaled, as described above and/or b) depending on a size of a partition to which the predetermined tree node 13i, 13 2 , ....
- 13g belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not.
- the encoder 10 and the decoder 110 may be configured to rule out the one of the no-split option and the split option, i.e. depending on the check whether the predetermined tree node meets 1 3i , 13 2 , ... . 1 3 9 the predetermined tree depth constraint.
- the depth of a given node 13i, 13 2 , 13 9 may be calculated differently, e.g. by omitting or weighting the count of nodes that adhere to one of the above characteristics. Thereby, search space is altered in an adaptive fashion adaptive to the given tree characteristics.
- both the encoder 10 and the decoder 1 10 may be configured to calculate the tree depth of the predetermined node 13i, 13 2 , . . . , 13 9 differently, namely depending on the above check whether the predetermined tree node 13i, 13 2 , .... 13g meets the predetermined tree depth constraint, wherein the calculating is performed by omitting or weighting the count of tree nodes 13i, 13 2 , .... 13g along the splitting tree if the predetermined tree node 13i, 13 2 , .... 13g adheres to at least one of the options a), b) or c).
- the encoder 10 and the decoder 110 may skip this tree node 13i, 13 2 , .... 13 s in counting the tree nodes along the splitting tree 13. Additionally or alternatively, the encoder 10 and the decoder 110 may weight the count of a predetermined tree node 13i, 13 2 , ... , 13g in counting the tree nodes along the splitting tree 13, wherein the weight may depend on the respective criterion to which the predetermined tree node 13i, 13 2 , ... , 13g adheres.
- the encoder 10 may be configured to, in discriminating according to the above mentioned option c), discriminate whether the partition has a height and a width which in number of samples is a power of two, i.e. whether the partition is a log2-partition.
- the partition is alog2-partition, it is allowed to be a leaf block.
- the partition is disallowed to be a leaf block and should be splitted at least one more time.
- the encoder may be configured to explicitly signal an indication in the data stream 14 for indicating whether a current partition is allowed to be a leaf block or not. For example, if the discrimination according to option c) reveals that the partition is allowed to be a leaf block, the encoder 10 may be configured to signal an indication in the data stream 14 for indicating that the partition is allowed to be a leaf block. Additionally or alternatively, if the discrimination according to option c) reveals that the partition is not allowed to be a leaf block, the encoder 10 may be configured to signal an indication in the data stream 14 for indicating that the partition is disallowed to be a leaf block. According to a further example, if the predetermined tree node 13i, 13 2 , ....
- the encoder 10 may be configured to omit signaling a split indication in the data stream 14. That is, if the current tree node 13i, 13 2 , .... 13g has not yet reached a minimum partitioning level in the splitting tree 13, the encoder 10 may perform implicit splits, as described above.
- the decoder 110 may infer a split option for said predetermined node 13i, 13 2 , . . . . 1 3g, i.e. the decoder 110 does not obtain a split option from the data stream 14 since the encoder 10 did not encode any split-option signal into the data stream 14.
- the decoder 110 may be configured to infer a split option without an explicit spiit-option-signal in the data stream 14. That is, the decoder 110 may perform an implicit split, as described above.
- the encoder 10 may be configured to rule out the one of the no-split option and the split option. In other words, the encoder 10 may not further provide the one of the no- split option and the split option for splitting the current partition.
- the decoder 1 10 may be configured to rule out the one of the no-split option and the split option. In other words, the decoder 110 may not further use the one of the no-split option and the split option for splitting the current partition.
- the encoder 10 may check whether the predetermined tree node 13i, 13 2 , .... 1 3g meets the predetermined tree depth constraint according to criterion b), i.e. b) depending on a size of the partition to which the predetermined tree node 13i, 13 2 , . . . . 1 3g belongs.
- the encoder 10 may check whether the size of the predetermined node 1 3i , 13 2 , . . . . 1 3g belongs to a given set of node sizes, the set being either pre-defined (e.g. sizes below 16 x 16) or dynamic (e.g. node sizes which presence in previously coded blocks lies below a threshold).
- both the encoder 10 and the decoder 110 may be configured to, in checking whether the predetermined tree node 13i, 13 2 , ... , 13Q meets the predetermined tree depth constraint according to b) depending on a size of a partition to which the predetermined tree node 13i, 13 2 , .... 13g belongs, check whether the size of the partition to which the predetermined tree node 13i, 13 2 , .... 13g belongs is equal to a predetermined partition size contained in a set of partition sizes.
- the above mentioned set of partition sizes may be static, wherein the set of partition sizes comprises one or more partition sizes, wherein each partition size has a static, i.e. fixed value.
- the above mentioned set of partition sizes may be dynamic, wherein the set of partition sizes comprises one or more partition sizes, each partition size having a dynamically changeable value, wherein said value may depend on a variable predetermined threshold (e.g. node sizes which presence in previously coded blocks lies below said threshold).
- a variable predetermined threshold e.g. node sizes which presence in previously coded blocks lies below said threshold.
- Figure 19 shows a block diagram representing a method for encoding and a method for decoding a picture into/from a data stream 14, according to the above described concept.
- the picture 12 is partitioned into one or more leaf blocks using recursive multitree partitioning along a splitting tree 13, by checking, for a predetermined tree node 13i, 13 2 , ... , 13g of the splitting tree 13, wherein the splitting tree 13 comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node 13 2 , 13g meets a predetermined tree depth constraint.
- Checking the predetermined tree depth constraint may depend on block 1902 in which a distance of said predetermined tree node 13i, 13 2 , .... 13g from a root node 13i of the splitting tree 13, or a number of splits of a partition to which said predetermined tree node 13i, 13 2 , ... , 1 3g belongs along the splitting tree 13.
- the predetermined tree node 13i, 13 2 , ... , 13g may further comprise one or more characteristics depending on which the predetermined tree node 13i, 13 13g may be handled differently by the encoder 10 and/or the decoder 1 10. These characteristics may be determined by the encoder 10 and the decoder 1 10, respectively, in block 1903a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or in block 1903b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or in block 1903c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not.
- the method then proceeds to block 1904 according to which the method further comprises a step of ruling out
- the state-of-the-art partitioning method QTBT describes an ordered combination of two partitioning tree types (namely QT and BT).
- the two tree types are used consecutively in the partitioning of large blocks into smaller blocks where the first level uses a QT partitioning followed by a second level of BT.
- Each tree type in QTBT is given with a corresponding set of constraints such as min and max tree depth, mode restrictions and so on.
- TTS tree type sets
- N levels of partitioning tree types out of a plurality of tree types such as QT, BT, GBS and so on, each with a corresponding set of constraints.
- Signaling of a TTS can be carried in any high-level syntax structure such as parameter sets or slice headers.
- Each coding partition such as tiles, slices, or CTUs within a bitstream can hence reference and thereby activate one of the available TTS, e.g. in high- level syntax structures such as parameter sets, slice header or also on CTU level.
- an encoder 10 for encoding a picture 12 into a data stream 14 wherein the encoder 10 is a block-based encoder.
- the encoder 10 is configured to partition the picture 12 along a splitting tree 13 from a tree root block 13i, at a root block partitioning level, into one or more leaf blocks 133, 13 4 , 13s, 13e, 13a, 13g, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, such as described above with reference to Figure 14.
- the encoder 10 is configured to select, for performing the multi-tree partitioning, a tree type set (in the following also referred to as TTS) comprising one or more tree types. Said tree type sets may be explained in more detail with reference to Figures 20 to 22.
- the encoder 10 of this aspect is further configured to signal the tree type set in the data stream 14, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level. That is, the signaling may be done at the root block partitioning level, and/or at a leaf block partitioning level, and/or at any level between the root block and leaf block partitioning levels.
- the decoder 110 may obtain said signaling from the data stream 14 in order to select the correct TTS for decoding the blocks of the picture using the one or more tree types contained in the signaled selected TTS.
- a decoder 110 for decoding the picture 12 from the data stream 14 is provided, wherein the decoder 110 is a block-based decoder.
- the decoder 110 is configured to partition the picture 12 along a splitting tree 13 from a tree root block 13i , at a root block partitioning level, into one or more leaf blocks 13s, 13 4 , 13s, 13e, 13e, 13g, each at a respective leaf block partitioning level, using recursive multitree partitioning, such as described above with reference to Figure 15.
- the decoder 110 is configured to perform the multi-tree partitioning using a tree type set (in the following also referred to as TTS) comprising one or more tree types.
- TTS tree type set
- the decoder 110 is configured to perform said multi-tree partitioning based on a signaling of the tree type set in the data stream 14, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level. That is, the signaling of the tree type set may be obtained from the data stream 14 at the root block partitioning level, and/or at a leaf block partitioning level, and/or at any level between the root block and leaf block partitioning levels.
- Figure 20 illustrates a first embodiment of the present concept with signaling on CTU level.
- FIG 20 shows a splitting tree 13 similar to the one described above with reference to Figures 14 and 15.
- the splitting tree 13 may comprise a root node 13i, wherein the root node 13i is at a first partitioning level 21 1 , which may therefore also be referred to as the root block partitioning level 21 1.
- the root block belonging to the root node 13i may be a CTU.
- a first split of the root block may be performed, i.e. the root node 13i may be partitioned at the root block partitioning level 211 into two or more further tree nodes 13 2 , 13 3 , 13 4 , 13s at a resulting second partitioning level 212.
- the tree nodes 13 3 , 13 4 , 13s may be leaf nodes which may not be partitioned any further.
- the tree node 13 2 at the second partitioning level 212 may be further partitioned along the splitting tree 13 which may result in two or more further nodes 13 6 , 13 7 at a resulting third partitioning level 213.
- the tree nodes 13 6 , 13 7 may be leaf nodes and the third partitioning level 213 may accordingly also be referred to as a leaf block partitioning level.
- the encoder 10 may select one or more partitioning parameters for splitting the blocks into subblocks.
- the encoder 10 may select one or more tree types for partitioning the tree nodes 13i , 13 2 , ... , 13 9 belonging to a respective block or subblock from one partitioning level to another partitioning level.
- a partitioning parameter set 201 may be provided which may, for instance, comprise one or more tree type set (TTS) 201A, 201 B, 201C.
- TTS tree type set
- Each TTS 201A, 201 B, 201 C may comprise one or more tree types.
- a first TTS 201 A may comprise one tree type, e.g. a quad split tree type (QT).
- QT quad split tree type
- a second TTS 201 B may also comprise one tree type, which may be the same tree type as contained in the first TTS 201A or a different tree type, e.g. a generalized binary split tree type (GBS).
- a third TTS 201 C may comprise two tree types, e.g. a QT and a GBS.
- the encoder 10 may select the third TTS 201 C for partitioning the tree nodes 13i, 13 2 , .... 13g along the splitting tree 13.
- the first partitioning of the root node 13i at the root block partitioning level 211 may be performed by using a QT.
- the root node 13i may be partitioned into four sub-nodes 13 2 , 13 3 13 4I 13 5 .
- the nodes 13i, 13 2 , 13g in the splitting tree 13 may also be referred to as partitions.
- the selected TTS 201 C provides a GBS tree type. Accordingly, the sub-node 13 2 is partitioned at the second partitioning level 212 by using a GBS tree type, such that two further sub-nodes 13e, 13 7 result therefrom at the third partitioning level 213. Since the two further sub-nodes 13e, 13 7 are leaf blocks, the third partitioning level is a leaf block partitioning level.
- the selected TTS 201 C comprises two tree types - QT and GBS - and the splitting tree 13 comprises three partitioning levels 211 , 212, 213, wherein a first partitioning is performed between the first and the second partitioning level 211 , 212 and a second partitioning is performed between the second and the third partitioning level 212, 213.
- the TTS 201 C comprises a number of tree types which number of tree types is equal to a number of partitioning levels 211 , 212, 213 minus one.
- the splitting tree 13 comprises a number of three partitioning levels 211 , 212,
- the selected TTS 201 C comprises a number of two tree types (QT, GBS).
- the encoder 10 may be configured to select a tree type set 201 C comprising a number of tree types which number is equal to a number of partitioning levels 211 , 212, 213 minus one.
- the selected TTS 201 C may be signaled at one or more partitioning levels 21 1 , 212, 213.
- the selected TTS 201C may be signaled in the data stream 14 at the root block partitioning level 211.
- a further embodiment provides a corresponding decoder 110, wherein the decoder 1 10 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 (or alternatively at one or more other partitioning levels 212, 213), a tree type set 201 C comprising a number of tree types which number of tree types is equal to a number of partitioning levels minus one.
- the decoder 110 may be configured to use, at each partitioning level 211 , 212 except the leaf block partitioning level, one of the tree types contained in the tree type set 201 C for performing the multi-tree partitioning.
- the encoder 10 may further be configured to use, at each partitioning level except the leaf block partitioning level 213, i.e. at the first partitioning level 211 and at the second partitioning level 212. one of the tree types contained in the tree type set 201 C.
- the encoder 10 is further configured to signal the selected tree type set 201 C in the data stream 14, at the root block partitioning level 211.
- an embodiment provides an encoder 10 which may be configured to signal in the data stream 14 the selected tree type set 201 C, at the root block partitioning level 211 , and to use the one or more tree types (e.g. QT, GBS) contained in the selected tree type set 201 C for partitioning the picture 12 from the tree root block into the one or more leaf blocks, i.e. for performing the multi-tree partitioning from the root node 13i to the one or more leaf nodes 13 3 , 13 4 , 13 5 , 13 6 , 13 7 .
- tree types e.g. QT, GBS
- a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 , the tree type set 201 C for performing the multi-tree partitioning, and to use the one or more tree types contained in the tree type set 201 C for partitioning the picture 12 from the tree root block into the one or more leaf blocks, i.e. for performing the multi-tree partitioning from the root node 13i to the one or more leaf nodes 13 3 , 13 4 , 13S, 13 6 , I SA .
- a further embodiment provides an encoder 10 that may be configured to select a tree type set 201 C comprising a first tree type, e.g. QT, and a different second tree type, e.g. GBS.
- a first tree type e.g. QT
- a different second tree type e.g. GBS.
- the encoder 10 may further be configured to use the first tree type (e.g. QT) for performing the multi-tree partitioning at a first partitioning level, for example at the root block partitioning level 211 , and to use the second tree type (e.g. GBS) for performing the multi-tree partitioning at a different second partitioning level, for example at the subsequent second partitioning level 212.
- first tree type e.g. QT
- second tree type e.g. GBS
- the encoder 10 may further be configured to signal the selected tree type set 201 C in the data stream 14, at the root block partitioning level 211.
- a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 , a tree type set 201 C comprising a first tree type and a different second tree type (e.g. QT and GBS).
- the decoder 1 10 may further be configured to use the first tree type (e.g. QT) for performing the multi-tree partitioning at a first partitioning level 211 , and to use the second tree type (e.g. GBS) for performing the multi-tree partitioning at a different second partitioning level 212.
- the first tree type e.g. QT
- GBS second tree type
- the tree types may be arranged in an order, e.g. the first tree type (e.g. QT) may be arranged prior to the second tree type (e.g. GBS).
- the tree types contained in the TTS 201 C are used in the splitting tree 13 in the order in which they are arranged in the TTS 201 C. That is, the first tree type (e.g. QT) was used at the first partitioning level 21 1 and the consecutive second tree type (e.g. GBS) was used at the second partitioning level 212.
- the first tree type e.g. QT
- the consecutive second tree type e.g. GBS
- the one or more tree types contained in a tree type set 201 C may be arranged in a consecutive order, wherein both the encoder 10 and the decoder 110 may be configured to consecutively use, at each partitioning level 211, 212 except the leaf block partitioning level 213, one of the one or more consecutively arranged tree types in the order as they are arranged in the tree type set 201 C.
- the third TTS 201C may comprise three tree types, e.g. QT, BT and GBS.
- the focus shall first be directed to the two tree types which are depicted in parentheses, i.e“BT or GBS".
- the tree types depicted in parentheses shall indicate that these two tree types may both be available at the same time for one and the same partition, or for one and the same partitioning level, respectively. That is, for partitioning a tree node, or for partitioning all tree nodes of a certain partitioning level 21 1 , 212, 213, the encoder 10 may be configured to selectively choose one of the two options, i.e. one of the two available tree types contained in the tree type set 201 C.
- a tree type set may comprise more than two exemplarily described available tree types.
- the encoder 10 may be configured to choose exactly one out of the available tree types for partitioning a tree node, or for partitioning all tree nodes of a certain partitioning level 21 1 , 212, 213.
- the encoder 10 may be configured to select a tree type set 201 C comprising at least a first tree type (e.g. BT) and a second tree type (e.g. GBS) to be used at a predetermined partitioning level 211 , 212, 213, in this example for the second partitioning level 212.
- the encoder 10 may further be configured to selectively choose exactly one out of the at least first and second tree types (e.g. BT or GBS) for performing the multi-tree partitioning at said predetermined partitioning level 211 , 212, 213.
- the encoder 10 may further be configured to signal the selected tree type set 201 C in the data stream 14, at the root block partitioning level 211.
- a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 21 1 , a tree type set 201 C comprising at least a first tree type (e.g. BT) and a second tree type (e.g. GBS) to be used at a predetermined partitioning level 21 1 , 212, 213.
- a first tree type e.g. BT
- GBS second tree type
- the decoder 110 may further be configured to selectively choose exactly one out of the at least first and second tree types (e.g. BT or GBS) for performing the multi-tree partitioning at said predetermined partitioning level 212, 213, 214.
- first and second tree types e.g. BT or GBS
- the encoder 10 may further encode an indication 205 into the data stream 14 for indicating which one of the available tree types (e.g. BT or GBS) was selected at the respective partitioning level 212 for partitioning the tree nodes 13 2 , 13s, 13 4 , 135 of said partitioning level 212.
- Said indication 205 may, for instance, be a flag, such as the depicted“ Pe r-le ve l_m od e_f I ag” .
- the encoder 10 may be configured to signal at the predetermined partitioning level 212 an indication 205 which one out of the first and second tree types (e.g. “BT or GBS”) is selectively chosen from the tree type set 201 C for performing the multi-tree partitioning at said predetermined partitioning level 212.
- an indication 205 which one out of the first and second tree types (e.g. “BT or GBS”) is selectively chosen from the tree type set 201 C for performing the multi-tree partitioning at said predetermined partitioning level 212.
- a further embodiment provides a corresponding decoder 110, wherein the decoder 1 10 may be configured to obtain from a signaling at the predetermined partitioning level 212 an indication 205 which one out of the first and second tree types (e.g.“BT or GBS") to selectively choose for performing the multi-tree partitioning at said predetermined partitioning level 212.
- the decoder 1 10 may be configured to obtain from a signaling at the predetermined partitioning level 212 an indication 205 which one out of the first and second tree types (e.g.“BT or GBS”) to selectively choose for performing the multi-tree partitioning at said predetermined partitioning level 212.
- the third TTS 201 C in Figure 21 may comprise three tree types, namely a first tree type (e.g. QT), and the aforementioned two available options (e.g.“BT or GBS”).
- the encoder 10 may use the first tree type (e.g. QT) for partitioning the root node 13i at the first partitioning level 211 , and to selectively choose exactly one out of the two available options (e.g.“BT or GBS”) for performing the multi-tree partitioning at the second partitioning level 212, as described above.
- the encoder 10 may be configured to select a tree type set 201 C comprising a first tree type (e.g. QT), a second tree type (e.g. BT) and a third tree type (e.g. GBS), to use the first tree type for performing the multi-tree partitioning at a predetermined partitioning level, e.g. the exemplarily depicted first partitioning level 21 1 , and to selectively choose exactly one out of the second and the third tree types (e.g.“BT or GBS”) for performing the multi-tree partitioning at one or more subsequent partitioning levels, e.g. at the exemplarily depicted second partitioning level 212, and to signal the selected tree type set 201 in the data stream 14, at the root block partitioning level 211.
- a first tree type e.g. QT
- BT a second tree type
- GBS third tree type
- the encoder 10 may be configured to signal at the second partitioning level 212 an indication, e.g. the“Per-level_mode_flag” 205”, to indicate which one out of the second and the third tree types (e.g.“BT or GBS”) is selectively chosen from the tree type set 201 C for performing the multi-tree partitioning at said second partitioning level 212.
- an indication e.g. the“Per-level_mode_flag” 205”
- a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 , a tree type set 201 C comprising a first tree type (e.g. QT), a second tree type (e.g. BT) and a third tree type (e.g. GBS), to use the first tree type for performing the multi-tree partitioning at a predetermined partitioning level, e.g. the exemplarily depicted first partitioning level 21 1 , and to selectively choose exactly one out of the second and the third tree types (e.g. “BT or GBS”) for performing the multi-tree partitioning at one or more subsequent partitioning levels, e.g. at the exemplarily depicted second partitioning level 212.
- a tree type set 201 C comprising a first tree type (e.g. QT), a second tree type (e.g. BT) and a third tree type (e.g.
- the decoder 1 10 may be configured to obtain from a signaling at the second partitioning level 212 an indication, e.g. the“Per-level_mode_flag” 205”, to indicate which one out of the second and the third tree types (e.g.“BT or GBS”) to selectively choose for performing the multi-tree partitioning at said second partitioning level 212.
- an indication e.g. the“Per-level_mode_flag” 205”
- the above described concept of tree type sets (TTS) 201 A, 201 B, 201 C could host multiple tree types for a given level 211 , 212, e.g. QT for the first coarse level of partitioning 21 1 and either BT or GBS for the second level of partitioning 212.
- Slice or CU-level type signaling such as a Per- level_mode_flag 205 could be used to index the used partitioning for levels 212 with multiple tree types (e.g.“BT or GBS”).
- indication of the tree partitioning type for the current split level and potentially following split levels may exist on each individual partitioning level.
- FIG 22 shows an example.
- the partition parameter set 201 may again comprise, by way of non-limiting example, three tree type sets (TTS) 201 A, 201 B, 201 C.
- TTS tree type sets
- Each TTS 201 A, 201 B, 201 C may comprise one or more tree types.
- each TTS 201 A, 201 B, 201 C may comprise only one tree type for the sake of simplification of the following description.
- a first TTS 201 A may be selected by the encoder 10. Accordingly, the encoder 10 use the tree type (e.g. QT) contained in the TTS 201 A for performing the multi-tree partitioning at the first partitioning level 211. The encoder may signal said selected TTS 201A in the data stream 14 at the first partitioning level. If a TTS may comprise only one tree type, then the encoder 10 may signal the tree type that is contained in the respective TTS, instead of signaling the TTS itself.
- the tree type e.g. QT
- a TTS 201 A, 201 B, 201 C may again be selected by the encoder 10.
- the encoder 10 may be configured to select one TTS 201 A, 201 B, 201 C for the entire partitioning level. Accordingly, the one or more tree types which are contained in the selected TTS may then be used for performing the multi-tree partitioning at the respective partitioning level. That is, one and the same tree type of the selected TTS may be used for partitioning all nodes of the respective partitioning level, except for leaf nodes because leaf nodes may per se not be further partitioned.
- the encoder 10 may select the third TTS 201 C at the second partitioning level 212.
- the encoder 10 may use the one or more tree types (e.g. GBS) contained in the selected third TTS 201 C for performing the multi-tree partitioning at the second partitioning level 212.
- GBS tree types
- each node of the second partitioning level 212 may be partitioned using the GBS tree type, except for leaf nodes. (Note that this example refers to a level-wise selection of TTS, which is not explicitly depicted)
- the encoder 10 may be configured to use the one or more tree types contained in the selected TTS 201 A, 201 B, 201 C for performing the multi-tree partitioning at the respective partitioning level 21 1 , 212, 213 and at one or more subsequent partitioning levels.
- the encoder 10 may select the first TTS 201 A comprising tree type QT for partitioning the root node 13i at the root partition level 211.
- the encoder 10 may also use the selected TTS 201 A for further subsequent partitioning levels.
- the encoder may use the one or more tree types (e.g. QT) contained in the first TTS 201 A as the tree type for performing the multi-tree partitioning at the second partitioning level 212 and optionally further subsequent partitioning levels.
- the encoder 10 may be configured to select a first tree type set 201 A at a first partitioning level 21 1 and to use one of the tree types contained in said first tree type set 201 A for performing the multi-tree partitioning at said first partitioning level 21 1 , or for performing the multi-tree partitioning at said first partitioning level 211 and one or more subsequent partitioning levels 212.
- the encoder 10 may further be configured to select the first or a second tree type set 201 A, 202B at a second partitioning level 212 and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 202B for performing the multitree partitioning at the second partitioning level 212, or for performing the multi-tree partitioning at said second partitioning level 212 and one or more subsequent partitioning levels. Furthermore, the encoder 10 may be configured to signal the selected tree type set 201 A, 201 B in the data stream 14, separately at the first partitioning level 21 1 and at the second partitioning level 212. That is, the encoder 10 may signal the selected TTS 201 A, 201 B level-wise.
- the encoder 10 may select a TTS for each node of a partitioning level separately, i.e. partition-wise.
- the encoder 10 may select the first TTS 201 A at the first partitioning level 21 1 , as described above. Further, the encoder 10 may select the third TTS 201 C for the node (which may also be referred to as a partition) 13 2 at the second partitioning level 212. Accordingly, the node 13 2 may further be partitioned, using the GBS tree type contained in the selected third TTS 201 C, into two subsequent sub- nodes 13e, 13 7 at a subsequent third partitioning level 213. However, the encoder 10 of this example may further be configured to select a TTS for another node of the second partitioning level 212.
- the encoder 10 may select the first TTS 201 A for a further node 13s at the second partitioning level 212. Accordingly, the node 13s may further be partitioned, using the QT tree type contained in the selected first TTS 201A, into four subsequent sub-nodes 13e, 13g, 13io, 13n at a subsequent third partitioning level 213.
- the encoder 10 may be configured to select for a first partition 13 2 of a first partitioning level 212 a first tree type set 201 C at the first partitioning level 212 and to use one of the tree types contained in said first tree type set 201 C for performing the multi-tree partitioning of said first partition 13 2 at said first partitioning level 212, or for performing the multi-tree partitioning of said first partition 13 2 at said first partitioning level 212 and of one or more subsequent partitions at one or more subsequent partitioning levels.
- the encoder 10 may further be configured to select for a second partition 13 5 of the first partitioning level 212 the first or a second tree type set 201 A, 201 C at said first partitioning level 212 and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 201 C for performing the multi-tree partitioning of said second partition 13 5 at said first partitioning level 212, or for performing the multi-tree partitioning of said second partition 13s at said first partitioning level 215 and of one or more subsequent partitions at one or more subsequent partitioning levels.
- the encoder 10 may further be configured to signal the respectively selected tree type set 201A, 201 C in the data stream 14, separately for the first partition 13 2 and the second partition 13 s at the first partitioning level 212.
- a further embodiment provides a corresponding decoder 1 10, wherein the decoder 1 10 may be configured to obtain from the signaling in the data stream 14, at a first partitioning level 21 1 , a first tree type set 201 A and to use one of the tree types contained in said first tree type set 201A for performing the multi-tree partitioning at said first partitioning level 21 1 , or for performing the multi-tree partitioning at said first partitioning level 21 1 and one or more subsequent partitioning levels 212, 213.
- the decoder 1 10 may further be configured to obtain from a separate signaling in the data stream 14, at a second partitioning level 212, the first or a second tree type set 201 A,
- the decoder 1 10 may be configured to select a TTS 201 A, 201 B, 201 C partition-wise. According to this example, the decoder 1 10 may be configured to obtain from the signaling in the data stream 14, at a first partitioning level 212, a first tree type set 201 C for a first partition 13 2 of said first partition level 212, and to use one of the tree types contained in said first tree type set 201 C for performing the multi-tree partitioning of said first partition 13 2 at said first partitioning level 212, or for performing the multi-tree partitioning of said first partition 13 z at said first partitioning level 212 and of one or more subsequent partitions at one or more subsequent partitioning levels.
- the decoder 1 10 may further be configured to obtain from the signaling in the data stream 14, at the first partitioning level 212, the first or a second tree type set 201 A, 201 C for a second partition 13s of said first partition level 212 and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 201 C for performing the multi-tree partitioning of said second partition 13s at said first partitioning level 212, or for performing the multi-tree partitioning of said second partition 13s at said first partitioning level 212 and one of one or more subsequent partitions at one or more subsequent partitioning levels.
- the decoder 1 10 may be configured to obtain the signaling of the respective tree type set from the data stream 14, separately for the first partition 13 2 and the second partition 13s at the first partitioning level 212.
- Figure 23 shows a block diagram of a method for block-based encoding a picture 12 into a data stream 14 according to the above described concept.
- Block 2301 the picture 12 is partitioned along a splitting tree 13 from a tree root block, at a root block partitioning level 21 1 , into one or more leaf blocks, each at a respective leaf block partitioning level 213, using recursive multi-tree partitioning.
- a tree type set 201 A, 201 B, 201 C comprising one or more tree types is selected for performing the multi-tree partitioning.
- FIG. 24 shows a block diagram of a method for block-based decoding a picture 12 from a data stream 14 according to the above described concept.
- the picture 12 is partitioned along a splitting tree 13 from a tree root block, at a root block partitioning level 211 , into one or more leaf blocks, each at a respective leaf block partitioning level 213, using recursive multi-tree partitioning.
- the multi-tree partitioning is performed using a tree type set 201 A, 201 B, 201 C comprising one or more tree types.
- the step of performing the multi-tree partitioning is based on a signaling of the tree type set 201 A, 201 B, 201 C in the data stream 14, at one or more partitioning levels 211 , 212, 213 between the root block partitioning level 211 and the leaf block partitioning level 213, including the root block partitioning level 211 and the leaf block partitioning level 213.
- an asymmetric variant of QT is introduced, i.e. a block may be divided into four partitions by two straight boundaries with a single split.
- An example is shown in Figure 25, which shows a block 250 that is partitioned into four subblocks 251 , 252, 253, 254.
- the block 250 may be a tree node block, e.g. a CTU, and the subblocks 251 , 252, 253, 254 may be leaf blocks.
- the splitting tree 13 depicted on the right side of the block 250 depicts the partitions belonging to the respective subblocks 251 , 252, 253, 254.
- a first horizontal split at the CTU partitioning level may be performed, wherein said first split may comprise a splitting ratio of 3 ⁇ 4.
- the partitioning into the depicted four partitions is comparatively expensive signaling-wise and requires signaling of three split decisions for each of the three partitions in the splitting tree 13, each with corresponding splitting ratios, i.e. 1 ⁇ 4 and 1/3.
- the invention in this aspect is a QT style partitioning that may be able to split the block 250 into the depicted four partitions 251 , 252, 253, 254 based on a single split indication, but in addition to state-of-the-art QT, the inventive principle may also allow for indicated two splitting ratios that define the exact position of the splitting boundaries that may be different than N/2 splits of conventional QT. Accordingly, asymmetric subblocks
- inventive principle of 5 this aspect may also be referred to as an Asymmetric Quad Split (AQT).
- AQT Asymmetric Quad Split
- an encoder 10 may be provided, wherein the encoder 0 is a block-based encoder for encoding a picture 12 into a data stream 14.
- the encoder 10 may be configured to partition the picture 12 into one or more leaf blocks 251 , 252, 253, 254 using recursive multi-tree partitioning along a splitting tree 13.
- the 10 encoder 10 may be configured to signal partitioning information of the splitting tree 13 in the data stream 14. This encoder 10 may be similar to the encoder as exemplarily described above with reference to Figure 14.
- the encoder 10 may be configured to perform for a predetermined block 250 an asymmetric quad split, in which the predetermined block 250 15 is partitioned by at least a first split boundary 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 , 252, 253, 254.
- the encoder 10 of this aspect may be configured to signal in the data stream 14 a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256.
- a corresponding decoder 110 For decoding the picture, a corresponding decoder 110 is provided.
- a block-based decoder 110 is provided for decoding the picture 12 from the data stream 14, wherein the decoder 110 may be configured to partition the picture 12 into one or more leaf blocks 251 , 252, 253, 254 using recursive multi-tree partitioning along a splitting tree 13.
- This decoder 1 10 may be similar to the decoder as exemplarily described 25 above with reference to Figure 15.
- the decoder 1 10 may be configured obtain a partitioning information of the splitting tree 13 from the data stream 14. Furthermore, the decoder 110 may further be configured to perform for a predetermined block 250 an asymmetric quad split, in which the predetermined block 250 is partitioned by at least a first split boundary 30 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 ,
- the decoder 1 10 may be configured to obtain from the data stream 14 a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256.
- this asymmetric quad split may be based on a single split indication. That is, a quad split is signaled together with two splitting ratios, namely a first splitting ratio for the first split boundary 255 and a second splitting ratio for the second split boundary 256.
- the encoder 10 may be configured to perform the asymmetric quad split based on a single split indication, and to signal in the data stream 14 the single split indication and the first splitting ratio of the first split boundary 255 and the second split ratio of the second split boundary 256.
- the decoder 1 10 of this embodiment may be configured to perform the asymmetric quad split based on a single split indication, and to obtain from the data stream 14 the single split indication and the first splitting ratio of the first split boundary 255, and the single split indication and the second split ratio of the second split boundary 256.
- An alternative embodiment of the invention is a constraint for GBS partitioning that allows for reduction of signaling overhead in such a scenario. This is achieved by restricting the splitting ratio of the third split to be equal to the second split and omit corresponding bitstream signaling in favor of inference on decoder side.
- Figure 25 illustrates the respective split tree 13. The set of available split ratios for the third split which leads to partitions 253 and 254 may be restricted to the actual split ratio of the second split.
- the encoder 10 may be configured to perform the asymmetric quad split by using a binary splitting tree 13 based on at least a first and a second binary split indication, i.e. based on at least a split indication for the first binary split of the first partition 13i at the first partitioning level, and based on at least a split indication for the second binary split of the second partition 13 2 at the second partitioning level.
- the encoder 10 may be configured to perform the first binary split of the first partition 13i which creates the first split boundary 255 having a first split ratio, e.g. 3 ⁇ 4.
- the encoder 10 may further be configured to perform the perpendicular second binary split of the second partition 13 2 which creates the second split boundary 256 having a second split ratio, e.g. 1/3.
- the encoder 10 may be configured to perform a perpendicular (relative to the first split boundary 255) third binary split of the third partition 13 3 creating a third split boundary having a third split ratio.
- the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split, which is indicated in the splitting tree 13 by means of the arrow 257.
- the encoder 10 is configured to omit signaling in the data stream 14 the third split ratio of the third binary split, since it is restricted to be the same as the second split ratio of the second binary split, as mentioned above.
- this implicit third split ratio may be inferred without an explicit split ratio indication for the third binary split in the data stream 14.
- the decoder 110 may be configured to perform the asymmetric quad split by using a binary splitting tree 13 based on at least a first and a second binary split indication, i.e. based on at least a split indication for the first binary split of the first partition 13i at the first partitioning level, and based on at least a split indication for the second binary split of the second partition 13 ⁇ at the second partitioning level.
- the decoder 1 10 may be configured to perform a first binary split creating a first split boundary 255 having a first split ratio, e.g. 3 ⁇ 4.
- the decoder 110 may further be configured to perform a perpendicular second binary split creating a second split boundary 256 having a second split ratio, e.g. 1/3.
- the decoder 1 10 may be configured to perform a perpendicular (relative to the first boundary 255) third binary split creating a third split boundary having a third split ratio.
- the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split, as mentioned above.
- the decoder 110 may be configured to infer, without an explicit split-ratio-signal in the data stream 14, the third split ratio of the third binary split to be equal to the second split ratio of the second binary split.
- Figure 26 shows a block diagram of an encoding method according to the above described concept.
- Figure 26 shows a block diagram of a method for block-based encoding a picture 12 into a data stream 14.
- the picture 12 is partitioned into one or more leaf blocks 251 , 252, 253, 254 using recursive multi -tree partitioning along a splitting tree 13, and partitioning information of the splitting tree 13 is signaled in the data stream 14.
- an asymmetric quad split is performed for a predetermined block 250, in which the predetermined block 250 is partitioned by at least a first split boundary 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 , 252, 253, 254.
- a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256 is signaled in the data stream 14, wherein both split ratios may be encoded into the data stream 14 during the same splitting step.
- Figure 27 shows a block diagram of a corresponding decoding method according to the above described concept.
- Figure 26 shows a block diagram of a method for block-based decoding the picture 12 from the data stream 14.
- the picture 12 is partitioned into one or more leaf blocks 251 , 252, 253, 254 using recursive multi-tree partitioning along a splitting tree 13, and partitioning information of the splitting tree 13 is obtained from the data stream 14.
- an asymmetric quad split is performed for a predetermined block 250, in which the predetermined block 250 is partitioned by at least a first split boundary 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 , 252, 253, 254.
- a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256 are obtained from the data stream 14, wherein both split ratios may be obtained from the data stream 14 during the same splitting step.
- This aspect intents to infer partitioning parameters of a block from its local neighborhood.
- Means for defining regions within a picture exists, e.g. tiling, slices or region-wise packing.
- ATS Adaptation Tree Sets
- TTS Adaptation Tree Sets
- partitioning parameters can be referenced/activated on a sub-picture basis.
- the current region definition capabilities may not be sufficient to efficiently capture the shape of image objects accurately enough, to maximize the coding-efficiency benefit of an adapted partitioning scheme.
- partitioning parameters of the current block are instead derived from previously coded blocks, e.g. in the spatial or temporal vicinity of the current block.
- Figure 28 shows an example for this aspect, which example is applicable to both an inventive encoder and an inventive decoder.
- a picture 12s is depicted which may be coded by using a spatial intra-prediction mode.
- the picture 12s may be subdivided into sub-picture regions, e.g. slices 281 , 282, 283.
- the slices 281 , 282, 283 may further be divided into blocks 284, 285, 286.
- Said blocks 284, 285, 286 may, for instance, be tree root blocks, e.g. CTUs, as previously explained above.
- the picture 12s may be partitioned, starting from a tree root block into one or more leaf blocks, along a splitting tree 13.
- partitioning parameters 291 , 292, 293, 294 may be used by the encoder and the decoder for partitioning the respective block.
- Partitioning parameters 291 , 292, 293, 294 may comprise, as non-limiting examples, one or more of the above described splitting indications of the other further aspects, such as, e.g. split ratios, split modes, tree types, tree type set (TTS), AQT-indications, implicit split indications for log2-partitions and non-log2-partitions, and many more.
- one or more partitioning parameters 291 , 292 may be available for partitioning said block 286.
- the one or more partitioning parameters 291 , 292 may be contained in a partitioning parameter set 290.
- At least one partitioning parameter 292 of said block 286 may be an inherited partitioning parameter, which shall be explained in more detail below.
- the picture 12s may comprise a further block 285 that may also be located in the same sub-picture region as block 286, e.g. located in slice 281.
- Said further block 285 is located in the spatial vicinity of the block 286.
- the further block 285 is a direct spatial neighbor of the block 286.
- the further block 285 is located directly adjacent to the block 286.
- the blocks 285, 286 may be coded along a coding order, wherein the further block 285 may be coded prior to the block 286. Accordingly, said further block 285 may be a previously coded block that has been previously coded prior to the block 286.
- the further block 285 may also comprise one or more partitioning parameters 293, 294.
- partitioning parameter‘B’ 294 may be used in partitioning the further block 285.
- the block 286 may be consecutively coded afterwards, e.g. subsequent to the further block 285.
- the block 286 may inherit from the previously coded further block 285 at least one partitioning parameter, e.g. the aforementioned partitioning parameter‘B’. 10 Said inheritance is indicated in Figure 28 by means of arrow 295. For the sake of symbolizing the inherited partitioning parameter in the Figure, an asterisk is used. Accordingly, the block 286 may comprise the inherited partitioning parameter‘B*’ 292. The inherited partitioning parameter‘B * ’ 292 may be a copy of partitioning parameter‘B’ 294 of the further block 285. Additionally or alternatively, block 286 may access partitioning 15 parameter‘B’ 294 by means of referencing.
- the block 286 may be partitioned along the splitting tree 13 by using the inherited partitioning parameter‘B*’ that has been inherited from the previously coded further block 285.
- Each of the pictures 12ti, 12t 2 may be subdivided into sub-picture regions, e.g. slices 281 , 282, 283, as explained above with reference to picture 12s.
- the slices 281 , 282, 283 may also be divided into blocks 284, 285, 286.
- Said blocks 284, 285, 286 may, for instance, be tree root blocks, e.g. CTUs, as previously explained above.
- each of the blocks of the pictures 12ti, 12t 2 may 25 be partitioned, starting from a tree root block into one or more leaf blocks, along a splitting tree 13.
- the pictures 12ti, 12t 2 may be coded along a coding order, wherein the first picture 12ti may be coded prior to the second picture 12t 2 .
- the second picture 12t 2 may comprise at least one block 286 wherein one or more partitioning parameters 291 , 292 may be available for partitioning said at least one block 286.
- the first picture 12ti may comprise a further block 285 wherein one or more partitioning parameters 293, 294 may be available for partitioning said further block 285.
- partitioning parameter‘B’ 294 may be used in partitioning the further block 285.
- the at least one block block 286 may be consecutively coded afterwards, e.g. subsequent to the further block 285.
- inter-prediction coding may be similar to the above described example of intra-prediction coding.
- the further block 285 may be located in the temporal vicinity of the at least one block 286. That Is, the further block 285 may be a block of a previously coded picture 12ti. However, said further block 285 may be spatially located in a corresponding sub-picture region (e.g. slice 2811) of the first picture 12ti corresponding to a respective sub-picture region (e.g. slice 2812) of the second picture 12t 2 .
- the block 286 of the second picture 12t 2 may inherit from the previously coded further block 285 of the first picture 12ti at least one partitioning parameter, e.g. the aforementioned partitioning parameter‘B’. Said inheritance is again indicated by means of arrow 295.
- the block 286 may comprise the inherited partitioning parameter‘B*’ 292.
- the inherited partitioning parameter‘B*’ 292 may be a copy of partitioning parameter ⁇ ’ 294 of the further block 285. Additionally or alternatively, block 286 may access partitioning parameter‘B’ 294 by means of referencing.
- the block 286 may be partitioned along the splitting tree 13 by using the inherited partitioning parameter‘B*’ that has been inherited from the previously coded further block 285.
- an encoder 10 is provided for block-based encoding a picture 12s, 12ti, 12t 2 into a data stream 14 using at least one of a spatial intra-prediction and a temporal inter-prediction coding.
- the encoder 10 is configured to partition the picture 12s, 12ti, 12t 2 along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multi-tree partitioning, similar to the encoder 10 as explained above with reference to Figure 14.
- the recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294. If at least one block 286 is encoded by means of spatial intra-prediction, the encoder 10 is configured to inherit one or more partitioning parameters 294 from a further block 285 in the spatial vicinity of the at least one block 286, and to use said inherited one or more partitioning parameters 292 for partitioning said at least one block 286 in the recursive multitree partitioning.
- the encoder 10 is configured to inherit one or more partitioning parameters 294 from a further block 285 in the temporal vicinity of the at least one block 286, and to use said inherited one or more partitioning parameters 292 for partitioning the at least one block 286 in the recursive multi-tree partitioning.
- the encoder 10 is configured to signal in the data stream 14 that one or more inherited partitioning parameters 294 are used for partitioning the at least one block 286.
- the corresponding decoder 1 10 may be instructed to infer, for at least one block 286, one or more partitioning parameters 292 of a block 285 from its local neighborhood.
- the decoder 1 10 may use one or more inherited partitioning parameters 292, which are inherited from a further block 285 in the spatial or temporal vicinity, for performing the multi-tree partitioning of the at least one block 286.
- one or more partitioning parameters 292 of the at least one current block 286 may be derived from the previously coded further block 285, e.g. in the spatial or temporal vicinity of the at least one current block 286.
- Such an inference may also be referred to as inheritance, and the one or more partitioning parameters 292 which are derived or inferred from the previously coded further block 285 may also be referred to as inherited partitioning parameters.
- the above mentioned signaling in the data stream 14 may indicate whether such inference or inheritance, respectively, is used over default partitioning using one or more default partitioning parameters of the picture 12s, 12ti , 12t 2 or sub-picture region (e.g. slices, tiles, blocks, etc.).
- a decoder 1 10 for block-based decoding a picture 12s, 12ti, 12t 2 from a data stream 14 using at least one of a spatial intra-prediction and a temporal inter-prediction coding.
- the decoder 110 is configured to partition the picture 12s, 12ti, 12t 2 along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multitree partitioning, which may be comparable to the decoder 110 as described above with reference to Figure 15.
- the recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294.
- the decoder 1 10 may be configured to use, based on the above mentioned signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block 286, which inherited one or more partitioning parameters 292 are inherited from a further block 285 in the spatial vicinity of the at least one block 286.
- the decoder 1 10 is configured to use, based on the above mentioned signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block 286, which inherited one or more partitioning parameters 292 are inherited from a further block 285 in the temporal vicinity of the at least one block 286.
- One example of this technique would be to optionally infer min/max tree depths from the partitioning trees of the spatial and/or temporal neighboring blocks 285 if advantageous.
- a first signaling on CTU level may indicate whether such inference is used over default partitioning parameters of the picture or subpicture.
- the one or more partitioning parameters 291 , 292, 293, 294 may comprise a predetermined (e.g. min/max) tree depth of the splitting tree 13 for partitioning the at least one block 286, wherein the encoder 10 may be configured to inherit from the further block 286 in the spatial vicinity or from the further block 268 in the temporal vicinity a predetermined tree depth 294 of a further splitting tree that is for partitioning the further block 286 in the spatial or temporal vicinity.
- the encoder 10 may further be configured to use said inherited predetermined tree depth 292 in the splitting tree for partitioning the at least one block 286.
- the encoder 10 may be configured to use an inherited partitioning parameter 292, i.e. the inherited tree depth of the splitting tree belonging to the previously coded further block 285, instead for partitioning the current at least one block 286.
- the default partitioning parameter e.g. a predetermined (e.g. min/max) tree depth
- the encoder 10 may be configured to use an inherited partitioning parameter 292, i.e. the inherited tree depth of the splitting tree belonging to the previously coded further block 285, instead for partitioning the current at least one block 286.
- the decoder 1 10 may derive from the signaling in the data stream 14 whether to use the default tree depth or the inherited tree depth 292 instead.
- the one or more partitioning parameters 291 , 292, 293, 294 may comprise a predetermined (e.g. min/max) tree depth of the splitting tree 13 for partitioning the at least one block 286, wherein the decoder 1 10 may be configured to, based on the signaling in the data stream 14, inherit from the further block 285 in the spatial vicinity or from the further block 285 in the temporal vicinity a predetermined tree depth of a further splitting tree that is for partitioning the further block 285 in the spatial or temporal vicinity, and to use said inherited predetermined (e.g. min/max) tree depth 292 in the splitting tree 13 for partitioning the at least one block 286.
- a predetermined (e.g. min/max) tree depth of the splitting tree 13 for partitioning the at least one block 286, wherein the decoder 1 10 may be configured to, based on the signaling in the data stream 14, inherit from the further block 285 in the spatial vicinity or from the further block 285 in the temporal vicinity a predetermined tree depth of a further splitting tree
- the signaling in the data stream 14 may be done at a tree root block level, e.g. on CTU level.
- the encoder 10 may be configured to signal, at a root node partitioning level (e.g. CTU level) in the splitting tree 13, that the one or more inherited partitioning parameters 292 are used for partitioning the at least one block 286 instead of one or more default partitioning parameters.
- a root node partitioning level e.g. CTU level
- the decoder 1 10 may also derive said signaling in the data stream 14 at a tree root block level, e.g. on CTU level.
- the decoder 1 10 may be configured to obtain the signaling from the data stream 14, at a root node partitioning level (e.g. CTU level) of the splitting tree 13, signaling that one or more inherited partitioning parameters 292 of the further block 285 in the spatial or temporal vicinity are used as the one or more partitioning parameters for partitioning the at least one block 286 instead of one or more default partitioning parameters.
- a root node partitioning level e.g. CTU level
- partitioning parameters 294, 295 of the neighboring blocks 285 into a set of partitioning parameter candidates.
- the set could also include mixtures/generalization/relaxations of some of the included candidates.
- the partitioning of the current block 286 could activate one of the partitioning parameter set candidates by reference through an index. Alternatively, the current block 286 could be partitioned using the most probable set out of the candidates as defined by other means.
- both the encoder 10 and the decoder 1 10 may be configured to access a set of inheritance partitioning parameter candidates comprising one or more partitioning parameters 294, 295 of the further block 285 in the spatial vicinity and/or one or more partitioning parameters 294, 295 of the further block 285 in the temporal vicinity.
- both the encoder 10 and the decoder 1 10 may be configured to select at least one 294 of the partitioning parameters 294, 295 contained in the set of inheritance partitioning parameter candidates, and to use said selected one 294 of the partitioning parameters from the set of inheritance partitioning parameter candidates as a partitioning parameter for partitioning the at least one block 286.
- the selected one partitioning parameter 294 may be inherited and may, thus, become the inherited partitioning parameter 292, as depicted in Figure 28.
- means for signaling may be required that indicate whether the sub-picture partitioning parameters through regions (defined for instance in parameter sets) are to be interpreted as to be applied to all CTUs or whether some CTUs (e.g. in the region borders) can inherit partitioning information from its neighboring CTUs that belong to another region (described inference/inheritance above).
- the encoder 10 may be configured to partition the picture 12s, 12ti, 12t 2 into a multitude of root blocks 284, 285, 286, each of which comprising one or more own partitioning parameters 291 , 293, 294, which may be non-inherited default partitioning parameters.
- the encoder 10 may be configured to signal an indication in the data stream 14 for indicating whether all root blocks of the multitude of root blocks 284, 285, 286 are partitioned by using their one or more own partitioning parameters 291 , 293, 294, or whether at least one root block 286 of the multitude of root blocks 284, 285, 286 is partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity of the at least one root block 286, wherein the one or more inherited partitioning parameters 292 are used instead of the one or more own partitioning parameters for partitioning the at least one root block 286.
- the decoder 110 may be configured to obtain a respective signaling comprising the above mentioned indication from the data stream 14, and the decoder 110 may then accordingly use the one or more inherited partitioning parameters 292 instead of one or more default partitioning parameters (also referred to as“own partitioning parameters").
- the decoder 1 10 may be configured to partition the picture 12s, 12ti, 12t 2 into a multitude of root blocks 284, 285, 286, each of which comprising one or more own partitioning parameters 291 , 293, 294, which may be non-inherited default partitioning parameters.
- the decoder 110 may be configured to obtain the above mentioned indication from the data stream 14 indicating whether all root blocks of the multitude of root blocks 284, 285, 286 are partitioned by using their one or more own partitioning parameters 291 , 293, 294, or whether at least one root block 286 of the multitude of root blocks 284, 285, 286 is partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity of the at least one root block 286, wherein the one or more inherited partitioning parameters 292 are used instead of the one or more own partitioning parameters for partitioning the at least one root block 286.
- Another embodiment would be to define a parameter that signals the number of CTUs at the border of the region for which syntax is present that allows for selecting between partitioning parameters of a neighboring CTU and default partitioning parameters of the corresponding region/subpicture.
- the encoder 10 may be configured to partition the picture 12s, 12ti, 12t 2 into a multitude of root blocks 284, 285, 286, and to signal an indication in the data stream 14 for indicating a number of root blocks which are partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity.
- the decoder 110 may be configured to obtain said indication from the data stream 14 to derive the number of blocks, e.g. CTUs, which are partitioned by using inherited partitioning parameters 292.
- the decoder 1 10 may be configured to partition the picture 12s, 12ti, 12t 2 into a multitude of root blocks 284, 285, 286, and to obtain an indication from the data stream 14 which indicates a number of root blocks which are partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity.
- ATS and TTS could be maintained per temporal layer.
- Figure 29 shows a block diagram of a method for encoding a picture 12s, 12ti, 12t 2 into the data stream 14 in accordance with the above described concept.
- the picture 12s, 12ti, 12t 2 is partitioned along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294.
- At least one block 286 is encoded by means of spatial intra- prediction
- one or more partitioning parameters are inherited from a further block 285 in the spatial vicinity of the at least one block 286, and said inherited partitioning parameters 292 are used for partitioning said at least one block 286 in the recursive multi-tree partitioning.
- one or more partitioning parameters are inherited from a further block 285 in the temporal vicinity of the at least one block 286, and said inherited partitioning parameters 292 are used for partitioning the at least one block 286 in the recursive multi- tree partitioning.
- one or more inherited partitioning parameters 292 are used for partitioning the at least one block 286, as shown in method block 2904.
- Figure 30 shows a block diagram of a corresponding method for decoding the picture 12s, 12ti, 12t 2 from the data stream 14 in accordance with the above described concept.
- the picture 12s, 12ti, 12t 2 is partitioned along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294. It is then differentiated whether a block is coded using spatial intra-prediction or temporal inter-prediction.
- the method comprises a step of using, based on a signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block
- the method comprises a step of using, based on a signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block
- Block-based encoder for encoding a picture into a data stream, wherein the
- encoder is configured to
- Block-based decoder for decoding a picture from a data stream, wherein the
- decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree into leaf blocks, a height and a width of which in number of samples is a power of two,
- aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
- Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.
- embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software.
- the implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
- embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.
- the program code may for example be stored on a machine readable carrier.
- inventions comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
- an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
- a further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
- the data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.
- a further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.
- the data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
- a further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
- a processing means for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
- a further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
- a further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver.
- the receiver may, for example, be a computer, a mobile device, a memory device or the like.
- the apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
- a programmable logic device for example a field programmable gate array
- a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein.
- the methods are preferably performed by any hardware apparatus.
- the apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
- the methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
- ITU-T and I SO/I EC High efficiency video coding.
- ITU-T and ISO/IEC Advanced video coding for audiovisual services.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention concerns partitioning concepts for partitioning blocks in block-based picture coding. In particular, the present invention is concerned with concepts for the block- based picture coding which concepts provide for a solution to the trade-off between signaling efficiency and coding flexibility and the trade-off between coding efficiency and coding complexity. The invention provides corresponding encoders, decoders, methods for encoding, methods for decoding, and respective computer program products.
Description
PARTITIONING CONCEPTS FOR BLOCK-BASED PICTURE CODING
The present invention concerns partitioning concepts for partitioning blocks in block-based picture coding. In particular, the present invention is concerned with concepts for the block- based picture coding which concepts provide for a solution to the trade-off between signaling efficiency and coding flexibility and the trade-off between coding efficiency and coding complexity.
BACKGROUND
In state-of-the-art video coding, such as the ITU-T H.265 | MPEG-H HE VC video coding standard [1], the pictures of a video sequence are coded in a certain order, which is referred to as coding order. The coding order of pictures can differ from the capture and display order.
For the actual coding, each video picture is partitioned into blocks. A block comprises the samples of a rectangular area of a particular color component. The entity of the blocks of all color components that correspond to the same rectangular picture area, together with the associated syntax, is often referred to as unit. Depending on the purpose of the block partitioning, in ITU-T H.265 | MPEG-H HEVC, it is differentiated between coding tree units (CTUs), coding units (CUs), prediction units (PUs), and transform units (TUs). The associated sample blocks are referred to as coding tree blocks (CTBs), coding blocks (CBs), prediction blocks (PBs), and transform blocks (TBs), respectively.
Typically, a video picture is initially partitioned into fixed sized units (i.e. , aligned fixed sized blocks for all color components). In ITU-T H.265 | MPEG-H HEVC, these units are referred to as coding tree units (CTUs). Their size (in luma samples) is transmitted in the sequence parameter set (SPS), which contains high-level syntax elements that are valid for an entire coding video sequence (which consists of an instantaneous decoding refresh (IDR) picture and all following pictures up to (but excluding) the next IDR picture). In ITU-T H.265 | MPEG-H HEVC, the CTU size can be chosen between 64x64, 32x32, and 16x16 luma samples. Note that both the width and height of a CTU represent an integer power of two. The luma and chroma blocks of samples that are contained in a CTU are called coding tree blocks (CTBs). For the purpose of coding mode selection, prediction, and transform coding
of the prediction residue, the fixed sized CTUs can be adaptively split into smaller units and blocks.
A picture can be comprised of one or multiple slices. A slice represents a collection of CTUs. A slice is associated with a slice type (e.g., I, P, or B slice) which determines the set of supported coding modes. Moreover, several types of prediction such as motion vector prediction and intra-picture prediction are only supported between blocks inside a slice.
A CTU can be split into multiple coding units (CUs). A coding unit is the entity for which a coding mode (e.g., intra- or inter-picture coding) is selected. The decomposition of a CTU into one or more CUs is specified by a quadtree (QT) and transmitted as part of the bitstream. In the CTU syntax, a first binary decision (bin) called split_cu_flag is transmitted, which signals whether the entire CTU forms one single CU (i.e., the CTU is not decomposed into multiple CUs) or whether the CTU is partitioned into four equally sized square units. If the CTU is split, then for each of the resulting units, a further split_cu_flag is transmitted, which indicates whether the corresponding unit is further partitioned into four equally sized square units. This multi-level splitting is continued until for each of the units it is either signaled that the unit is not further split or a minimum CU size is reached. If a unit has the minimum CU size, no split_cu_flag is transmitted, but it is inferred that the unit is not further split. In ITU-T H.265 | MPEG-H HEVC, the minimum allowed CU size is signaled in the sequence parameter set; it can range from the size of the CTU to 8x8 luma samples (and always presents a square with an integer power of two for both the width and height). The resulting units of the described splitting are the coding units (CUs), for which a coding mode and associated parameters are selected and transmitted. Note that the partitioning represents a quadtree (QT) structure.
Each of the resulting CUs is associated with a coding mode, which is selected at the encoder side and transmitted as part of the bitstream. In ITU-T H.265 | MPEG-H HEVC, the binary decisions split_cu_flag, which specify the partitioning of a CTU into one or multiple CUs are coded using context-based adaptive binary arithmetic coding (CABAC). The CUs of a CTU are processed in the so-called z-scan order. That means, the four blocks that result from a split are processed in raster-scan order; and if any of the blocks is further partitioned, the corresponding four blocks (including the included smaller blocks) are processed before the next block of the higher splitting level is processed.
If a CU is coded in an intra-coding mode, an intra prediction mode for the luma signal and, if the video signal includes chroma components, another intra prediction mode for the chroma signals is transmitted. In ITU-T H.265 | MPEG-H HEVC, if the CU size is equal to the minimum CU size (as signaled in the sequence parameter set), the luma coding block can also be split into four equally sized blocks, in which case, for each of these blocks, a separate luma intra prediction mode is transmitted. This splitting is indicated by a binary syntax element partjmode. The actual intra prediction and coding is done on the basis of transform blocks.
For the purpose of transform coding, a CU can be partitioned into multiple transform blocks. Similarly to the splitting of a CTU into CUs, the subdivision into transform blocks is transmitted using a quadtree syntax. The root of the quadtree is formed by the coding unit. The syntax of the coding unit includes a binary decision called split_transform_f lag , which specifies whether the luma component of the CU consists of a single luma transform block or whether it is split into four equally sized square blocks. In the latter case, for each of the resulting luma blocks, another split_transform_flag is transmitted specifying whether the corresponding block is further split. This quadtree splitting is continued until for each of the resulting luma blocks, it is signaled that the block is not further split or the block reaches a minimum transform block size (indicated in the sequence parameter set). For intra-coded CUs for which four luma intra prediction modes are coded, the first split_transform_flag (at the CU level) is not transmitted, but it is inferred that the luma coding block is split into four luma blocks. The splitting of the chroma blocks basically follows the splitting of the colocated luma block, i.e. , for each luma block the associated chroma blocks cover the same picture area. The luma block and the associated chroma blocks that cover the same picture area form a transform unit. There is, however, one exception in the ITU-T H.265 | MPEG-H HEVC standard: If the resulting chroma blocks would be smaller than the minimum transform block size (as specified in the sequence parameter set), only the luma block is split. In ITU-T H.265 | MPEG-H HEVC, the binary syntax elements split_transform_flag are coded using CABAC. A transform block (TB) is the entity to which a single 2d transform is applied for coding the prediction residual signal. Note that all transform blocks in ITU-T H.265 | MPEG-H HEVC are square blocks and the width and height of the blocks represent integer powers of two.
The TUs of an intra-coded CU are processed in z-scan order (see above). For each transform block (luma or chroma), first a prediction signal is generated using already reconstructed neighboring samples of the same color component. The algorithm that is
used for generating the prediction signal is determined by the intra prediction mode. The resulting prediction error / residual signal is transmitted using transform coding. A two- dimensional transform is applied to the corresponding block of residual samples, the resulting transform coefficients are quantized (typically using uniform reconstruction quantizers), and the resulting quantization indexes, which are also referred to as transform coefficient levels, are entropy-coded. The entropy coding in ITU-T H.265 j MPEG-H HEVC uses the concept of context-based adaptive binary arithmetic coding (CABAC).
Inter-picture coded CUs can be split into multiple prediction units. A prediction unit is the entity of a luma and, for color video, two associated chroma blocks (covering the same picture area), for which a single set of prediction parameters is used. A CU can be coded as a single prediction unit, or it can be split into two non-square or four square prediction units. The latter (four square PUs) is only supported if the CU has the minimum CU size specified in the sequence parameter set. The chosen splitting is indicated by the syntax element partjnode. ITU-T H.265 j MPEG-H HE VC supports in total 8 modes for partitioning a CU into PUs. An NxN CU can be partitioned into:
• a single NxN PU;
• two (N/2)xN PUs;
• two Nx(N/2) PUs;
• a (3N/4)xN PU (left) and a (N/4)xN PU (right);
• a (N/4)xN PU (left) and a (3N/4)xN PU (right);
• a Nx(3N/4) PU (top) and a Nx(N/4) PU (bottom);
• a Nx(N/4) PU (top) and a Nx(3N/4) PU (bottom);
• four (N/2)x(N/2) PUs.
The asymmetric splittings are only supported for CUs larger than or equal to 16x16 luma samples. The smallest supported PU sizes in ITU-T H.265 | MPEG-H HE VC are 8x4 and 4x8.
For each PU, an individual set of motion parameters is transmitted. Each set of motion parameters includes the number of motion hypotheses (one or two in ITU-T H.265 | MPEG-H HE VC) and, for each motion hypothesis, the reference picture (indicated via a reference picture index into a list of reference pictures) and the associated motion vector. First a merge_flag is transmitted, which indicates whether the motion parameters are completely derived based on already coded motion parameters or whether the motion
parameters are transmitted. If the merge_flag is equal to 1 , a list of motion parameter set candidates is derived, which typically includes motion parameters sets of already coded spatially neighboring blocks as well as one or more motion parameter sets that are derived from temporally co-located blocks. The used motion parameter set is indicated by a syntax element mergejdx, which specifies an index into the candidate list of motion parameter sets. If merge_flag is equal to 0, the motion parameters are transmitted as part of the bitstream. A first syntax element (inter_pred_idc in ITU-T H.265 | MPEG-H HEVC) specifies the number of motion hypotheses. In ITU-T H.265 | MPEG-H HEVC, one or two motion hypothesis are supported. For each motion hypothesis, the used reference picture is indicated via a reference picture index, which represents an index into a reference picture list. In ITU-T H.265 | MPEG-H HEVC, there is one reference picture list for P slices and two reference picture lists for B slices, which are referred to as list 0 and list 1. For each slice, the reference picture lists are constructed based on slice header syntax elements. Each reference picture list contains one or more already coded pictures (which are stored in a decoded picture buffer) in a specified order. In B slices, the syntax element inter_predjdc does not only indicate single-hypothesis prediction or bi-prediction, but for single-hypothesis prediction it additionally indicates whether list 0 or list 1 is used. For bi-prediction, the first motion hypothesis refers to list 0 and the second motion hypothesis refers to list 1. For each motion hypothesis, in addition to the reference picture index, an associated motion vector is transmitted as part of the bitstream. The motion vectors are differentially coded. Given the already coded reference picture index, a list of motion vector predictor candidates is derived. It can include (potentially scaled) motion vectors of neighboring PUs as well as scaled motion vectors of temporally co-located blocks. The actually chosen motion vector predictor is signaled via an index into the motion vector predictor list. Finally the difference between the actual motion vector for the PU and the selected motion vector predictor is transmitted. In single hypothesis prediction, the prediction signal for a PU is formed by a displaced block in the selected reconstructed reference picture, where the displacement relative to the current PU is specified by the motion vector. Note that the displacement is typically specified with sub-sample accuracy (in ITU-T H.265 | MPEG-H HEVC, the motion vectors have a precision of a quarter luma sample). For non-integer motion vectors, the prediction signal is generated by interpolating the reconstructed reference picture (typically, using separable FIR filters). The final prediction signal of PUs with multi-hypothesis prediction is formed by a weighted sum of the prediction signals for the individual motion hypothesis. Note that luma and chroma blocks of a PU share the same set of motion parameters.
The prediction signal for an inter-picture coded CU is composed of the prediction signals for the contained PUs. The prediction error signal (also called residual signal) for the luma and chroma blocks of a coding unit is transmitted via transform coding. For that purpose, the block of luma residual samples of a CU as well as the blocks of chroma residual samples (if present) are partitioned into transform blocks. The partitioning into transform blocks is the same as for intra-picture coded CUs described above. The partitioning of a CU into transform block is indicated by a quadtree syntax, which is also referred to as residual quadtree (RQT). The resulting transform blocks are coded using transform coding: A 2d transform is applied to the block of residual samples, the resulting transform coefficients are quantized, and the resulting transform coefficient levels (quantization indexes) are entropy coded. In P and B slices, at the beginning of the CU syntax, a skip_flag is transmitted. If this flag is equal to 1 , it indicates that the corresponding CU consists of a single prediction unit coded in merge mode (i.e., merge_flag is inferred to be equal to 1) and that all transform coefficients are equal to zero (i.e., the reconstruction signal is equal to the prediction signal). In that case, only the mergejdx is transmitted in addition to the skip_flag. If skip_flag is equal to 0, the prediction mode (inter or intra) is signaled, followed by the syntax features described above.
In ITU-T H.265 | MPEG-H HEVC, all low-level syntax elements are coded using context- based adaptive binary arithmetic coding (CABAC). Non-binary syntax elements are first binarized, i.e., they are mapped onto a series of binary decisions, which are also referred to as bins. Then, each bin (including binary syntax elements) is coded using a binary arithmetic coder. There are two modes of the arithmetic coding engine. In one mode, the so-called bypass mode, the bins are coded with a non-adaptive binary probability model with o = Pi = 0.5. In the other mode, the so-called regular mode, one adaptive binary probability model is chosen per bin; after coding the corresponding bin, the binary probability model is updated using the actual value of the bin. The entropy coder maintains a certain number of binary probability models. The selected probability model depends on the syntax element and the bin number (for non-binary syntax elements). It can additionally depend on already coded syntax elements or bins. The selection of binary probability models is also referred to as context modelling.
The syntax of ITU-T H.265 | MPEG-H HEVC provides a multitude of possibilities for coding a block of samples (such as a coding tree unit). The encoder has to select the partitionings (into CUs, PUs, and TUs), the coding modes and associated parameters (such as intra prediction modes and motion parameters), and the transform coefficient levels (quantization
indexes). The selected coding parameters determine both the reconstruction quality as well as the number of bits required for representing the coding parameter inside the bitstream. Modern encoding algorithms are typically based on Lagrangian bit allocation. While the employed encoding algorithm determines the rate-distortion efficiency for the generated bitstream, the maximum achievable coding efficiency of a codec is determined by the supported syntax features, such as the set of coding modes, the partitioning options, the supported intra prediction modes, etc. One of the most important syntax features are the supported partitioning options. On one hand, a flexible partitioning allows an adaptation of the block sizes used for prediction and transform coding to the video signal. But on the other hand, the partitioning has to be transmitted as part of the bitstream. Hence, the gain in prediction and transform coding for providing additional partitioning options has to outweigh the additional signaling overhead. Due to that reason, state-of-the-art video coding standards support only very simple partitioning concepts, such as quadtree partitionings.
In this invention, a partitioning concept based on generalized binary trees is described it provides more flexibility than conventional partitioning concepts (such as the quadtree approach) for adapting the blocks used for prediction and transform coding to the video signal to be coded. And since the syntax for transmitting the selected partitioning is still very simple, the gain in prediction and transform coding typically outweighs the additional signaling overhead. Hence, the generalized partitioning concept is capable of increasing the achievable coding efficiency of a video codec.
The described partitioning concept can be used for partitioning a block for coding mode signaling, intra-picture prediction, inter-picture prediction, transform coding, or any combination thereof. It can completely replace conventional partitioning concepts, or it can be combined with other partitioning concepts, such as a quadtree partitioning.
STATE-OF-THE-ART
In the state-of-the-art video coding standard ITU-T H.265 | MPEG-H HE VC [1], the partitioning into coding units and transform units / transform blocks follows a simple quadtree (QT) concept. A square block can be split into four square blocks of the same size, the resulting blocks can be further split into four square blocks of the same size, etc. For
prediction units, the standard also supports the splitting of a square block into two non- square blocks (either of the same size or of different sizes). However, the resulting non- square blocks cannot be further split. A similar one-level splitting of a square block into two non-square blocks was already supported in ITU-T H.264 | MPEG-4 AVC [2]
While the quadtree splitting is very simple, it has the disadvantage that often an object or motion boundary can only be reasonably approximated by splitting a large block into many smaller blocks. This potentially increases the bit rate for signaling the prediction parameters (such as motion parameters) as well as the bit rate for the transform coding of the prediction error signals. With the goal of increasing the coding efficiency, the Joint Exploration Test Model [3] specifies a combined quadtree/binary tree (QTBT) approach. In this approach, prediction units (PUs) and transform units (TUs) are always aligned with coding units (CUs); hence, there is basically only one block type. A CTU is first split using the conventional quadtree approach. Then, the quadtree leaf nodes can be further partitioned by a binary tree (BT) structure. For the binary splitting, two splitting types are supported: A horizontal split, which horizontally splits a NxM block into two Nx(M/2) blocks, and a vertical split, which vertically splits a NxM block into two (N/2)xM blocks. The binary splitting can be continued until a minimum block width/block height is reached or a maximum number of (binary) splits were executed. The resulting blocks are used for coding mode signaling, intra-picture prediction, inter-picture prediction, and transform coding.
A further extension of the QTBT concept as suggested in [4], where two further splitting modes where introduced for the binary tree part: A horizontal triple split and a vertical triple split. In the horizontal triple split, an NxM block is split into three blocks: an Nx(M/4), an Nx(M/2), and an Nx(M/4) block (from top to bottom). The vertical triple split, vertically partitions an NxM block into an (N/4)xM, an (N/2)xM, and an (N/4)xM block (from left to right).
Another enhancement of QTBT was proposed in [5] In this proposal, asymmetric binary splits that split an NxM block into two subblocks of size Nx(M/4) and Nx(3M/4), as well as (N/4)xM and (3N/4)xM, were introduced in addition to the symmetric binary splits.
However, these conventional partitioning methods may only provide for a limited flexibility and a limited coding efficiency.
INVENTION
The present invention solves the above mentioned problems by providing encoders, decoders, methods for encoding, methods for decoding and corresponding computer programs according to the independent claims.
The generalized binary splitting that is described herein represents a superset of all mentioned splitting approaches QT, QTBT, QTBT with additional triple splits, and QTBT with additional asymmetric splits into subblocks of sizes 1/4 and 3/4. It is characterized by a simple syntax and is capable of better adapting the block sizes to object and motion boundaries, which results in an improved coding efficiency. The partitioning concept as described herein may also be referred to as Generalized Binary Splitting (GBS). The inventive aspects of the present invention may, at least partially, be based on said GBS partitioning.
The generalized binary splitting can be combined with any other splitting. For example, the quadtree splitting (or any other splitting concept) can be used on a first level and the resulting blocks can be further partitioned using the generalized binary splitting. Or, blocks obtained by the generalized binary splitting can be further partitioned using any other splitting concept.
All of the following aspects of the present invention are interconnected by a common general inventive concept that provides a solution for handling the trade-off between coding efficiency versus coding complexity.
A first aspect is concerned with providing implicit splits and consecutive implicit splits wherein one or more implicit splits may be derived at the decoder side.
According to said first aspect, a block-based encoder is provided for encoding a picture into a data stream, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, signal partitioning information of the binary splitting tree in the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, signal the split option or the
no-split option in the data stream, and if not, refrain from signaling the split option in the data stream.
Furthermore, a block-based decoder is provided for decoding a picture from a data stream, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, obtain partitioning information of the binary splitting tree from the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, obtain the split option or the no-split option from the data stream, and if not, implicitly infer the split option without an explicit split-option-signal in the data stream.
Furthermore, a method for block-based encoding block-based encoding a picture into a data stream is provided, wherein the method comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, signaling partitioning information of the binary splitting tree in the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, the method comprises a step of signaling the split option or the no-split option in the data stream, and if not, the method comprises a step of refraining from signaling the split option in the data stream.
Furthermore, a method for block-based decoding a picture from a data stream is provided, wherein the method comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, obtaining partitioning information of the binary splitting tree from the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, the method comprises a step of obtaining the split option or the no-split option from the data stream, and if not, the method comprises a step of implicitly inferring the split option without an explicit split-option-signal in the data stream.
Furthermore, a computer readable digital storage medium is provided, having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding.
Furthermore, a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
A second concept is concerned with differentiating nodes in a tree depth evaluation, wherein tree nodes according to this aspect may be handled differently than other tree nodes which do not fulfil the requirements of this aspect.
According to this second concept, a block-based encoder is provided for encoding a picture into a data stream, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including at least one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not,
and wherein the encoder is further configured to rule out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
Furthermore, a corresponding block-based decoder is provided for decoding a picture from a data stream, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including at least one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are available in the data stream or not, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not, and wherein the decoder is further configured to rule out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
Furthermore, a method for block-based encoding a picture into a data stream is provided, wherein the method comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and
a) by discriminating whether coding options of one or more proceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not, and wherein the method further comprises a step of ruling out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
Furthermore, a method for block-based decoding a picture from a data stream is provided, wherein the method comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more proceeding tree nodes, via which the predetermined tree node is connected to the root node, are available in the data stream or not, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not,
and wherein the method further comprises a step of ruling out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint. Furthermore, a computer readable digital storage medium is provided, having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding. Furthermore, a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
A third concept is concerned with providing tree type sets comprising one or more tree types, wherein different tree types for partitioning a block may be selected at one or more different partitioning levels.
According to the third aspect, a block-based encoder is provided for encoding a picture into a data stream, wherein the encoder is configured to partition the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, select, for performing the multi-tree partitioning, a tree type set comprising one or more tree types, and signal the tree type set in the data stream, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
Furthermore, a corresponding block-based decoder is provided for decoding a picture from a data stream, wherein the decoder is configured to partition the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, perform the multi-tree partitioning using a tree type set comprising one or more tree types based on a signaling of the tree type set in the data stream, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
Furthermore, a method for block-based encoding a picture into a data stream is provided, wherein the method comprises the steps of partitioning the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, selecting, for performing the multi-tree partitioning, a tree type set comprising one or more tree types, and signaling the tree type set in the data stream, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
Furthermore, a corresponding method for block-based decoding a picture from a data stream is provided, wherein the method comprises the steps of partitioning the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, performing the multi-tree partitioning using a tree type set comprising one or more tree types, wherein the step of performing the multi-tree partitioning is based on a signaling of the tree type set in the data stream, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
Furthermore, a computer readable digital storage medium is provided, having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding.
Furthermore, a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
A fourth aspect is concerned with providing asymmetric quad splits having a splitting ratio different from ½, wherein one or more implicit splits may be derived at the decoder side.
According to said fourth aspect, a block-based encoder for encoding a picture into a data stream is provided, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and signal partitioning information of the splitting tree in the data stream, wherein the encoder is further configured to perform for a predetermined block an asymmetric quad split, in which the
predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and to signal in the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
Furthermore, a corresponding decoder for decoding a picture from a data stream is provided, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and obtain a partitioning information of the splitting tree from the data stream, wherein the decoder is further configured to perform for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and to obtain from the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
Furthermore, a method for block-based encoding a picture into a data stream is provided, wherein the method comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and signaling partitioning information of the splitting tree in the data stream, wherein the method further comprises steps of performing for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and signaling in the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
Furthermore, a corresponding method for block-based decoding a picture from a data stream is provided, wherein the method comprises the steps of partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and obtaining a partitioning information of the splitting tree from the data stream, wherein the method further comprises steps of performing for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and obtaining from the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
Furthermore, a computer readable digital storage medium is provided, having stored thereon a computer program having a program code for performing, when running on a
computer, at least one of the above mentioned methods of block-based encoding and block- based decoding.
Furthermore, a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
A fifth aspect is concerned with an inheritance of partitioning parameters, wherein a currently coded block may inherit one or more partitioning parameters from a further block of a spatial or temporal vicinity.
According to said fifth aspect, a block-based encoder for encoding a picture into a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding is provided, wherein the encoder is configured to partition the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is encoded by means of spatial intra-prediction, the encoder is configured to inherit one or more partitioning parameters from a further block in the spatial vicinity of the at least one block, and to use said inherited one or more partitioning parameters for partitioning said at least one block in the recursive multi-tree partitioning, and if the at least one block is encoded by means of temporal inter-prediction, the encoder is configured to inherit one or more partitioning parameters from a further block in the temporal vicinity of the at least one block, and to use said inherited one or more partitioning parameters for partitioning the at least one block in the recursive multi-tree partitioning, and wherein the encoder is further configured to signal in the data stream that one or more inherited partitioning parameters are used for partitioning the at least one block.
Furthermore, a corresponding block-based decoder for decoding a picture from a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding is provided, wherein the decoder is configured to partition the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is decoded by means of spatial intra-prediction, the decoder is configured to use, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one block, which inherited one or more partitioning parameters are inherited from a further block in the spatial vicinity of the at least one block, and if the at least one block is decoded by means of temporal inter-
prediction, the decoder is configured to use, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one block, which inherited one or more partitioning parameters are inherited from a further block in the temporal vicinity of the at least one block.
Furthermore, a method for block-based encoding a picture into a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding is provided, wherein the method comprises the steps of partitioning the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is encoded by means of spatial intra-prediction, the method comprises further steps of inheriting one or more partitioning parameters from a further block in the spatial vicinity of the at least one block, and using said inherited one or more partitioning parameters for partitioning said at least one block in the recursive multi-tree partitioning, and if the at least one block is encoded by means of temporal inter-prediction, the method comprises further steps of inheriting one or more partitioning parameters from a further block in the temporal vicinity of the at least one block, and using said inherited partitioning parameters for partitioning the at least one block in the recursive multi-tree partitioning, and signaling in the data stream that one or more inherited partitioning parameters are used for partitioning the at least one block.
Furthermore, a corresponding method for block-based decoding a picture from a data stream using at least one of a spatial intra-prediction and a temporal inter-prediction coding is provided, wherein the method comprises the steps of partitioning the picture along a splitting tree from a tree root block into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters, wherein if at least one block is decoded by means of spatial intraprediction, the method comprises a further step of using, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one block, which inherited partitioning parameters are inherited from a further block in the spatial vicinity of the at least one block, and if the at least one block is decoded by means of temporal inter-prediction, the method comprises a further step of using, based on a signaling in the data stream, one or more inherited partitioning parameters for partitioning the at least one block, which inherited partitioning parameters are inherited from a further block in the temporal vicinity of the at least one block.
Furthermore, a computer readable digital storage medium is provided, having stored thereon a computer program having a program code for performing, when running on a computer, at least one of the above mentioned methods of block-based encoding and block- based decoding.
Furthermore, a data stream is provided, which data stream is generated according to the above mentioned method of block-based encoding.
In the following, embodiments of the present invention are described in more detail with reference to the figures, in which
Fig. 1 shows an exemplary partitioning of a 128x128 CTU, Fig. 2 shows the partitioning tree describing the topology in fig. 1 , Fig. 3. shows a description of non-leaf nodes of the partitioning tree in fig. 2
(including block sizes and the splits used to subdivide a particular node described by the split direction and the split ratio),
Fig. 4 shows an example of signaling of a split direction, Fig. 5 shows an exemplary signaling of the split ratio for a particular block size, Fig. 6 shows a reduced signaling from fig. 5 (for cases where not all split ratios are available), Fig. 7 shows three template layouts used to define prohibited partitioning tree
patterns,
Fig. 8 shows exemplary context modeling for a simple binary decision tree for
signaling the split ratio,
Fig. 9 shows exemplary context modeling for another binary decision tree for
signaling the split ratio,
Fig. 10 shows exemplary locations of neighboring blocks, Fig. 1 1 shows an example for redundant parallel splits, Fig. 12 shows an example for redundant perpendicular splits,
Fig. 13 shows an example for higher-order redundancies,
Fig. 14 shows an embodiment of an encoder according to an aspect,
Fig. 15 shows an embodiment of a decoder according to an aspect,
Fig. 16A shows a triple tree split according to the prior art,
Fig. 16B shows a triple tree like split according to an embodiment,
Fig. 17 shows a block diagram of an encoding method according to an embodiment,
Fig. 18 shows a block diagram of a decoding method according to an embodiment,
Fig. 19 shows a block diagram of an encoding and a decoding method according to an embodiment,
Fig. 20 shows a selection of tree type sets according to an embodiment,
Fig. 21 shows a further selection of tree type sets according to an embodiment,
Fig. 22 shows a further selection of tree type sets according to an embodiment,
Fig. 23 shows a block diagram of an encoding method according to an embodiment,
Fig. 24 shows a block diagram of a decoding method according to an embodiment,
Fig. 25 shows an asymmetric quad split based on a reduced GBS partitioning
according to an embodiment,
Fig. 26 shows a block diagram of an encoding method according to an embodiment,
Fig. 27 shows a block diagram of a decoding method according to an embodiment,
Fig. 28 shows picture partitioning according to an embodiment, in which one or more partitioning parameters for partitioning a current block may be inherited by a further previously coded block,
Fig. 29 shows a schematic block diagram of a method for encoding a picture using the concept of Figure 28 in which one or more partitioning parameters for partitioning a current block may be inherited by a further previously coded block, according to an embodiment, and
Fig. 30 shows a schematic block diagram of a method for decoding a picture using the concept of Figure 28 in which one or more partitioning parameters for partitioning a current block may be inherited by a further previously coded block, according to an embodiment.
DESCRIPTION OF THE FIGURES
Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals.
A method of video coding using generalized binary splitting (GBS) is presented in the following passages. Said GBS may provide the basic framework for the concepts of the present invention. Thus, an introduction into GBS is presented in chapters A to E, wherein the concepts of the present invention, that may be based on GBS, may be described in detail afterwards, starting with the chapter“F. FURTHER ADDITIONAL ASPECTS”.
A. GENERALIZED BINARY SPLITTING
The presented generalized binary splitting can be applied for all block partitionings required in video coding. In particular, it can be applied for partitioning a block into coding units (for signaling the coding mode), prediction units (for signaling prediction parameters such as motion parameters), or transform units (for transform coding of the residual signal). In an example, the signaling of the coding mode, the signaling of prediction parameters and the transform coding is applied to the same blocks (i.e., such a block represents a CU, PU, and TU) and the generalized binary splitting is used for specifying the corresponding partitioning of a fixed sized block into coding units (which also represent PUs and TUs). It is, however, also possible to use the generalized binary splitting for specifying the partitioning into CUs, while the resulting CUs can be further partitioned into PUs and/or TUs (either using the generalized binary splitting or any other partitioning concept).
In another example, the generalized binary splitting can be applied individually to color components and auxiliary components, such as alpha channel or depth masks, respectively. Alternatively, generalized binary splitting can be applied to groups of those components. For example, in combination with subsampled chrominance components as used in common frame formats like YUV420 or YUV422, two different splitting trees can be
used: one for partitioning the luma component and other for partitioning the chroma components. In another example, it might be useful to apply a different partitioning (different from the partitioning of luma and chroma components) to auxiliary components (such as alpha blending data or depth maps).
Furthermore, the generalized binary splitting can be combined with any other splitting. For example, the quadtree splitting (or any other splitting concept) can be used on a first level and the resulting blocks can be further partitioned using the generalized binary splitting. Or, blocks obtained by the generalized binary splitting can be further partitioned using any other splitting concept.
Without loss of generality, for the following description, we assume that fixed-sized blocks (such as CTUs) are partitioned using the concept of generalized binary splitting. The splitting structure can be represented as a binary splitting tree (also referred to as “partitioning tree” in the following). Each node represents a binary split with a split direction (horizontal or vertical) and a split ratio (which is the ratio of the width (or height) of the first resulting block and the width (or height) of the original block).
In a further example, the split direction is coded relative to the previous split direction (i.e. , the corresponding split on the next higher level). That means a split is coded as“parallel" or as "perpendicular" split, which always translates to either a“vertical” (splitting the width) or a“horizontal” split (splitting the height). If we assume that the current block was obtained by a horizontal split (at the last split level), then, if a“parallel split” is signaled, the current block is again horizontally split into two subblocks, and if a "perpendicular split" is signaled, the current block is vertically split into two subblocks. If the last level split was a vertical split, the“parallel split" specifies a vertical split and the“perpendicular split” specified a horizontal split. Since on the highest level (i.e., for the root level block) no previous split was signaled, it has to be defined by convention whether the last split level is inferred to represent a horizontal or a vertical split.
In addition to the split direction, a binary split is characterized by the split ratio. The split ratio specifies the ratio of the sizes (in number of samples) of the first resulting subblock (from top/left) and the block being split. Note that, since a block is always split into two subblocks, the split ratio also specifies ratio of the block widths (for vertical splits) or the ratio of the block heights (for horizontal blocks). The leaf nodes of the partitioning tree represent the actual units (consisting of a luma sample block and, for non-monochrome
video, the co-located chroma sample blocks) being coded. These units, which shall be referred to as coding units in the following, are characterized by their positions and sizes, which can be inferred from the partitioning tree. Coding, prediction and transform coding can be applied at the leaf level of the splitting structure. It is also possible to apply the generalized binary splitting to a single color component or a subset of color components (e.g., both chroma components). A perpendicular split with a split ratio of 1/2 is the split that can be signaled using the minimum number of bits (or bins) in most configurations and will referred to as“default split” in the following.
In a further example, starting at the root blocks of the partitioning tree (which may be a CTU), the partitioning is signaled as follows. Without loss of generality, we define that the last split for the root block (which did not result from an actual split) is considered as vertical split. But note that it could also be defined as horizontal split. First a binary decision perpend_split_flag is coded, which indicates whether a perpendicular split is applied. If perpend_split_flag is equal to 1 and the last split (i.e., the split from which a current block resulted) was a horizontal split, a vertical split is applied to the current block. If perpend_split_flag is equal to 1 and the last split was a vertical split (with our above defined convention this includes the case that our current block is the root blocks), this indicates that a horizontal split is applied.
If perpend_sp!it_flag is equal to 0, another binary decision parallel_split_flag is transmitted. If parallel_split_flag is equal to 1 and the last split was a horizontal split, the current block is again horizontally split. If parallel_split_flag is equal to 1 and the last split was a vertical split (or was inferred as vertical split by our convention), the current block is vertically split. If parallel_split_flag is equal to 0, the current block is not further split, but represents a leaf block (equivalent to a“no-split” signaling). If high level syntax restriction (e.g., restrictions on the minimum block width or block height or restrictions on the maximum split level) prevent a horizontal or vertical split, the corresponding binary decisions are not transmitted, but can be inferred at the decoder side. For example, if the last split was a vertical split and the width of the current block is equal to the minimum supported block width, the binary decision parallel_split_flag (which indicates whether the current block is again vertically split) is not transmitted, but inferred to be equal to 0.
Either or both of the binary decisions perpend_split_flag and parallel_split_flag can be inferred if this is indicated by high level parameters, which may be signaled in the PPS (picture parameter set), SPS (sequence parameter set) or slice header. A binary decision
tree for signaling the split direction is shown in fig. 4. If a horizontal split is to be applied, the split size M is defined as the length of the vertical edge of the block to be split (as the split will introduce a new horizontal edge). For a vertical split, the split size M is the length of the horizontal edge of the block to be split.
In a further example, only the perpend_split_flag is coded. Consequently, the split pattern consists of alternating horizontal and vertical splits. Given a particular block, the block is either not further split or it is split using a split direction that is perpendicular to the previous split direction (at the higher level). An exception could be made for the root block (where no previous split exists); for the root block it could be explicitly signaled whether a horizontal or a vertical split is applied (or whether the block is not split at all).
In a further example, the split direction is coded using binary decisions h o r__s pi it_f I a g (indicating whether or not the block is horizontally split) and ver_split_flag (indicating whether or not the block is vertically split). These flags could be coded in any order; the second flag is only transmitted if the first flag indicates no splitting in the corresponding direction. In a further example, the split and the split direction are coded using a split_flag and a sp!it_direction_flag, where the splitj!ag indicates whether the current block is split at all and the split_direction_flag indicates whether the split is applied in horizontal or vertical direction. In a variation, the split_direction_flag indicates whether the split is applied in perpendicular direction to the last split or in the same direction as the last split.
Note that the following description (regarding the signaling of the split ratio) focusses on a non-limiting example using the perpend_split_flag and parallel_split_flag, but it can also be used in connection with any other signaling of the split direction, e.g., (hor_split_flag and ver_split_flag) or (splitjiag and split_direction_flag).
In a further example, the split flags are coded using context-based adaptive arithmetic coding (as CABAC in H.265 | MPEG-H HEVC).
If the signaling of the split direction (using any of the above described approaches) indicates that the current block is split (for this non-limiting example this means either perpend_split_flag or parallel_split_flag is equal to 1 or inferred to be equal to 1 ), a split ratio is coded in addition to the split direction. The set of possible split ratios can be arbitrarily chosen for every block size provided the resulting subblocks can be encoded using the video encoding framework (e.g., an appropriate transformation size has to be available).
The split ratio may be encoded as an element of a set as well as by binarization of the nominator and the denominator (in the former case, the set has to be predefined). In a further example, the split ratio is coded based on a binary decision tree (with adaptive tree pruning according to the available split ratios) as will be described below in more detail. In another example, the split ratio can be coded as an index into an ordered list of available split ratios. The index can be mapped onto a series of binary decisions using any binarization scheme.
In a further example, the overall set of split ratios is defined as: {1/2, 1/4, 3/4, 3/8, 5/8, 1/3, 2/3, 1/5, 2/5, 3/5, 4/5}. For each actual split, the available set of split ratios represents a subset of this overall set. The subset for a particular block and a particular split direction can be determined by the size of the current block and/or the series of previous splits. If some of the ratios are not available for a particular split size (e.g., because restrictions on transform or prediction sizes or minimum block sizes prevent these split ratios or because the splits would result in redundant partitionings), the corresponding signaling flags are not transmitted in order to reduce the bit rate. Especially, if only one split ratio is available, all bins of the ratio signaling can be inferred and are not transmitted (e.g., for block sizes larger than the maximum block size for a non-default split). If, for a split direction, none of the split ratios can be applied (e.g., due to prohibitions enforced by high level parameters), the signaling of this split direction can be inferred as false.
The binary splitting will be applied recursively until no further splitting is inferred or signaled. I.e., if a block is split into two subblocks, for each of the resulting subblocks another split consisting of a split direction and a split ratio is coded (where some or all of the corresponding binary decisions can be inferred based on syntax constraints). This process is continued until it is signaled (or inferred) that no block is further split.
In a further example, the generalized binary splitting concept is applied for partitioning fixed sized blocks (such as CTUs) into coding units, which also represent prediction units and transform units. For each of the resulting blocks, a coding mode and associated prediction parameters (such as intra prediction modes or motion parameters) are transmitted and the blocks are also used for transform coding of the residual signal. In another example, the resulting blocks are used as coding blocks (i.e., for signaling the coding mode), but the coding blocks can be further partitioned into prediction and/or transform blocks (either using the generalized binary splitting concept or any other splitting method).
The high-level parameters controlling the behavior of generalized binary splitting can include a minimum block size (e.g., specified by a minimum block width and a minimum block height, or specified by a minimum number of samples), a maximum partition tree depth, a maximum partition tree depth following a first non-default split, a maximum block size for a non-default split, maximum and minimum allowed aspect ratios for the resulting blocks as well as constraints regarding the applicable split ratios. The high-level parameters may be signaled in a picture parameter set (PPS), sequence parameter set (SPS), slice header or any other high-level syntax structure. To reduce the number of bits required for transmitting the high-level parameters, boundaries and specific domains may be defined for the high-level parameters. For example, the maximum aspect ratio may be defined as a power-of-2 bigger than 4 thus significantly reducing the number of possible options. Or the minimum aspect ratio may be defined as the reciprocal of the maximum aspect ratio or vice versa, in which case only the minimum or maximum aspect ratio needs to be transmitted.
In a further example, both the split direction and the split ratio are coded using context- based adaptive arithmetic coding (CABAC). I.e., these parameters are coded as a series of binary decisions with adaptive binary probability models (a subset of the bins can also be coded with non-adaptive probability models). The used binary probability models for the split decision may be selected based on the size of neighboring blocks. The underlying assumption is that small blocks often occur in the neighborhood of other small blocks (i.e., in finely partitioned areas of a video picture). This or a similar assumption may also be used to model the contexts for the split ratio coding. Note that non-symmetric splits result in a smaller and a larger block and smaller blocks may tend to be located in the neighborhood of other small blocks.
Due to the complex patterns that can be generated using a generalized binary splitting framework, some final partitioning patterns could be generated using more than one partitioning tree. In order to reduce this phenomenon (and to minimize the number of bits required for signaling a particular block partitioning), in a further example, some combinations of splits are prohibited by the syntax. This can be achieved by prohibiting some tree patterns from being introduced into the partitioning tree. A tree pattern is defined using either the actual“verticar’fhorizontai” or the directional '‘parallel7"perpendicular” split direction semantics, possibly mixed. A tree pattern also includes split ratios. If a split direction, split ratio, or a combination of split direction and split ratio would cause a prohibited pattern to occur, it will be inferred as false and the corresponding binary decisions
are not transmitted, but inferred at the decoder side. Note that both the encoder and decoder have to know the prohibited tree patterns (or prohibited split combinations).
To prevent redundant split representations, simple rules can be used to identify such partitionings as will be exemplarily described in the following.
B. CONSTRAINTS ON AVAILABLE SPLIT RATIOS
An example for preventing redundant split representations and constraining the allowed split directions and split ratios is presented in the following. It should illustrate possible implementations of the discussed example and is to be understood in an exemplary, but not limiting, way.
A specific partial partitioning of a 128 x 128 block (e.g., a CTU of size 128 x 128) using the means described in this example is shown in Figure 1. The partitioning tree describing the shown topology is shown in Figure 2, and Figure 3 explains which split directions and ratios were used at specific nodes.
In the following description, we focus (without loss of generality) on restricting the set of split ratios for a given split direction. In principle, for a given block size (width and height), and a given series of splits that led to the current block, an allowed set of split ratios is derived by removing prohibited splits from an overall set of available split ratios. In case the resulting set of split ratios is empty, the corresponding split direction is completely prohibited and the corresponding split direction flag (i.e., perpend_split_flag or parallel_split_flag or any other corresponding binary decision is not coded, but inferred to be equal to 0).
For the general case we can assume that all split ratios a/b are given as irreducible fractions (numerator and denominator are coprime). In addition to the symmetric split with split ratio 1/2, several non-symmetric split ratios are included in the overall set of split ratios. In a further example, if a non-symmetric split with split ratio a/b is included, the split ratio (b- a)/b is also included. Both splits yield the same subblock sizes, but for one of the splits, the
larger subblock represents the left/top part, and for the other split, the larger subblock represents the right/bottom part of the block to be split.
Each pair of split ratios akl/bk and ak2/bk, with ak2 = bk - akl, represents two complementary non-symmetric splits, which result in two subblocks of the same sizes but in different order. Note that it is possible to have bk = bj for k ¹ j, as long as akl ¹ a 1 and akl ¹ Uj2 (see the set in eq. (1 ), where we have /½_]_ = bn = 5). If no symmetric split (with ratio 1/2) is supported, the overall set of split ratios is equal to the overall set of non- symmetric splits, A = Ansym. In a further example, the symmetric split is included in the overall set of split ratios, in which case we have
Criterion 1 : Granularity of block sizes
A first criterion for reducing the set of potential split ratios is related to the resulting block widths or heights. It is rather obvious that the block width (and block height) must be integer numbers, since one cannot code blocks with a fractional number of samples. Typically, it is advantageous that the width/height of each subblock represents an integer multiple of a certain granularity g e N, where M = {1,2, ··· } represents the set of natural numbers. In a further example, we set g = 4. Even though it is reasonable to have the same granularity for block width and block heights, it is also possible to use different granularities for the horizontal and vertical block dimensions.
Consequently, a split with split ratio a/b is only supported if the split size M (i.e., the width (for vertical splits) or height (for horizontal splits) of the block to be split) satisfies the following equations
As an example, we consider the overall set of split ratios given in eq. (1) and assume a granularity of g = 4. For the split size of M = 32, the set of available splits is reduced to the set For a split size of M = 20, the set of available splits is reduced to the set
reduced to the set
Criterion 2: Block sizes, split depth and aspect ratios
In addition to the granularity, the set of available splits can be constraint by a minimum block size. That means, if a split would results in a block size that is smaller than a minimum block size, it is prohibited and removed from the list of available splits. The minimum block size can be specified by a minimum block width and a minimum block height, or it can be specified by a minimum number of samples. For example, with a granularity of g = 4 and a minimum block size of 32 samples, the smallest supported block sizes would be 8x4 and 4x8, such a configuration could not be achieved by separately specifying a minimum block width and minimum block height.
Another possible constraint is the minimum and maximum aspect ratio. If a split would result in any subblock with an aspect ratio (block width)/(block height) greater than a maximum aspect ratio or less than a minimum aspect ratio, the corresponding split is removed from the set of available splits. As noted earlier, the minimum aspect ratio could be set equal to the reciprocal of the maximum aspect ratio. As an example, the maximum aspect ratio could be set equal to 8.
A further constraint could restrict the maximum split depth. The split depth represents the depth of the binary split tree structure. Or in other words, for a particular block, the split depth represents the number of splits required to reach the current block from the root block. If the split depth of a current block is already equal to the maximum split depth, no further split is allowed. As an example, the maximum split depth could be set equal to 3.
It is also possible to introduce constraints so that for the first split levels (up to a certain depth) or, alternatively, for all split levels greater than a particular value, only symmetric
splits are allowed. In this case, no split ratio is transmitted for the corresponding splits (but the ratio is inferred to be equal to 1/2).
Parameters which control the minimum block size, the minimum and maximum aspect ratio, the maximum split depth, and/or values restricting the use of non-symmetric splits are preferably transmitted in a high-level syntax structure, similar to the sequence parameter set, the picture parameter set, or the slice header of H.265 | MPEG-H HEVC.
Criterion 3: Redundant parallel splits
Without further restrictions, the generalized binary splitting typically results in redundant representations for certain partitionings. In order to avoid such redundancies, additional restrictions need to be introduced. Note that redundancies in the syntax make the code for specifying the partitioning less efficient and should be avoided. Additionally, if redundant partitionings are tested in the encoding algorithm, the complexity is unnecessarily increased.
An exemplary set of restrictions for the overall set of split ratios given in eq. (1) is summarized in Table 1. Each restriction corresponds to one of three layouts (a, b, c), i.e., subsets of a binary splitting tree; these layouts are depicted in Fig. 7.
As a first aspect, we concentrate on redundancies in consecutive parallel splits. As an example, we consider a 16x16 block, as illustrated in Fig. 11. If the block is first split using a split ratio of 3/4 in horizontal direction and the first of the resulting subblocks is further split with a split ratio of 1/3 in the same direction (see "tree A" in Fig. 11), we obtain a 16x4, a 16x8, and a 16x4 block (in that order). Exactly the same partitioning is obtained if the first split uses a split ratio of 1/4 in horizontal direction and the second of the resulting blocks is split (in the same direction) with a split ratio of 2/3 (see“tree B” in Fig. 11).
to the first subblock. As a consequence, we obtain three blocks with block dimensions dx = M SQSx, d2 = MSq( 1 - Si), and d3 = M( 1 - s0),
where M is the split size, i.e., the block dimension of the original block. Now, if we apply a first split s'0 and then split the second subblock with a split s in the same direction, we yield three blocks with dimensions
Whenever it is possible to yield the same block dimensions d = cfl t d2 = cf2, and d3 = d'3 with two parallel splits s0 and sx (s1 is applied to the first subblock) and two parallel splits s' o and s (s is applied to the second subblock), one of the combinations (s0, s1) or (s'o/ s'i) should be prohibited.
This is always the case whenever s'0 = s0s! and 1 - s0 = (1 - s'i)(l - 's0). The decision which combination is prohibit could depend (for example) on which of the redundant combinations requires a smaller number of binary decisions for signaling the partitioning.
For the overall set of split ratios given in eq. (1), the first part of Table 1 specifies an example for prohibited parallel splits. The parallel splits that are prohibited for the first subblock are labeled by“layout a)” and the parallel splits that are prohibited for the second subblock are labeled by“layout b)”. Note that the redundancy for consecutive parallel splits could also be removed by only prohibiting splits for the first subblock, or only prohibiting splits for the second subblock, or by using another combination of prohibiting splits for the first and second subblock.
Criterion 4: Redundant perpendicular splits
In addition to redundant parallel splits, there are also redundant perpendicular splits. Assume we have a block that is first split using a split ratio s0 in horizontal direction and then both of the resulting subblocks are split in vertical direction using the split ratio s (see Fig. 12a). The same partitioning is obtained if the original block is first split in vertical direction using the split ratio s± and then both of the resulting subblocks are split in horizontal direction using the split ratio s0 (see Fig. 12b).
Note, however, that although the resulting partitionings of both discussed options are the same, the corresponding coding options can be different. The reason is the different coding order of the subblocks. Typically, the coding order is defined in a way that for each split, the first subblock (top or left subblock) or all blocks contained in the first subblock i s/a re coded
before the second subblock (bottom or right subblock) or all block contained in the second subblock. Hence, if we first apply a vertical split and then a horizontal split for each of the subblocks, the four resulting blocks would be transmitted in the following order: top-left, bottom-left, top-right, bottom-right (see Fig. 12b). In contrast to that, if we first apply the horizontal split and then the vertical split to both subblocks, we obtain the coding order; top- left, top-right, bottom-left, bottom-right (see Fig. 12a). Due to the different coding order of the resulting subblocks, the methods for intra-picture prediction, motion vector prediction, derivation of merge candidates, etc. could yield different results and, hence, the different split could result in different representations of the original block.
Nonetheless, the described redundant perpendicular splits are typically still very similar, so that prohibiting these splits could actually improve coding efficiency. Depending on the actual codec and the employed encoding algorithm, the loss in coding efficiency due to not considering both of the described coding options may be outweighed by the bit rate savings obtained by using shorter codewords for signaling the partitioning. Furthermore, a potential gain by considering both ways of splitting is only achievable if the encoder evaluates both options, which would increase the encoding complexity.
So, in summary, it depends on the actual coding options and the application whether it is preferable to allow both options in the syntax. In a further example, the described redundant perpendicular splits are prohibited. In another example, the potential redundancy is supported in the syntax. And in another example, the bitstream contains a syntax element, which indicates whether redundant perpendicular splits are allowed or prohibited by the syntax. In the latter case, the actually used split ratio syntax is depending on the value of a corresponding syntax element. The syntax element that indicates whether redundant perpendicular splits are allowed or prohibited could be transmitted as part of any high-level syntax structure, such as the SPS, PPS, or slice header. It is also possible that this syntax element is transmitted as part of the low-level syntax, as for example, the CTU syntax or the CU syntax.
If redundant perpendicular splits are prohibited (either by convention or indicated by a high- level syntax element), it has to be defined which of the two variants is prohibited and which of the variants is supported. This decision could depend on which variant requires less binary decisions for coding or which variant most likely provides better intra-prediction signals or motion vector predictors, etc. For the example of the overall set of split ratios given in eq. (1), Table 1 gives an example for specifying prohibited redundant splits. The
prohibited perpendicular splits are marked with“layout c)”. They have to be interpreted in the following way. If a given block is split in a particular direction with split ratio A (“A-Ratio” in the table) and the first of the resulting subblocks is split in perpendicular direction with split ratio B (“B-Ratio” in the table), then the second subblock (i.e. the second subblock of the original block) cannot be split in perpendicular direction (perpendicular to the split of the original block) with split ratio C (“C-Ratio” in the table). Note that the split ratio B is always identical to the split ratio C. Hence, in other words, if a block was split in direction da with split ratio a and the first resulting subblock was split in perpendicular direction (perpendicular to da) with split ratio b, then the perpendicular split (perpendicular to da) with split ratio c = b is removed from the set of available splits for the second subblock (of the split with direction da and ratio a) if indicated in the table. As noted above, the convention specified in Table 1 has to be considered as a particular example. There are other possibilities to achieve a very similar prohibition of redundant splits.
Figure 7 shows three different partitioning tree patterns that are prohibited if specific split directions and ratios for the nodes A, B, and C are met. In Table 1 , all configurations prohibited in the described example are listed. The local topology produced by those patterns can be exactly replicated using a different partitioning tree pattern. Note that the prohibitions are chosen in correspondence to the least expensive splits. In this particular case, Layout a) with A-Ratio 3/4 and B-Ratio 2/3 yields the same pattern as Layout b) with A-Ratio 1/2 and B-Ratio 1/2. However, the former is prohibited whereas the latter is allowed due to the fact that 1/2 is less expensive in signaling (when using the signaling scheme described below).
Table 1 - Example for prohibited splits
Layout A-Dir A-Ratio B-Dir B-Ratio C-Dir C-Ratio
Layout A-Dir A-Ratio B-Dir B-Ratio C-Dii atio
Potential additional constraints
It is possible to further constrain the available splits for a current block, either in order to prevent additional redundancies or in order to obtain a more restricted concept for partitioning. Some important examples are listed below:
1. Higher-order redundancies
Beyond the above described redundancies in consecutive parallel and consecutive perpendicular splits, there might be more redundancies when considering more splitting orders. As an example, consider a 16x16 block at location (0,0) and the following two ways of splitting (which are both not prohibited according to the rules in Table 1 ).
Method A: (1) Vertical split with ratio 1/4 applied to 16x16 block at location (0,0)
(2) Horizontal split with ratio 1/2 applied to 4x16 block at location (0,0)
(3) Vertical split with ratio 2/3 applied to 12x16 block at location (4,0)
(4) Horizontal split with ratio 1/2 applied to 8x16 block at location (4,0)
Method B: (1) Vertical split with ratio 3/4 applied to 16x16 block at location (0,0)
(2) Horizontal split with ratio 1/2 applied to 12x16 block at location (0,0)
(3) Vertical split with ratio 1/3 applied to 12x8 block at location (0,0)
(4) Vertical split with ratio 1/3 applied to 12x8 block at location (0,8)
As illustrated in Fig. 13, both methods yield the following partitioning into 5 subblocks: 4x8 block at location (0,0), 8x8 block at location (4,0), 4x8 block at location (0,8), 8x8 block at location (4,8), and 4x16 block at location (12,0). Similar as for the redundant perpendicular splits discussed above, the coding order for both methods is usually different (in Fig. 13, the coding order is indicated by the numbers inside the resulting blocks), so that both ways of splitting actually represent different coding modes. Such higher-order redundancies could be prohibited, or they could be allowed by the syntax.
2. Restrictions for split chains
For restricting the partitioning to selected split patterns, it can be beneficial to constrain series of splits. As a simple example, consider the overall set of splits An example
for restricted split series is the following: The symmetric split 1/2 is always allowed, as long as other conditions (e.g., a minimum block size) are not violated. But the split 3/4 is only allowed if it is followed by a 1/3 split in the same direction for the first subblock. The split 1/3 is only allowed if the current block represents the first subblock of a previous 3/4 split in the same direction. Such a configuration/syntax would actually lead to the same partitioning scheme as QTBT with additional triple splits (triple split are splits that partition a block into three subblocks with sizes 1/4, 1/2, 1/4, see state-of-the-art [4]).
There are, however, many more possibilities for introducing constraints in order to achieve a certain restriction of the available partitionings.
According to an example, only such chains of splits are allowed that result in partitionings for which all resulting blocks have a width w and a height h that represent an integer power of 2 (i.e., w = 2",n e N and h = 2m,m e N). In order to keep the signaling efficient, corresponding constraints can be directly included into the syntax. They can be specified using tables or any other means.
3. Tree adaptation sets
Beside the tree modification due to the elimination of redundant branches and constraints introduced by aspect ratio limitation, further modification of the default generalized binary split tree might be summed up in a tree adaption set, that can be activated implicitly or explicitly by transmitting corresponding syntax elements. A tree adaption set could define a specific set of leaf nodes or branches, or a set of rules to select leaf nodes or branches
added and/or removed from the default tree. The aspect of an explicitly signaled tree adaption set might include the signaling of the use of a tree adaption set and/or the signaling of an index, selecting a tree adaption set out of a predefined record of tree adaption sets, using variable or fixed length codes embedded in any high level syntax structure. Implicit signaling might range from deriving the use of additional tree adaption sets as well as selecting a tree adaption set out of a record by evaluation of syntax elements coded in any high level syntax structure and/or on CTU-level or below. One aspect could link specific profile and level syntax elements to specific tree adaption sets, in order to limit complexity or eliminate unsupported block width, block height or block aspect ratios, which might not be compatible with certain profile-level combinations. Furthermore, a tree adaption set might be derived from syntax elements of the temporal or spatial neighborhood, restricting or expanding the default tree, according to assumed local requirements.
In a further example, an implicit tree adaption set could be activated by the extraordinary shape of a block at a picture, tile or shape boundary. This might also include the overriding of generally applied restrictions like aspect-ratio restrictions. Note that at picture or tile boundaries, a coding tree unit often has a different block size. For example, assume an HD picture of 1920x1080 luma samples and a CTU size of 128x128 luma samples. Then, the CTUs at the bottom of the picture only have a size of 128x56 luma samples. Here, while the CTUs are initially still considered to consist of 128x128 luma samples, an implicit splitting could be inferred. As an example, a horizontal split with split ratio 1/2 and a following horizontal split with split ratio 7/8 for the first subblock could be inferred, which results in a 128x56 block; the remaining blocks that only include samples outside the picture are not coded. Instead of inferring implicit splits, the binary decision tree for transmitting the split ratio could also be adapted on a picture or tile boundary. For example, splits that would introduce a block border outside the picture could be eliminated for the blocks at a boundary. This concept could also be used in connection with implicit splits. For example, while a block does not completely lie inside the picture/tile region, it is inferred that the block is vertically (at right boundary) or horizontally (at bottom boundary) split, but for transmitting the split ratio, all splits that would introduce a block border outside the picture/tile region are eliminated from the corresponding binary decision tree (i.e., the corresponding binary decision in the tree used for signaling the split ratio are inferred to be equal to 0 or 1 ).
High-level syntax elements constraining the binary split syntax
According to an example, the high-level syntax includes syntax elements by which the encoder tells the decoder the actual configuration for the binary splitting tree. In certain applications, the encoder complexity only allows to evaluate a subset of the splitting options that are supported by the syntax. In order to not transmit binary decisions that will always have the same values, the encoder signals the decoder, which subset of the splitting options are used. The high-level syntax elements are transmitted in one or more high-level syntax structures, such as the sequence parameter set (SPS: valid for all pictures inside a coded video sequence), the picture parameter set (PPS: valid for one or more pictures), the slice header (valid for a slice), etc.
The high-level syntax elements that restrict the available splitting option can include one or more of the following parameters:
• a minimum block sizes (e.g., specified by a minimum block width and a minimum block height, or specified by a minimum number of samples);
• a maximum partition tree depth;
• a maximum partition tree depth following a first non-default split (i.e., the tree depth after the first split with a split ratio not equal to ½ is limited);
• a maximum block size for a non-default split (i.e., splits with a ratio not equal to 1/2 are only supported if the block size is greater than or equal to a given size);
• a minimum block size for a non-default split (i.e., splits with a ratio not equal to 1/2 are only supported if the block size is less than or equal to a given size);
• maximum and minimum allowed aspect ratios;
• a parameter indicating a subset of the split ratios (i.e., only split ratios included in the specified subset are supported);
• a parameter indicating that only perpendicular splits are supported, possibly for a restricted set of block sizes (i.e., with exception of the first split or the first N splits);
• a parameter constraining series of splits (see above);
• a parameter specifying whether redundant perpendicular splits (or generally, splits that result in the same partitioning, but with different coding orders for the resulting subblocks) are allowed.
The parameters can be included in more than one high-level syntax structures. For example, global parameters specified in the SPS can be overridden by parameters specified in the PPS. C. CODING OF THE SPLIT RATIO
Fig. 5 depicts signaling diagrams for an exemplary split size of 64 (assuming no further restrictions). In order to avoid the transmission of obsolete binary decisions, a special algorithm for pruning the binary decision tree can be employed. A non-limiting example uses a special algorithm to prune the signaling tree (an example for a full decision tree is shown in fig. 5). The concept uses basically the same tree of binary decisions for signaling all split ratios, but before any binary decision (bin) is actually coded, the tree is pruned based on the availability of the split ratios. Or in other words, binary decisions that can be inferred to be equal to 0 or 1 based on the availability of the split ratios are not transmitted, but inferred at the decoder side.
Hence, starting with the full binary decision tree, the outputs of the signaling tree (leaf nodes of the tree) are marked according to their availability, where the availability can be derived according to any combination of the rules described above (including, minimum block size, block granularity, minimum and maximum aspect ratio, redundant parallel splits, redundant perpendicular splits, etc.). Each node contains an information about the context model that is used to encode the specific bin using the CABAC engine.
The reduction (or pruning) algorithm reduces the decision tree as follows. Initially, it is assumed that all nodes of the binary decision tree for signaling the split ratio are available. Then, the leaf nodes of the tree are marked as available or unavailable according to the availability of the corresponding split ratios (which is derived according to any rules, for example, the rules or a subset of the rules specified above). Afterwards, the decision tree is pruned from the bottom to the top according to the following two rules: (1 ) If a node has no available child node (i.e. , both child nodes are marked as unavailable), the node is marked as unavailable (i.e., it is finally removed from the decision tree). (2) If a node has only one available child, it is replaced with that child (the child is moved up by one level in the decision tree). As outcome of the described tree pruning algorithm, a new binary decision tree is obtained. Each non-leaf node of the tree has two available child nodes and each leaf node represents one available split ratio. Note that the obtained binary tree can represent the original full decision tree or a smaller binary decision tree.
In figure 6, three examples of reduced signaling decision trees are shown (for the full decision tree of fig. 5). In fig. 6a), an example of a reduced tree without possible outcomes {3/8, 5/8} is presented. In figures 6b) the outcome {1/4} is not available. In figure 6c), the ratios {1/2, 3/8, 1/4} are not available. By comparing the trees in fig. 6 with the full tree in fig. 5, it can be observed which nodes are eliminated and which nodes are retained (the nodes are labelled by a context model id, which could be used to code the particular decision). In figures 5, 8 and 9, binary decision tree for coding the split ratio for different split sizes are presented. In a further example, the binary decisions are arithmetically coded using adaptive binary probability models. The selection of the probability model could depend on any of the following (or a combination thereof):
• Node in decision tree;
• Current depth of binary split tree;
• Split direction;
• Split direction in spatial neighboring blocks;
• Block sizes in spatial neighboring blocks;
• Split direction in temporal neighboring blocks;
• Block sizes in temporal neighboring blocks.
In a further example, the assignment of probability model sets (also referred to as context model sets) to binary decision nodes is not modified for reduced decision trees. That means, for each binary decision, the same context model set (among which a probability model is selected based on additional parameters) is used, regardless of whether the binary decision is coded as part of the full decision tree or any form of a reduced decision tree.
According to an example, the depth of a tree following a non-default split is restricted. Thus, a first non-default split would mean that scarce partitioning is done and the last few available depths will be used to fine-adjust the block edges. Translating this reasoning into the context of the present invention, selection of probability models for the binary decisions labelled with“ctxO” and“ctx1 " (in figs. 5 and 6) is based on the current depth of the split tree, but
different sets of probability models are used for“ctxO" and“ctx1”. The binary decisions labeled with“ctx2”,“ctx3”,“ctx5”,“ctx7" and“ctx8” (in figs. 5, 8, 9) are all coded using the same probability model; for each decision tree, the probability model is selected based on the current depth of the splitting tree and the sizes of the neighboring blocks. If a subdivided block has small neighboring blocks on one end and large neighboring block on the other, it is assumed that the smaller of the blocks resulting from current subdivision will be placed at the side neighboring with other small blocks. The probability model for the binary decision labelled with“ctx6” is selected based on how much do the sizes of neighboring blocks vary (if they vary much, it is assumed that the 1/5 or 4/5 ratio could be more probable as they result in subdivisions with higher difference of areas). A specific example could define the context models as follows. A split area is shown in fig. 10.
The anchors for neighbor access are named A1 , A2, B1 , and B2. The area is assumed to a special representation of a specific node in a split tree with a specific depth d. On the path to this node, first n £ d splits were default splits, thus simulating a quad-tree of depth q =
[ J For coding of decision“ctxO” and“ctx1” probabilities P(ctx 0 = 1) = P0(ctx 0 = l \ q),
P(ctx 1 = 1) = P^ctxl = 1 ) q) are selected. For coding of decision“ctx2”,“ctx3”,“ctx5", “ctx6”,“ctx7” and“ctx8”, the neighbors are inspected (the neighbors are defined as the units anchoring at the split block according to fig. 10). If the split direction is horizontal,
is the neighbor at anchor B1 and N2 is the neighbor at anchor B2. If the split direction is vertical Ni is the neighbor at anchor A1 and /V2 is the neighbor at anchor A2. The neighboring blocks have a specific areas area^ and area(/V2). The context offset ctx0ff is defined as 1 if area (Ay equals area(iV2). If area(JVt) is greater than area(JV2) the ctxoff is defined as 0. In all other cases it is defined as 2. Probability of contexts“ctx6” is defined as P(ctx 6 = 1) = P2 (ctx 6 - 1 1 ctXgff). It is also defined that P2(ctx 6 = 1 1 0) = P2{ctx 6 = 1 12). For the other contexts the probability is defined as follows P(ctx 2 = 1) = P(ctx 3 = 1) = P(ctx 5 = 1) =
In another example, the selection of the binary probability models used for coding the binary decision is based on the assumption that a partition tree of a specific CTU in a picture is similar structured compared to a corresponding partition tree of a so called collocated CTU in an already decoded picture. Without loss of generality, a collocated CTU might be defined as CTU that is obtained when accessing a CTU in a given reference picture using the coordinates derived from the current CTU’s position and size, for example the CTU’s center position. From the obtained collocated CTU, split depth or CTU dimension, or both, or any
other syntax element of the collocated CU might be used to select probability models for signaling split decisions (in fig.4). Independently, syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element from the obtained collocated CU might be used to select probability models for signaling split ratio decisions (in fig.5-fig.9).
In another example, syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element might be used to infer one or all binary split decisions (in fig.4) without additional signaling. Independently, syntax elements from the obtained collocated CTU, split depth or CTU dimension, or both, or any other syntax element from the obtained collocated CU might be used to infer one, any or all split ratio decisions (in fig.5-fig.9).
D. ENCODER OPERATION
In a typical modern video encoder, the decision between multiple coding modes is based on a minimization of a Lagrange function J = D + l · R, where is a distortion measure between the original and reconstructed block (for example, the sum of squared differences (SSD)) for a given mode and R represents the number of bits (or an estimate thereof) that are required for transmitting the block in the given mode (including the bits for partitioning, side information such as intra prediction modes or motion parameters, and quantization indexes for the transform coefficients). This Lagrangian approach can also be used for deciding between different split modes. For obtaining optimal decisions, it is important to proceed the encoding using a depth-first approach. That means for calculating the rate- distortion cost / for a certain split mode, the best rate distortion cost for each of the subblocks (including further splits) has to be considered. This can be ensured by using a depth-first approach. The cost for a certain split mode is calculated as follows. First the cost of the first subblock is determined. At this point, the best coding mode (in rate-distortion sense) for the first subblock including all possible split modes is determined. Given the best coding mode for the first subblock, the best coding mode for the second subblock is determined (again, potential splits have to be taken into account). The rate-distortion cost J for a split mode is the sum of the rate-distortion costs for the subblocks. This concept can be implemented using a recursive function, similarto the following (simplified) pseudo-code: function rd_cost( block ) {
bestjnode =“no split”
min_cost - distortion( block ) + lambda * rate( block ) for( split_mode ) // loop over all split modes
{
cost = rd_cost( first_subblock( split_mode ) )
+ rd_cost( second_subblock( split_mode ) )
if( cost < min_cost )
{
best_mode = split_mode
min_cost = cost
}
}
return min_cost
}
Note that an encoder does not need to evaluate all coding options supported in the syntax. For decreasing the encoder complexity, it is typically beneficial to only test the most likely splitting modes and skip the evaluation of unlikely modes. Most notably, the maximum depth of a tree after the first non-default split is a very important parameter to balance the expected rd-performance and runtime. The maximum block size at which a non-default split is allowed can be used to restrict the testing of partitionings with large blocks, when much smaller blocks are more probable (e.g. for intra slices). Furthermore, the maximum aspect ratio tested in an encoder may be smaller than that supported by the syntax (correspondingly, the minimum aspect ratio tested may be larger than the minimum aspect ratio supported by the syntax).
Beyond the reduction of redundancies additional heuristic rules that do not change signaling may be incorporated to speed up the split decision process. For instance, during the rd- search a complete set of splits along one direction may be excluded if the corresponding symmetric split gives no improvement compared to the symmetric split in perpendicular direction. This, in particular, implies that a block will not be further divided after the two symmetric splits have been tried and did not result in an rd-cost smaller than that for coding the block without any split. The more asymmetric splits are supported by the syntax, the more such heuristic rules speed-up the encoding process. Another constraint may be set on the aspect ratio of the resulting splitting scheme (if one implements it as encoder only optimization, this would have no impact on signaling).
E. ADDITIONAL ASPECTS
The generalized binary splitting framework for partitioning blocks into coding units, prediction units, transform units, etc. , may affect other parts of a video codec. Some of these aspects are described in the following.
1, Adaptation of transform coding The described partitioning with generalized binary splits typically results in block width and block heights that do not represent integer powers of 2. In state-of-the-art video codecs (HEVC [1] or JEM [3]), such transform sizes are not supported. Hence, some modifications of the transform coding are required. Both the transforms as well as the entropy coding of the transform coefficient levels (quantization indexes) need to be modified/extended.
While state-of-the-art video codecs typically only specify power-of-two transform sizes, the generalized framework requires that additional transform sizes are specified.
Another aspect is related to a coding of the last non-zero transform coefficient in a transform block. In HEVC, the position of the last non-zero transform coefficient is indicated by transmitting the x-coordinate and the y-coordinate of the last non-zero transform coefficient. The corresponding entropy codes were designed for blocks for which both the width and height represent integer powers of two. The following table is taken from Recommendation ITU-T H.265 1 1 SO/I EC 23008-2; the bold marked part specifies the binarization process for the syntax elements last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, !ast_sig_coe ff_x_.su ffix, last_sig_coeff_y_suffix, which specify the location of the last nonzero transform coefficient in a transform block.
In a further example, the binarization for the syntax elements last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix is modified as follows:
Version 1
Version 2: Note the correction of the suffix cMax-value is only done for the T rafoSize 10, 20, 40, etc. (3/8-5/8 splits).
This results in the following binarization tables.
Code-tables for version 1 (for selected values of Trafosize):
Code-table for version 2 (for selected values of Trafosize):
The binarization codes for TrafoSize equal to an integer power of 2 are not changed. Note that version 1 includes redundancies (unnecessary bins for certain codewords), while the codeword tables for version 2 do not include redundant bins.
2. Hadamard transforms for motion search / evaluation of intra prediction modes
In consensus with the previous section the new partition structure requires various new transforms among them Hadamard transform matrices for prediction residuals. Often the Hadamard transform is used as a very simple approximation of a DOT. In typical encoding algorithms, the sum of absolute Hadamard coefficients is, for example, used as rate- distortion measure in the sub-sample motion search or the pre-selection of intra prediction modes.
For the particular example with available split ratios {1/2, 1/4, 3/4, 3/8, 5/8, 1/3, 2/3, 1/5, 4/5, 2/5, 3/5 }, the new luma block dimensions 96, 80, 48, 40, 24, 20, and 12 appear if the maximal CTU size is 128. The corresponding Hadamard matrices are not all explicitly constructed. In fact it is sufficient to construct Hadamard matrices of size 20 and 12 using Paley’s construction via finite fields. The other sizes are then constructed by Kronecker products since Kronecker products of Hadamard matrices are Hadamard matrices again. The construction of the other matrices via Kronecker products have the advantage that a full large sized Hadamard matrix never has to be constructed entirely. For example, instead of applying an 80x80 Hadamard matrix it is sufficient to use a known 4x4 Hadamard matrix for the respective subblocks. For the final part, the summation of subblocks is done with respect to the correct signs given in the 20x20 Hadamard matrix. All other sizes can be constructed in the same manner using either the new 20x20 or 12x12 Hadamard transform.
3 Derivation of merge candidates
Due to more complex block partitioning (compared to state-of-the-art partitioning methods), the number and topology of neighbors of a particular block (leaf of a partitioning tree) may vary. Coding parameters of neighboring block are, for example, used for deriving merge candidates, motion vector prediction candidates, or most probable intra prediction modes. In connection with the generalized splitting framework, an adaption of the state-of-the-art strategy for selecting candidates among the neighboring blocks (see HEVC) could led to an improved coding efficiency. The state-of-the-art strategies often implicitly assume that blocks have a square shape. But with the generalized splitting concept, the block width can
be much larger than the block height or vice versa. In that case, it can be preferable to consider more candidate block along the long block edge than along the short block edge. This may include but is not limited to: Scanning for all neighbors along the edges of a block (e.g. to find an extremum or a mean of a measure derived from the neighboring blocks); defining additional neighbor anchor points (e.g. in the case of scanning for merge candidates, adding anchors in the middle of the left and upper edge of a block).
F, FURTHER ASPECTS
As outlined above, encoder side operation, and in particular encoder side operation of the GBS concept, may be restricted in a trade-off between complexity and coding efficiency, e.g. through reduction of considered modes. At the same time, such restrictions can be facilitated in the signaling to reduce necessary bits, e.g. through implicit derivation of syntax elements on decoder side. The following aspects introduce embodiments and examples for further restrictions and extensions.
1. Further restricting sp lit chains with respect to node size or node type for trade off coding efficiency versus complexity
1.1 Implicit splits
As mentioned above, a size of a block, and in particular a size of a leaf block (which may be a CU, for example), may be restricted to powers of 2, i.e. the binary logarithm of height and width in number of samples must be an integer number, which blocks may then also be referred to as log2 partitions.
Accordingly, such blocks may comprise (in number of samples) a height and/or a width of, for example, 2°, 2\ 22, .... 2" samples. As a non-limiting example, a block comprising 64 x 128 samples (height x width) may be referred to as a log2-partition, while a block comprising 64 x 72 samples (height x width) may be referred to as a non-log2-partition.
According to an aspect of the invention, if restricting a codec, such as a GBS based codec, to CU sizes to powers of 2, the signaling of specific split decisions on encoder side may be omitted in favor of implicit derivation on decoder side. For instance, in the case that a node of a splitting tree is a non-log2 partition with respect to either its height or width, further splitting is inevitable to progress towards log2 partitions that can host a CU as per the given constraint. Accordingly, the encoder may perform an implicit split of a non-log2-partition in order to create two or more log2-partitions. Split indications for such
implicit splits of non-log2 partitions can hence be inferred at decoder side and explicitly signaling is necessary only for log2 partitions.
Figure 14 shows an embodiment of an inventive encoder 10 for performing such implicit splits. The encoder 10 is a block-based encoder for block-based encoding a picture 12 into a data stream 14.
The block-based encoder 10 is configured to partition the picture 12 into one or more leaf blocks 15 using recursive multi-tree partitioning along a binary splitting tree 13, wherein a height and a width of the leaf blocks 133, 134, 13s, 13e, 13s, 13g in number of samples is a power of two.
The blocks in which the picture 12 is partitioned may also be referred to as partitions.
Each of the nodes 13i, 132, .... 13g of the splitting tree 13 may belong to one partition.
For example, the partitioning of the picture 12 may comprise four root partitions, e.g. four CTUs. In Figure 14, only the CTU in the upper right corner of the picture 12 will be exemplarily described in more detail now.
The splitting tree 13 may comprise a root node 13i at a first partitioning level, or root block level, which root node 13i belongs to the aforementioned CTU. This root block may be partitioned along the splitting tree 13 to one or more leaf blocks 133, 134, 135, 13e, 13e,
1 3g, which may be located at different partition levels.
Therefore, the aforementioned CTU may first be split, at a first partitioning level, by a quad split (QT) such that four subblocks result. In the splitting tree 13, the four tree nodes 132,
133, 134, 13d at the second partitioning level result from said quad split. For the purpose of easier referencing, the reference numbers of the respective tree nodes of the splitting tree 13 are indicated inside each block of the picture 12 to which the respective tree node belongs.
For example, the blocks to which the tree nodes 133, 134, 13s belong may already be leaf blocks, wherein no further partitioning takes part. According to the convention of this aspect, these blocks to which the tree nodes 133, 134, 13s belong may be Iog2-partitions.
However, the block to which the tree node 132 belongs may not yet be a leaf block.
Furthermore, it may be a non-log2 partition. Since it is not yet a leaf block, it will therefore be subjected to further partitioning. In the depicted non-limiting example, this block is further partitioned, at the second partitioning level, using a bi-split, for instance a BT or a
GBS split. Accordingly, the tree node 132 may be partitioned at the second partitioning level resulting into two further nodes 13e, 137.
The block to which the tree node 13e belongs may be a leaf block, wherein no further partitioning takes part. According to the convention of this aspect, the block to which the tree node 136 belongs may be a log2-partition.
However, the block to which the tree node 137 belongs may not yet be a leaf block.
Furthermore, it may be a non-log2 partition. Since it is not yet a leaf block, it will therefore be subjected to further partitioning. In the depicted non-limiting example, this block is further partitioned using a bi-split, for instance a BT or a GBS split. Accordingly, the tree node 13e may be partitioned at the third partitioning level resulting into two further nodes 13e, 139.
The blocks to which the tree nodes 130, 13g belong may be leaf blocks, wherein no further partitioning takes part. According to the convention of this aspect, these blocks to which the tree nodes 13s, 13g belong may be log2-partitions.
The encoder 10 according to this aspect may explicitly signal log2-partitions in the data stream 14. However, the encoder 10 may implicitly split non-log2-partitions further in order to create two or more log2-partitions at subsequent levels. The one or more splits are done implicitly. That is, for said one or more implicit splits the encoder 10 refrains from signaling such implicit splits in the data stream 14. Therefore, the encoder 10 may have to check whether a partition is a log2-partition or a non-log2-partition.
In other words, the encoder 10 of this aspect is configured to signal partitioning
information of the binary splitting tree 13 in the data stream 14 by checking, for a tree node 13i, 132, ... , 139 of the binary splitting tree 13 comprising a set of split options including a no-split option and a split option, whether a partition to which the respective tree node 13i, 132, ... , 130 belongs, has a height and a width which are in number of samples a power of two.
If yes, i.e. if the current partition (or block) is a log2-partition, then the encoder 10 is configured to signal the split option or the no-split option in the data stream 14.
If not, i.e. if the current partition (or block) is a non-log2-partition, the encoder 10 is configured to refrain from signaling the split option in the data stream 14.
Figure 15 shows the corresponding decoder 110 of this aspect. The decoder 1 10 is a block-based decoder for block-based decoding a picture 12 from a data stream 14. The decoder 110 reconstructs the original picture, except for possible quantization losses, such that a reconstructed picture 12' may be output by the decoder 1 10.
According to an embodiment, if the encoder 10 determines that the current partition is a log2-partition, then the encoder 10 may split the log2-partition and signal the split option in the data stream 14. Alternatively, the encoder may 10 not split the log2-partition (e.g. if it is already a leaf block) and signal the no-split option in the data stream 14.
Otherwise, if the encoder 10 determines that the current partition is a non-log2-partition, then the encoder 10 may implicitly split the non-log2-partition so as to create two or more subsequent log2-partitions at a subsequent partition level. Since this split is implicit, the encoder 10 may not signal any split option in the data stream 10. Thus, the decoder 110 may infer the split option itself.
In other words, if the partition to which a predetermined tree node 13i, 132, .... 139 belongs has a height and a width which are in number of samples a power of two, the encoder 10 may be configured to split the partition and to signal, in the data stream 14, the applied split option, or not split the partition and to signal the no-split option in the data stream 14.
Alternatively, if the partition to which said predetermined tree node 13i, 132. 13g belongs does not have a height and/or a width which are in number of samples a power of two, the encoder 10 is configured to split the partition such that a resulting sub-partition belonging to a tree node 132, ... , 139 at a subsequent tree level has a height and a width which are in number of samples a power of two, and refrain from signaling the applied split option in the data stream 14.
The decoder 110 is configured to partition the picture 12 into one or more leaf blocks 133, 134, 13s, 13e, 138, 13g using recursive multi-tree partitioning along a binary splitting tree 13, wherein a height and a width of the leaf blocks 133, 134, 135, 136, 138, 139 in number of samples is a power of two.
According to this aspect, the decoder 1 10 may check whether a partition to which a tree node 13i, 132, .... 139 of the splitting tree 13 belongs is a log2-partition or a non-log2- partition.
If it is a log2-partition, then the decoder 110 may read from the data stream 14 an indication of the split option, e.g. which split mode and/or which split ratio to apply to said partition. In other words, the decoder 110 may obtain a split-option-signal from the data stream 14.
If the partition is a non-log2-partition, then the decoder 1 10 may implicitly infer, without obtaining an explicit split-option-signal from the data stream 14, the split option to be applied to the current partition.
In other words, the decoder 1 10 is generally configured to obtain partitioning information of the binary splitting tree 13 from the data stream 14 for a tree node 1 3i , 132, . . . . 1 3g of the binary splitting tree 13, wherein the tree node 13 comprises a set of split options including a no-split option and a split option.
The decoder 110 may do so by checking whether a partition to which the tree node 13i, 132, .... 13g belongs, has a height and a width which are in number of samples a power of two.
If yes, i.e. if the partition is a log2-partition, the decoder 110 is configured to obtain, from the data stream 14, the split option or the no-split option for splitting the current partition.
If not, i.e. if the partition is a non-log2-partition, the decoder 110 is configured to implicitly infer the split option without obtaining an explicit split-option-signal from the data stream 14.
According to an embodiment, if the decoder 110 determines that the current partition is a log2-partition, then the decoder 1 10 may split said log2-partition based on a split signal in the data stream 14. Alternatively, the decoder 110 may not split said partition if a no-split signal is signaled in the data stream 14.
If the decoder 110 determines that the current partition is a non-log2-partition, then the decoder 110 may split said non-log2-partition into two or more log2-partitions on a next partitioning level. For this splitting, the decoder 1 10 may use a split option is implicitly inferred. That is, the decoder 110 may implicitly infer a split option and the decoder 110 may not obtain an explicit split-option-signal from the data stream 14 for the current non-log2- partition.
In other words, if the partition to which the tree node 13i, 132, .... 1 3g belongs has a height and a width which are in number of samples a power of two, i.e. if the partition is a log2- partition, the decoder 110 is configured to split the partition based on a split signal in the data stream 14, or not split the partition based on a no-split signal in the data stream 14.
If the partition to which the tree node 13i , 132, ... , 13g belongs does not have a height and/or a width which are in number of samples a power of two, i.e. if the partition is a non-log2- partition, the decoder 110 is configured to implicitly infer, without an explicit split-option- signal in the data stream 14, a split option that results in a sub-partition belonging to a tree node 132, .... 139 at a subsequent tree level, which sub-partition has a height and a width which are in number of samples a power of two, and split the partition into the sub-partition according to said inferred split option.
1.2 Consecutive splits of non-log2 partitions Another aspect targets to allow a GBS based codec to efficiently signal a triple tree like portioning at certain tree depths, e.g. indicating a (N/4, N/2, N/4) - split of a node into three partitions as illustrated in Figures 16A and 16B depicting a 64x64 block.
In such a case, as shown in Figure 16A, a conventional GBS partitioning according to the prior art would require indication of each split individually, e.g. starting with a N/4, 3N/4 - split indicated in the left side of the 64 x 64 block in Figure 16A and followed by a 2N/3, N/3 - split indicated in the right side of the 64 x 64 block in Figure 16A.
To lessen the signaling effort, in this aspect, a non-log2-partition such as the 3 N/4 partition resulting from the first split, may mandatorily be split by a parallel split into two log2-partitions with the split (and possibly the splitting ratio indication) being implicitly derived at decoder side. The described mandatory consecutive split could be restricted to certain node sizes, e.g. towards leaf level of the split tree.
Figure 16B shows an example according to this aspect. The left block 164 in Figure 16B represents a first partition. Said first block 164 is again a 64 x 64 block that is split by a first vertical split having a split ratio of N/4, 3 N/4 into a first sub-partition 161 having a size of N/4, i.e. 64 x 16, and into a second sub-partition 162 having a size of 3N/4, i.e. 64 x 48.
The first sub-partition 161 is a log2-partition, while the second sub-partition 162 is a non- log2-partition. According to this aspect, the second sub-partition 162 is implicitly split into two further sub-partitions 162a, 162b such that these two resulting further sub-partitions 162a, 162b are log2-partitions. For example, the second sub-partition 162 may be implicitly split by an implicit split having a split ratio of 2 N/3, N/3. Thus, the block 165 as depicted on the right side of Figure 16B
may result therefrom. That is, the block 165 may comprise, after said implicit split, three partitions 161 , 162a, 162b, wherein each partition 161 , 162a, 162b is a log2-partition.
Accordingly, the encoder of this aspect may refrain from signaling a split indication (e.g. split mode and split ratio) of the implicit second split in the data stream 14. The decoder 1 10, in turn, may infer a split indication (e.g. split mode and split ratio) for the implicit second split without obtaining an explicit split-indication-signal from the data stream 14, since the second split is an implicit split for splitting the non-log2-partition 162 into two or more log2-partitions 162a, 162b.
The above described concept was only exemplarily described with reference to vertical splits. Of course, it also holds true for horizontal splits or combinations thereof.
Furthermore, also the sizes were only mentioned by way of non-limiting examples.
Thus, according to an embodiment, the encoder 10 may be configured to split a predetermined partition 164 into at least three sub-partitions 161 , 162a, 162b, each of which having a height and a width which are in number of samples a power of two.
The encoder 10 may do so by applying a first split to the predetermined partition 164, wherein said first split creates a first sub-partition 162 which does not have a height and a width which are in number of samples a power of two. The encoder 10 may signal the applied split option of the first split in the data stream 14.
Furthermore, the encoder 10 is configured to subsequently apply a consecutive second split to the first sub-partition 162, wherein said consecutive second split creates a second and a third sub-partition 162a, 162b, each of which having a height and a width which are in number of samples a power of two. In this case, the encoder 10 may refrain from signaling the applied split option of the consecutive second split in the data stream 14.
According to an embodiment, the decoder 1 10 may be configured to split a predetermined partition 164 into at least three sub-partitions 161 , 162a, 162b, each of which having a height and a width which are in number of samples a power of two.
The decoder 1 10 may do so by obtaining a first split option from the data stream 14 and applying a first split according to said first split option to the predetermined partition 164, wherein said first split option creates a first sub-partition 162 which does not have a height and a width which are in number of samples a power of two.
Furthermore, the decoder 110 is configured to implicitly infer, without an explicit split- option-signal in the data stream 14, a split option for a subsequent second split and to subsequently apply said second split according to the split option to the first sub-partition 162, wherein said second split creates a second and a third sub-partition 162a, 162b, each of which having a height and a width which are in number of samples a power of two.
As can be seen in Figure 16B, the implicit second split is a parallel split relative to the first split. Thus, according to an embodiment, the encoder 10 may be configured to choose a parallel split as the split option for the consecutive second split, wherein the consecutive second split is parallel to the first split.
Also the decoder 110 may be configured to choose a parallel split as the split option for the consecutive second split, wherein the consecutive second split is parallel to the first split. Again, the consecutive second split is an implicit split, wherein the split option may be inferred from the decoder 110 without an explicit split-option-signaling in the data stream 14.
Generally, the split option may represent an option whether a split shall be applied or not. Accordingly, the split option may either indicate whether a split or a no split shall be performed. Therefore, the split option may comprise a split indication or a no-split indication. The split option may be contained in the partitioning information.
Furthermore, the partitioning information may also comprise information which kind of split to apply, e.g. a quad split, a binary split, a generalized binary split, a triple split, or the like. This may also be referred to as the split mode. Optionally additionally, the partitioning information may also comprise information about the split ratio.
If the consecutive second split is an implicit split, for example if the partition to be splitted is a non-log2-partition as explained above, the encoder may refrain from encoding a split mode and/or a split ratio indication into the data stream. Accordingly, for implicit splits, the decoder may infer the split mode and/or the split ratio without any explicit split mode signal and/or split ratio signal in the data stream.
Thus, according to an embodiment, the partitioning information may comprise at least one of a split mode and a split ratio, wherein the encoder 10 may be configured to omit signaling in the data stream 14 at least one of the split mode and the split ratio for the partition to
which the respective tree node 13i, 132. 139 belongs, if said partition does not have a height and a width which are in number of samples a power of two.
The decoder 1 10 may accordingly be configured to infer from the data stream 14, without an explicit signaling in the data stream 14, at least one of the split mode and the split ratio for the partition to which the respective tree node 13i, 132, ... , 139 belongs, if said partition does not have a height and a width which are in number of samples a power of two.
According to an example of this aspect, implicit splits shall only be applied to tree nodes at a certain tree depth level, or in terms of blocks, the implicit splits shall only be applied to partitions having a predetermined size, for example a size that lies above a minimum block size to be coded. This minimum block size may be pre-defined.
Thus, according to an embodiment, the encoder 10 may be configured to refrain from signaling the split option in the data stream 14 only if the partition or sub-partition, which shall be split according to said split option, comprises a predetermined size or if the corresponding tree node 13i, 132, .... 13g of said partition or sub-partition is at a predetermined tree-depth level of the binary tree 13.
Accordingly, the decoder 1 10 may be configured to implicitly infer the split option only if the partition or sub-partition, which shall be split according to the split option, comprises a predetermined size or if the corresponding tree node 13i, 132, .... 139 of said partition or sub-partition is at a predetermined tree-depth level of the binary tree 13.
For instance, the implicit splits may be performed towards a leaf block level. That is, the closer the current partitioning level is to the leaf block level, the more probable an implicit split will be.
According to an embodiment, the above mentioned predetermined tree-depth level may be the level of the last tree node before a tree node belonging to a leaf block. That is, the implicit splits may only be performed for the last block that, if split, may lead to a leaf block.
Alternatively, the above mentioned predetermined tree-depth level may be one level or two levels above the last tree node before a leaf block. In other words, the implicit splits may only be performed for the penultimate, i.e. the second to last, or even for the third to last level before the tree node belonging to a leaf block.
Figure 17 shows a block diagram of a method for encoding a picture 12 into the data stream 14 according to the above described concept.
In block 1701 the picture 12 is partitioned into one or more leaf blocks using recursive multitree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two. That is, the picture 12 is partitioned into blocks, wherein a leaf block is a log2-partition.
In block 1702 partitioning information of the binary splitting tree 13 is signaled in the data stream 14 by checking, for a tree node 13i, 132, .... 139 of the binary splitting tree 13 comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node 13i, 132, .... 139 belongs, has a height and a width which are in number of samples a power of two, i.e. whether the respective partition is a log2- partition.
The block diagram proceeds to a selection node 1703. If yes, i.e. if the current partition is a log2-partition, the method proceeds to block 1704 according to which the split option or the no-split option for said partition is signaled in the data stream 14.
If not, i.e. if the partition is a non-log2-partition, the method proceeds to block 1705 according to which it is refrained from signaling the split option in the data stream 14, i.e. the split option is not signaled in the data stream 14.
Figure 18 shows a block diagram of a method for decoding a picture 12 from the data stream 14 according to the above described concept.
In block 1801 the picture 12 is partitioned into one or more leaf blocks using recursive multitree partitioning along a binary splitting tree 13, wherein a height and a width of the leaf blocks in number of samples is a power of two. That is, the picture 12 is partitioned into blocks, wherein a leaf block is a log2-partition.
In block 1802 partitioning information of the binary splitting tree 13 is obtained from the data stream 14 by checking, for a tree node 13i , 132, .... 13g of the binary splitting tree 13 comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node 13i, 132, ... , 139 belongs, has a height and a width which are in number of samples a power of two, i.e. by checking whether the current partition is a log2-partition.
The block diagram proceeds to a selection node 1803. If yes, i.e. if the current partition is a log2-partition, the method proceeds to block 1804 according to which the split option or the no-split option for said partition is obtained from the data stream 14.
If not, i.e. if the partition is a non-log2-partition, the method proceeds to block 1805 according to which the split option is implicitly inferred from the data stream 14 without an explicit split-option-signal in the data stream 14. That is, the split option is not obtained from the data stream 14.
1.3 Differentiating nodes in tree depth evaluation
Another aspect of the invention targets to constrain a GBS based codec towards a better trade-off between coding efficiency and complexity. A state-of-the-art approach to control this trade-off is to restrict min/max depth of trees when searching and signaling, where tree depth is the maximum distance from a leaf node 13g to the root node 13i. A splitting indication can be inferred when a block does not yet fulfill the minimum tree depth and the splitting indication can be ruled out once the maximum tree depth is reached. These restrictions allow on the one hand to control encoder runtime through limiting the search space which decreases coding efficiency, while on the other hand the restrictions impact the necessity to signal certain split decisions.
The invention in this aspect is to differentiate tree nodes 1 3i , 132, . . . . 139 when evaluating a depth of a tree 13, The characteristics by which tree nodes 13i, 132, .... 13g could be handled differently could encompass but are not limited to: a) whether split signaling is implicit for a node 13i , 132, .... 13g b) whether node size belongs to a given set of node sizes, the set being either predefined (e.g. sizes below 16 x 16) or dynamic (e.g. node sizes which presence in previously coded blocks lies below a threshold) c) whether a node 13i, 132, .... 13g is disallowed to contain a CU
Depending on the above exemplary characteristics, the depth of a given node may be calculated differently, e.g. by omitting or weighting the count of nodes that adhere to one of the above characteristics. Thereby, search space is altered in an adaptive fashion adaptive to the given tree characteristics.
Thus, according to an embodiment, an encoder 10 for encoding a picture 12 into a data stream 14 is presented, wherein the encoder 10 is configured to partition the picture 12 into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree 13,
as it was previously explained above with reference to Figure 14. Also in this example, the splitting tree 13 comprises a set of split options including at least one of a no-split option and a split option.
Furthermore, a decoder 1 10 for decoding a picture 12 from the data stream 14 is presented, wherein the decoder 110 is configured to partition the picture 12 into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree 13, as it was previously explained above with reference to Figure 15. Also in this example, the splitting tree 13 comprises a set of split options including at least one of a no-split option and a split option.
According to this aspect, the encoder 12 and the decoder 100 may be configured to check, for a predetermined tree node 13i, 132, .... 13g of the splitting tree 13, whether said predetermined tree node 13i, 132, .... 139 meets a predetermined tree depth constraint. This check of the encoder 10 may depend on a distance of said predetermined tree node 13i, 132, ... , 13g from a root node 13i of the splitting tree 13, or a number of splits of a partition to which said predetermined tree node 13i, 132, .... 139 belongs along the splitting tree 13.
Furthermore, as mentioned above, there may be some further characteristics that may be checked by the encoder 10 and the decoder 110, respectively. The encoder 10 and the decoder 110 may evaluate said characteristics, e.g.: a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node 132, .... 139 is connected to the root node 13i, are implicitly signaled or explicitly signaled, as described above and/or b) depending on a size of a partition to which the predetermined tree node 13i, 132, .... 13g belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not.
Depending on the result of the evaluation of the encoder 10 and the decoder 110, respectively, i.e. based on the criteria above, the encoder 10 and the decoder 110 may be configured to rule out the one of the no-split option and the split option, i.e. depending on the check whether the predetermined tree node meets 1 3i , 132, ... . 1 39 the predetermined tree depth constraint.
In other words, depending on the above exemplary characteristics, the depth of a given node 13i, 132, 139 may be calculated differently, e.g. by omitting or weighting the count of nodes that adhere to one of the above characteristics. Thereby, search space is altered in an adaptive fashion adaptive to the given tree characteristics.
According to an embodiment, both the encoder 10 and the decoder 1 10 may be configured to calculate the tree depth of the predetermined node 13i, 132, . . . , 139 differently, namely depending on the above check whether the predetermined tree node 13i, 132, .... 13g meets the predetermined tree depth constraint, wherein the calculating is performed by omitting or weighting the count of tree nodes 13i, 132, .... 13g along the splitting tree if the predetermined tree node 13i, 132, .... 13g adheres to at least one of the options a), b) or c).
For instance, if a predetermined tree node 13i, 132, .... 139 fulfils at least one of the above criteria, the encoder 10 and the decoder 110 may skip this tree node 13i, 132, .... 13s in counting the tree nodes along the splitting tree 13. Additionally or alternatively, the encoder 10 and the decoder 110 may weight the count of a predetermined tree node 13i, 132, ... , 13g in counting the tree nodes along the splitting tree 13, wherein the weight may depend on the respective criterion to which the predetermined tree node 13i, 132, ... , 13g adheres.
According to an embodiment, the encoder 10 may be configured to, in discriminating according to the above mentioned option c), discriminate whether the partition has a height and a width which in number of samples is a power of two, i.e. whether the partition is a log2-partition.
If the partition is alog2-partition, it is allowed to be a leaf block.
If the partition is a non-log2-partition, the partition is disallowed to be a leaf block and should be splitted at least one more time.
According to a further embodiment, the encoder may be configured to explicitly signal an indication in the data stream 14 for indicating whether a current partition is allowed to be a leaf block or not. For example, if the discrimination according to option c) reveals that the partition is allowed to be a leaf block, the encoder 10 may be configured to signal an indication in the data stream 14 for indicating that the partition is allowed to be a leaf block. Additionally or alternatively, if the discrimination according to option c) reveals that the partition is not allowed to be a leaf block, the encoder 10 may be configured to signal an indication in the data stream 14 for indicating that the partition is disallowed to be a leaf block.
According to a further example, if the predetermined tree node 13i, 132, .... 139, upon splitting the partitions of the picture 12 along the splitting tree 13, does not yet fulfil a minimum tree depth constraint, the encoder 10 may be configured to omit signaling a split indication in the data stream 14. That is, if the current tree node 13i, 132, .... 13g has not yet reached a minimum partitioning level in the splitting tree 13, the encoder 10 may perform implicit splits, as described above.
Accordingly, for such nodes 13i, 132, .... 13g which do not yet fulfil a minimum tree depth constraint, the decoder 110 may infer a split option for said predetermined node 13i, 132, . . . . 1 3g, i.e. the decoder 110 does not obtain a split option from the data stream 14 since the encoder 10 did not encode any split-option signal into the data stream 14.
In other words, if the predetermined tree node 13i, 132, .... 1 3g, upon splitting along the splitting tree 13, does not yet fulfil a minimum tree depth constraint, the decoder 110 may be configured to infer a split option without an explicit spiit-option-signal in the data stream 14. That is, the decoder 110 may perform an implicit split, as described above.
Otherwise, if a predetermined node 13i, 132, .... 139 fulfils a maximum tree depth constraint, the encoder 10 may be configured to rule out the one of the no-split option and the split option. In other words, the encoder 10 may not further provide the one of the no- split option and the split option for splitting the current partition.
The same holds true for the decoder 110, i.e. if a predetermined node 1 3i , 132, .... 1 3g fulfils a maximum tree depth constraint, the decoder 1 10 may be configured to rule out the one of the no-split option and the split option. In other words, the decoder 110 may not further use the one of the no-split option and the split option for splitting the current partition.
As mentioned above, the encoder 10 may check whether the predetermined tree node 13i, 132, .... 1 3g meets the predetermined tree depth constraint according to criterion b), i.e. b) depending on a size of the partition to which the predetermined tree node 13i, 132, . . . . 1 3g belongs.
In other words, the encoder 10 may check whether the size of the predetermined node 1 3i , 132, . . . . 1 3g belongs to a given set of node sizes, the set being either pre-defined (e.g. sizes below 16 x 16) or dynamic (e.g. node sizes which presence in previously coded blocks lies below a threshold).
According to an embodiment, both the encoder 10 and the decoder 110 may be configured to, in checking whether the predetermined tree node 13i, 132, ... , 13Q meets the predetermined tree depth constraint according to b) depending on a size of a partition to which the predetermined tree node 13i, 132, .... 13g belongs, check whether the size of the partition to which the predetermined tree node 13i, 132, .... 13g belongs is equal to a predetermined partition size contained in a set of partition sizes.
According to yet a further embodiment of both the encoder 10 and the decoder 110, the above mentioned set of partition sizes may be static, wherein the set of partition sizes comprises one or more partition sizes, wherein each partition size has a static, i.e. fixed value.
Alternatively, the above mentioned set of partition sizes may be dynamic, wherein the set of partition sizes comprises one or more partition sizes, each partition size having a dynamically changeable value, wherein said value may depend on a variable predetermined threshold (e.g. node sizes which presence in previously coded blocks lies below said threshold).
Figure 19 shows a block diagram representing a method for encoding and a method for decoding a picture into/from a data stream 14, according to the above described concept.
In block 1901 the picture 12 is partitioned into one or more leaf blocks using recursive multitree partitioning along a splitting tree 13, by checking, for a predetermined tree node 13i, 132, ... , 13g of the splitting tree 13, wherein the splitting tree 13 comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node 132, 13g meets a predetermined tree depth constraint.
Checking the predetermined tree depth constraint may depend on block 1902 in which a distance of said predetermined tree node 13i, 132, .... 13g from a root node 13i of the splitting tree 13, or a number of splits of a partition to which said predetermined tree node 13i, 132, ... , 1 3g belongs along the splitting tree 13.
Furthermore, the predetermined tree node 13i, 132, ... , 13g may further comprise one or more characteristics depending on which the predetermined tree node 13i, 13 13g may be handled differently by the encoder 10 and/or the decoder 1 10. These characteristics may be determined by the encoder 10 and the decoder 1 10, respectively,
in block 1903a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or in block 1903b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or in block 1903c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not. The method then proceeds to block 1904 according to which the method further comprises a step of ruling out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
2. Tree type sets The state-of-the-art partitioning method QTBT describes an ordered combination of two partitioning tree types (namely QT and BT). The two tree types are used consecutively in the partitioning of large blocks into smaller blocks where the first level uses a QT partitioning followed by a second level of BT. Each tree type in QTBT is given with a corresponding set of constraints such as min and max tree depth, mode restrictions and so on.
The present aspect provides a concept according to which such tree type combinations, which may in the following be referred to as tree type sets (TTS), may be used to indicate an ordered combination of N levels of partitioning tree types out of a plurality of tree types such as QT, BT, GBS and so on, each with a corresponding set of constraints. Signaling of a TTS can be carried in any high-level syntax structure such as parameter sets or slice headers. Each coding partition such as tiles, slices, or CTUs within a bitstream can hence reference and thereby activate one of the available TTS, e.g. in high- level syntax structures such as parameter sets, slice header or also on CTU level.
Thus, according to this concept, an encoder 10 for encoding a picture 12 into a data stream 14 is provided, wherein the encoder 10 is a block-based encoder.
The encoder 10 is configured to partition the picture 12 along a splitting tree 13 from a tree root block 13i, at a root block partitioning level, into one or more leaf blocks 133, 134, 13s, 13e, 13a, 13g, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, such as described above with reference to Figure 14.
However, according to this aspect, the encoder 10 is configured to select, for performing the multi-tree partitioning, a tree type set (in the following also referred to as TTS) comprising one or more tree types. Said tree type sets may be explained in more detail with reference to Figures 20 to 22.
The encoder 10 of this aspect is further configured to signal the tree type set in the data stream 14, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level. That is, the signaling may be done at the root block partitioning level, and/or at a leaf block partitioning level, and/or at any level between the root block and leaf block partitioning levels.
The decoder 110 may obtain said signaling from the data stream 14 in order to select the correct TTS for decoding the blocks of the picture using the one or more tree types contained in the signaled selected TTS.
Thus, according to an embodiment, a decoder 110 for decoding the picture 12 from the data stream 14 is provided, wherein the decoder 110 is a block-based decoder.
The decoder 110 is configured to partition the picture 12 along a splitting tree 13 from a tree root block 13i , at a root block partitioning level, into one or more leaf blocks 13s, 134, 13s, 13e, 13e, 13g, each at a respective leaf block partitioning level, using recursive multitree partitioning, such as described above with reference to Figure 15.
However, according to this aspect, the decoder 110 is configured to perform the multi-tree partitioning using a tree type set (in the following also referred to as TTS) comprising one or more tree types.
The decoder 110 is configured to perform said multi-tree partitioning based on a signaling of the tree type set in the data stream 14, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level. That is, the signaling of the tree type set may be obtained from the data stream 14 at the root block partitioning level, and/or at
a leaf block partitioning level, and/or at any level between the root block and leaf block partitioning levels.
Figure 20 illustrates a first embodiment of the present concept with signaling on CTU level.
Figure 20 shows a splitting tree 13 similar to the one described above with reference to Figures 14 and 15. The splitting tree 13 may comprise a root node 13i, wherein the root node 13i is at a first partitioning level 21 1 , which may therefore also be referred to as the root block partitioning level 21 1. In this non-limiting example, the root block belonging to the root node 13i may be a CTU.
Along the splitting tree 13, a first split of the root block (CTU) may be performed, i.e. the root node 13i may be partitioned at the root block partitioning level 211 into two or more further tree nodes 132, 133, 134, 13s at a resulting second partitioning level 212. In this non-limiting example, the tree nodes 133, 134, 13s may be leaf nodes which may not be partitioned any further.
However, the tree node 132 at the second partitioning level 212 may be further partitioned along the splitting tree 13 which may result in two or more further nodes 136, 137 at a resulting third partitioning level 213. In this non-limiting example, the tree nodes 136, 137 may be leaf nodes and the third partitioning level 213 may accordingly also be referred to as a leaf block partitioning level.
The encoder 10 may select one or more partitioning parameters for splitting the blocks into subblocks. In terms of the corresponding splitting tree 13, the encoder 10 may select one or more tree types for partitioning the tree nodes 13i , 132, ... , 139 belonging to a respective block or subblock from one partitioning level to another partitioning level.
According to this aspect, a partitioning parameter set 201 may be provided which may, for instance, comprise one or more tree type set (TTS) 201A, 201 B, 201C. Each TTS 201A, 201 B, 201 C may comprise one or more tree types. In the non-limiting example shown in Figure 20, a first TTS 201 A may comprise one tree type, e.g. a quad split tree type (QT).
A second TTS 201 B may also comprise one tree type, which may be the same tree type as contained in the first TTS 201A or a different tree type, e.g. a generalized binary split tree type (GBS). A third TTS 201 C may comprise two tree types, e.g. a QT and a GBS.
In the present example, the encoder 10 may select the third TTS 201 C for partitioning the tree nodes 13i, 132, .... 13g along the splitting tree 13. As can be seen, the first
partitioning of the root node 13i at the root block partitioning level 211 may be performed by using a QT. Accordingly, the root node 13i may be partitioned into four sub-nodes 132, 133 134I 135. The nodes 13i, 132, 13g in the splitting tree 13 may also be referred to as partitions.
For partitioning the sub-node 132 at the second partitioning level 212 the selected TTS 201 C provides a GBS tree type. Accordingly, the sub-node 132 is partitioned at the second partitioning level 212 by using a GBS tree type, such that two further sub-nodes 13e, 137 result therefrom at the third partitioning level 213. Since the two further sub-nodes 13e, 137 are leaf blocks, the third partitioning level is a leaf block partitioning level.
In the present example, the selected TTS 201 C comprises two tree types - QT and GBS - and the splitting tree 13 comprises three partitioning levels 211 , 212, 213, wherein a first partitioning is performed between the first and the second partitioning level 211 , 212 and a second partitioning is performed between the second and the third partitioning level 212, 213.
Accordingly, the TTS 201 C comprises a number of tree types which number of tree types is equal to a number of partitioning levels 211 , 212, 213 minus one. In the present example, the splitting tree 13 comprises a number of three partitioning levels 211 , 212,
213 and, thus, the selected TTS 201 C comprises a number of two tree types (QT, GBS).
Thus, according to an embodiment, the encoder 10 may be configured to select a tree type set 201 C comprising a number of tree types which number is equal to a number of partitioning levels 211 , 212, 213 minus one. As mentioned above, the selected TTS 201 C may be signaled at one or more partitioning levels 21 1 , 212, 213. For example, the selected TTS 201C may be signaled in the data stream 14 at the root block partitioning level 211.
Accordingly, a further embodiment provides a corresponding decoder 110, wherein the decoder 1 10 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 (or alternatively at one or more other partitioning levels 212, 213), a tree type set 201 C comprising a number of tree types which number of tree types is equal to a number of partitioning levels minus one.
Furthermore, the decoder 110 may be configured to use, at each partitioning level 211 , 212 except the leaf block partitioning level, one of the tree types contained in the tree type set 201 C for performing the multi-tree partitioning.
The encoder 10 may further be configured to use, at each partitioning level except the leaf block partitioning level 213, i.e. at the first partitioning level 211 and at the second partitioning level 212. one of the tree types contained in the tree type set 201 C.
As shown in Figure 20, the encoder 10 is further configured to signal the selected tree type set 201 C in the data stream 14, at the root block partitioning level 211.
In more general terms, an embodiment provides an encoder 10 which may be configured to signal in the data stream 14 the selected tree type set 201 C, at the root block partitioning level 211 , and to use the one or more tree types (e.g. QT, GBS) contained in the selected tree type set 201 C for partitioning the picture 12 from the tree root block into the one or more leaf blocks, i.e. for performing the multi-tree partitioning from the root node 13i to the one or more leaf nodes 133, 134, 135, 136, 137.
Accordingly, a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 , the tree type set 201 C for performing the multi-tree partitioning, and to use the one or more tree types contained in the tree type set 201 C for partitioning the picture 12 from the tree root block into the one or more leaf blocks, i.e. for performing the multi-tree partitioning from the root node 13i to the one or more leaf nodes 133, 134, 13S, 136, I SA .
According to the above described example as depicted in Figure 20, a further embodiment provides an encoder 10 that may be configured to select a tree type set 201 C comprising a first tree type, e.g. QT, and a different second tree type, e.g. GBS.
The encoder 10 may further be configured to use the first tree type (e.g. QT) for performing the multi-tree partitioning at a first partitioning level, for example at the root block partitioning level 211 , and to use the second tree type (e.g. GBS) for performing the multi-tree partitioning at a different second partitioning level, for example at the subsequent second partitioning level 212.
The encoder 10 may further be configured to signal the selected tree type set 201 C in the data stream 14, at the root block partitioning level 211.
Accordingly, a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 , a tree type set 201 C comprising a first tree type and a different second tree type (e.g. QT and GBS).
The decoder 1 10 may further be configured to use the first tree type (e.g. QT) for performing the multi-tree partitioning at a first partitioning level 211 , and to use the second tree type (e.g. GBS) for performing the multi-tree partitioning at a different second partitioning level 212.
As can be seen in Figure 20, if two or more tree types (e.g. QT and GBS) are contained in one tree type set 201 C, the tree types may be arranged in an order, e.g. the first tree type (e.g. QT) may be arranged prior to the second tree type (e.g. GBS).
In the above described example the tree types contained in the TTS 201 C are used in the splitting tree 13 in the order in which they are arranged in the TTS 201 C. That is, the first tree type (e.g. QT) was used at the first partitioning level 21 1 and the consecutive second tree type (e.g. GBS) was used at the second partitioning level 212.
Thus, according to an embodiment, the one or more tree types contained in a tree type set 201 C may be arranged in a consecutive order, wherein both the encoder 10 and the decoder 110 may be configured to consecutively use, at each partitioning level 211, 212 except the leaf block partitioning level 213, one of the one or more consecutively arranged tree types in the order as they are arranged in the tree type set 201 C.
Figure 21 shows a further example. As can be seen, the third TTS 201C may comprise three tree types, e.g. QT, BT and GBS. However, the focus shall first be directed to the two tree types which are depicted in parentheses, i.e“BT or GBS". The tree types depicted in parentheses shall indicate that these two tree types may both be available at the same time for one and the same partition, or for one and the same partitioning level, respectively. That is, for partitioning a tree node, or for partitioning all tree nodes of a certain partitioning level 21 1 , 212, 213, the encoder 10 may be configured to selectively choose one of the two options, i.e. one of the two available tree types contained in the tree type set 201 C. Of course, a tree type set may comprise more than two exemplarily described available tree types. However, the encoder 10 may be configured to choose exactly one out of the available tree types for partitioning a tree node, or for partitioning all tree nodes of a certain partitioning level 21 1 , 212, 213.
Thus, according to an embodiment, the encoder 10 may be configured to select a tree type set 201 C comprising at least a first tree type (e.g. BT) and a second tree type (e.g. GBS) to be used at a predetermined partitioning level 211 , 212, 213, in this example for the second partitioning level 212.
The encoder 10 may further be configured to selectively choose exactly one out of the at least first and second tree types (e.g. BT or GBS) for performing the multi-tree partitioning at said predetermined partitioning level 211 , 212, 213.
The encoder 10 may further be configured to signal the selected tree type set 201 C in the data stream 14, at the root block partitioning level 211.
Accordingly, a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 21 1 , a tree type set 201 C comprising at least a first tree type (e.g. BT) and a second tree type (e.g. GBS) to be used at a predetermined partitioning level 21 1 , 212, 213.
The decoder 110 may further be configured to selectively choose exactly one out of the at least first and second tree types (e.g. BT or GBS) for performing the multi-tree partitioning at said predetermined partitioning level 212, 213, 214.
As can be seen in Figure 21 , the encoder 10 may further encode an indication 205 into the data stream 14 for indicating which one of the available tree types (e.g. BT or GBS) was selected at the respective partitioning level 212 for partitioning the tree nodes 132, 13s, 134, 135 of said partitioning level 212. Said indication 205 may, for instance, be a flag, such as the depicted“ Pe r-le ve l_m od e_f I ag” .
According to an embodiment, the encoder 10 may be configured to signal at the predetermined partitioning level 212 an indication 205 which one out of the first and second tree types (e.g. “BT or GBS") is selectively chosen from the tree type set 201 C for performing the multi-tree partitioning at said predetermined partitioning level 212.
Accordingly, a further embodiment provides a corresponding decoder 110, wherein the decoder 1 10 may be configured to obtain from a signaling at the predetermined partitioning level 212 an indication 205 which one out of the first and second tree types (e.g.“BT or GBS") to selectively choose for performing the multi-tree partitioning at said predetermined partitioning level 212.
As mentioned above, the third TTS 201 C in Figure 21 may comprise three tree types, namely a first tree type (e.g. QT), and the aforementioned two available options (e.g.“BT or GBS”). In this case, the encoder 10 may use the first tree type (e.g. QT) for partitioning the root node 13i at the first partitioning level 211 , and to selectively choose exactly one out
of the two available options (e.g.“BT or GBS”) for performing the multi-tree partitioning at the second partitioning level 212, as described above.
Thus, according to an embodiment, the encoder 10 may be configured to select a tree type set 201 C comprising a first tree type (e.g. QT), a second tree type (e.g. BT) and a third tree type (e.g. GBS), to use the first tree type for performing the multi-tree partitioning at a predetermined partitioning level, e.g. the exemplarily depicted first partitioning level 21 1 , and to selectively choose exactly one out of the second and the third tree types (e.g.“BT or GBS”) for performing the multi-tree partitioning at one or more subsequent partitioning levels, e.g. at the exemplarily depicted second partitioning level 212, and to signal the selected tree type set 201 in the data stream 14, at the root block partitioning level 211.
Also in this example, the encoder 10 may be configured to signal at the second partitioning level 212 an indication, e.g. the“Per-level_mode_flag” 205”, to indicate which one out of the second and the third tree types (e.g.“BT or GBS”) is selectively chosen from the tree type set 201 C for performing the multi-tree partitioning at said second partitioning level 212.
Accordingly, a further embodiment provides a corresponding decoder 110, wherein the decoder 110 may be configured to obtain from the signaling in the data stream 14, at the root block partitioning level 211 , a tree type set 201 C comprising a first tree type (e.g. QT), a second tree type (e.g. BT) and a third tree type (e.g. GBS), to use the first tree type for performing the multi-tree partitioning at a predetermined partitioning level, e.g. the exemplarily depicted first partitioning level 21 1 , and to selectively choose exactly one out of the second and the third tree types (e.g. “BT or GBS”) for performing the multi-tree partitioning at one or more subsequent partitioning levels, e.g. at the exemplarily depicted second partitioning level 212.
Also in this example, the decoder 1 10 may be configured to obtain from a signaling at the second partitioning level 212 an indication, e.g. the“Per-level_mode_flag” 205”, to indicate which one out of the second and the third tree types (e.g.“BT or GBS”) to selectively choose for performing the multi-tree partitioning at said second partitioning level 212.
According to the embodiment as depicted in Figure 21 , the above described concept of tree type sets (TTS) 201 A, 201 B, 201 C could host multiple tree types for a given level 211 , 212, e.g. QT for the first coarse level of partitioning 21 1 and either BT or GBS for the second level of partitioning 212. Slice or CU-level type signaling such as a Per-
level_mode_flag 205 could be used to index the used partitioning for levels 212 with multiple tree types (e.g.“BT or GBS”).
In a further variant of this concept, indication of the tree partitioning type for the current split level and potentially following split levels may exist on each individual partitioning level.
Figure 22 shows an example. The partition parameter set 201 may again comprise, by way of non-limiting example, three tree type sets (TTS) 201 A, 201 B, 201 C. Each TTS 201 A, 201 B, 201 C may comprise one or more tree types. In the example as depicted in Figure 22 each TTS 201 A, 201 B, 201 C may comprise only one tree type for the sake of simplification of the following description.
For example, at the first partitioning level 21 1 , a first TTS 201 A may be selected by the encoder 10. Accordingly, the encoder 10 use the tree type (e.g. QT) contained in the TTS 201 A for performing the multi-tree partitioning at the first partitioning level 211. The encoder may signal said selected TTS 201A in the data stream 14 at the first partitioning level. If a TTS may comprise only one tree type, then the encoder 10 may signal the tree type that is contained in the respective TTS, instead of signaling the TTS itself.
At the second partitioning level 212, a TTS 201 A, 201 B, 201 C may again be selected by the encoder 10. The encoder 10 may be configured to select one TTS 201 A, 201 B, 201 C for the entire partitioning level. Accordingly, the one or more tree types which are contained in the selected TTS may then be used for performing the multi-tree partitioning at the respective partitioning level. That is, one and the same tree type of the selected TTS may be used for partitioning all nodes of the respective partitioning level, except for leaf nodes because leaf nodes may per se not be further partitioned.
For example, the encoder 10 may select the third TTS 201 C at the second partitioning level 212. The encoder 10 may use the one or more tree types (e.g. GBS) contained in the selected third TTS 201 C for performing the multi-tree partitioning at the second partitioning level 212. In this example, each node of the second partitioning level 212 may be partitioned using the GBS tree type, except for leaf nodes. (Note that this example refers to a level-wise selection of TTS, which is not explicitly depicted)
If the encoder 10 has selected one TTS 201 A, 201 B, 201 C, the encoder 10 may be configured to use the one or more tree types contained in the selected TTS 201 A, 201 B,
201 C for performing the multi-tree partitioning at the respective partitioning level 21 1 , 212, 213 and at one or more subsequent partitioning levels.
With reference to Figure 22, the encoder 10 may select the first TTS 201 A comprising tree type QT for partitioning the root node 13i at the root partition level 211. The encoder 10 may also use the selected TTS 201 A for further subsequent partitioning levels. For example, the encoder may use the one or more tree types (e.g. QT) contained in the first TTS 201 A as the tree type for performing the multi-tree partitioning at the second partitioning level 212 and optionally further subsequent partitioning levels.
Thus, according to an embodiment, the encoder 10 may be configured to select a first tree type set 201 A at a first partitioning level 21 1 and to use one of the tree types contained in said first tree type set 201 A for performing the multi-tree partitioning at said first partitioning level 21 1 , or for performing the multi-tree partitioning at said first partitioning level 211 and one or more subsequent partitioning levels 212.
The encoder 10 may further be configured to select the first or a second tree type set 201 A, 202B at a second partitioning level 212 and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 202B for performing the multitree partitioning at the second partitioning level 212, or for performing the multi-tree partitioning at said second partitioning level 212 and one or more subsequent partitioning levels. Furthermore, the encoder 10 may be configured to signal the selected tree type set 201 A, 201 B in the data stream 14, separately at the first partitioning level 21 1 and at the second partitioning level 212. That is, the encoder 10 may signal the selected TTS 201 A, 201 B level-wise.
Additionally or alternatively, the encoder 10 may select a TTS for each node of a partitioning level separately, i.e. partition-wise.
This example is depicted in Figure 22. For instance, the encoder 10 may select the first TTS 201 A at the first partitioning level 21 1 , as described above. Further, the encoder 10 may select the third TTS 201 C for the node (which may also be referred to as a partition) 132 at the second partitioning level 212. Accordingly, the node 132 may further be partitioned, using the GBS tree type contained in the selected third TTS 201 C, into two subsequent sub- nodes 13e, 137 at a subsequent third partitioning level 213.
However, the encoder 10 of this example may further be configured to select a TTS for another node of the second partitioning level 212. For instance, the encoder 10 may select the first TTS 201 A for a further node 13s at the second partitioning level 212. Accordingly, the node 13s may further be partitioned, using the QT tree type contained in the selected first TTS 201A, into four subsequent sub-nodes 13e, 13g, 13io, 13n at a subsequent third partitioning level 213.
According to this embodiment, the encoder 10 may be configured to select for a first partition 132 of a first partitioning level 212 a first tree type set 201 C at the first partitioning level 212 and to use one of the tree types contained in said first tree type set 201 C for performing the multi-tree partitioning of said first partition 132 at said first partitioning level 212, or for performing the multi-tree partitioning of said first partition 132 at said first partitioning level 212 and of one or more subsequent partitions at one or more subsequent partitioning levels.
The encoder 10 may further be configured to select for a second partition 135 of the first partitioning level 212 the first or a second tree type set 201 A, 201 C at said first partitioning level 212 and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 201 C for performing the multi-tree partitioning of said second partition 135 at said first partitioning level 212, or for performing the multi-tree partitioning of said second partition 13s at said first partitioning level 215 and of one or more subsequent partitions at one or more subsequent partitioning levels.
The encoder 10 may further be configured to signal the respectively selected tree type set 201A, 201 C in the data stream 14, separately for the first partition 132 and the second partition 13s at the first partitioning level 212.
Accordingly, a further embodiment provides a corresponding decoder 1 10, wherein the decoder 1 10 may be configured to obtain from the signaling in the data stream 14, at a first partitioning level 21 1 , a first tree type set 201 A and to use one of the tree types contained in said first tree type set 201A for performing the multi-tree partitioning at said first partitioning level 21 1 , or for performing the multi-tree partitioning at said first partitioning level 21 1 and one or more subsequent partitioning levels 212, 213.
The decoder 1 10 may further be configured to obtain from a separate signaling in the data stream 14, at a second partitioning level 212, the first or a second tree type set 201 A,
201 B and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 201 B for performing the multi-tree partitioning at the second
partitioning level 212, or for performing the multi-tree partitioning at said second partitioning level 212 and one or more subsequent partitioning levels 213.
According to yet a further embodiment, the decoder 1 10 may be configured to select a TTS 201 A, 201 B, 201 C partition-wise. According to this example, the decoder 1 10 may be configured to obtain from the signaling in the data stream 14, at a first partitioning level 212, a first tree type set 201 C for a first partition 132 of said first partition level 212, and to use one of the tree types contained in said first tree type set 201 C for performing the multi-tree partitioning of said first partition 132 at said first partitioning level 212, or for performing the multi-tree partitioning of said first partition 13z at said first partitioning level 212 and of one or more subsequent partitions at one or more subsequent partitioning levels.
The decoder 1 10 may further be configured to obtain from the signaling in the data stream 14, at the first partitioning level 212, the first or a second tree type set 201 A, 201 C for a second partition 13s of said first partition level 212 and to use one of the tree types contained in the selected one of the first and the second tree type sets 201 A, 201 C for performing the multi-tree partitioning of said second partition 13s at said first partitioning level 212, or for performing the multi-tree partitioning of said second partition 13s at said first partitioning level 212 and one of one or more subsequent partitions at one or more subsequent partitioning levels.
According to this example, the decoder 1 10 may be configured to obtain the signaling of the respective tree type set from the data stream 14, separately for the first partition 132 and the second partition 13s at the first partitioning level 212.
Figure 23 shows a block diagram of a method for block-based encoding a picture 12 into a data stream 14 according to the above described concept.
In Block 2301 the picture 12 is partitioned along a splitting tree 13 from a tree root block, at a root block partitioning level 21 1 , into one or more leaf blocks, each at a respective leaf block partitioning level 213, using recursive multi-tree partitioning.
In block 2302 a tree type set 201 A, 201 B, 201 C comprising one or more tree types is selected for performing the multi-tree partitioning.
In block 2303 the selected tree type set 201 A, 201 B, 201C is signaled in the data stream 14, wherein the signaling is done at one or more partitioning levels 21 1 , 212, 213 between the root block partitioning level 21 1 and the leaf block partitioning level 213, including the root block partitioning level 21 1 and the leaf block partitioning level 213.
Figure 24 shows a block diagram of a method for block-based decoding a picture 12 from a data stream 14 according to the above described concept.
In block 2401 the picture 12 is partitioned along a splitting tree 13 from a tree root block, at a root block partitioning level 211 , into one or more leaf blocks, each at a respective leaf block partitioning level 213, using recursive multi-tree partitioning.
In block 2402 the multi-tree partitioning is performed using a tree type set 201 A, 201 B, 201 C comprising one or more tree types.
According to block 2403 the step of performing the multi-tree partitioning is based on a signaling of the tree type set 201 A, 201 B, 201 C in the data stream 14, at one or more partitioning levels 211 , 212, 213 between the root block partitioning level 211 and the leaf block partitioning level 213, including the root block partitioning level 211 and the leaf block partitioning level 213.
3. Asymmetric QT In another aspect, an asymmetric variant of QT is introduced, i.e. a block may be divided into four partitions by two straight boundaries with a single split. An example is shown in Figure 25, which shows a block 250 that is partitioned into four subblocks 251 , 252, 253, 254. The block 250 may be a tree node block, e.g. a CTU, and the subblocks 251 , 252, 253, 254 may be leaf blocks. The splitting tree 13 depicted on the right side of the block 250 depicts the partitions belonging to the respective subblocks 251 , 252, 253, 254.
For example a first horizontal split at the CTU partitioning level may be performed, wherein said first split may comprise a splitting ratio of ¾. This results in two subblocks, each of which is further splitted by a respective vertical split comprising a splitting ratio of 1/3 resulting the four subblocks 251 , 252, 253, 254 as depicted. When the above described partitioning would be based on conventional GBS, then the partitioning into the depicted four partitions is comparatively expensive signaling-wise and requires signaling of three split decisions for each of the three partitions in the splitting tree 13, each with corresponding splitting ratios, i.e. ¼ and 1/3.
However, the invention in this aspect is a QT style partitioning that may be able to split the block 250 into the depicted four partitions 251 , 252, 253, 254 based on a single split
indication, but in addition to state-of-the-art QT, the inventive principle may also allow for indicated two splitting ratios that define the exact position of the splitting boundaries that may be different than N/2 splits of conventional QT. Accordingly, asymmetric subblocks
251 , 252, 253, 254, as depicted in Figure 25, may result. Thus, the inventive principle of 5 this aspect may also be referred to as an Asymmetric Quad Split (AQT).
According to such an embodiment, an encoder 10 may be provided, wherein the encoder 0 is a block-based encoder for encoding a picture 12 into a data stream 14. The encoder 10 may be configured to partition the picture 12 into one or more leaf blocks 251 , 252, 253, 254 using recursive multi-tree partitioning along a splitting tree 13. Furthermore, the 10 encoder 10 may be configured to signal partitioning information of the splitting tree 13 in the data stream 14. This encoder 10 may be similar to the encoder as exemplarily described above with reference to Figure 14.
However, according to this aspect, the encoder 10 may be configured to perform for a predetermined block 250 an asymmetric quad split, in which the predetermined block 250 15 is partitioned by at least a first split boundary 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 , 252, 253, 254.
Furthermore, the encoder 10 of this aspect may be configured to signal in the data stream 14 a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256.
20 For decoding the picture, a corresponding decoder 110 is provided. According to an embodiment, a block-based decoder 110 is provided for decoding the picture 12 from the data stream 14, wherein the decoder 110 may be configured to partition the picture 12 into one or more leaf blocks 251 , 252, 253, 254 using recursive multi-tree partitioning along a splitting tree 13. This decoder 1 10 may be similar to the decoder as exemplarily described 25 above with reference to Figure 15.
:
However, the decoder 1 10 according to this aspect may be configured obtain a partitioning information of the splitting tree 13 from the data stream 14. Furthermore, the decoder 110 may further be configured to perform for a predetermined block 250 an asymmetric quad split, in which the predetermined block 250 is partitioned by at least a first split boundary 30 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 ,
252, 253, 254.
The decoder 1 10 may be configured to obtain from the data stream 14 a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256.
As mentioned above, this asymmetric quad split (AQT) may be based on a single split indication. That is, a quad split is signaled together with two splitting ratios, namely a first splitting ratio for the first split boundary 255 and a second splitting ratio for the second split boundary 256.
Thus, according to an embodiment, the encoder 10 may be configured to perform the asymmetric quad split based on a single split indication, and to signal in the data stream 14 the single split indication and the first splitting ratio of the first split boundary 255 and the second split ratio of the second split boundary 256.
Accordingly, the decoder 1 10 of this embodiment may be configured to perform the asymmetric quad split based on a single split indication, and to obtain from the data stream 14 the single split indication and the first splitting ratio of the first split boundary 255, and the single split indication and the second split ratio of the second split boundary 256.
An alternative embodiment of the invention is a constraint for GBS partitioning that allows for reduction of signaling overhead in such a scenario. This is achieved by restricting the splitting ratio of the third split to be equal to the second split and omit corresponding bitstream signaling in favor of inference on decoder side. Figure 25 illustrates the respective split tree 13. The set of available split ratios for the third split which leads to partitions 253 and 254 may be restricted to the actual split ratio of the second split.
According to this embodiment, the encoder 10 may be configured to perform the asymmetric quad split by using a binary splitting tree 13 based on at least a first and a second binary split indication, i.e. based on at least a split indication for the first binary split of the first partition 13i at the first partitioning level, and based on at least a split indication for the second binary split of the second partition 132 at the second partitioning level.
The encoder 10 may be configured to perform the first binary split of the first partition 13i which creates the first split boundary 255 having a first split ratio, e.g. ¾. The encoder 10 may further be configured to perform the perpendicular second binary split of the second partition 132 which creates the second split boundary 256 having a second split ratio, e.g. 1/3.
Furthermore, the encoder 10 may be configured to perform a perpendicular (relative to the first split boundary 255) third binary split of the third partition 133 creating a third split boundary having a third split ratio.
According to this aspect of the invention, the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split, which is indicated in the splitting tree 13 by means of the arrow 257.
The encoder 10 is configured to omit signaling in the data stream 14 the third split ratio of the third binary split, since it is restricted to be the same as the second split ratio of the second binary split, as mentioned above.
Accordingly, at the decoder side, this implicit third split ratio may be inferred without an explicit split ratio indication for the third binary split in the data stream 14. According to this embodiment, the decoder 110 may be configured to perform the asymmetric quad split by using a binary splitting tree 13 based on at least a first and a second binary split indication, i.e. based on at least a split indication for the first binary split of the first partition 13i at the first partitioning level, and based on at least a split indication for the second binary split of the second partition 13å at the second partitioning level.
The decoder 1 10 may be configured to perform a first binary split creating a first split boundary 255 having a first split ratio, e.g. ¾. The decoder 110 may further be configured to perform a perpendicular second binary split creating a second split boundary 256 having a second split ratio, e.g. 1/3.
Furthermore, the decoder 1 10 may be configured to perform a perpendicular (relative to the first boundary 255) third binary split creating a third split boundary having a third split ratio.
According to this aspect of the invention, the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split, as mentioned above.
Thus, the decoder 110 may be configured to infer, without an explicit split-ratio-signal in the data stream 14, the third split ratio of the third binary split to be equal to the second split ratio of the second binary split.
Figure 26 shows a block diagram of an encoding method according to the above described concept. In particular, Figure 26 shows a block diagram of a method for block-based encoding a picture 12 into a data stream 14.
In block 2601 the picture 12 is partitioned into one or more leaf blocks 251 , 252, 253, 254 using recursive multi -tree partitioning along a splitting tree 13, and partitioning information of the splitting tree 13 is signaled in the data stream 14.
In block 2602 an asymmetric quad split is performed for a predetermined block 250, in which the predetermined block 250 is partitioned by at least a first split boundary 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 , 252, 253, 254.
In block 2603 a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256 is signaled in the data stream 14, wherein both split ratios may be encoded into the data stream 14 during the same splitting step.
Figure 27 shows a block diagram of a corresponding decoding method according to the above described concept. In particular, Figure 26 shows a block diagram of a method for block-based decoding the picture 12 from the data stream 14.
In block 2701 the picture 12 is partitioned into one or more leaf blocks 251 , 252, 253, 254 using recursive multi-tree partitioning along a splitting tree 13, and partitioning information of the splitting tree 13 is obtained from the data stream 14.
In block 2702 an asymmetric quad split is performed for a predetermined block 250, in which the predetermined block 250 is partitioned by at least a first split boundary 255 and a perpendicular second split boundary 256 into four asymmetric subblocks 251 , 252, 253, 254.
In block 2703 a first split ratio of the first split boundary 255 and a second split ratio of the second split boundary 256 are obtained from the data stream 14, wherein both split ratios may be obtained from the data stream 14 during the same splitting step.
4, ATS and TTS inheritance
This aspect intents to infer partitioning parameters of a block from its local neighborhood. Means for defining regions within a picture exists, e.g. tiling, slices or region-wise packing. Based on such sub-picture partitioning, all of the constraints and extensions herein, subsumed as Adaptation Tree Sets (ATS) and TTS or, in general, partitioning parameters, can be referenced/activated on a sub-picture basis. However, the current region definition
capabilities may not be sufficient to efficiently capture the shape of image objects accurately enough, to maximize the coding-efficiency benefit of an adapted partitioning scheme.
In this aspect, partitioning parameters of the current block are instead derived from previously coded blocks, e.g. in the spatial or temporal vicinity of the current block. Figure 28 shows an example for this aspect, which example is applicable to both an inventive encoder and an inventive decoder.
On the left side in Figure 28 a picture 12s is depicted which may be coded by using a spatial intra-prediction mode. The picture 12s may be subdivided into sub-picture regions, e.g. slices 281 , 282, 283. The slices 281 , 282, 283 may further be divided into blocks 284, 285, 286. Said blocks 284, 285, 286 may, for instance, be tree root blocks, e.g. CTUs, as previously explained above. As also explained above, the picture 12s may be partitioned, starting from a tree root block into one or more leaf blocks, along a splitting tree 13.
For each block, which may be a root block 284, 285, 286, a leaf block or any block between, one or more partitioning parameters 291 , 292, 293, 294 may be used by the encoder and the decoder for partitioning the respective block.
Partitioning parameters 291 , 292, 293, 294 may comprise, as non-limiting examples, one or more of the above described splitting indications of the other further aspects, such as, e.g. split ratios, split modes, tree types, tree type set (TTS), AQT-indications, implicit split indications for log2-partitions and non-log2-partitions, and many more.
For example in root block 286 one or more partitioning parameters 291 , 292 may be available for partitioning said block 286. The one or more partitioning parameters 291 , 292 may be contained in a partitioning parameter set 290.
According to this aspect of the invention, at least one partitioning parameter 292 of said block 286 may be an inherited partitioning parameter, which shall be explained in more detail below.
As can be seen, the picture 12s may comprise a further block 285 that may also be located in the same sub-picture region as block 286, e.g. located in slice 281. Said further block 285 is located in the spatial vicinity of the block 286. In this example, the further block 285 is a direct spatial neighbor of the block 286. In other words, the further block 285 is located directly adjacent to the block 286.
The blocks 285, 286 may be coded along a coding order, wherein the further block 285 may be coded prior to the block 286. Accordingly, said further block 285 may be a previously coded block that has been previously coded prior to the block 286.
The further block 285 may also comprise one or more partitioning parameters 293, 294. For 5 example, partitioning parameter‘B’ 294 may be used in partitioning the further block 285.
The block 286 may be consecutively coded afterwards, e.g. subsequent to the further block 285.
According to this aspect, the block 286 may inherit from the previously coded further block 285 at least one partitioning parameter, e.g. the aforementioned partitioning parameter‘B’. 10 Said inheritance is indicated in Figure 28 by means of arrow 295. For the sake of symbolizing the inherited partitioning parameter in the Figure, an asterisk is used. Accordingly, the block 286 may comprise the inherited partitioning parameter‘B*’ 292. The inherited partitioning parameter‘B*’ 292 may be a copy of partitioning parameter‘B’ 294 of the further block 285. Additionally or alternatively, block 286 may access partitioning 15 parameter‘B’ 294 by means of referencing.
Thus, the block 286 may be partitioned along the splitting tree 13 by using the inherited partitioning parameter‘B*’ that has been inherited from the previously coded further block 285.
On the right side in Figure 28, two pictures 12ti, 12t2 are depicted which may be coded by
20 using a temporal inter-prediction mode. Each of the pictures 12ti, 12t2 may be subdivided into sub-picture regions, e.g. slices 281 , 282, 283, as explained above with reference to picture 12s. The slices 281 , 282, 283 may also be divided into blocks 284, 285, 286. Said blocks 284, 285, 286 may, for instance, be tree root blocks, e.g. CTUs, as previously explained above. As also explained above, each of the blocks of the pictures 12ti, 12t2 may 25 be partitioned, starting from a tree root block into one or more leaf blocks, along a splitting tree 13.
In this example regarding temporal inter-prediction coding, the pictures 12ti, 12t2 may be coded along a coding order, wherein the first picture 12ti may be coded prior to the second picture 12t2.
30 The second picture 12t2 may comprise at least one block 286 wherein one or more partitioning parameters 291 , 292 may be available for partitioning said at least one block
286. The first picture 12ti may comprise a further block 285 wherein one or more partitioning parameters 293, 294 may be available for partitioning said further block 285.
For example, partitioning parameter‘B’ 294 may be used in partitioning the further block 285. The at least one block block 286 may be consecutively coded afterwards, e.g. subsequent to the further block 285.
This example of inter-prediction coding may be similar to the above described example of intra-prediction coding. However, in this example the further block 285 may be located in the temporal vicinity of the at least one block 286. That Is, the further block 285 may be a block of a previously coded picture 12ti. However, said further block 285 may be spatially located in a corresponding sub-picture region (e.g. slice 2811) of the first picture 12ti corresponding to a respective sub-picture region (e.g. slice 2812) of the second picture 12t2.
According to this aspect, the block 286 of the second picture 12t2 may inherit from the previously coded further block 285 of the first picture 12ti at least one partitioning parameter, e.g. the aforementioned partitioning parameter‘B’. Said inheritance is again indicated by means of arrow 295. For the sake of symbolizing the inherited partitioning parameter in the Figure, an asterisk is used. Accordingly, the block 286 may comprise the inherited partitioning parameter‘B*’ 292. The inherited partitioning parameter‘B*’ 292 may be a copy of partitioning parameter Έ’ 294 of the further block 285. Additionally or alternatively, block 286 may access partitioning parameter‘B’ 294 by means of referencing.
Thus, the block 286 may be partitioned along the splitting tree 13 by using the inherited partitioning parameter‘B*’ that has been inherited from the previously coded further block 285.
Thus, according to an embodiment, an encoder 10 is provided for block-based encoding a picture 12s, 12ti, 12t2 into a data stream 14 using at least one of a spatial intra-prediction and a temporal inter-prediction coding.
The encoder 10 is configured to partition the picture 12s, 12ti, 12t2 along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multi-tree partitioning, similar to the encoder 10 as explained above with reference to Figure 14.
The recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294.
If at least one block 286 is encoded by means of spatial intra-prediction, the encoder 10 is configured to inherit one or more partitioning parameters 294 from a further block 285 in the spatial vicinity of the at least one block 286, and to use said inherited one or more partitioning parameters 292 for partitioning said at least one block 286 in the recursive multitree partitioning.
Additionally or alternatively, if the at least one block 286 is encoded by means of temporal inter-prediction, the encoder 10 is configured to inherit one or more partitioning parameters 294 from a further block 285 in the temporal vicinity of the at least one block 286, and to use said inherited one or more partitioning parameters 292 for partitioning the at least one block 286 in the recursive multi-tree partitioning.
Furthermore, the encoder 10 is configured to signal in the data stream 14 that one or more inherited partitioning parameters 294 are used for partitioning the at least one block 286.
Based on said signaling in the data stream 14, the corresponding decoder 1 10 may be instructed to infer, for at least one block 286, one or more partitioning parameters 292 of a block 285 from its local neighborhood. In other words, the decoder 1 10 may use one or more inherited partitioning parameters 292, which are inherited from a further block 285 in the spatial or temporal vicinity, for performing the multi-tree partitioning of the at least one block 286.
Stated differently, one or more partitioning parameters 292 of the at least one current block 286 may be derived from the previously coded further block 285, e.g. in the spatial or temporal vicinity of the at least one current block 286.
Such an inference may also be referred to as inheritance, and the one or more partitioning parameters 292 which are derived or inferred from the previously coded further block 285 may also be referred to as inherited partitioning parameters.
The above mentioned signaling in the data stream 14 may indicate whether such inference or inheritance, respectively, is used over default partitioning using one or more default partitioning parameters of the picture 12s, 12ti , 12t2 or sub-picture region (e.g. slices, tiles, blocks, etc.).
Thus, for decoding according to this aspect of the invention, a decoder 1 10 is provided for block-based decoding a picture 12s, 12ti, 12t2 from a data stream 14 using at least one of a spatial intra-prediction and a temporal inter-prediction coding.
The decoder 110 is configured to partition the picture 12s, 12ti, 12t2 along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multitree partitioning, which may be comparable to the decoder 110 as described above with reference to Figure 15.
The recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294.
If at least one block 286 is decoded by means of spatial intra-prediction, the decoder 1 10 may be configured to use, based on the above mentioned signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block 286, which inherited one or more partitioning parameters 292 are inherited from a further block 285 in the spatial vicinity of the at least one block 286.
Additionally or alternatively, if the at least one block 286 is decoded by means of temporal inter-prediction, the decoder 1 10 is configured to use, based on the above mentioned signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block 286, which inherited one or more partitioning parameters 292 are inherited from a further block 285 in the temporal vicinity of the at least one block 286.
One example of this technique would be to optionally infer min/max tree depths from the partitioning trees of the spatial and/or temporal neighboring blocks 285 if advantageous. A first signaling on CTU level may indicate whether such inference is used over default partitioning parameters of the picture or subpicture.
Thus, according to an embodiment, the one or more partitioning parameters 291 , 292, 293, 294 may comprise a predetermined (e.g. min/max) tree depth of the splitting tree 13 for partitioning the at least one block 286, wherein the encoder 10 may be configured to inherit from the further block 286 in the spatial vicinity or from the further block 268 in the temporal vicinity a predetermined tree depth 294 of a further splitting tree that is for partitioning the further block 286 in the spatial or temporal vicinity. The encoder 10 may further be configured to use said inherited predetermined tree depth 292 in the splitting tree for partitioning the at least one block 286.
In other words, instead of using the default partitioning parameter, e.g. a predetermined (e.g. min/max) tree depth, the encoder 10 may be configured to use an inherited partitioning
parameter 292, i.e. the inherited tree depth of the splitting tree belonging to the previously coded further block 285, instead for partitioning the current at least one block 286.
Accordingly, the decoder 1 10 may derive from the signaling in the data stream 14 whether to use the default tree depth or the inherited tree depth 292 instead.
Thus, according to an embodiment, the one or more partitioning parameters 291 , 292, 293, 294 may comprise a predetermined (e.g. min/max) tree depth of the splitting tree 13 for partitioning the at least one block 286, wherein the decoder 1 10 may be configured to, based on the signaling in the data stream 14, inherit from the further block 285 in the spatial vicinity or from the further block 285 in the temporal vicinity a predetermined tree depth of a further splitting tree that is for partitioning the further block 285 in the spatial or temporal vicinity, and to use said inherited predetermined (e.g. min/max) tree depth 292 in the splitting tree 13 for partitioning the at least one block 286.
As mentioned above, the signaling in the data stream 14 may be done at a tree root block level, e.g. on CTU level.
Thus, according to an embodiment, if one or more inherited partitioning parameters 292, which are inherited from the further block 285 of the spatial or temporal vicinity are used as the one or more partitioning parameters for partitioning the at least one block 286, the encoder 10 may be configured to signal, at a root node partitioning level (e.g. CTU level) in the splitting tree 13, that the one or more inherited partitioning parameters 292 are used for partitioning the at least one block 286 instead of one or more default partitioning parameters.
Accordingly, the decoder 1 10 may also derive said signaling in the data stream 14 at a tree root block level, e.g. on CTU level.
Thus, according to an embodiment, the decoder 1 10 may configured to obtain the signaling from the data stream 14, at a root node partitioning level (e.g. CTU level) of the splitting tree 13, signaling that one or more inherited partitioning parameters 292 of the further block 285 in the spatial or temporal vicinity are used as the one or more partitioning parameters for partitioning the at least one block 286 instead of one or more default partitioning parameters.
In case such inference or inheritance of partitioning parameters 292 from neighboring blocks 285 is used, one more advanced embodiment would be to collect partitioning parameters 294, 295 of the neighboring blocks 285 into a set of partitioning parameter candidates. The set could also include mixtures/generalization/relaxations of some of the included candidates. The partitioning of the current block 286 could activate one of the
partitioning parameter set candidates by reference through an index. Alternatively, the current block 286 could be partitioned using the most probable set out of the candidates as defined by other means.
Thus, according to an embodiment, both the encoder 10 and the decoder 1 10 may be configured to access a set of inheritance partitioning parameter candidates comprising one or more partitioning parameters 294, 295 of the further block 285 in the spatial vicinity and/or one or more partitioning parameters 294, 295 of the further block 285 in the temporal vicinity.
Furthermore, both the encoder 10 and the decoder 1 10 may be configured to select at least one 294 of the partitioning parameters 294, 295 contained in the set of inheritance partitioning parameter candidates, and to use said selected one 294 of the partitioning parameters from the set of inheritance partitioning parameter candidates as a partitioning parameter for partitioning the at least one block 286.
For example, the selected one partitioning parameter 294 may be inherited and may, thus, become the inherited partitioning parameter 292, as depicted in Figure 28.
With this respect, means for signaling may be required that indicate whether the sub-picture partitioning parameters through regions (defined for instance in parameter sets) are to be interpreted as to be applied to all CTUs or whether some CTUs (e.g. in the region borders) can inherit partitioning information from its neighboring CTUs that belong to another region (described inference/inheritance above).
Thus, according to an embodiment, the encoder 10 may be configured to partition the picture 12s, 12ti, 12t2 into a multitude of root blocks 284, 285, 286, each of which comprising one or more own partitioning parameters 291 , 293, 294, which may be non-inherited default partitioning parameters.
The encoder 10 may be configured to signal an indication in the data stream 14 for indicating whether all root blocks of the multitude of root blocks 284, 285, 286 are partitioned by using their one or more own partitioning parameters 291 , 293, 294, or whether at least one root block 286 of the multitude of root blocks 284, 285, 286 is partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity of the at least one root block 286, wherein the one or more inherited partitioning parameters 292 are used instead of the one or more own partitioning parameters for partitioning the at least one root block 286.
The decoder 110 may be configured to obtain a respective signaling comprising the above mentioned indication from the data stream 14, and the decoder 110 may then accordingly use the one or more inherited partitioning parameters 292 instead of one or more default partitioning parameters (also referred to as“own partitioning parameters").
Thus, according to an embodiment, the decoder 1 10 may be configured to partition the picture 12s, 12ti, 12t2 into a multitude of root blocks 284, 285, 286, each of which comprising one or more own partitioning parameters 291 , 293, 294, which may be non-inherited default partitioning parameters.
Furthermore, the decoder 110 may be configured to obtain the above mentioned indication from the data stream 14 indicating whether all root blocks of the multitude of root blocks 284, 285, 286 are partitioned by using their one or more own partitioning parameters 291 , 293, 294, or whether at least one root block 286 of the multitude of root blocks 284, 285, 286 is partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity of the at least one root block 286, wherein the one or more inherited partitioning parameters 292 are used instead of the one or more own partitioning parameters for partitioning the at least one root block 286.
Besides, another embodiment would be to define a parameter that signals the number of CTUs at the border of the region for which syntax is present that allows for selecting between partitioning parameters of a neighboring CTU and default partitioning parameters of the corresponding region/subpicture.
Thus, according to an embodiment, the encoder 10 may be configured to partition the picture 12s, 12ti, 12t2 into a multitude of root blocks 284, 285, 286, and to signal an indication in the data stream 14 for indicating a number of root blocks which are partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity.
Accordingly, the decoder 110 may be configured to obtain said indication from the data stream 14 to derive the number of blocks, e.g. CTUs, which are partitioned by using inherited partitioning parameters 292.
Thus, according to an embodiment, the decoder 1 10 may be configured to partition the picture 12s, 12ti, 12t2 into a multitude of root blocks 284, 285, 286, and to obtain an indication from the data stream 14 which indicates a number of root blocks which are
partitioned using one or more inherited partitioning parameters 292 being inherited from a further root block 285 in the temporal or spatial vicinity.
Alternatively, different sets of ATS and TTS could be maintained per temporal layer.
Figure 29 shows a block diagram of a method for encoding a picture 12s, 12ti, 12t2 into the data stream 14 in accordance with the above described concept.
In block 2901 the picture 12s, 12ti, 12t2 is partitioned along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294.
It is then differentiated whether a block is coded using spatial intra-prediction or temporal inter-prediction.
According to block 2902, if at least one block 286 is encoded by means of spatial intra- prediction, one or more partitioning parameters are inherited from a further block 285 in the spatial vicinity of the at least one block 286, and said inherited partitioning parameters 292 are used for partitioning said at least one block 286 in the recursive multi-tree partitioning.
According to block 2903, if the at least one block 286 is encoded by means of temporal inter-prediction, one or more partitioning parameters are inherited from a further block 285 in the temporal vicinity of the at least one block 286, and said inherited partitioning parameters 292 are used for partitioning the at least one block 286 in the recursive multi- tree partitioning.
In both cases, however, it is signaled in the data stream 14 that one or more inherited partitioning parameters 292 are used for partitioning the at least one block 286, as shown in method block 2904.
Figure 30 shows a block diagram of a corresponding method for decoding the picture 12s, 12ti, 12t2 from the data stream 14 in accordance with the above described concept.
In block 3001 the picture 12s, 12ti, 12t2 is partitioned along a splitting tree 13 from a tree root block 284, 285, 286 into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters 291 , 292, 293, 294.
It is then differentiated whether a block is coded using spatial intra-prediction or temporal inter-prediction.
As shown in block 3002, if at least one block 286 is decoded by means of spatial intra- prediction, the method comprises a step of using, based on a signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block
286, which inherited partitioning parameters 292 are inherited from a further block 285 in the spatial vicinity of the at least one block 286.
As shown in block 3003, if the at least one block 286 is decoded by means of temporal inter- prediction, the method comprises a step of using, based on a signaling in the data stream 14, one or more inherited partitioning parameters 292 for partitioning the at least one block
286, which inherited partitioning parameters 292 are inherited from a further block 285 in the temporal vicinity of the at least one block 286.
Further embodiments of an encoder and a decoder, respectively, that may be used in the aspects 1.1 and/or 1.2 of chapter F may be:
1. Block-based encoder for encoding a picture into a data stream, wherein the
encoder is configured to
5 partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree into leaf blocks, a height and a width of which in number of samples is a power of two, signal partitioning information on the splitting tree in the data stream by, for a tree node of the splitting tree involving a set of split options including a no-split 10 option, checking whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, signal the split option out of the set of split options, and if not, remove the no-split option from the set of split options to 15 obtain a reduced set of split options and if the number of split options in the reduced set is greater than one, signal a split option out of the reduced set of split options, using which the partition is split according to the splitting tree, or if the number of split options in the reduced set is one, refrain from signaling the split option.
20 2. Block-based decoder for decoding a picture from a data stream, wherein the
decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree into leaf blocks, a height and a width of which in number of samples is a power of two,
25 obtaining partitioning information on the splitting tree from the data stream by, for a tree node of the splitting tree involving a set of split options including a nosplit option, checking whether a partition to which the tree node belongs, has a height : and a width which are in number of samples a power of two,
if yes, obtaining the split option out of the set of split options based on a signaling in the data stream, and if not, remove the no-split option from the set of split options to obtain a reduced set of split options and if the number of split options in the reduced set is greater than one, obtain a split option out of the reduced set of split options from the data stream based on a split-option-signaling in the data stream, and using said split option for splitting the partition according to the splitting tree, or if the number of split options in the reduced set is one, infer the split option without obtaining a split-option-signal from the data stream.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.
Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
Some embodiments according to the invention comprise a data carrier having
electronically readable control signals, which are capable of cooperating with a
programmable computer system, such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier,
the digital storage medium or the recorded medium are typically tangible and/or non-transitory.
A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.
The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore,
to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.
References
[1 ] ITU-T and I SO/I EC. High efficiency video coding. ITU-T Recommendation H.265 | ISO/IEC 23008-10 (HEVC), edition 1 , 2013; edition 2, 2014.
[2] ITU-T and ISO/IEC. Advanced video coding for audiovisual services. ITU-T Recommendation H.264 | ISO/IEC 14406-10 (AVC), edition 1 , 2003.
[3] J. Chen, E. Alshina, G.J. Sullivan, J.-R. Ohm, J. Boyce, Algorithm Description of Joint Exploration Test Model 6 (JEM 6), JVET, doc. JVET-F1001 , April 2017.
[4] X. Li, H.-C. Chuang, J. Chen, M. Karczewicz, L. Zhang, X. Zhao, A. Said, Multi- Type-Tree, JVET, doc. JVET-D01 17, Oct. 2016. [5] F. Le L6annec, T. Poirier, F. Urban, Asymmetric Coding Units in QTBT, JVET, doc.
JVET-D0064, Oct. 2016.
Claims
1. Block-based encoder for encoding a picture into a data stream, configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, signal partitioning information of the binary splitting tree in the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, signal the split option or the no-split option in the data stream, and if not, refrain from signaling the split option in the data stream. 2. The encoder of claim 1 , wherein, if the partition to which the tree node belongs has a height and a width which are in number of samples a power of two, the encoder is configured to split the partition and signal the applied split option, or not split the partition and signal the no-split option, and if the partition to which the tree node belongs does not have a height and/or a width which are in number of samples a power of two, split the partition such that a resulting sub-partition belonging to a tree node at a subsequent tree level has a height and a width which are in number of samples a power of two, and refrain from signaling the applied split option in the data stream.
3 The encoder of claim 1 or 2, wherein the encoder is configured to split a predetermined partition into at least three sub-partitions, each of which having a height and a width which are in number of samples a power of two, by
applying a first split to the predetermined partition, wherein said first split creates a first sub-partition which does not have a height and a width which are in number of samples a power of two, and signaling the applied split option of the first split in the data stream, and subsequently applying a consecutive second split to the first sub-partition, wherein said consecutive second split creates a second and a third sub-partition, each of which having a height and a width which are in number of samples a power of two, and refraining from signaling the applied split option of the consecutive second split in the data stream.
4. The encoder of claim 3, wherein the encoder is configured to apply the consecutive second split such that at least one of the resulting second and third sub-partitions is symmetrical to the first sub-partition with respect to a width and a height of the respective sub-partitions.
5. The encoder of claim 3 or 4, wherein the encoder is configured to choose a parallel split as the split option for the consecutive second split, wherein the consecutive second split is parallel to the first split.
6 The encoder of any one of claims 1 to 5, wherein the partitioning information comprises at least one of a split mode and a split ratio, and wherein the encoder is configured to omit signaling at least one of the split mode and the split ratio for the partition to which the tree node belongs, if said partition does not have a height and a width which are in number of samples a power of two.
7. The encoder of any one of claims 1 to 6, wherein the encoder is configured to refrain from signaling the split option in the data stream (14) only if the partition or subpartition, which shall be split according to said split option, comprises a predetermined size or if the corresponding tree node of said partition or sub-partition is at a predetermined tree-depth level of the binary tree.
8 The encoder of claim 7, wherein the predetermined tree-depth level Is the level of the last tree node before a tree node belonging to a leaf block, or wherein the predetermined tree-depth level is one level or two levels above the last tree node before a tree node belonging to a leaf block.
9. The encoder of any one of claims 1 to 8, wherein the encoder is configured to signal in the data stream an indication for indicating whether the partition to which the tree node belongs is allowed to be a leaf block.
10. The encoder of any one of claims 1 to 9, wherein the encoder is configured to use a codec that is based on Generalized Binary Splitting, in which binary splits with uneven splitting ratios are allowed.
11. Block-based decoder for decoding a picture from a data stream, configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, obtain partitioning information of the binary splitting tree from the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, obtain the split option or the no-split option from the data stream, and if not, implicitly infer the split option without an explicit split- option-signal in the data stream. 12. The decoder of claim 11 , wherein, if the partition to which the tree node belongs has a height and a width which are in number of samples a power of two, the decoder is configured to split the partition based on a split signal in the data stream, or not split the partition based on a no-split signal in the data stream, and if the partition to which the tree node belongs does not have a height and/or a width which are in number of samples a power of two,
implicitly infer, without an explicit split-option-signal in the data stream, a split option that results in a sub-partition belonging to a tree node at a subsequent tree level, which sub-partition has a height and a width which are in number of samples a power of two, and split the partition into the sub-partition according to said inferred split option.
13. The decoder of claim 11 or 12, wherein the decoder is configured to split a predetermined partition into at least three sub-partitions, each of which having a height and a width which are in number of samples a power of two, by obtaining a first split option from the data stream and applying a first split according to said first split option to the predetermined partition, wherein said first split option creates a first sub-partition which does not have a height and a width which are in number of samples a power of two, and implicitly inferring, without an explicit split-option-signal in the data stream, a split option for a subsequent second split and subsequently applying said second split to the first sub-partition, wherein said second split creates a second and a third sub-partition, each of which having a height and a width which are in number of samples a power of two.
14. The decoder of claim 13, wherein the decoder is configured to apply the consecutive second split such that at least one of the resulting second and third sub-partitions is symmetrical to the first sub-partition with respect to a width and a height of the respective sub-partitions.
15. The decoder of claim 13 or 14, wherein the decoder is configured to choose a parallel split as the split option for the consecutive second split, wherein the consecutive second split is parallel to the first split. 16. The decoder of any one of claims 1 1 to 15, wherein the partitioning information comprises at least one of a split mode and a split ratio, and wherein the decoder is configured to infer from the data stream, without an explicit signaling in the data stream, at least one of the split mode and the split ratio for the partition to which the tree node belongs, if said partition does not have a height and a width which are in number of samples a power of two.
17 The decoder of any one of claims 11 to 16, wherein the decoder is configured to implicitly infer the split option only if the partition or sub-partition, which shall be split
according to the split option, comprises a predetermined size or if the corresponding tree node of said partition or sub-partition is at a predetermined tree-depth level of the binary tree.
18. The decoder of claim 17, wherein the predetermined tree-depth level is the level of the last tree node before a tree node belonging to a leaf block, or wherein the predetermined tree-depth level is one level or two levels above the last tree node before a tree node belonging to a leaf block.
19. The decoder of any one of claims 11 to 18, wherein the decoder is configured to obtain from the data stream an indication for indicating whether the partition to which the tree node belongs is allowed to be a leaf block.
20. The decoder of any one of claims 1 1 to 19, wherein the decoder is configured to use a codec that is based on Generalized Binary Splitting, in which binary splits with uneven splitting ratios are allowed.
21. A method for block-based encoding a picture into a data stream, wherein the method comprises the steps of: partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, signaling partitioning information of the binary splitting tree in the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, the method comprises a step of signaling the split option or the no-split option in the data stream, and if not, the method comprises a step of refraining from signaling the split option in the data stream.
22 A method for block-based decoding a picture from a data stream, wherein the method comprises the steps of:
partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a binary splitting tree, wherein a height and a width of the leaf blocks in number of samples is a power of two, obtaining partitioning information of the binary splitting tree from the data stream by checking, for a tree node of the binary splitting tree comprising a set of split options including a no-split option and a split option, whether a partition to which the tree node belongs, has a height and a width which are in number of samples a power of two, if yes, the method comprises a step of obtaining the split option or the no-split option from the data stream, and if not, the method comprises a step of implicitly inferring the split option without an explicit split-option-signal in the data stream.
23. A computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, a method according to claim 21 or 22.
24. A data stream generated according to the method of claim 21.
25. Block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including at least one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on: a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is
connected to the root node, are implicitly signaled or explicitly signaled, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not, and wherein the encoder is further configured to rule out the one of the no- split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
26. The encoder of claim 25, wherein the encoder is configured to calculate the tree depth of the predetermined node depending on the check whether the predetermined tree node meets the predetermined tree depth constraint, wherein the calculating is performed by omitting or weighting the count of tree nodes along the splitting tree if the predetermined tree node adheres to at least one of the options a), b) or e).
27. The encoder of claim 25 or 26, wherein the encoder is configured to, in discriminating according to option c), discriminate whether the partition has a height and a width which in number of samples is a power of two, wherein: if yes, the partition is allowed to be a leaf block, and if not, the partition is disallowed to be a leaf block.
28. The encoder of any one of claims 25 to 27, wherein, if the discrimination according to option c) reveals that the partition is allowed to be a leaf block, the encoder is configured to signal an indication in the data stream for indicating that the partition is allowed to be a leaf block.
29 The encoder (10) of any one of claims 25 to 28, wherein, if the predetermined tree node (13i, 132, ... , 139), upon splitting along the splitting tree (13), does not yet fulfil a minimum tree depth constraint, the encoder (10) is configured to omit signaling a split indication in the data stream (14).
30. The encoder of any one of claims 25 to 29, wherein, if the predetermined tree node fulfils a maximum tree depth constraint, the encoder is configured to rule out the one of the no-split option and the split option.
31. The encoder of any one of claims 25 to 30, wherein the encoder is configured to, in checking whether the predetermined tree node meets the predetermined tree depth constraint according to b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, check whether the size of the partition to which the predetermined tree node belongs and/or the a size of the one or more intermediate partitions belonging to the one or more intermediate tree nodes between the root node and the predetermined tree node is equal to a predetermined partition size contained in a set of partition sizes.
32. The encoder of claim 31 , wherein the set of partition sizes is static, wherein the set of partition sizes comprises one or more partition sizes, each partition size having a static value, or dynamic, wherein the set of partition sizes comprises one or more partition sizes, each partition size having a dynamically changeable value, wherein said value depends on a variable predetermined threshold.
33. Block-based decoder for decoding a picture from a data stream, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including at least one of a no- split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on:
a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are available in the data stream or not, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not, and wherein the decoder is further configured to rule out the one of the no- split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
34. The decoder of claim 33, wherein the decoder is configured to calculate the tree depth of the predetermined node depending on the check whether the predetermined tree node meets the predetermined tree depth constraint, wherein the calculating is performed by omitting or weighting the count of tree nodes along the splitting tree if the predetermined tree node adheres to at least one of the options a), b) or c).
35. The decoder of claim 33 or 34, wherein the decoder is configured to, in discriminating according to option c), discriminate whether the partition has a height and a width which in number of samples is a power of two, wherein: if yes, the partition is allowed to be a leaf block, and if not, the partition is disallowed to be a leaf block.
36. The decoder of any one of claims 33 to 35, wherein the decoder is configured to obtain an indication from the data stream indicating whether the partition is allowed to be a leaf block.
37. The decoder of any one of claims 33 to 36, wherein, if the predetermined tree node (13i, 132, ... , 13g), upon splitting along the splitting tree, does not yet fulfil a minimum tree depth constraint, the decoder (110) is configured to infer a split option without an explicit split-option-signal in the data stream (14).
5 38. The decoder of any one of claims 33 to 37, wherein, if the predetermined tree node fulfils a maximum tree depth constraint, the decoder is configured to rule out the one of the no-split option and the split option.
39. The decoder of any one of claims 33 to 38, wherein the decoder is configured to, in checking whether the predetermined tree node meets the predetermined tree depth 10 constraint according to b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node,
15 check whether the size of the partition to which the predetermined tree node belongs and/or the size of the one or more intermediate partitions belonging to the one or more intermediate tree nodes between the root node and the predetermined tree node is equal to a predetermined partition size contained in a set of partition sizes.
20 40. The decoder of claim 39, wherein the set of partition sizes is static, wherein the set of partition sizes comprises one or more partition sizes, each partition size having a static value, or dynamic, wherein the set of partition sizes comprises one or more partition sizes, each partition size having a dynamically changeable value, wherein said value 25 depends on a variable predetermined threshold.
41 , Method for block-based encoding a picture into a data stream, wherein the method comprises the steps of: partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by
checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on: a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are implicitly signaled or explicitly signaled, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not, and wherein the method further comprises a step of ruling out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
A method for block-based decoding a picture from a data stream, wherein the method comprises the steps of: partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, by checking, for a predetermined tree node of the splitting tree, wherein the splitting tree comprises a set of split options including one of a no-split option and a split option, whether said predetermined tree node meets a predetermined tree depth constraint depending on:
a distance of said predetermined tree node from a root node of the splitting tree, or a number of splits of a partition to which said predetermined tree node belongs along the splitting tree, and a) by discriminating whether coding options of one or more preceeding tree nodes, via which the predetermined tree node is connected to the root node, are available in the data stream or not, and/or b) depending on a size of a partition to which the predetermined tree node belongs and/or on a size of one or more intermediate partitions belonging to one or more intermediate tree nodes between the root node and the predetermined tree node, and/or c) by discriminating whether a partition to which the predetermined tree node belongs is allowed to be a leaf block, or not, and wherein the method further comprises a step of ruling out the one of the no-split option and the split option depending on the check whether the predetermined tree node meets the predetermined tree depth constraint.
43. A computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, a method according to claim 41 or 42. 44. A data stream generated according to the method of claim 41.
45. Block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to partition the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, select, for performing the multi-tree partitioning, a tree type set comprising one or more tree types, and signal the tree type set in the data stream, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf
block partitioning level, including the root block partitioning level and the leaf block partitioning level.
46. The encoder of claim 45, wherein the encoder is configured to signal in the data stream the selected tree type set, at the root block partitioning level, and to use the one or more tree types contained in the selected tree type set for partitioning the picture from the tree root block into the one or more leaf blocks.
47. The encoder of claim 45 or 46, wherein the encoder is configured to select a tree type set comprising a first tree type and a different second tree type, to use the first tree type for performing the multi-tree partitioning at a first partitioning level, and to use the second tree type for performing the multi-tree partitioning at a different second partitioning level, and to signal the selected tree type set in the data stream, at the root block partitioning level. 48. The encoder of any one of claims 45 to 47, wherein the encoder is configured to select a tree type set comprising a number of tree types which number of tree types is equal to a number of partitioning levels minus one, to use, at each partitioning level except the leaf block partitioning level, one of the tree types contained in the tree type set, and to signal the selected tree type set in the data stream, at the root block partitioning level.
49. The encoder of any one of claims 45 to 48, wherein the encoder is configured to select a tree type set comprising at least a first tree type and a second tree type to be used at a predetermined partitioning level, and to selectively choose exactly one out of the at least first and second tree types for performing the multi-tree partitioning at said predetermined partitioning level, and
to signal the selected tree type set in the data stream, at the root block partitioning level.
50. The encoder of claim 49, wherein the encoder is configured to signal at the predetermined partitioning level an indication which one out of the first and second tree types is selectively chosen from the tree type set for performing the multi-tree partitioning at said predetermined partitioning level.
51. The encoder of any one of claims 45 to 50, wherein the encoder is configured to select a tree type set comprising a first tree type, a second tree type and a third tree type, to use the first tree type for performing the multi -tree partitioning at a predetermined partitioning level, and to selectively choose exactly one out of the second and the third tree types for performing the multi-tree partitioning at one or more subsequent partitioning levels, and to signal the selected tree type set in the data stream, at the root block partitioning level.
52. The encoder of claim 51 , wherein the encoder is configured to signal at the one or more subsequent partitioning levels an indication which one out of the second and the third tree types is selectively chosen from the tree type set for performing the multi-tree partitioning at said second partitioning level.
53. The encoder of any one of claims 45 to 52, wherein the encoder is configured to select a first tree type set at a first partitioning level and to use one of the tree types contained in said first tree type set for performing the multi-tree partitioning at said first partitioning level, or for performing the multi -tree partitioning at said first partitioning level and one or more subsequent partitioning levels, and select the first or a second tree type set at a second partitioning level and to use one of the tree types contained in the selected one of the first and the second tree type sets for performing the multi-tree partitioning at the second partitioning level, or for performing the multi-tree partitioning at said second partitioning level and one or more subsequent partitioning levels, and
to signal the selected tree type set in the data stream, separately at the first partitioning level and the second partitioning level.
54. The encoder of any one of claims 45 to 53, wherein the encoder is configured to select for a first partition of a first partitioning level a first tree type set at the first partitioning level and to use one of the tree types contained in said first tree type set for performing the multi-tree partitioning of said first partition at said first partitioning level, or for performing the multi-tree partitioning of said first partition at said first partitioning level and of one or more subsequent partitions at one or more subsequent partitioning levels, and select for a second partition of the first partitioning level the first or a second tree type set at said first partitioning level and to use one of the tree types contained in the selected one of the first and the second tree type sets for performing the multi- tree partitioning of said second partition at said first partitioning level, or for performing the multi-tree partitioning of said second partition at said first partitioning level and of one or more partitions at one or more subsequent partitioning levels, and to signal the respectively selected tree type set in the data stream, separately for the first partition and the second partition at the first partitioning level.
55 The encoder of any one of claims 45 to 54, wherein the one or more tree types contained in a tree type set are arranged in a consecutive order, wherein the encoder is configured to consecutively use, at each partitioning level except the leaf block partitioning level, one of the one or more consecutively arranged tree types in the order as they are arranged in the tree type set. 56. The encoder of any one of claims 45 to 55, wherein each tree type set comprises at least one of a quad split tree type, a binary split tree type, a generalized binary split tree type, and a triple split tree type.
57 Block-based decoder for decoding a picture from a data stream, wherein the decoder is configured to
partition the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, perform the multi-tree partitioning using a tree type set comprising one or more tree types based on a signaling of the tree type set in the data stream, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level. 58. The decoder of claim 57, wherein the decoder is configured to obtain from the signaling in the data stream, at the root block partitioning level, the tree type set for performing the multi -tree partitioning, and to use the one or more tree types contained in the tree type set for partitioning the picture from the tree root block into the one or more leaf blocks. 59. The decoder of claim 57 or 58, wherein the decoder is configured to obtain from the signaling in the data stream, at the root block partitioning level, a tree type set comprising a first tree type and a different second tree type, and to use the first tree type for performing the multi-tree partitioning at a first partitioning level, and to use the second tree type for performing the multi-tree partitioning at a different second partitioning level.
60. The decoder of any one of claims 57 to 59, wherein the decoder is configured to obtain from the signaling in the data stream, at the root block partitioning level, a tree type set comprising a number of tree types which number of tree types is equal to a number of partitioning levels minus one, to use, at each partitioning level except the leaf block partitioning level, one of the tree types contained in the tree type set for performing the multi-tree partitioning.
61. The decoder of any one of claims 57 to 60, wherein the decoder is configured to
obtain from the signaling in the data stream, at the root block partitioning level, a tree type set comprising at least a first tree type and a second tree type to be used at a predetermined partitioning level, and to selectively choose exactly one out of the at least first and second tree types for performing the multi-tree partitioning at said predetermined partitioning level.
62. The decoder of claim 61 , wherein the decoder is configured to obtain from a signaling at the predetermined partitioning level an indication which one out of the first and second tree types to selectively choose for performing the multi-tree partitioning at said predetermined partitioning level.
63. The decoder of any one of claims 57 to 62, wherein the decoder is configured to obtain from the signaling in the data stream, at the root block partitioning level, a tree type set comprising a first tree type, a second tree type and a third tree type, to use the first tree type for performing the multi-tree partitioning at a predetermined partitioning level, and to selectively choose exactly one out of the second and the third tree types for performing the multi-tree partitioning at one or more subsequent partitioning levels. 64. The decoder of claim 63, wherein the decoder is configured to obtain from a signaling at the one or more subsequent partitioning levels an indication which one out of the second and the third tree types to selectively choose for performing the multi-tree partitioning at said second partitioning level.
65 The decoder of any one of claims 57 to 64, wherein the decoder is configured to obtain from the signaling in the data stream, at a first partitioning level, a first tree type set and to use one of the tree types contained in said first tree type set for performing the multi-tree partitioning at said first partitioning level, or for performing the multi-tree partitioning at said first partitioning level and one or more subsequent partitioning levels, and
obtain from a separate signaling in the data stream, at a second partitioning level, the first or a second tree type set and to use one of the tree types contained in the selected one of the first and the second tree type sets for performing the multitree partitioning at the second partitioning level, or for performing the multi-tree partitioning at said second partitioning level and one or more subsequent partitioning levels.
The decoder of any one of claims 57 to 65, wherein the decoder is configured to obtain from the signaling in the data stream, at a first partitioning level, a first tree type set for a first partition of said first partition level, and to use one of the tree types contained in said first tree type set for performing the multi-tree partitioning of said first partition at said first partitioning level, or for performing the multi-tree partitioning of said first partition at said first partitioning level and of one or more subsequent partitions at one or more subsequent partitioning levels, and obtain from the signaling in the data stream, at the first partitioning level, the first or a second tree type set for a second partition of said first partition level and to use one of the tree types contained in the selected one of the first and the second tree type sets for performing the multi-tree partitioning of said second partition at said first partitioning level, or for performing the multi-tree partitioning of said second partition at said first partitioning level and one of one or more subsequent partitions at one or more subsequent partitioning levels, wherein the decoder is further configured to obtain the signaling of the respective tree type set from the data stream, separately for the first partition at the first partitioning level and for the second partition at said first partitioning level.
The decoder of any one of claims 57 to 66, wherein the one or more tree types contained in a tree type set are arranged in a consecutive order, wherein the decoder is configured to consecutively use, at each partitioning level except the leaf block partitioning level, one of the one or more consecutively arranged tree types in the order as they are arranged in the tree type set.
The decoder of any one of claims 57 to 67, wherein each tree type set comprises at least one of a quad split tree type, a binary split tree type, a generalized binary split tree type, and a triple split tree type.
69. A method for block-based encoding a picture into a data stream, wherein the method comprises the steps of partitioning the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, selecting, for performing the multi-tree partitioning, a tree type set comprising one or more tree types, and signaling the tree type set in the data stream, wherein the signaling is done at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
70. A method for block-based decoding a picture from a data stream, wherein the method comprises the steps of partitioning the picture along a splitting tree from a tree root block, at a root block partitioning level, into one or more leaf blocks, each at a respective leaf block partitioning level, using recursive multi-tree partitioning, performing the multi-tree partitioning using a tree type set comprising one or more tree types, wherein the step of performing the multi-tree partitioning is based on a signaling of the tree type set in the data stream, at one or more partitioning levels between the root block partitioning level and the leaf block partitioning level, including the root block partitioning level and the leaf block partitioning level.
71. A computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, a method according to claim 69 or 70.
72. A data stream generated according to the method of claim 69.
73. Block-based encoder for encoding a picture into a data stream, wherein the encoder is configured to
partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and signal partitioning information of the splitting tree in the data stream, wherein the encoder is further configured to perform for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and to signal in the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary. 74. The encoder of claim 71 , wherein the encoder is configured to perform the asymmetric quad split based on a single split indication, and to signal in the data stream the single split indication and the first splitting ratio of the first split boundary and the second split ratio of the second split boundary.
75. The encoder of claim 73 or 74, wherein the encoder is configured to perform the asymmetric quad split by using a binary splitting tree based on at least a first and a second binary split indication, wherein the encoder is configured to perform a first binary split creating a first split boundary having a first split ratio, and to perform a perpendicular second binary split creating a second split boundary having a second split ratio, and a perpendicular third binary split creating a third split boundary having a third split ratio, wherein the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split, and wherein the encoder is configured to omit signaling in the data stream the third split ratio of the third binary split.
76. The encoder of any one of claims 73 to 75, wherein at least one of the first and the second binary splits is a generalized binary split such that the resulting subblocks are asymmetric subblocks.
77. Block-based decoder for decoding a picture from a data stream, wherein the decoder is configured to partition the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and obtain a partitioning information of the splitting tree from the data stream, wherein the decoder is further configured to perform for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and to obtain from the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
78. The decoder of claim 77, wherein the decoder is configured to perform the asymmetric quad split based on a single split indication, and to obtain from the data stream the single split indication and the first splitting ratio of the first split boundary, and the single split indication and the second split ratio of the second split boundary.
79. The decoder of claim 77 or 78, wherein the decoder is configured to perform the asymmetric quad split by using a binary splitting tree based on at least a first and a second binary split indication, wherein the decoder is configured to perform a first binary split creating a first split boundary having a first split ratio, and to perform a perpendicular second binary split creating a second split boundary having a second split ratio, and a perpendicular third binary split creating a third split boundary having a third split ratio, wherein the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split,
and wherein the decoder is configured to infer, without an explicit split-ratio- signal in the data stream, the third split ratio of the third binary split to be equal to the second split ratio of the second binary split.
80. The decoder of any one of claims 77 to 79, wherein at least one of the first and the second binary splits is a generalized binary split such that the resulting subblocks are asymmetric subblocks.
81. A method for block-based encoding a picture into a data stream, wherein the method comprises the steps of: partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and signaling partitioning information of the splitting tree in the data stream, wherein the method further comprises steps of performing for a predetermined block an asymmetric quad split, in which the predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and signaling in the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
82. The method of claim 81 , wherein the method further comprises steps of: performing the asymmetric quad split based on a single split indication, and signaling in the data stream the single split indication and the first splitting ratio of the first split boundary and the second splitting ratio of the second split boundary.
83. The method of claim 81 or 82, wherein the method further comprises steps of: performing the asymmetric quad split by using a binary splitting tree based on at least a first and a second binary split indication, wherein the method further comprises steps of performing a first binary split creating a first split boundary having a first split ratio, and
performing a perpendicular second binary split creating a second split boundary having a second split ratio, and a perpendicular third binary split creating a third split boundary having a third split ratio, wherein the third split ratio of the third binary split is restricted to be equal to 5 the second split ratio of the second binary split, and wherein the method comprises a step of omit signaling in the data stream the third split ratio of the third binary split.
84. A method for block-based decoding a picture from a data stream, wherein the method comprises the steps of:
10 partitioning the picture into one or more leaf blocks using recursive multi-tree partitioning along a splitting tree, and obtaining a partitioning information of the splitting tree from the data stream, wherein the method further comprises steps of performing for a predetermined block an asymmetric quad split, in which the
15 predetermined block is partitioned by at least a first split boundary and a perpendicular second split boundary into four asymmetric subblocks, and obtaining from the data stream a first split ratio of the first split boundary and a second split ratio of the second split boundary.
:
85. The method of claim 84, wherein the method further comprises steps of:
20 performing the asymmetric quad split based on a single split indication, and obtaining from the data stream the single split indication and the first splitting ratio of the first split boundary, and the single split indication and the second split ratio of the second split boundary.
86. The method of claim 84 or 85, wherein the method further comprises steps of:
25 performing the asymmetric quad split by using a binary splitting tree based on at least a first and a second binary split indication, wherein the method further comprises steps of performing a first binary split creating a first split boundary having a first split ratio, and
performing a perpendicular second binary split creating a second split boundary having a second split ratio, and a perpendicular third binary split creating a third split boundary having a third split ratio, wherein the third split ratio of the third binary split is restricted to be equal to the second split ratio of the second binary split, and wherein the method comprises a step of inferring, without an explicit split-ratio-signal in the data stream, the third split ratio of the third binary split to be equal to the second split ratio of the second binary split.
87. A computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, a method according to claim 78 or 81.
88. A data stream generated according to the method of claim 81.
89. Block-based encoder (10) for encoding a picture (12s, 12ti, 12t2) into a data stream (14) using at least one of a spatial intra-prediction and a temporal inter-prediction coding, wherein the encoder (10) is configured to partition the picture (12s, 12ti, 12t2) along a splitting tree (13) from a tree root block (284, 285, 286) into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters (291 , 292, 293, 294), wherein if at least one block (286) is encoded by means of spatial intra-prediction, the encoder (10) is configured to inherit one or more partitioning parameters (294) from a further block (285) in the spatial vicinity of the at least one block (286), and to use said inherited one or more partitioning parameters (292) for partitioning said at least one block (286) in the recursive multi-tree partitioning, and if the at least one block (286) is encoded by means of temporal interprediction, the encoder (10) is configured to inherit one or more partitioning parameters (294) from a further block (285) in the temporal vicinity of the at least one block (286), and to use said inherited one or more partitioning parameters (292) for partitioning the at least one block (286) in the recursive multi-tree partitioning, and
wherein the encoder (10) is further configured to signal in the data stream (14) that one or more inherited partitioning parameters (292) are used for partitioning the at least one block (286).
90. The encoder (10) of claim 89, wherein the further block (285) in the spatial vicinity is a block that is located in the same spatial sub-picture region (281 ) as the at least one block (286).
91 . The encoder (10) of claim 89 or 90, wherein the further block (285) is located directly adjacent to the at least one block (286), or wherein one or two other blocks are arranged between the at least one block (286) and the further block (285). 92. The encoder (10) of any one of claims 89 to 91 , wherein the one or more partitioning parameters (291 , 293, 294) comprise a predetermined tree depth of the splitting tree (13) for partitioning the at least one block (286), and wherein the encoder (10) is configured to inherit from the further block (285) in the spatial vicinity or from the further block (285) in the temporal vicinity a predetermined tree depth (292) of a further splitting tree that is for partitioning the further block (285) in the spatial or temporal vicinity, to use said inherited predetermined tree depth in the splitting tree (13) for partitioning the at least one block (286).
93. The encoder (10) of any one of claims 89 to 92, wherein, if one or more inherited partitioning parameters (292), which are inherited from the further block (285) of the spatial or temporal vicinity are used as the one or more partitioning parameters for partitioning the at least one block (286), the encoder (10) is configured to signal, at a root node partitioning level in the splitting tree (13), that the one or more inherited partitioning parameters (292) are used for partitioning the at least one block (286).
94. The encoder (10) of any one of claims 89 to 93, wherein the encoder (10) is configured to access a set of inheritance partitioning parameter candidates comprising one or more partitioning parameters of the further block (285) in the spatial vicinity and/or one or more partitioning parameters of the further block (285) in the temporal vicinity, and wherein the encoder (10) is further configured to select at least one (292) of the partitioning parameters contained in the set of inheritance partitioning parameter candidates, and to use said selected one (292) of the partitioning parameters from
the set of inheritance partitioning parameter candidates as a partitioning parameter for partitioning the at least one block (286).
95. The encoder (10) of any one of claims 89 to 94, wherein the encoder (10) is configured to partition the picture (12s, 12h, 12t2) into a multitude of root blocks (284, 285 286), each of which comprising one or more own partitioning parameters (291 ,
293, 294), and to signal an indication in the data stream (14) for indicating whether all root blocks of the multitude of root blocks (284, 285, 286) are partitioned by using their one or more own partitioning parameters (291 , 293, 294), or whether at least one root block (286) of the multitude of root blocks (284, 285, 286) is partitioned using one or more inherited partitioning parameters (292) being inherited from a further root block (285) in the temporal or spatial vicinity of the at least one root block (286), wherein the one or more inherited partitioning parameters (292) are used for partitioning the at least one root block (286).
96. The encoder (10) of any one of claims 89 to 95, wherein the encoder (10) is configured to partition the picture (12s, 12ti, 12t2) into a multitude of root blocks (284,
285, 286), and to signal an indication in the data stream (14) for indicating a number of root blocks which are partitioned using one or more inherited partitioning parameters (292) being inherited from a further root block (285) in the temporal or spatial vicinity. 97. Block-based decoder (110) for decoding a picture (12s, 12ti, 12t2) from a data stream (14) using at least one of a spatial intra-prediction and a temporal interprediction coding, wherein the decoder (1 10) is configured to partition the picture (12s, 12ti, 12t2) along a splitting tree (13) from a tree root block (284, 285, 286) into one or more leaf blocks using recursive multi -tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters (291 , 292, 293, 294), wherein if at least one block (286) is decoded by means of spatial intra-prediction, the decoder (110) is configured to use, based on a signaling in the data stream (14), one or more inherited partitioning parameters (292) for partitioning the at least one block (286), which inherited one or more partitioning parameters (292) are inherited from a further block (285) in the spatial vicinity of the at least one block (286), and
if the at least one block (286) is decoded by means of temporal inter- prediction, the decoder (110) is configured to use, based on a signaling in the data stream (14), one or more inherited partitioning parameters (292) for partitioning the at least one block (286), which inherited one or more partitioning parameters (292) are inherited from a further block (285) in the temporal vicinity of the at least one block (286).
98. The decoder (110) of claim 97, wherein the further block (285) in the spatial vicinity is a block that is located in the same sub-picture region as the at least one block (286).
99. The decoder (110) of claim 97 or 98, wherein the further block (285) is located directly adjacent to the at least one block (286), or wherein one or two other blocks are arranged between the at least one block (286) and the further block (285).
100. The decoder (110) of any one of claims 97 to 99, wherein the one or more partitioning parameters comprise a predetermined tree depth of the splitting tree (13) for partitioning the at least one block (286), and wherein the decoder (110) is configured to, based on the signaling in the data stream (14), inherit from the further block (285) in the spatial vicinity or from the further block (285) in the temporal vicinity a predetermined tree depth (292) of a further splitting tree that is for partitioning the further block (285) in the spatial or temporal vicinity, and to use said inherited predetermined tree depth (292) in the splitting tree (13) for partitioning the at least one block (286).
101. The decoder (110) of any one of claims 97 to 100, wherein, the decoder (110) is configured to obtain a signal from the data stream (14), at a root node partitioning level of the splitting tree (13), signaling that one or more inherited partitioning parameters (292) of the further block (285) in the spatial or temporal vicinity are used as the one or more partitioning parameters for partitioning the at least one block (286).
102. The decoder (110) of any one of claims 97 to 101 , wherein the decoder (110) is configured to access a set of inheritance partitioning parameter candidates comprising one or more partitioning parameters of the further block (285) in the spatial vicinity and/or one or more partitioning parameters of the further block (285) in the temporal vicinity, and
wherein the decoder (110) is further configured to select, based on the signaling in the data stream (14), at least one (292) of the partitioning parameters contained in the set of inheritance partitioning parameter candidates, and to use said selected one (292) of the partitioning parameters from the set of inheritance partitioning parameter candidates as a partitioning parameter for partitioning the at least one block (286).
103. The decoder (110) of any one of claims 97 to 102, wherein the decoder (1 10) is configured to partition the picture (12s, 12ti , 12t2) into a multitude of root blocks (284, 285, 286), each of which comprising one or more own partitioning parameters (291 , 293, 294), and to obtain an indication from the data stream (14) indicating whether all root blocks of the multitude of root blocks (284, 285, 286) are partitioned by using their one or more own partitioning parameters (291 , 293, 294), or whether at least one root block (286) of the multitude of root blocks (284, 285, 286) is partitioned using one or more inherited partitioning parameters (292) being inherited from a further root block (285) in the temporal or spatial vicinity of the at least one root block (286), wherein the one or more inherited partitioning parameters (292) are used for partitioning the at least one root block (286).
104. The decoder (110) of any one of claims 97 to 103, wherein the decoder (110) is configured to partition the picture (12s, 12ti, 12t2) into a multitude of root blocks (284, 285, 286), and to obtain an indication from the data stream (14) which indicates a number of root blocks which are partitioned using one or more inherited partitioning parameters (292) being inherited from a further root block (285) in the temporal or spatial vicinity.
105. A method for block-based encoding a picture (12s, 12ti, 12t2) into a data stream (14) using at least one of a spatial intra-prediction and a temporal inter-prediction coding, wherein the method comprises the steps of: partitioning the picture (12s, 12ti, 12t2) along a splitting tree (13) from a tree root block (284, 285, 286) into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters (291 , 292, 293, 294), wherein if at least one block (286) is encoded by means of spatial intra-prediction, the method comprises further steps of inheriting one or more partitioning parameters
from a further block (285) in the spatial vicinity of the at least one block (286), and using said inherited one or more partitioning parameters (292) for partitioning said at least one block (286) in the recursive multi-tree partitioning, and if the at least one block (286) is encoded by means of temporal inter- prediction, the method comprises further steps of inheriting one or more partitioning parameters from a further block (285) in the temporal vicinity of the at least one block (286), and using said inherited partitioning parameters (292) for partitioning the at least one block (286) in the recursive multi-tree partitioning, and signaling in the data stream (14) that one or more inherited partitioning parameters (292) are used for partitioning the at least one block (286).
106. A method for block-based decoding a picture (12s, 12ti, 12t2) from a data stream (14) using at least one of a spatial intra-prediction and a temporal inter-prediction coding, wherein the method comprises the steps of: partitioning the picture (12s, 12ti, 12t2) along a splitting tree (13) from a tree root block (284, 285, 286) into one or more leaf blocks using recursive multi-tree partitioning, wherein the recursive multi-tree partitioning is based on one or more partitioning parameters (291 , 292, 293, 294), wherein if at least one block (286) is decoded by means of spatial intra-prediction, the method comprises a further step of using, based on a signaling in the data stream (14), one or more inherited partitioning parameters (292) for partitioning the at least one block (286), which inherited partitioning parameters (292) are inherited from a further block (285) in the spatial vicinity of the at least one block (286), and if the at least one block (286) is decoded by means of temporal interprediction, the method comprises a further step of using, based on a signaling in the data stream (14), one or more inherited partitioning parameters (292) for partitioning the at least one block (286), which inherited partitioning parameters (292) are inherited from a further block (285) in the temporal vicinity of the at least one block (286).
107. A computer readable digital storage medium having stored thereon a computer program having a program code for performing, when running on a computer, a method according to claim 105 or 106.
WO 2019/185815 PCT/EP2019/057891
108. A data stream generated according to the method of claim 105.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP18165249 | 2018-03-29 | ||
| EP18165249.6 | 2018-03-29 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2019185815A1 true WO2019185815A1 (en) | 2019-10-03 |
Family
ID=61868359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/EP2019/057891 Ceased WO2019185815A1 (en) | 2018-03-29 | 2019-03-28 | Partitioning concepts for block-based picture coding |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2019185815A1 (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020044255A1 (en) * | 2018-08-28 | 2020-03-05 | Beijing Bytedance Network Technology Co., Ltd. | Context coding in partitioning with extended quadtree |
| US20210360243A1 (en) * | 2019-02-03 | 2021-11-18 | Beijing Bytedance Network Technology Co., Ltd. | Condition-dependent unsymmetrical quad-tree partitioning |
| WO2021253373A1 (en) * | 2020-06-19 | 2021-12-23 | Alibaba Group Holding Limited | Probabilistic geometric partitioning in video coding |
| US20220046239A1 (en) * | 2018-12-18 | 2022-02-10 | Mediatek Inc. | Method and Apparatus of Encoding or Decoding Video Blocks with Constraints during Block Partitioning |
| US11336890B2 (en) | 2018-07-09 | 2022-05-17 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | Encoder and decoder, encoding method and decoding method for versatile spatial partitioning of coded pictures |
| WO2022174701A1 (en) * | 2021-02-21 | 2022-08-25 | 腾讯科技(深圳)有限公司 | Video coding method and apparatus, video decoding method and apparatus, and computer-readable medium and electronic device |
| US20220345704A1 (en) * | 2020-06-26 | 2022-10-27 | Google Llc | Extended Transform Partitions for Video Compression |
| CN116881422A (en) * | 2023-07-24 | 2023-10-13 | 浙江大学 | Knowledge visual question-answering method and system generated by triple asymmetry and principle |
| US12321870B2 (en) | 2021-12-01 | 2025-06-03 | Nokia Technologies Oy | Apparatus method and computer program product for probability model overfitting |
| CN116881422B (en) * | 2023-07-24 | 2026-01-23 | 浙江大学 | Knowledge visual question-answering method and system generated by triple asymmetry and principle |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180063528A1 (en) * | 2014-12-10 | 2018-03-01 | Mediatek Singapore Pte. Ltd. | Method of Video Coding Using Binary Tree Block Partitioning |
| US20180109812A1 (en) * | 2016-10-14 | 2018-04-19 | Media Tek Inc. | Block partitioning using tree structures |
| US20180199072A1 (en) * | 2017-01-06 | 2018-07-12 | Qualcomm Incorporated | Multi-type-tree framework for video coding |
-
2019
- 2019-03-28 WO PCT/EP2019/057891 patent/WO2019185815A1/en not_active Ceased
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180063528A1 (en) * | 2014-12-10 | 2018-03-01 | Mediatek Singapore Pte. Ltd. | Method of Video Coding Using Binary Tree Block Partitioning |
| US20180109812A1 (en) * | 2016-10-14 | 2018-04-19 | Media Tek Inc. | Block partitioning using tree structures |
| US20180199072A1 (en) * | 2017-01-06 | 2018-07-12 | Qualcomm Incorporated | Multi-type-tree framework for video coding |
Non-Patent Citations (7)
| Title |
|---|
| "Advanced video coding for audiovisual services. ITU-T Recommendation H.264", 2003 |
| "High efficiency video coding. ITU-T Recommendation H.265", 2013 |
| "HIGH EFFICIENCY VIDEO CODING. ITU-T RECOMMENDATION H.265", 2014 |
| F. LE L6ANNEC; T. POIRIER; F. URBAN: "Asymmetric Coding Units in QTBT, JVET", JVET-D0064, October 2016 (2016-10-01) |
| J. CHEN; E. ALSHINA; G.J. SULLIVAN; J.-R. OHM; J. BOYCE: "Algorithm Description of Joint Exploration Test Model 6 (JEM 6), JVET, doc.", JVET-F1001, April 2017 (2017-04-01) |
| LI X ET AL: "Multi-Type-Tree", 4. JVET MEETING; 15-10-2016 - 21-10-2016; CHENGDU; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/,, no. JVET-D0117-v3, 20 October 2016 (2016-10-20), XP030150367 * |
| X. LI; H.-C. CHUANG; J. CHEN; M. KARCZEWICZ; L. ZHANG; X. ZHAO; A. SAID: "Multi-Type-Tree, JVET", JVET-D0117, October 2016 (2016-10-01) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11336890B2 (en) | 2018-07-09 | 2022-05-17 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | Encoder and decoder, encoding method and decoding method for versatile spatial partitioning of coded pictures |
| US12120298B2 (en) | 2018-07-09 | 2024-10-15 | Fraunhofer-Gesellschaft zur Förderung derangewandten Forschung e.V. | Encoder and decoder, encoding method and decoding method for versatile spatial partitioning of coded pictures |
| US11770528B2 (en) | 2018-07-09 | 2023-09-26 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Encoder and decoder, encoding method and decoding method for versatile spatial partitioning of coded pictures |
| US11632575B2 (en) | 2018-08-28 | 2023-04-18 | Beijing Bytedance Network Technology Co., Ltd. | Binarization of partitioning related to extended quadtree |
| WO2020044253A1 (en) * | 2018-08-28 | 2020-03-05 | Beijing Bytedance Network Technology Co., Ltd. | Redundancy removal of partitioning methods |
| WO2020044255A1 (en) * | 2018-08-28 | 2020-03-05 | Beijing Bytedance Network Technology Co., Ltd. | Context coding in partitioning with extended quadtree |
| WO2020044254A1 (en) * | 2018-08-28 | 2020-03-05 | Beijing Bytedance Network Technology Co., Ltd. | Binarization of partitioning related to extended quadtree |
| US20220046239A1 (en) * | 2018-12-18 | 2022-02-10 | Mediatek Inc. | Method and Apparatus of Encoding or Decoding Video Blocks with Constraints during Block Partitioning |
| US11589050B2 (en) * | 2018-12-18 | 2023-02-21 | Hfi Innovation Inc. | Method and apparatus of encoding or decoding video blocks with constraints during block partitioning |
| US11870991B2 (en) | 2018-12-18 | 2024-01-09 | Hfi Innovation Inc. | Method and apparatus of encoding or decoding video blocks with constraints during block partitioning |
| US20210360243A1 (en) * | 2019-02-03 | 2021-11-18 | Beijing Bytedance Network Technology Co., Ltd. | Condition-dependent unsymmetrical quad-tree partitioning |
| US12238346B2 (en) * | 2019-02-03 | 2025-02-25 | Beijing Bytedance Network Technology Co., Ltd. | Condition-dependent unsymmetrical quad-tree partitioning |
| US12137257B2 (en) | 2019-02-03 | 2024-11-05 | Beijing Bytedance Network Technology Co., Ltd | Signaling for video block partition mode |
| US12108090B2 (en) | 2019-02-03 | 2024-10-01 | Beijing Bytedance Network Technology Co., Ltd | Unsymmetrical quad-tree partitioning |
| WO2021253373A1 (en) * | 2020-06-19 | 2021-12-23 | Alibaba Group Holding Limited | Probabilistic geometric partitioning in video coding |
| US20220345704A1 (en) * | 2020-06-26 | 2022-10-27 | Google Llc | Extended Transform Partitions for Video Compression |
| WO2022174701A1 (en) * | 2021-02-21 | 2022-08-25 | 腾讯科技(深圳)有限公司 | Video coding method and apparatus, video decoding method and apparatus, and computer-readable medium and electronic device |
| US12244838B2 (en) | 2021-02-21 | 2025-03-04 | Tencent Technology (Shenzhen) Company Limited | Video decoding method, apparatus, and computer-readable medium with sub-block transform partitioning |
| US12321870B2 (en) | 2021-12-01 | 2025-06-03 | Nokia Technologies Oy | Apparatus method and computer program product for probability model overfitting |
| CN116881422A (en) * | 2023-07-24 | 2023-10-13 | 浙江大学 | Knowledge visual question-answering method and system generated by triple asymmetry and principle |
| CN116881422B (en) * | 2023-07-24 | 2026-01-23 | 浙江大学 | Knowledge visual question-answering method and system generated by triple asymmetry and principle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019185815A1 (en) | Partitioning concepts for block-based picture coding | |
| KR101382101B1 (en) | Methods and apparatus for reduced resolution partitioning | |
| JP7581470B2 (en) | Intra-prediction-based video signal processing method and apparatus | |
| CN102939754B (en) | Sample Region Merging Technique | |
| TWI830012B (en) | Apparatus and method of coding of pictures | |
| CN117221590A (en) | Image encoding/decoding method and image data transmission method | |
| CN102939618A (en) | Inheritance Techniques in Multivariate Tree Subdivision of Sample Arrays | |
| US11843774B2 (en) | Apparatus and method for encoding and decoding a picture using picture boundary handling | |
| TWI731361B (en) | Shape dependent intra coding | |
| CN106412606A (en) | Plane-crossing predication | |
| US20240314313A1 (en) | Coding Concepts for a Transformed Representation of a Sample Block | |
| CN105915918B (en) | Method and device for cross-plane prediction | |
| HK40067190B (en) | Decoder, encoder and methods comprising a coding for intra subpartitions | |
| HK40067190A (en) | Decoder, encoder and methods comprising a coding for intra subpartitions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19712794 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 19712794 Country of ref document: EP Kind code of ref document: A1 |