WO2026005571A1 - Image encoding/decoding method, device, and recording medium storing bitstream - Google Patents

Abstract

Provided are an image encoding/decoding method, a device, a recording medium storing a bitstream, and a transmission method. The image decoding method may comprise an image decoding method comprising the steps of: determining, with respect to bypass decoding of a bin of a syntax element parsed for a current data unit, whether a current appearance probability for a predetermined value of the bin is initialized; in response to determining that the current appearance probability is initialized, obtaining a reference appearance probability for the predetermined value of the bin from a reference data unit decoded before the current data unit; and initializing the current appearance probability on the basis of the reference appearance probability.

Description

Video encoding/decoding method, device, and recording medium storing bitstream

The present disclosure relates to a video encoding/decoding method, a device, and a recording medium storing a bitstream. Specifically, the present disclosure relates to a video encoding/decoding method, a device, and a recording medium storing a bitstream that apply syntax element probability initialization for bypass coding.

Recently, the demand for high-resolution, high-quality images, such as UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired or wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image encoding/decoding technologies for higher-resolution and higher-quality images are required.

In image encoding/decoding technology, syntax elements can be efficiently encoded based on the occurrence probability of symbols through entropy coding. In entropy coding, a series of syntax elements are binarized into a bin string consisting of bins. Then, for entropy coding of each bin, either context-based adaptive binary arithmetic coding (CABAC) or bypass coding is applied. Bypass coding is an encoding method that fixes the occurrence probability of a bin to 0.5 and outputs a bitstream identical to the bin string. By improving the efficiency of bypass coding, the performance of the overall entropy encoder can be improved.

The present disclosure aims to provide a video encoding/decoding method and device with improved encoding/decoding efficiency.

In addition, the present disclosure aims to provide a recording medium storing a bitstream generated by an image decoding method or device according to the present disclosure.

A video decoding method according to one embodiment of the present disclosure may be characterized by including, in bypass decoding of a bin of a syntax element parsed for a current data unit, a step of determining whether a current appearance probability for a predetermined value of the bin is initialized; a step of obtaining a reference appearance probability for a predetermined value of the bin from a reference data unit decoded before the current data unit in response to the current appearance probability being determined to be initialized; and a step of initializing the current appearance probability based on the reference appearance probability.

In the above image decoding method, the current data unit may be characterized as being a tile or a slice.

In the above image decoding method, the current data unit may be characterized as being a VPDU (Virtual pipeline data unit).

In the above image decoding method, the reference data unit may be characterized as being a data unit that was decoded immediately before according to the decoding order of the current data unit.

In the above image decoding method, the reference data unit may be characterized in that, when the current data unit is the first data unit of the current picture, it is the last data unit of the reference picture decoded before the current picture.

In the above image decoding method, it may further include a step of obtaining information on whether to use empty probability initialization from a high level syntax, and whether the current appearance probability is initialized is determined based on the information on whether to use empty probability initialization.

In the above image decoding method, the method may further include a step of obtaining information on a bin probability initialization method from a high level syntax, and a method of obtaining a reference appearance probability for a predetermined value of the bin is determined according to the bin probability initialization method indicated by the information on the bin probability initialization method.

In the above image decoding method, it may be characterized in that it is determined whether the current appearance probability is initialized based on a reference picture referenced by the current picture of the current data unit.

In the above image decoding method, it may be characterized in that whether the current appearance probability is initialized is determined based on a temporal difference between the current picture and the reference picture, and the temporal difference is a POC (Picture Order Counter) of the current picture and the reference picture, and the POC is a number assigned to a picture according to a display order.

In the above image decoding method, it may be characterized in that whether the current appearance probability is initialized is determined based on the difference in quantization parameters of the current picture and the reference picture.

In the above image decoding method, it may be characterized in that whether the current appearance probability is initialized is determined based on whether the quantization parameter of the reference picture referenced by the current picture is greater than a predetermined threshold value.

In the above image decoding method, it may be characterized in that whether the current appearance probability is initialized is determined based on whether the GOP (Group of Picture) including the current picture of the current data unit is an open GOP or a closed GOP.

A video encoding method according to one embodiment of the present disclosure may be characterized by including, in bypass coding of a bin of a syntax element parsed for a current data unit, a step of determining whether a current appearance probability for a predetermined value of the bin is initialized; a step of obtaining a reference appearance probability for a predetermined value of the bin from a reference data unit decoded before the current data unit in response to the current appearance probability being determined to be initialized; and a step of initializing the current appearance probability based on the reference appearance probability.

A non-transitory computer-readable recording medium according to one embodiment of the present disclosure can store a bitstream generated by an image encoding method according to various embodiments of the present disclosure.

A bitstream transmission method according to one embodiment of the present disclosure includes a step of transmitting the bitstream, and can transmit a bitstream generated by an image encoding method according to various embodiments of the present disclosure.

The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

According to the present disclosure, a video encoding/decoding method and device with improved encoding/decoding efficiency can be provided.

Additionally, according to the present disclosure, a method for initializing syntax element probabilities for bypass coding can be provided.

Additionally, according to the present disclosure, the efficiency of entropy coding can be improved by bypass coding using adaptive probability.

Additionally, according to the present disclosure, the efficiency of entropy coding can be improved by initializing bypass coding for each specific data unit.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

FIG. 1 is a block diagram showing a configuration according to one embodiment of an encoding device to which the present disclosure is applied.

FIG. 2 is a block diagram showing a configuration according to one embodiment of a decryption device to which the present disclosure is applied.

FIG. 3 is a diagram schematically illustrating a video coding system to which the present disclosure can be applied.

Figure 4 illustrates the structure of a context-based adaptive binary arithmetic coding.

FIG. 5 is a diagram illustrating a method for performing probability initialization of bins for bypass coding when one picture is divided into multiple slice or tile structures.

Figure 6 describes a method for setting a probability initialization value for bypass coding when encoding and decoding are performed based on a VPDU (Virtual pipeline data unit).

Figure 7 shows a reference picture structure for explaining the probability initialization of bins according to the reference picture structure.

FIGS. 8 and 9 illustrate an example of inter-screen prediction of a current block to explain a method of setting a probability initialization value based on reference block information.

FIG. 10 is a flowchart illustrating an image decoding method according to an embodiment of the present disclosure.

FIG. 11 is a drawing exemplifying a content streaming system to which an embodiment according to the present disclosure can be applied.

A video decoding method according to one embodiment of the present disclosure may be characterized by including, in bypass decoding of a bin of a syntax element parsed for a current data unit, a step of determining whether a current appearance probability for a predetermined value of the bin is initialized; a step of obtaining a reference appearance probability for a predetermined value of the bin from a reference data unit decoded before the current data unit in response to the current appearance probability being determined to be initialized; and a step of initializing the current appearance probability based on the reference appearance probability.

The present disclosure is susceptible to various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure. In the drawings, similar reference numerals designate the same or similar functions throughout. The shapes and sizes of elements in the drawings may be provided by way of example only for clarity. The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments, while different from one another, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present disclosure. Furthermore, it should be understood that the positions or arrangements of individual components within each disclosed embodiment may be modified without departing from the spirit and scope of the embodiment. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope equivalents to which such claims are entitled.

While terms such as "first" and "second" may be used herein to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and/or" includes a combination of multiple related items described herein or any of multiple related items described herein.

The components shown in the embodiments of the present disclosure are independently depicted to represent different characteristic functions, and do not imply that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

The terminology used in this disclosure is merely used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, some components of the present disclosure may not be essential components that perform essential functions in the present disclosure and may be optional components merely for performance enhancement. The present disclosure may be implemented by including only components essential to implementing the essence of the present disclosure, excluding components used solely for performance enhancement, and a structure including only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present disclosure.

In an embodiment, the term "at least one" may mean one of a number greater than or equal to 1, such as 1, 2, 3, and 4. In an embodiment, the term "a plurality of" may mean one of a number greater than or equal to 2, such as 2, 3, and 4.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In describing the embodiments of this specification, if a detailed description of a related known configuration or function is judged to obscure the gist of this specification, the detailed description will be omitted. The same reference numerals will be used for identical components in the drawings, and duplicate descriptions of the same components will be omitted.

Glossary of Terms

Hereinafter, “video” may mean a single picture constituting a video, or may refer to the video itself. For example, “encoding and/or decoding of a video” may mean “encoding and/or decoding of a video,” or may mean “encoding and/or decoding of one of the videos constituting the video.”

Hereinafter, the terms "video" and "movie" may be used interchangeably and have the same meaning. Furthermore, the target image may be an encoding target image, which is the target of encoding, and/or a decoding target image, which is the target of decoding. Furthermore, the target image may be an input image input to an encoding device, or an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

Hereinafter, the terms encoder and image encoding device may be used interchangeably and have the same meaning.

Hereinafter, the terms decoder and image decoding device may be used interchangeably and interchangeably.

Hereinafter, “image”, “picture”, “frame” and “screen” may be used with the same meaning and may be used interchangeably.

Hereinafter, the term "target block" may refer to an encoding target block, which is the target of encoding, and/or a decoding target block, which is the target of decoding. Furthermore, the target block may refer to a current block, which is the target of current encoding and/or decoding. For example, the terms "target block" and "current block" may be used interchangeably and have the same meaning.

Hereinafter, "block" and "unit" may be used with the same meaning and may be used interchangeably. In addition, "unit" may mean including a luminance component block and a corresponding chroma component block to distinguish it from a block. For example, a coding tree unit (CTU) may be composed of one luma component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated with it.

Hereinafter, the terms “sample,” “pixel,” and “pixel” may be used interchangeably and have the same meaning. Here, a sample may represent a basic unit that constitutes a block.

Hereinafter, “inter” and “between screens” may be used interchangeably and have the same meaning.

Hereinafter, “intra” and “within screen” may be used interchangeably and have the same meaning.

FIG. 1 is a block diagram showing a configuration according to one embodiment of an encoding device to which the present disclosure is applied.

The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device (100) may sequentially encode one or more images.

Referring to FIG. 1, the encoding device (100) may include an image segmentation unit (110), an intra prediction unit (120), a motion prediction unit (121), a motion compensation unit (122), a switch (115), a subtractor (113), a transformation unit (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse transformation unit (170), an adder (117), a filter unit (180), and a reference picture buffer (190).

Additionally, the encoding device (100) can generate a bitstream including encoded information through encoding an input image and output the generated bitstream. The generated bitstream can be stored in a computer-readable recording medium or can be streamed via a wired/wireless transmission medium.

The video segmentation unit (110) can segment the input video into various forms to increase the efficiency of video encoding/decoding. That is, the input video is composed of multiple pictures, and one picture can be hierarchically segmented and processed for compression efficiency, parallel processing, etc. For example, one picture can be segmented into one or more tiles or slices, which can then be segmented into multiple Coding Tree Units (CTUs). Alternatively, one picture can first be segmented into multiple sub-pictures, which are defined as groups of rectangular slices, and each sub-picture can then be segmented into the tiles/slices. Here, the sub-pictures can be utilized to support the function of partially independently encoding/decoding and transmitting the picture. Since multiple sub-pictures can each be individually restored, there is an advantage of easy editing in applications that configure multi-channel input into a single picture. In addition, tiles can be segmented horizontally to generate bricks. Here, a brick can be utilized as the basic unit of intra-picture parallel processing. In addition, one CTU can be recursively split into a quadtree (QT), and the terminal node of the split can be defined as a coding unit (CU). The CU can be split into a prediction unit (PU) and a transformation unit (TU), and prediction and splitting can be performed. Meanwhile, the CU can be utilized as a prediction unit and/or a transformation unit itself. Here, for flexible splitting, each CTU can be recursively split into a multi-type tree (MTT) as well as a quadtree (QT). Splitting of a CTU into a multi-type tree can start from the terminal node of a QT, and the MTT can be composed of a binary tree (BT) and a triple tree (TT). For example, the MTT structure can be divided into vertical binary split mode (SPLIT_BT_VER), horizontal binary split mode (SPLIT_BT_HOR), vertical ternary split mode (SPLIT_TT_VER), and horizontal ternary split mode (SPLIT_TT_HOR). In addition, the minimum block size (MinQTSize) of the quad tree of the luma block during splitting can be set to 16x16, the maximum block size (MaxBtSize) of the binary tree can be set to 128x128, and the maximum block size (MaxTtSize) of the triple tree can be set to 64x64. In addition, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the triple tree can be set to 4x4, and the maximum depth (MaxMttDepth) of the multi-type tree can be set to 4. In addition, in order to increase the encoding efficiency of the I slice, a dual tree that uses different CTU split structures for luma and chrominance components can be applied. On the other hand, in P and B slices, the luminance and chrominance CTBs (Coding Tree Blocks) within the CTU can be split into a single tree that shares the coding tree structure.

The encoding device (100) may perform encoding on the input image in intra mode and/or inter mode. Alternatively, the encoding device (100) may perform encoding on the input image in a third mode (e.g., IBC mode, Palette mode, etc.) other than the intra mode and inter mode. However, if the third mode has functional characteristics similar to the intra mode or inter mode, it may be classified as intra mode or inter mode for convenience of explanation. In the present disclosure, the third mode will be classified and described separately only when a specific description is required.

When the intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when the inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, the intra mode can mean an intra-screen prediction mode, and the inter mode can mean an inter-screen prediction mode. The encoding device (100) can generate a prediction block for an input block of an input image. In addition, after the prediction block is generated, the encoding device (100) can encode a residual block using a residual of the input block and the prediction block. The input image can be referred to as a current image that is currently a target of encoding. The input block can be referred to as a current block that is currently a target of encoding or an encoding target block.

When the prediction mode is intra mode, the intra prediction unit (120) can use samples of blocks already encoded/decoded around the current block as reference samples. The intra prediction unit (120) can perform spatial prediction on the current block using the reference samples, and can generate prediction samples for the input block through spatial prediction. Here, intra prediction can mean prediction within the screen.

As an intra prediction method, non-directional prediction modes such as DC mode and Planar mode, as well as directional prediction modes (e.g., 65 directions) can be applied. Here, the intra prediction method can be expressed as an intra prediction mode or an intra-screen prediction mode.

When the prediction mode is inter mode, the motion prediction unit (121) can search for an area that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched area. At this time, the area can be used as a search area. The reference image can be stored in the reference picture buffer (190). Here, when encoding/decoding for the reference image is processed, it can be stored in the reference picture buffer (190).

The motion compensation unit (122) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter prediction may mean inter-screen prediction or motion compensation.

The above motion prediction unit (121) and motion compensation unit (122) can generate a prediction block by applying an interpolation filter to a portion of an area within a reference image when the value of the motion vector does not have an integer value. In order to perform inter-screen prediction or motion compensation, it is possible to determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) mode, and Intra Block Copy (IBC) mode based on the encoding unit, and perform inter-screen prediction or motion compensation according to each mode.

In addition, based on the above inter-screen prediction method, the AFFINE mode of sub-PU based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and the GPM (Geometric Partitioning Mode) mode of PU based prediction can be applied. In addition, in order to improve the performance of each mode, the HMVP (History based MVP), the PAMVP (Pairwise Average MVP), the CIIP (Combined Intra/Inter Prediction), the AMVR (Adaptive Motion Vector Resolution), the BDOF (Bi-Directional Optical-Flow), the BCW (Bi-predictive with CU Weights), the LIC (Local Illumination Compensation), the TM (Template Matching), and the OBMC (Overlapped Block Motion Compensation) can be applied.

Among these, AFFINE mode is a technology that is used in both AMVP and MERGE modes and also has high encoding efficiency. In the existing video coding standard, since MC (Motion Compensation) is performed by considering only the parallel translation of the block, there was a disadvantage in that it could not properly compensate for motions that occur in reality, such as zoom in/out and rotation. To supplement this, a 4-parameter affine motion model using two control point motion vectors (CPMV) and a 6-parameter affine motion model using three control point motion vectors can be applied to inter prediction. Here, CPMV is a vector representing the affine motion model of one of the upper left, upper right, and lower left of the current block.

The subtractor (113) can generate a residual block using the difference between the input block and the predicted block. The residual block may also be referred to as a residual signal. The residual signal may refer to the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming and quantizing the difference between the original signal and the predicted signal. The residual block may be a residual signal in block units.

The transform unit (130) can perform a transform on the residual block to generate a transform coefficient and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit (130) may also skip the transform on the residual block.

Quantized levels can be generated by applying quantization to transform coefficients or residual signals. In the following embodiments, quantized levels may also be referred to as transform coefficients.

For example, a 4x4 luminance residual block generated through intra prediction can be transformed using a basis vector based on DST (Discrete Sine Transform), and the remaining residual blocks can be transformed using a basis vector based on DCT (Discrete Cosine Transform). In addition, through RQT (Residual Quad Tree) technology, the transform block is divided into a quad tree shape for one block, and after performing transformation and quantization on each transform block divided through RQT, a coded block flag (cbf) can be transmitted to increase encoding efficiency when all coefficients become 0.

Another alternative is to apply Multiple Transform Selection (MTS) technology, which selectively performs transformation using multiple transformation bases. That is, instead of dividing CUs into TUs via RQT, a Sub-block Transform (SBT) technology can perform a function similar to TU division. Specifically, SBT is applied only to inter-screen prediction blocks, and unlike RQT, it can divide the current block into ½ or ¼ blocks vertically or horizontally, and then perform transformation on only one of the blocks. For example, in a vertically divided block, the transformation can be performed on the leftmost or rightmost block, and in a horizontally divided block, the transformation can be performed on the topmost or bottommost block.

Additionally, LFNST (Low Frequency Non-Separable Transform), a secondary transform technique that further transforms the residual signal converted to the frequency domain through DCT or DST, can be applied. LFNST additionally performs a transform on the low-frequency region of 4x4 or 8x8 in the upper left, which allows the residual coefficients to be concentrated in the upper left.

The quantization unit (140) can generate a quantized level by quantizing a transform coefficient or residual signal according to a quantization parameter (QP), and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transform coefficient using a quantization matrix.

For example, a quantizer with QP values of 0 to 51 can be used. Alternatively, if the image size is larger and high encoding efficiency is required, a QP of 0 to 63 can be used. In addition, a Dependent Quantization (DQ) method that uses two quantizers instead of a single quantizer can be applied. DQ performs quantization using two quantizers (e.g., Q0 and Q1), but even without signaling information about the use of a specific quantizer, the quantizer to be used for the next transform coefficient can be selected based on the current state through a state transition model.

The entropy encoding unit (150) can generate a bitstream by performing entropy encoding according to a probability distribution on values produced by the quantization unit (140) or coding parameter values produced during the encoding process, and can output the bitstream. The entropy encoding unit (150) can perform entropy encoding on information about image samples and information for decoding the image. For example, the information for decoding the image can include syntax elements, etc.

When entropy encoding is applied, a small number of bits are allocated to symbols with a high occurrence probability, and a large number of bits are allocated to symbols with a low occurrence probability, thereby representing the symbols, whereby the size of the bit string for the symbols to be encoded can be reduced. The entropy encoding unit (150) can use an encoding method such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) for entropy encoding. For example, the entropy encoding unit (150) can perform entropy encoding using a Variable Length Coding/Code (VLC) table. In addition, the entropy encoding unit (150) may perform arithmetic encoding using the binarization method, probability model, and context model derived from the binarization method of the target symbol and the probability model of the target symbol/bin.

In this regard, when applying CABAC, the table probability update method can be changed to a simple formula-based table update method to reduce the size of the probability table stored in the decryption device. Furthermore, two different probability models can be used to obtain more accurate symbol probability values.

The entropy encoding unit (150) can change a two-dimensional block form coefficient into a one-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

Coding parameters may include not only information (flags, indexes, etc.) encoded in an encoding device (100) and signaled to a decoding device (200), such as syntax elements, but also information derived during an encoding process or a decoding process, and may mean information necessary when encoding or decoding an image.

Here, signaling a flag or index may mean that the encoder entropy encodes the flag or index and includes it in the bitstream, and that the decoder entropy decodes the flag or index from the bitstream.

The encoded current image can be used as a reference image for other images to be processed later. Accordingly, the encoding device (100) can reconstruct or decode the encoded current image again and store the reconstructed or decoded image as a reference image in the reference picture buffer (190).

The quantized level can be dequantized in the dequantization unit (160) and inversely transformed in the inverse transformation unit (170). The dequantized and/or inversely transformed coefficients can be combined with a prediction block through an adder (117), and a reconstructed block can be generated by combining the dequantized and/or inversely transformed coefficients and the prediction block. Here, the dequantized and/or inversely transformed coefficients refer to coefficients on which at least one of dequantization and inverse transformation has been performed, and may refer to a reconstructed residual block. The dequantization unit (160) and the inverse transformation unit (170) can be performed in the reverse process of the quantization unit (140) and the transformation unit (130).

The restoration block may pass through a filter unit (180). The filter unit (180) may apply a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), a bilateral filter (BIF), a Luma Mapping with Chroma Scaling (LMCS), etc. as a filtering technique, in whole or in part, to the restoration sample, restoration block, or restoration image. The filter unit (180) may also be referred to as an in-loop filter. In this case, the in-loop filter is also used as a name excluding LMCS.

A deblocking filter can remove block distortion that occurs at the boundaries between blocks. Whether to apply a deblocking filter to the current block can be determined based on the samples contained in several columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering strength.

Sample adaptive offset can be used to compensate for encoding errors by adding an appropriate offset value to sample values. Sample adaptive offset can compensate for the offset from the original image on a sample-by-sample basis for deblocked images. This can be done by dividing the samples contained in the image into a fixed number of regions, determining the regions to be offset, and applying the offset to those regions. Alternatively, the offset can be applied by considering the edge information of each sample.

Bilateral filter (BIF) can also compensate for the offset from the original image on a sample-by-sample basis for the deblocked image.

An adaptive loop filter can perform filtering based on a comparison between a reconstructed image and the original image. By dividing the samples contained in the image into predetermined groups and determining the filter to be applied to each group, filtering can be performed differentially for each group. Information regarding whether to apply an adaptive loop filter can be signaled for each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter applied to each block can vary.

In LMCS (Luma Mapping with Chroma Scaling), luma mapping (LM) refers to remapping luminance values through a piece-wise linear model, and chroma scaling (CS) refers to a technique that scales the residual values of chrominance components according to the average luminance value of the prediction signal. In particular, LMCS can be utilized as an HDR correction technique that reflects the characteristics of HDR (High Dynamic Range) images.

The restored block or restored image that has passed through the filter unit (180) may be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be a part of the reference image. In other words, the reference image may be a restored image composed of restored blocks that have passed through the filter unit (180). The stored reference image may be used for inter-screen prediction or motion compensation thereafter.

FIG. 2 is a block diagram showing a configuration according to one embodiment of a decryption device to which the present disclosure is applied.

The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (201), a switch (203), a filter unit (260), and a reference picture buffer (270).

The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium, or a bitstream streamed through a wired/wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. In addition, the decoding device (200) can generate a restored image or a decoded image through decoding, and can output the restored image or the decoded image.

If the prediction mode used for decryption is intra mode, the switch (203) can be switched to intra. If the prediction mode used for decryption is inter mode, the switch (203) can be switched to inter.

The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as a current block.

The entropy decoding unit (210) can generate symbols by performing entropy decoding according to a probability distribution for the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the reverse process of the entropy encoding method described above.

The entropy decoding unit (210) can change a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a transform coefficient scanning method to decode a transform coefficient level (quantized level).

The quantized level can be inversely quantized in the inverse quantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of performing inverse quantization and/or inverse transformation. At this time, the inverse quantization unit (220) can apply a quantization matrix to the quantized level. The inverse quantization unit (220) and inverse transformation unit (230) applied to the decoding device can apply the same technology as the inverse quantization unit (160) and inverse transformation unit (170) applied to the encoding device described above.

When intra mode is used, the intra prediction unit (240) can generate a predicted block by performing spatial prediction on the current block using sample values of already decoded blocks surrounding the block to be decoded. The intra prediction unit (240) applied to the decoding device can apply the same technology as the intra prediction unit (120) applied to the encoding device described above.

When the inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer (270) on the current block. The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. In order to perform motion compensation, it is possible to determine whether the motion compensation method of the prediction unit included in the corresponding encoding unit is skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and motion compensation can be performed according to each mode. The motion compensation unit (250) applied to the decoding device can apply the same technology as the motion compensation unit (122) applied to the encoding device described above.

The adder (201) can add the restored residual block and the predicted block to generate a restored block. The filter unit (260) can apply at least one of an Inverse-LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the restored block or restored image. The filter unit (260) applied to the decoding device can apply the same filtering technology as that applied to the filter unit (180) applied to the encoding device described above.

The filter unit (260) can output a restored image. The restored block or restored image can be stored in the reference picture buffer (270) and used for inter prediction. The restored block that has passed through the filter unit (260) can be a part of the reference image. In other words, the reference image can be a restored image composed of restored blocks that have passed through the filter unit (260). The stored reference image can be used for inter-screen prediction or motion compensation thereafter.

FIG. 3 is a diagram schematically illustrating a video coding system to which the present disclosure can be applied.

A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and/or image information or data to the decoding device (20) in the form of a file or streaming through a digital storage medium or a network.

An encoding device (10) according to one embodiment may include a video source generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a reception unit (21), a decoding unit (22), and a rendering unit (23). The encoding unit (12) may be referred to as a video/image encoding unit, and the decoding unit (22) may be referred to as a video/image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The reception unit (21) may be included in the decoding unit (22). The rendering unit (23) may include a display unit, and the display unit may be configured as a separate device or an external component.

The video source generation unit (11) can obtain video/images through a process of capturing, synthesizing, or generating video/images. The video source generation unit (11) can include a video/image capture device and/or a video/image generation device. The video/image capture device can include, for example, one or more cameras, a video/image archive including previously captured video/images, etc. The video/image generation device can include, for example, a computer, a tablet, a smartphone, etc., and can (electronically) generate video/images. For example, a virtual video/image can be generated through a computer, etc., in which case the video/image capture process can be replaced with a process of generating related data.

The encoding unit (12) can encode the input video/image. The encoding unit (12) can perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit (12) can output encoded data (encoded video/image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can also be configured in the same manner as the encoding device (100) of FIG. 1 described above.

The transmission unit (13) can transmit encoded video/image information or data output in the form of a bitstream to the reception unit (21) of the decoding device (20) via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) can include an element for generating a media file through a predetermined file format and can include an element for transmission via a broadcasting/communication network. The reception unit (21) can extract/receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

The decoding unit (22) can decode video/image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can also be configured in the same manner as the decoding device (200) of FIG. 2 described above.

The rendering unit (23) can render the decrypted video/image. The rendered video/image can be displayed through the display unit.

Hereinafter, with reference to FIGS. 4 to 10, a method for initializing bin probability for bins to which bypass coding is applied according to an embodiment of the present disclosure will be specifically described.

Figure 4 illustrates the structure of a context-based adaptive binary arithmetic coding (400).

A context-based adaptive binary arithmetic encoder (400) is composed of syntax element binarization (402), a regular coding mode (404) based on a context model and probability update, and a bypass coding mode (406). In the syntax element binarization (402) step, syntax elements generated during an image encoding process are converted into a bin string, which is an array of bins consisting of only 0s and 1s, through a binarization process. Then, a bitstream is generated by applying the regular coding mode (404) or the bypass coding mode (406) to each bin of the bin string. In the regular coding mode (404), the bin string is encoded according to the probability of the bin according to context modeling. Then, in the bypass coding mode (406), the binarized bin string is output as a bitstream. That is, bypass coding is a simple coding method applied to specific syntax elements to reduce the complexity of encoding and decoding.

Bypass coding is performed without a context model. In one embodiment, the probability of the current bin can be fixed to 0.5 (the probability of 0 and the probability of 1 are each set to 0.5). In this case, the bin string can be output as a bitstream. Bypass coding can be applied when it is difficult to improve encoding efficiency or when low complexity is required even when context-based adaptive binary arithmetic coding is applied. However, since bypass coding applies a fixed probability (0.5) regardless of the data unit, encoding efficiency may be limited.

Therefore, by setting the probability of the bins to which bypass coding is applied differently depending on the data unit, the efficiency of bypass coding can be improved. In this case, when determining the probability of the bins to which bypass coding is applied by referencing the data unit most closely related to the current data unit among the data units encoded or decoded prior to the current data, the efficiency of bypass coding is likely to be improved.

In particular, by variably controlling whether to apply the syntax element probability initialization method of bypass coding at a high level, the complexity of the encoder/decoder for ultra-low latency video can be reduced. For example, for ultra-low latency video, the bypass coding engine can be applied to all syntax elements. Alternatively, information on whether to use empty probability initialization and information on the empty probability initialization method can be determined for at least one syntax element.

Therefore, in the present disclosure, a method for determining whether to initialize or change the probability of a bin to which bypass coding is applied to a probability other than a fixed probability is described. In addition, when the initialization or change of the probability of a bin is determined, a method for determining the probability of a bin used for bypass coding for current data is proposed. In the present disclosure, the value of a bin is 0 or 1, and the probability of a bin represents the probability of occurrence of the bin having a predetermined value (0 or 1). In addition, in the present disclosure, initializing the probability of a bin means changing the probability of the bin to a probability other than a fixed default probability, and for a data unit to which the bin probability initialization is applied, the initialized (or changed) probability is applied to bypass coding of the bin. In addition, the current occurrence probability represents the probability of occurrence of a predetermined value of a bin applied to the current data unit, and the reference occurrence probability (or probability initialization value) represents the probability of occurrence of a predetermined value of a bin applied to a reference (previous) data unit.

In one embodiment, information indicating whether empty probability initialization is used for bypass coding may be parsed. At the decoding stage, based on the information indicating whether empty probability initialization is used, it may be determined whether empty probability initialization is applied to bypass coding.

Additionally, information on the empty probability initialization method, which indicates the empty probability initialization method applied to the current data unit among multiple empty probability initialization methods, can be parsed. At the decoding stage, the empty probability initialization method used for bypass coding can be determined based on the empty probability initialization method information.

In one embodiment, the information on whether to use empty probability initialization and the information on how to initialize empty probability may be implemented as separate syntax elements. Alternatively, the information on whether to use empty probability initialization and the information on how to initialize empty probability may be implemented as a single syntax element.

A syntax element that implements information on whether to use empty probability initialization and information on how to initialize empty probability can be signaled in at least one high-level syntax. For example, the syntax element can be transmitted in the form of a flag or an index in at least one of a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a picture header (PH), and a slice header (SH). According to an embodiment, if the information on whether to use empty probability initialization and the empty probability initialization method are implemented as separate syntax elements, they can be included in high-level syntaxes of different levels.

According to one embodiment, the information on whether to use bin probability initialization and the information on the bin probability initialization method may be applied to one or more bins among bins to which bypass coding is applied, in which probability initialization is allowed. In addition, for the bins to which probability initialization is allowed, in order to determine whether to perform probability initialization, one or more syntax elements corresponding to the information on whether to use bin probability initialization may be defined for each of the bins. In addition, in order to determine the bin probability initialization method for the bins on which probability initialization is performed, one or more syntax elements corresponding to the information on the bin probability initialization method may be defined for each of the bins. The syntax elements may be applied to one or more bins to which probability initialization is allowed, according to an embodiment.

According to one embodiment, information on whether to use empty probability initialization and information on the empty probability initialization method may be transmitted on a picture-by-picture basis. If a current picture or a reference picture is divided into specific parallelization units (e.g., slice units, tile units, or VPDU (virtual pipeline data unit) units), information on whether to use empty probability initialization and information on the empty probability initialization method may be transmitted for each parallelization unit. In addition, information on whether to use empty probability initialization and information on the empty probability initialization method may be applied on a video sequence basis.

In one embodiment, bypass coding may be performed on two or more bins among the bins constituting a single syntax element. In this case, whether to use bin probability initialization and the probability initialization method may be determined collectively for the bins of the syntax element to which bypass coding is applied. However, the probability initialization values for each bin may be determined independently.

Alternatively, for each bin of a syntax element to which bypass coding is applied, whether to use bin probability initialization and the probability initialization method can be determined. In addition, for each bin of a syntax element to which bypass coding is applied, the probability initialization value can be determined independently.

In the present disclosure, when the partition structure of a picture is partitioned into slices or tiles, it is described how a probability initialization value for bypass coding of a bin of a specific syntax element of a current slice or current tile is set.

FIG. 5 is a diagram illustrating a method for performing probability initialization of bins for bypass coding when one picture is divided into multiple slice or tile structures.

The current picture (500) and the previous picture (510) of FIG. 5 are illustrated as being divided into, for example, nine data units (slices or tiles). However, depending on the embodiment, the picture to which the probability initialization decision method for bypass coding of bins is applied may be divided into a number of data units other than nine. In FIG. 5, a small square block within a picture represents a coding tree unit (CTU) block. And a thick straight line within a picture represents the boundary of a slice or tile.

According to one embodiment, a probability initialization value (current occurrence probability of a specific value) for bypass coding of a bin of a syntax element of a current slice or current tile may be set based on the reference occurrence probability of a specific value of a bin of a syntax element of a previous slice unit or a previous tile unit. Hereinafter, the embodiment is described assuming that bypass coding is applied to bin A of syntax element A and the specific value is 0.

For example, if nine data units of a picture in FIG. 5 are slices, the reference appearance probability that bin A will have a value of 0 in the first slice (slice1) (502) is determined. Then, based on the reference appearance probability for bin A in the first slice (slice1) (502), the initial probability value (current appearance probability) for bin A in the second slice (slice2) (504) can be determined.

Alternatively, if the nine data units of the picture in FIG. 5 are tiles, the reference appearance probability that the blank A in the first tile (tile1) (502) will have a value of 0 is determined. Then, based on the reference appearance probability for the blank A in the first tile (tile1) (502), the initial probability value (current appearance probability) for the blank A in the second tile (tile2) (504) can be determined.

According to one embodiment, the probability initialization value (current appearance probability) for bypass coding of the bin of the first slice or the first tile of the current picture can be determined based on the reference appearance probability of the bin of the last slice or the last tile of the previous picture.

For example, if nine data units of a picture in FIG. 5 are slices, the reference appearance probability that bin A will have a value of 0 in the last slice (slice9) (512) of the previous picture (510) is determined. Then, according to the reference appearance probability of bin A, the initial probability value (current appearance probability) applied to bin A of the first slice (slice1) (502) of the current picture (500) can be determined.

Alternatively, if the nine data units of the picture in FIG. 5 are tiles, the reference appearance probability that the blank A will have a value of 0 in the last tile (tile9) (512) of the previous picture (510) is determined. Then, according to the reference appearance probability of the blank A, the initial probability value (current appearance probability) applied to the blank A of the first tile (tile1) (502) of the current picture (500) can be determined.

In the above embodiment, it was described that one previous data unit was referenced to determine the reference appearance probability. However, according to one embodiment, two or more previous data units may be referenced to determine the reference appearance probability. For example, in FIG. 5, when nine data units of a picture are tiles, the reference appearance probability may be determined from the first tile (tile1) (502) and the second tile (tile2) (504) to determine the probability initial value for the bin A of the third tile (tile3) (506). The same applies when the data unit is a slice.

According to one embodiment, in order to determine the reference occurrence probability, when two or more previous data units are referenced, the reference occurrence probability may be determined by averaging the occurrence probabilities of predetermined values of the bins of each previous data unit.

Alternatively, the reference occurrence probability may be determined based on the weight of each previous data unit. For example, a high weight may be assigned to a previous data unit that is close to the current data unit in encoding order, and a low weight may be assigned to a previous data unit that is far from the current data unit in encoding order. In addition, the reference occurrence probability may be determined by weighting and averaging the occurrence probabilities of a predetermined value in the bins of each previous data unit based on the weights. For example, when determining the probability initial value for bin A of the third tile (tile3) (506), a higher weight may be assigned to the occurrence probability of a predetermined value in bin A of the second tile (tile2) (504) than to the occurrence probability of a predetermined value in bin A of the first tile (tile1) (502). The same applies when the data unit is a slice.

According to one embodiment, when a previous data unit is referenced, the reference occurrence probability of the previous data unit may be determined based on the reference occurrence probability referenced for bypass coding of the bin of the previous data unit and the actual occurrence probability of the bin in the previous data unit. For example, when only the second tile (tile2) (504) is referenced to determine the probability initial value for bin A of the third tile (tile3) (506), the reference occurrence probability to be referenced by the third tile (tile3) (506) may be determined based on the reference occurrence probability referenced for determining the probability initial value of the second tile (tile2) (504) and the actual occurrence probability of the predetermined value of the bin in the second tile (tile2) (504). The same applies when the data unit is a slice.

In one embodiment, to determine the reference occurrence probability, the actual occurrence probability of previous data units may be accumulated to determine the probability initial value. The probability initial value may be initialized on a picture-by-picture basis, a specific slice basis, or a specific tile basis.

According to one embodiment, the probability initialization value for bypass coding of a bin of a first slice or a first tile of a picture may be determined based on the actual occurrence probability of that bin within the entire or a portion of the previous picture.

According to embodiments of the present disclosure, the occurrence probability for a 0 value of a bin parsed in a previous decryption process can be calculated according to mathematical expression 1.

[Mathematical Formula 1]

In mathematical expression 1, total_count _{bin_A} represents the total number of bins A parsed for a given range of data units decoded before the current data unit. occurrence_count[0] _{bin_A} represents the number of bins A having a value of 0 parsed for a given range of data units decoded before the current data unit. And probability[0] _{bin_A} represents the probability of occurrence of bin A having a value of 0 in a given range of data units decoded before the current data unit. Conversely, probability[1] _{bin_A} represents the probability of occurrence of bin A having a value of 1 in a given range of data units decoded before the current data unit.

The above probability[0] _{bin_A} and probability[1] _{bin_A} can be determined as the initial probability values for 0 and 1 of bin A of the current picture. Alternatively, the initial probability values for 0 and 1 of bin A for performing bypass coding of the current picture can be determined by transforming probability[0] _{bin_A} and probability[1] _{bin_A} .

Bypass coding for bin A can be performed based on the initial probability values for 0 and 1 of bin A of the current data unit determined based on probability[0] _{bin_A} and probability[1] _{bin_A} . The data unit includes a picture, a slice, a tile, etc. In addition, here, bin_A means a bin of any syntax element for which bypass coding is performed.

Figure 6 describes a method for setting a probability initialization value for bypass coding when encoding and decoding are performed based on a VPDU (Virtual pipeline data unit).

VPDU is a parallel data processing unit that was introduced as the size of the coding tree unit (CTU) gradually increased. By defining multiple VPDUs within one coding tree unit and processing multiple VPDUs in parallel, the implementation complexity and size of the hardware can be reduced. The small square block in Fig. 6 represents a VPDU, and a square block (a block shaped like a dotted line) containing 2 x 2 VPDUs represents one coding tree unit block. Hereinafter, as in Fig. 5, an embodiment will be described assuming that bypass coding is applied to empty A of syntax element A and that a specific value is 0.

According to one embodiment, in the decoding stage, the probability initialization value for bypass coding of the corresponding syntax element of the current VPDU may be set based on the probability of the corresponding syntax element of the corresponding VPDU of the previous coding tree unit block. For example, the probability initialization value of a specific syntax element of the first VPDU (612) of the second coding tree unit block (610) may be set based on the probability of the corresponding specific syntax element of the first VPDU (602) of the first coding tree unit block (600). Similarly, the probability initialization values of the specific syntax elements of the VPDUs (614, 616, 618) of the second coding tree unit block (610) may be set based on the probability of the specific syntax elements of the VPDUs (614, 616, 618) of the first coding tree unit block (600), respectively.

At this time, the probability initialization value of a specific syntax element of a VPDU at an arbitrary position of the current coding tree unit block can be determined by accumulating the probability of a specific syntax element of VPDUs at corresponding arbitrary positions of at least one coding tree unit block that was previously coded. Hereinafter, mathematical expression 2 is an expression for setting the probability initialization value of the empty A of the syntax element A of the third VPDU (3rd VPDU) of the Nth coding tree unit block (Nth CTU). At this time, the probability initialization value uses M coding tree unit blocks that were previously coded. Here, N and M are positive integers such that N > M.

[Equation 2]

According to mathematical expression 2, the syntax element of the third VPDU of the M coding tree unit blocks encoded or decoded before the Nth coding tree unit block is used. Although mathematical expression 2 is explained based on the third VPDU as an example, it can be applied to other VPDUs as well.

In one embodiment, the same probability initialization value may be applied to all VPDUs within a coding tree unit block. For example, the probability initialization value may be determined from one or more previously encoded or decoded coding tree unit blocks.

According to one embodiment, A probability initialization value for bypass coding of a specific bin can be set based on reference picture structure information.

Figure 7 shows a reference picture structure for explaining the probability initialization of bins according to the reference picture structure.

According to Fig. 7, a series of pictures arranged in chronological order can be encoded and decoded according to a separate encoding order different from the chronological order. Fig. 7 illustrates a reference picture structure according to one embodiment of a video encoding scenario corresponding to random access. By configuring the encoding order and the chronological order differently, the convenience of random location access in the video sequence and the encoding efficiency can be improved.

At this time, considering the human visual perception characteristics, one or more temporal layers can be configured according to the flow of time. And by assigning different quantization parameter (QP) values to each temporal layer, compression performance can be improved. Therefore, a method for performing probability initialization of bins is described, considering that different quantization parameter values are applied to the time order of each picture or to each temporal layer. The reference picture structure of Fig. 7 is only one example, and any reference picture structure different from Fig. 7 can be applied to video encoding and decoding.

The numbers displayed on the left side of the pictures (700 to 780) in FIG. 7 indicate the order of the pictures. The numbers above indicate the encoding order, and the numbers displayed in the "" below indicate the POC (Picture Order Counter), which indicates the display order. Furthermore, the arrows between the pictures in FIG. 7 indicate the pictures referenced by a given picture.

Picture (700) is an IDR or Intra picture, and is encoded and decoded without reference to other pictures. Picture (710) is encoded and decoded only with reference to picture (700). Picture (720) is encoded and decoded with reference to picture (700) and picture (710). Reference pictures referenced by the remaining pictures can also be identified according to the arrows.

The pictures in Fig. 7 can be divided into temporal layers. A temporal layer refers to a set of pictures grouped by the same temporal ID. As the level of the temporal layer increases, the frame rate doubles stepwise for each level. For example, pictures (700, 710) correspond to temporal layer level 0. And picture (720) corresponds to temporal layer level 1. And pictures (730, 740) correspond to temporal layer level 2. And pictures (750, 760, 770, 780) correspond to temporal layer level 3.

In one embodiment, the probability initialization of the bins for bypass coding may be performed based on the temporal difference between the current picture and a reference picture. For example, if the temporal distance difference between the current picture and a given reference picture A is less than or equal to S, the probability initialization value of bin A in the current picture may be calculated, where S is an arbitrary positive integer.

Hereinafter, the above embodiment is described assuming that the S value is 2. According to Fig. 7, when the current picture is a picture (720) whose temporal order is 4, the reference pictures restored before the current picture are a picture (700) whose temporal order is 0 and a picture (710) whose temporal order is 8. At this time, the temporal distance difference between the current picture (720) whose temporal order is 4 and the reference picture (700) whose temporal order is 0 and the temporal distance difference between the current picture (720) whose temporal order is 4 and the reference picture (710) whose temporal order is 8 are both 4. That is, when the current picture is a picture (720) whose temporal order is 4, the temporal distance between the current picture and the two reference pictures is greater than 2, so probability initialization for the empty A of the current picture is not performed.

When the current picture is a picture (750) with a temporal order of 1, the reference pictures restored before the current picture are a picture (700) with a temporal order of 0 and a picture (730) with a temporal order of 2.

At this time, since the temporal distance between the current picture and the two reference pictures is less than 2, probability initialization for the empty A of the current picture can be performed.

When it is determined that the probability initialization of bin A in the current picture is performed based on the temporal distance between the reference picture and the current picture, the initial probability value calculation of bin A in the current picture can be performed based on the appearance probability of bin A in the reference picture.

According to one embodiment, if the current picture refers only to the two closest reference pictures in determining whether to perform probability initialization, the initial probability value of bin A of the current picture may be calculated based on the appearance probability of bin A obtained from the two reference pictures. Alternatively, the initial probability value of bin A of the current picture may be calculated based on the appearance probability of bin A obtained from one of the two reference pictures. In this case, which of the two reference pictures is referenced may be determined based on probability initialization reference picture information explicitly transmitted from the bitstream. The probability initialization reference picture information may be transmitted/parsed at the picture or slice level. Which of the two reference pictures is referenced may be implicitly determined based on the temporal layer level, quantization parameter value, etc. of the current reference picture.

According to one embodiment, probability initialization for a bin to which bypass coding is applied may be performed based on the difference in quantization parameters between the current picture and a reference picture. For example, if the difference in quantization parameters between the current picture and a given reference picture A is less than or equal to T, a probability initialization value for bin A in the current picture may be calculated, where T is an arbitrary positive integer.

Hereinafter, the above embodiment is described assuming that the T value is 1. According to Fig. 7, when the current picture is a picture (720) with a temporal order of 4, the reference pictures restored before the current picture are a picture (700) with a temporal order of 0 and a picture (710) with a temporal order of 8. At this time, the quantization parameter difference between the current picture (720) with a temporal order of 4 and the reference picture (700) with a temporal order of 0 and the quantization parameter difference between the current picture (720) with a temporal order of 4 and the reference picture (710) with a temporal order of 8 are 2 and 1, respectively.

According to one embodiment, if the quantization parameter difference between at least one of the two reference pictures closest to the current picture (720) is less than a predetermined T value, probability initialization of a bin for the current picture may be performed. According to the example of FIG. 7, since the quantization parameter difference between the current picture (720) and the reference picture (710) having a temporal order of 8 is 1, probability initialization of a bin A for the current picture (720) may be performed. And the initial probability value of the bin A may be determined according to the appearance probability of each value of the bin A of the reference picture (710) that satisfies the quantization parameter difference condition.

According to one embodiment, if both reference pictures (700, 710) satisfy the quantization parameter difference condition, the initial probability value of bin A for the current picture (720) can be determined according to the appearance probability of each value of bin A of the two reference pictures (700, 710). And if both reference pictures (700, 710) do not satisfy the quantization parameter difference condition, the probability initialization of the bin for the current picture is not performed.

According to one embodiment, even if the difference in quantization parameters between a reference picture and a current picture is less than or equal to T, if the quantization parameter of the reference picture is greater than or equal to a predetermined threshold value U, the appearance probability of the bin of the reference picture may not be considered in initializing the probability of the bin for the current picture. This is because reference pictures with a quantization parameter greater than a predetermined threshold value have a large data loss, and thus the appearance probability of the bin derived from the reference picture is unreliable and close to noise. Here, the threshold value U is an arbitrary positive integer.

According to one embodiment, whether to apply empty probability initialization of bypass coding may be determined based on the reference structure of a Group of Pictures (GOP). In the present disclosure, an open GOP refers to a GOP that can refer to a previous GOP, and a closed GOP refers to a GOP that cannot refer to a previous GOP. If the current GOP is an open GOP, a method for determining whether to initialize empty probability based on a temporal difference between the current picture and a reference picture or a method for determining whether to initialize empty probability based on a difference in quantization parameters between the current picture and the reference picture may be performed on the current picture. On the other hand, if the current GOP is a closed GOP, the two methods for determining whether to initialize empty probability described above may not be applied to the current picture.

According to one embodiment, if the current picture is a non-reference picture, the two methods for determining whether to initialize the empty probability described above may not be applied to the current picture due to issues such as buffer management and reference dependency.

According to one embodiment, a probability initialization value for bypass coding of a specific bin can be set based on reference block information.

FIG. 8 illustrates an example of inter-screen prediction of a current block (800) to explain a method for setting a probability initialization value based on reference block information.

According to one embodiment, based on a given bin parsed for a corresponding reference block (810) of a current block (800), the probability of occurrence of the given bin is determined. And, based on the probability of occurrence of the given bin, an initial probability value of the given bin applied to the current block (800) can be calculated.

FIG. 9 illustrates an example of inter-screen prediction of a current block (900) to explain a method for setting a probability initialization value based on reference block information.

According to one embodiment, based on a parsed predetermined bin for a collection area for probability occurrence frequency (920) including not only the reference block (910) but also the surrounding area of the reference block (910), the occurrence probability of the predetermined bin can be determined. Then, based on the occurrence probability of the predetermined bin, an initial probability value of the predetermined bin applied to the current block (900) can be calculated.

According to one embodiment, the size of the probability occurrence frequency collection area (920) may have any shape. For example, the probability occurrence frequency collection area (920) may be set to include samples located within a distance of U samples from the boundary of the reference block. In this case, when the size of the current block (900) is MxN, the size of the probability occurrence frequency collection area (920) may be (M+2U)x(N+2U).

FIG. 10 is a flowchart illustrating an image decoding method according to an embodiment of the present disclosure. The image decoding method of FIG. 10 can be performed by an image decoding device.

In step 1010, in the bypass coding of the bin of the syntax element being parsed for the current data unit, it is determined whether the current occurrence probability for a given value of the bin is initialized.

According to one embodiment, the current data unit may be a tile or a slice. Alternatively, the current data unit may be a Virtual Pipeline Data Unit (VPDU). Alternatively, the current data unit may be a data unit of various levels, such as a picture or a coding tree unit.

According to one embodiment, information on whether to use empty probability initialization can be obtained from the high-level syntax. Based on the information on whether to use empty probability initialization, it can be determined whether the current occurrence probability is initialized.

According to one embodiment, information on a method for initializing a bin probability can be obtained from a high-level syntax. Furthermore, based on the method for initializing a bin probability indicated by the information on the method for initializing a bin probability, a reference occurrence probability for a given value of the bin can be obtained.

According to one embodiment, it can be determined whether the current appearance probability is initialized based on a reference picture referenced by the current picture of the current data unit.

According to one embodiment, whether the current appearance probability is initialized can be determined based on the temporal difference between the current picture and the reference picture. The temporal difference is a Picture Order Counter (POC) of the current picture and the reference picture, and the POC is a number assigned to the picture according to the display order.

According to one embodiment, it can be determined whether the current appearance probability is initialized based on a difference in quantization parameters of the current picture and the reference picture.

According to one embodiment, it may be determined whether the current appearance probability is initialized based on whether a quantization parameter of a reference picture referenced by the current picture is greater than a predetermined threshold.

According to one embodiment, it may be determined whether the current occurrence probability is initialized based on whether the GOP (Group of Pictures) containing the current picture of the current data unit is an open GOP or a closed GOP.

In step 1020, corresponding to the determination that the current occurrence probability is initialized, a reference occurrence probability for a given value of the bin is obtained from a reference data unit decrypted before the current data unit.

According to one embodiment, the reference data unit may be a data unit that was decoded immediately before the current data unit in the decoding order of the current data unit. In addition, the reference data unit may be the last data unit of a reference picture decoded before the current picture when the current data unit is the first data unit of the current picture.

At step 1030, based on the above reference appearance probability, the current appearance probability is initialized.

Meanwhile, the steps described in FIG. 10 can be performed in the same manner in an image encoding method. Furthermore, a bitstream can be generated by an image encoding method including the steps described in FIG. 10. The bitstream can be stored on a non-transitory computer-readable recording medium and can also be transmitted (or streamed).

FIG. 11 is a diagram exemplifying a content streaming system to which an embodiment according to the present disclosure can be applied.

As illustrated in FIG. 11, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and CCTVs into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and CCTVs directly generate bitstreams, the encoding server may be omitted.

The above bitstream can be generated by an image encoding method and/or an image encoding device to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server may control commands/responses between each device within the content streaming system.

The streaming server can receive content from a media repository and/or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

The above embodiments can be performed in the same or corresponding manner in an encoding device and a decoding device. In addition, an image can be encoded/decoded using at least one or a combination of at least one of the above embodiments.

The order in which the above embodiments are applied may be different in the encoding device and the decoding device. Alternatively, the order in which the above embodiments are applied may be the same in the encoding device and the decoding device.

The above embodiments can be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments can be performed identically for the luminance and chrominance signals.

In the above embodiments, the methods are described based on a flowchart as a series of steps or units. However, the present disclosure is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will understand that the steps depicted in the flowchart are not exclusive, and that other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the present disclosure.

The above embodiments may be implemented in the form of program commands that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be those specifically designed and configured for the present disclosure, or may be known and usable by those skilled in the art of computer software.

The bitstream generated by the encoding method according to the above embodiment can be stored in a non-transitory computer-readable recording medium. In addition, the bitstream stored in the non-transitory computer-readable recording medium can be decoded by the decoding method according to the above embodiment.

Here, examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

Although the present disclosure has been described above with specific details such as specific components and limited examples and drawings, these are provided only to help a more general understanding of the present disclosure, and the present disclosure is not limited to the above examples, and a person having ordinary knowledge in the technical field to which the present disclosure belongs can make various modifications and variations from this description.

Therefore, the spirit of the present disclosure should not be limited to the embodiments described above, and all modifications that are equivalent or equivalent to the following claims as well as the claims are considered to fall within the scope of the spirit of the present disclosure.

The present disclosure can be used in a device for encoding/decoding an image and a recording medium storing a bitstream.

Claims (15)

In bypass decoding of a bin of a syntax element being parsed for the current data unit, a step of determining whether the current occurrence probability for a given value of the bin is initialized; In response to the current occurrence probability being determined to be initialized, a step of obtaining a reference occurrence probability for a predetermined value of the bin from a reference data unit decrypted before the current data unit; and An image decoding method comprising a step of initializing the current appearance probability based on the above reference appearance probability. In the first paragraph, An image decoding method, characterized in that the current data unit is a tile or a slice. In the first paragraph, A video decoding method characterized in that the current data unit is a VPDU (Virtual pipeline data unit). In the first paragraph, The above reference data unit is, An image decoding method characterized in that the current data unit is a data unit that was decoded immediately before in the decoding order of the current data unit. In the first paragraph, The above reference data unit is, A video decoding method, characterized in that when the current data unit is the first data unit of the current picture, it is the last data unit of a reference picture decoded before the current picture. In the first paragraph, The above video decryption method is, Further comprising a step of obtaining information on whether to use empty probability initialization from the high level syntax, An image decoding method characterized in that whether the current appearance probability is initialized is determined based on information on whether the above empty probability initialization is used. In the first paragraph, The above video decryption method is, Further comprising a step of obtaining information on how to initialize the empty probability from the high level syntax, An image decoding method characterized in that a method for obtaining a reference appearance probability for a predetermined value of the bin is determined according to the bin probability initialization method indicated by the above bin probability initialization method information. In the first paragraph, An image decoding method characterized in that it is determined whether the current appearance probability is initialized based on a reference picture referenced by the current picture of the current data unit. In paragraph 8, Based on the temporal difference between the current picture and the reference picture, it is determined whether the current appearance probability is initialized, A video decoding method, characterized in that the above-mentioned temporal difference is a POC (Picture Order Counter) of the current picture and the reference picture, and the POC is a number assigned to the picture according to the display order. In paragraph 8, An image decoding method characterized in that it is determined whether the current appearance probability is initialized based on the difference in quantization parameters of the current picture and the reference picture. In paragraph 8, An image decoding method characterized in that it is determined whether the current appearance probability is initialized based on whether the quantization parameter of the reference picture referenced by the current picture is greater than a predetermined threshold value. In the first paragraph, A video decoding method characterized in that it is determined whether the current appearance probability is initialized based on whether the GOP (Group of Picture) including the current picture of the current data unit is an open GOP or a closed GOP. In bypass coding of a bin of a syntax element being parsed for the current data unit, a step of determining whether the current occurrence probability for a given value of the bin is initialized; In response to the current occurrence probability being determined to be initialized, a step of obtaining a reference occurrence probability for a predetermined value of the bin from a reference data unit decrypted before the current data unit; and An image encoding method comprising a step of initializing the current appearance probability based on the above reference appearance probability. In a non-transitory computer-readable recording medium storing a bitstream generated by a video encoding method, The above image encoding method is, A step of determining a reference candidate block of the current block; In bypass coding of a bin of a syntax element being parsed for the current data unit, a step of determining whether the current occurrence probability for a given value of the bin is initialized; In response to the current occurrence probability being determined to be initialized, a step of obtaining a reference occurrence probability for a predetermined value of the bin from a reference data unit decrypted before the current data unit; and A non-transitory computer-readable recording medium, characterized in that it comprises a step of initializing the current appearance probability based on the above reference appearance probability. In a bitstream transmission method for transmitting a bitstream generated by a video encoding method, A step of encoding an image based on the above image encoding method; and A step of transmitting a bitstream including the encoded image, The above image encoding method is, In bypass coding of a bin of a syntax element being parsed for the current data unit, a step of determining whether the current occurrence probability for a given value of the bin is initialized; In response to the current occurrence probability being determined to be initialized, a step of obtaining a reference occurrence probability for a predetermined value of the bin from a reference data unit decrypted before the current data unit; and A transmission method, characterized in that it includes a step of initializing the current appearance probability based on the above reference appearance probability.