GB2501125A - Providing adaptation parameters to a decoder by including an identifier to a relevant characteristic set in a bit stream portion. - Google Patents
Providing adaptation parameters to a decoder by including an identifier to a relevant characteristic set in a bit stream portion. Download PDFInfo
- Publication number
- GB2501125A GB2501125A GB1206571.0A GB201206571A GB2501125A GB 2501125 A GB2501125 A GB 2501125A GB 201206571 A GB201206571 A GB 201206571A GB 2501125 A GB2501125 A GB 2501125A
- Authority
- GB
- United Kingdom
- Prior art keywords
- parameter set
- aps
- identifier
- encoding
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000006978 adaptation Effects 0.000 title abstract description 12
- 238000000034 method Methods 0.000 claims abstract description 265
- 230000008569 process Effects 0.000 claims abstract description 182
- 238000001914 filtration Methods 0.000 claims abstract description 20
- 239000000872 buffer Substances 0.000 claims description 82
- 238000013139 quantization Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 6
- 230000001419 dependent effect Effects 0.000 claims description 5
- 230000003139 buffering effect Effects 0.000 claims description 2
- 230000004048 modification Effects 0.000 abstract description 6
- 238000012986 modification Methods 0.000 abstract description 6
- 208000031212 Autoimmune polyendocrinopathy Diseases 0.000 abstract description 2
- 235000019395 ammonium persulphate Nutrition 0.000 abstract description 2
- 238000000261 appearance potential spectroscopy Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 17
- 239000013598 vector Substances 0.000 description 17
- 230000003044 adaptive effect Effects 0.000 description 16
- 238000004891 communication Methods 0.000 description 13
- 230000007246 mechanism Effects 0.000 description 11
- 230000006835 compression Effects 0.000 description 9
- 238000007906 compression Methods 0.000 description 9
- 230000002123 temporal effect Effects 0.000 description 8
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- HAAITRDZHUANGT-UHFFFAOYSA-N 1-[2-[(7-chloro-1-benzothiophen-3-yl)methoxy]-2-(2,4-dichlorophenyl)ethyl]imidazole;nitric acid Chemical compound O[N+]([O-])=O.ClC1=CC(Cl)=CC=C1C(OCC=1C2=CC=CC(Cl)=C2SC=1)CN1C=NC=C1 HAAITRDZHUANGT-UHFFFAOYSA-N 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 230000001364 causal effect Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000000153 supplemental effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- FMYKJLXRRQTBOR-UBFHEZILSA-N (2s)-2-acetamido-4-methyl-n-[4-methyl-1-oxo-1-[[(2s)-1-oxohexan-2-yl]amino]pentan-2-yl]pentanamide Chemical group CCCC[C@@H](C=O)NC(=O)C(CC(C)C)NC(=O)[C@H](CC(C)C)NC(C)=O FMYKJLXRRQTBOR-UBFHEZILSA-N 0.000 description 1
- 101000658124 Apomastus schlingeri Mu-cyrtautoxin-As1a Proteins 0.000 description 1
- 101000849579 Arabidopsis thaliana 30S ribosomal protein S13, chloroplastic Proteins 0.000 description 1
- 240000002791 Brassica napus Species 0.000 description 1
- 101100001794 Neurospora crassa (strain ATCC 24698 / 74-OR23-1A / CBS 708.71 / DSM 1257 / FGSC 987) aps-2 gene Proteins 0.000 description 1
- 108010063123 alfare Proteins 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003116 impacting effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
- H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Decoding a bit stream representative of video data comprising decoding an encoded image obtained using at least two different encoding processes and in which the encoded image includes a bit stream portion corresponding to a set of coding units. In the case where a parameter set comprising a set of characteristics of one of the used encoding processes (such as an Adaptation Parameter Set , APS) is relevant, the bit stream portion will include an identifier providing a reference to the relevant parameter set. This identifier may be provided in the slice header. The decoder will obtain a set of characteristics for each active encoding process of the encoding processes 1004 and decode the image in using the obtained characteristics 1005. Also disclosed is an encoder in which the bit stream portion is provided with an identifier giving a reference to a relevant parameter set corresponding to the relevant encoding process. The invention relates to the new HEVC (High Efficiency Video Coding) Standard and is devised to overcome the problem of redundancies that can be introduced between APSs when there are frequent modifications of the filtering parameters during a sequence.
Description
tM:;: INTELLECTUAL S... * PROPERTY OFFICE Application No. 0B1206571.0 RTM Date:27 Scptcnibcr 2012 The following terms are registered trademarks and should be read as such wherever they occur in this document: Wifi Intellectual Properly Office is an operaling name of Ihe Patent Office www.ipo.gov.uk
METHOD AND DEVICE FOR PROVIDING ADAPTATION PARAMETERS TO
A DECODER
The present invention relates to a method and device for providing parameters to a set of decoding processes to be applied by a decoder. More precisely, the present invention relates to the signalization and the encoding in a video stream compatible with the upcoming ITU-T/MPEG video standard, HEVC (High Efficiency Video Coding), of parameters representative of adaptive characteristics of some decoding processes.
The invention may be applied in the field of digital signal processing, and in particular in the field of video compression and in the field of the streaming of compressed video data.
Many video compression formats, such as for example H.263, H.264, MPEG-i, MPEG-2, MPEG-4, SVC, use block-based discrete cosine transform (DCT) and motion compensation to remove spatial and temporal redundancies.
They are often referred to as predictive video formats.
As represented in figure 1, each frame or image 102 of the video signal 101 is divided into slices 103 which are encoded and can be decoded independently. A slice is typically a rectangular portion of the frame, or more generally, a portion of a frame or an entire frame.
In HEVC, blocks of pixels from 64x64, to 4x4 can be used. The partitioning is organized according to a quad-tree structure based on the largest coding unit ([CU) 104. An [CU corresponds to a square block of 64x64. If an [CU needs to be divided, a split flag indicates that the [CU is split into 4 32x32 blocks. In the same way, if any of these 4 blocks need to be split, the split flag is set to true and the 32x32 block is divided into 4 16x16 blocks etc. When a split flag is set to false, the current block is a coding unit CU (105). A CU has a size equal to 64x64, 32x32, 16x16 or 8x8 pixels. CU can be divided in Prediction Units (PU) 106 for spatial or temporal prediction and in Transform Units (107) for transformation in the frequency domain.
There are two families of coding modes for coding blocks of an image: coding modes based on spatial prediction, referred to as INTRA prediction and coding modes based on temporal prediction (INTER, Merge, Skip). In both spatial and temporal prediction modes, a residual is computed by subtracting the prediction from the original block.
An INTRA block is generally predicted by an INTRA prediction process from the encoded pixels at its causal boundary. In INTRA prediction, a prediction direction is encoded.
Temporal prediction consists in finding in a reference frame, either a previous or a future frame of the video sequence, an image portion or reference area which is the closest to the block to be encoded. This step is typically known as motion estimation. Next, the block to be encoded is predicted using the reference area in a step typically referred to as motion compensation. The difference between the block to be encoded and the reference portion is encoded, along with an item of motion information relative to the motion vector which indicates the reference area to use for motion compensation. In temporal prediction, at least one motion vector is encoded.
In order to further reduce the cost of encoding motion information, rather than directly encoding a motion vector, assuming that motion is locally homogeneous, the motion vector are encoded in terms of a difference between the motion vector and a motion vector predictor, typically computed from one or more motion vectors of the blocks surrounding the block to be encoded.
In H.264, for instance motion vectors are encoded with respect to a median predictor computed from the motion vectors situated in a causal neighborhood of the block to be encoded, for example from the three blocks situated above and on the left of the block to be encoded. Only the difference, referred to as a residual motion vector, between the median predictor and the current block motion vector is encoded in the bitstream to reduce the encoding cost.
In order to further reduce the encoding cost, a quantization is applied to the residual information. The quantization process may be preceded by a scaling stage: quantization scaling matrixes are used to scale the residual before applying the quantization process. Then Syntax, quantized residual and prediction information are entropy coded generally with a context adaptive entropy coder.
The main advantage of prediction and context adaptive entropy coding is to take benefit of the local correlation in the video signal to reduce the coding cost. The counterpart of this compression improvement is the reduction of error resilience of the video stream due to inter CU dependencies.
The concept of slices was proposed in previous standards to cut block dependencies and as a consequence improves the error resilience. In HEVC, two types of slices were defined: coding slices and entropy slices. In the example of figure 1, the frame 103 has been divided in 16 slices: CS1, CS7 and CS13 correspond to coding slices and others to entropy slices. Each coding slice starts by a slice header 108 that indicates the coding parameters of the slice. These slices break the coding dependencies on their boundaries which are represented by dash lines 109 and 110. Thus, CUs of slice CS7 cannot use CU5 pixels of slice ES4 for intra prediction.
Entropy slice allows the syntax decoding of the bitstream independently of other entropy slices. The coding parameters of entropy slices are those of the latest coding slice in raster scan order. To achieve such functionality, each entropy slice resets the entropy context and considers neighborhood data as unavailable. Thus, entropy slices allow to decode syntax information of the bitstream in parallel. For instance, slice ES2, ES3 and ES4 can be decoded in parallel.
Both encoding and decoding processes involves a decoding process of an encoded image. This process is typically performed at the encoder side for the purpose of future motion estimation which enables an encoder and a corresponding decoder to have the same reference frames.
To reconstruct the coded frame, the residual is inverse quantized and inverse transformed in order to provide the "decoded" residual in the pixel domain. The first reconstruction is then filtered by one or several kinds of loop filtering processes. These loop filters are applied on the reconstructed frame at encoder and decoder side in order that the same reference frame is used at both sides. The aim of this loop filtering is to remove compression artifacts. For example, H.264/AVC uses a deblocking filter. This filter can remove blocking artifacts due to the quantization of the transformed residual and to block motion compensation. In the current HEVC standard, 3 types of loop filters are used: deblocking filter, sample adaptive offset (SAO) and adaptive loop filter (ALE).
In order to facilitate the encapsulation of video data in network packets, H.264/AVC has defined the concept of NAL (Network Abstraction layer) Unit. The NAL unit structure definition specifies a generic format for use in both packet-oriented and bitstream-oriented transport systems. The first part of each NAL unit is a header that contains an indication of the type of data in the NAL unit, and the remaining part contains payload data of the type indicated by the header. NAL units are classified into VCL and non-VCL NAL units. The VCL NAL units contain the data that represents the values of the samples in the video pictures. The non-VOL NAL units contain any associated additional information such as parameter sets (important header data that can apply to a large number of VOL NAL units) and supplemental enhancement information (timing information and other supplemental data that may enhance usability of the decoded video signal but are not necessary for decoding the values of the samples in the video pictures).
In HEVO and H.264/AVC two kinds of parameter sets NAL units are employed: first, the Sequence Parameter Set (SPS) NAL unit that gathers all parameters that are unchanged during the whole video sequence. Typically, it handles the coding profile, the size of the video frames and other parameters.
Secondly, Picture Parameter Sets (PPS) codes the different values that may change from one frame to another. In addition these picture and sequence parameter sets, HEVO has introduced Adaptation Parameter Sets (APS) which contains parameters that may change from one slice to another.
Figure 2 is a flow chart illustrating the steps performed by an HEVO en coder.
Each frame of the original video sequence 101 is first divided into a grid of largest coding units (LOU) during stage 201. This stage controls also the definition of coding and entropy slices.
The subdivision of the LCU in CUs and the partitioning of the CU in TUs and PUs is determined based on a rate distortion criterion. Each PU of the CU being processed is predicted spatially by an INTRA predictor 217, or temporally by an INTER predictor 218. Each predictor is a block of pixels determined using encoded pixels from the same image or blocks of encoded pixels of another image, from which a difference block (or "residual") is computed. An encoded block is represented by an identifier representative of the predictor to be used and a residual block.
The encoded frames are of two types: temporally predicted frames (either predicted from one reference frame called P-frames or predicted from two reference frames called B-frames) and non-temporally predicted frames (called INTRA frames or I-frames). In I-frames, only INTRA prediction is considered for coding CUs/PUs. In P-frames and B-frames, INTRA and INTER prediction are considered for coding CUs/PUs.
In the INTRA prediction module 217, the current block is predicted by means of an INTRA predictor obtained using encoded pixels at the boundary of the current block. A prediction direction allowing identifying the pixels at the boundary is determined in module 202 and encoded in the bitstream in module 203 along with residual data. A further compression improvement is obtained by predictively encoding the INTRA prediction direction from INTRA prediction directions of surrounding PU.
Prediction of intra direction is not allowed for neighbor CUs that are not in the same entropy slices.
With regard to the second processing module 218 related to INTER coding, two prediction types are possible. Mono-prediction (P-type) consists in predicting a block of pixels (i.e. a PU) by referring to one reference block of pixels from one reference picture. Bi-prediction (B-type) consists in predicting a block of pixels (i.e. a PU) by referring to two reference blocks of pixels from one or two reference pictures. An estimation of motion 204 between the current PU and reference images 215 is made in order to identify, in one or several of these reference images, one (P-type) or several (B-type) blocks of pixels to use as predictors of this current PU. In case where several block predictors are used (B-type), these blocks are merged to generate one single prediction block.
The reference block is identified in the reference frame by a motion vector that is equal to the displacement between the PU in current frame and the reference block. Next stage (205) of inter prediction process consists in computing the difference between the prediction block and current block to obtain a residual block. At the end of the inter prediction process the current PU is composed of at least one motion vector and a residual.
In order to reduce the cost of motion information, HEVC proposes in a process called AMVP (Adaptive Motion Vector Predictor) to select a motion predictor in a set of candidate motion predictors. The motion information are then encoded in the form of an index on a motion vector predictor and a motion vector difference. Again, this prediction process creates inter CU dependencies.
Prediction of motion information is not allowed for neighbor CUs that are not in the same entropy slices.
The prediction steps is followed by a selection step (216) selecting the type of encoding (INTRA or INTER) minimizing a rate distortion criterion.
When selected, the residual obtained is transformed 207 using a DCT based transform. The transform applies to a Transform Unit (TU) that is included into a CU and that can be split into smaller TUs using a so-called Residual QuadTree (ROT) decomposition. In HEVC, generally 2 or 3 levels of decompositions are used and authorized transform sizes are from 32x32, 16x16, 8x8 and 4x4.
The residual transformed coefficients are then quantized (208). The coefficients of the quantized transformed residual are then coded by means of an entropy coding 209 and then inserted in the compressed bitstream 210.
Syntax elements are also coded with help of the stage 209.
The encoder includes a decoding loop (211, 212, 213, 214, 215) to insure that there is no drift between the encoder and the decoder.
Thus the quantized transformed residual is dequantized 211 by applying the inverse quantization to that provided at step 208 and reconstructed 212 by applying the inverse transform to that of the step 207.
If the residual comes from an INTRA coding 217, the used INTRA predictor is identified thanks to the INTRA prediction information and added to the residual to recover a reconstructed block.
If the residual comes from an INTER coding 218, the prediction block(s) is (are) identified using the motion information, merged if necessary and then added to the decoded residual.
A final loop filter processing module 219 is applied to the reconstructed signal in order to reduce the encoding artifacts. In the current HEVC standard, 3 types of loop filters are used: deblocking filter 213, sample adaptive offset (SAO) 220 and adaptive loop filter (ALF) 214. These loop filters are described in the following. The parameters of the filters are coded and transmitted in one header of the bitstream typically slice header or in an adaptation parameter set.
The filtered images, also called reconstructed images, are then stored as reference images 215 in order to allow the subsequent INTER predictions.
The resulting bitstream 210 of the encoder 20 is encapsulated in VCL NAL units, and complemented by Non VCL NAL units comprising SPS, PPS and APS NAL units.
Figure 3 is a flow chart illustrating a classical video decoder 30 of HEVC type. The decoder 30 receives as an input a bit stream 210 corresponding to a video sequence 101.
During the decoding process, the bit stream 210 is first of all parsed with help of the entropy decoding module (301). This processing module uses the previously entropy decoded elements to decode the encoded data. It decodes in particular the parameter sets of the video sequence to initialize the decoder. Each VCL NAL unit that corresponds to coding slices or entropy slices are then decoded. The parsing process that consists in 301, 302 and 304 stages can be done in parallel for each slice but block prediction processes module 305 and 303 and loop filter module 319 are generally sequential to avoid the issue of neighbor data availability.
The partitioning of LCU in CU, PU and TU and the coding modes (INTER or INTRA) are obtained from the bitstream 210 with help of the entropy decoding module 301. Depending on the coding mode, either INTRA prediction processing module 307 or INTER prediction processing module 306 is employed. If the coding mode of the current block is INTRA, the prediction direction is extracted from the bitstream and decoded with help of neighbors' prediction direction during stage 303. The intra predicted block is then computed (303) with the decoded prediction direction and the already decoded pixels at the boundaries of current PU. The residual associated with the current block is recovered from the bitstream 301 and then entropy decoded.
If the coding mode of the current block is INTER, the motion information are extracted from the bitstream 301 and decoded (304) applying the AMVP method. The obtained motion vector(s) is (are)used in the reverse motion compensation module 305 in order to determine the INTER predictor block contained in the reference image(s) 315 of the decoder 30. In a similar manner to the encoder, these reference images 315 are composed of images that precede in decoding order the image currently being decoded and that are reconstructed from the bit stream.
Next decoding step consists in decoding the residual block that has been transmitted in the bitstream. The parsing module 301 extracts the residual coefficients from the bitstream and performs successively the inverse quantization (311) and inverse transform (312) to obtain the residual block. This residual block is added to the predicted block.
At the end of the decoding of all the blocks of the current image, the loop filter processing module 219 is used to eliminate the artifacts and improves the signal quality in order to obtain the reference images 315. As done at the encoder, this processing module employs the deblocking filter 213, then SAD 220 filter and finally the ALF 214.
The images thus decoded constitute the output video signal 308 of the decoder, which can then be displayed and used.
Figure 4 represents the steps of the different loop filters applied by the encoder and the decoder. A first frame reconstruction 401 is obtained by the combination of block predictors (identified using either motion information or INTRA prediction information) with decoded residual data. This frame suffers of encoding artifacts such as the well-known block artifacts at the block boundaries or other quantization artifacts. As already mentioned HEVC applies successively 3 loop filters: The deblocking filter 402, the SAO filter 404 and the ALF filter 406.
The purpose of deblocking filter (DBF) described in figure 5 is to remove block artifacts.
The DBF is iteratively applied on each coding units (in decoding order) of the frame in a processing loop constituted of stages 501 to 504.
Vertical edges of the CU are filtered first from left to right and then horizontal edges are processed starting from the edge on the top of CU to the bottom. The first stage 501 consists in selecting edges to be filtered. Except for some particular edges (boundary of the picture, boundary of slices...), all edges corresponding to edges of prediction and transform unit (PU and TU) are selected.
In step 502, the strength of the boundary is determined. Indeed, the deblocking filter process employs two kinds of filters: a strong and a weak filter.
Strong filter is applied when the block artifacts are high while weak filter is applied when the block artifacts are low or to preserve contours in the edge neighborhood. The determination of the boundary strength takes into account several parameters such as the edge type (TV or PU), the coding mode (INTRA or INTER) of CUs on each side of the edge, the transform coefficients of TUs on each side of the edge or the motion information.
The filtering decision which indicates if the edge is filtered with a strong or weak filter is then taken (503) based on the determined boundary strength, the edge samples, the Quantization Parameter (OP) and a predetermined table of parameters. Deblocking filter can be tuned with two parameters beta_offset and tc_offset which permits to select a value in the table of parameters. These two parameters will be referred in this document as deblocking filter parameters.
The final stage 504 consists in applying either the strong or weak filter for samples around the edge to be processed depending on filtering decision taken in 503.
Figure 6 illustrates the principle of SAD. The aim of SAD is to improve the quality of the reconstructed frame 403 reconstruction by sending additional data in the bitstream.
The principle of SAD is to classify each pixel in a predetermined class and to add the same offset value to each pixel of the class. Each class is associated to one offset which is encoded in the bitstream.
SAO loop filtering has 2 kinds of classification for a frame area: Edge Offset and Band Offset.
The Edge Offset consists in determining a class for each pixel by comparing its value with the values of 2 neighboring pixels, the 2 neighboring pixels being determined using an estimation of the signal direction. These directions depicted in the top part of figure 6 are the horizontal direction 605, 45-degree direction 606, vertical direction 607 and 135-degree direction 608.
For the Edge Offset mode, the absolute value of each offset is encoded in the bitstream.
The second SAD classification, represented in the bottom part of figure 6, is called the Band Offset. This classification also depends on the pixel value of the sample to be processed. In the current version of the SAD band offset classification the full range of pixel values (from 0 to max=255) is divided in 32 classes of 8 consecutive values. In this full range, a band of 4 consecutive classes is selected. A same offset is applied to all pixel values concerned by a same class. Pixel values outside the selected band of 4 classes are not filtered.
A value representative of the position of the band (SAO_Band_Position) of 4 classes is encoded in the bitstream along with the corresponding offset values.
The SAO tool is locally adapted to the frame. Some pixel area of a frame could be filtered while other could be not. In addition, SAO parameters (offset values, band position) could vary from one pixel area to another.
The current version of HEVC offers two alternative solutions to encode SAD parameters in the bitstream: either SAD parameters are coded in an APS or they are coded interleaved within LCU coded data of slice data.
The Adaptive Parameter Set mode representation is used to associate one single set of SAO parameters per frame area. For this mode, the set of SAO parameters is valid for a rectangular frame area which size is multiple of the size of the Largest Coding Unit (LCU). In order to reduce the amount of bits to store all the SAD parameters of a frame for this particular mode a run-length coding technique is used.
The Largest Coding Unit (LCU) mode representation is used to associate a set of SAO parameters to each LCU. In order to avoid encoding one set of SAD parameters for each LCU, a predictive scheme is used. This predictive mode consists in predicting the SAO parameters of a current LCU with the SAD parameters of the LCU on the left or of the LCU above the current LCU. This predictive technique enables to reduce the amount of data to represent the SAD parameters for the LCU mode.
Figure 7 illustrates the principle of Adaptive Loop Filter (ALF). ALF aims at further improving the reconstructed frame 405 after SAD processing.
Similarly to SAD, ALF relies on additional information that are encoded in the bitstream.
ALF employs a convolution filter whose shape 701 is represented in figure 7. The principle of adaptive loop filter is to classify samples to be filtered in up to 16 classes. Each class is associated with one set of filter coefficients.
Consequently one filter is selected among 16 different filters to process each 4x4 block of pixels of the frame.
Adaptive Loop Filter employs two filter adaptation modes which are region based adaptation (RA) or block-based adaptation (BA) mode. Region based adaption consists in dividing the picture in a regular grid of 16 rectangular regions 702. Each region represents one class and could be merged with another region as represented in 703. RA mode classification is therefore based on the position of the 4x4 blocks of pixels in the frame.
The second mode is block-based mode. All blocks are classified into 16 groups based on the block activity (determined with a Laplacian filter) and main direction. In both adaptation modes, the optimal coefficient filters are determined for each class with a least mean square method.
Coefficients of each filter are coded either in Adaptation Parameter Set NAL Unit or in slice headers.
As can be seen above, loop filters parameters (deblocking filter parameters, SAO parameters and ALF parameters) can be stored in an APS NAL unit. Similarly to SPS and PPS, an APS NAL unit can be relevant for several frames. But while practically, SF5 and PPS never or rarely change during a sequence, the nature of data stored in APS NAL units may induce more frequent changes. For instance, since SAO and ALF filters may have new parameters at each frame, APS is very likely to change at each frame in most applications.
The table I below represents an existing syntax of an APS NAL unit.
Each AFS NAL unit, starts by aps_id syntax element which is a value identifying the NAL unit. The aps_id value is recommended to be incremented for each new APS NAL unit. In the current implementation of HEVC, when decoding an APS NAL unit having the same aps_id than a previous one, the previous APS parameters are replaced by the parameters in the new APS NAL unit.
APS contains 4 types of parameters in parsing order: * Quantization Scaling Matrices parameters: scaling_list_param() set of syntax elements.
* Deblocking filter parameters: d isable_deblocking_filter_flag, beta_offset_div2 and tc_offset_div2 syntax elements * SAO parameters: aps_sao_param() set of syntax elements.
Adaptive Loop filter parameters: Alf_param() set of syntax elements.
aps_rbsp( ) { Descript or aps_id ue(v) aps_scaling_list_data_present_flag u(1) if(_aps_scaling_list_data_present_flag_) _________ scaling_list_pararn(_) __________ aps_deblocking_filter_flag u(1) if(aps_deblocking_filter_flag)_{ _________ d isable_deblocking_filter_flag u(1) if(_!disable_deblocking_filter_flag_)_{ __________ beta_offset_d iv2 se(v) tc_offset_d iv2 se(v) } __________________ } ____________________ aps_sao_interleaving_flag u(1) if(_!aps_sao_interleaving_flag_)_{ __________ aps_sample_adaptive_offset_flag u(1) if(_aps_sample_adaptive_ofiset_flag_) _________ apssaoparam() } ____________________ aps_adaptive_loop_filter_flag u(1) if(_aps_adaptive_loop_filter_flag _________ alf_param( _________ aps_extens ion_flag u(1) if(_aps_extension_flag_) __________ while(_more_rbsp_data()) _________ aps_extension_data_flag u(1) rbsp_trailing_bits(_) __________ } __________________
Table 1
A flag (i.e. aps scaling list data present flag, aps_deblocking_filter_flag, aps_sao_interleaving_flag, aps_sample_adaptive_offset_ilag and aps_adaptive_loop_filter_flag) is defined before each type of parameters to indicate if corresponding syntax elements are defined in the APS.
Consequently, an APS NAL unit may contain only SAO parameters or only OSM parameters for example. In addition, any combination of parameters may be used. For instance, one APS NAL unit may contain QSM and DBF parameters while another may contain SAO and ALF parameters.
The use of APS is not necessary to define the quantization or loop filter parameters. Indeed the slice header or the slice data may contain the parameters described in APS. The table 2 is an abstract of slice header syntax showing essential parameters providing the same information.
The slice header contains an aps_id syntax element which refers to the APS NAL unit to be used for decoding the slice. This syntax element is present in the slice header bitstream when any one of the following conditions is true: * scaling_list_enable_flag is true (defined in SPS) which states that QSM are modified * deblocking_filter_in_aps_enabled_flag is true (defined in SPS) which indicates that deblocking filter parameters are coded in the APS.
* adaptive_loop_filter_enabled_flag is true (defined in SPS) which indicates that ALF processing is employed * When sample_adaptive_offset_enabled (defined in SPS) is true and slice_sao_interleaving_flag (defined in slice header) is false, which states that SAO process is enabled and that LCU mode representation is used for SAO parameters.
Deblocking filter parameters may be defined in slice header when inherit_dbl_params_from_aps_flag is set to false. This flag is defined in slice header and allows to define deblocking filter parameters in slice header when an APS is employed and the deblocking filter parameters are not defined in this APS. When alf coef in slice flag (defined in SPS) is set to true, ALE parameters are coded in the slice header otherwise they are coded in APS.
slice_header( ) { Descrip tor first_slice_in_pic_flag u(1) if( first_slice_in_pic_flag = = 0) _________ slice_address u(v) slice_type ue(v) entropy_slice_flag u(1) if( entropy sliceflag) { ________ pic_parameter_set_id ue(v) [. __________________ if(_sample_adaptive_offset_enabled_flag_)_{ ________ slice_sao_interleaving_flag u(1) slice_sample_adaptive_offset_flag u( 1) [. __________________ } __________________ if( scaling_list_enable_flag I I deblocking_filter_in_aps_enabled_flag I I (sample_adaptive_offset_enabled_flag && !slice_sao_interleaving_flag) I I adaptive_loop_filter_enabled_flag) ________ aps_id ue(v) [. .1 __________________ } __________________ [. __________________ if( entropy_slice_flag) { _________ slice_qp_delta se(v) if(_deblocking filtercontrol present_flag_)_{ _________ if(_deblocking_filter_in_aps_enabled_flag) ________ inherit_dbl_params_from_aps_flag u(1) if(_!inherit_dbl_params_from_aps_flag ________ disable_deblocking_filter_flag u(1) if(_!disable_deblocking_filter_flag_)_{ _________ beta_offset_div2 se(v) tc_offset_d iv2 se(v) } __________________ } ________________ } __________________ [. .1 __________________ } __________________ [. *1 ________________ if(_adaptive_loop_filter_enabled_flag_)_{ ________ slice_adaptive_loop_filter_flag u(1) if( slice_adaptive_loop_filter_flag && alf_coef_in_slice_flag ________ alf_param() ________ if( slice adaptive loop filter flag && !alf_coef_in_slice_flag ________ alfcucontrolparam() ________ } __________________ if( seq_loop_filter_across_si ices_enabled_flag && (slice_adaptive_loop_filter_flag I I slice_sample_adaptive_offset_flag II disable_deblocking_filter_flag) _________ slice_loop_filter_across_slices_enabled_flag u(1) if(_tiles_or_entropy_cod ing_sync_idc > 0)_{ ________ num_entry_point_offsets ue(v) if( num_entry_point_ofisets > 0) { ________ offset_len_minusl ue(v) for( i = 0; i < num_entry_point_ofisets; i++ entry_point_offset[ i] u(v) } __________________ } __________________ } __________________
Table 2
A characteristic associated with the current APS implementation in HEVC is limited flexibility and a tendency to introduce redundancies between APS when there are frequent modifications of the filtering parameters during a sequence.
The present invention has been devised to address one or more of the foregoing concerns.
According to a first aspect of the invention there is provided a method for encoding a bitstream representative of video data comprising Encoding an image using at least two different encoding processes, said encoded image resulting from the encoding comprising at least one bitstream portion corresponding to a set of coding units, Obtaining at least one parameter set comprising for at least one of the two different encoding processes a set of characteristics, wherein the at least one bitstream portion corresponding to a set of coding units comprises for each of the at least two different encoding processes when this encoding process is active, an identifier providing, when a parameter set is relevant for providing the characteristics, a reference on the at least one parameter set relevant for providing a set of characteristics to said encoding process.
In another embodiment the reference on the at least one parameter set corresponds to a parameter set index comprised in the parameter set.
In another embodiment the at least one parameter set is stored in a buffer of parameter sets, the position of the parameter set in the buffer depending on the parameter set index value.
In another embodiment when no parameter set is relevant for providing the characteristics associated to an encoding process, then a given value of the identifier indicates that the characteristics are comprised in the at least one bitstream portion. Accordingly, in a related aspect, a method for encoding a bitstream representative of video data may be provided independently, without requiring a parameter set, the characteristics being provided in the bitstream portion, for example in a slice header.
In another embodiment any other identifier value different from the given value, indicates that the characteristics are comprised in a parameter set.
In another embodiment for each of the at least two different encoding processes, a syntax elements indicates if the identifier is comprised in the bitstream portion.
In another embodiment if the identifier is not comprised in the bitstream portion, the characteristics corresponding to the encoding process are comprised in the bitstream portion.
In another embodiment the step of obtaining at least one parameter set comprises encoding a current parameter set In an embodiment the current parameter set comprises characteristics relevant for at least one bistream portion of the encoded image.
In another embodiment when the index of the current parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a replacement process is applied in which the current parameter set replaces the parameter set with the same index in the buffer of parameter sets.
In another embodiment when the index of the current parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a merging process is applied in which the characteristics comprised in the current parameter set replaces the corresponding characteristics stored in the parameter set with the same index in the buffer of parameter sets.
In another embodiment a syntax element comprised in parameter sets indicates which process between the replacement and the merging process has to be applied.
In another embodiment the current parameter set comprises a syntax element indicating if the buffer of parameter sets has to be emptied before storing the current parameter set.
In another embodiment the buffer of parameter sets is emptied periodically.
In another embodiment the emptying period is indicated in a sequence parameter set or a picture parameter set.
In another embodiment the emptying period depends on resynchronisation images.
In another embodiment a single identifier providing a reference on the at least one parameter set relevant for providing a set of characteristics, is used for a plurality of encoding processes of the at least two encoding processes.
In another embodiment said single identifier is associated to a syntax element indicating which encoding processes are considered in the at least one parameter set.
In another embodiment the syntax element associated with said single identifier is encoded with a variable length code depending on probabilities of changes of the characteristics values of each encoding process.
In another embodiment the at least one parameter set comprises a syntax element representing the priority of the parameter set.
In another embodiment the priority depends on the image containing the bitstream portion to which is associated the parameter set.
In another embodiment the priority depends of the type of the encoding processes for which the parameter set is relevant to provide characteristics.
In another embodiment the size of the buffer of parameters sets is fixed to a given size.
In another embodiment the at least two encoding processes comprises, loop filtering processes and a quantization process. As noted above, loop filtering processes may include deblocking filtering and/or sample adative offset.
According to a second aspect of the invention there is provided a method of encoding a bitstream representative of video data comprising: an encoded image of pixels, encoded applying at least two different encoding processes, said encoded image comprising coding units, and at least one parameter set comprising a set of characteristics for each of the at least two different encoding processes applied, wherein, for at least one coding unit on which at least one of the two different encoding processes was applied, the bitstream comprises when a parameter set is necessary for decoding said image, at least one first syntax element representing a reference to one of the at least one parameter set relevant for providing a set of characteristics of at least one of the at least two encoding processes applied and at least one second syntax element related to the at least one first syntax element indicating which of the sets of characteristics comprised in the parameter set represented by the first syntax element have to be considered.
In another embodiment the at least one second syntax element is encoded using a VLC code.
In another embodiment the at least one and second syntax element are stored in a slice header.
In an embodiment, the at least one and second syntax element are stored in a intermediate parameter set associated to the at least one coding unit.
According to a third aspect of the invention there is provided a method of decoding a bitstream representative of video data comprising decoding an encoded image obtained using at least two different encoding processes, said encoded image comprising at least one bitstream portion corresponding to a set of coding units, wherein the at least one bitstream portion corresponding to a set of coding units comprises for each of the at least two different encoding processes when this encoding process is active, an identifier providing, when a parameter set is relevant for providing a set of characteristics for at least one of the two different encoding processes, a reference to at least one parameter set relevant for providing said set of characteristics, the decoding comprising obtaining a set of characteristics for each active encoding process of the at least two encoding processes and, decoding the image in function of the obtained characteristics.
In another embodiment the method comprises obtaining at least one parameter set.
In another embodiment the reference on the at least one parameter set corresponds to a parameter set index comprised in one of the obtained at least one parameter set.
In another embodiment the at least one obtained parameter set is stored in a buffer of parameter sets, the position of the parameter set in the buffer depending on the parameter set index value.
In another embodiment when no parameter set is relevant for providing the characteristics associated to an encoding process, then a given value of the identifier indicates that the characteristics are comprised in the at least one bitstream portion. Accordingly, in a related aspect, a method for decoding a bitstream representative of video data may be provided independently, without requiring a parameter set, the characteristics being provided in the bitstream portion, for example in a slice header.
In another embodiment any other identifier value different from the given value, indicates that the characteristics are comprised in an obtained parameter set.
In another embodiment for each of the at least two different encoding processes, a syntax elements indicates if the identifier is comprised in the bitstream portion.
In another embodiment if the identifier is not comprised in the bitstream portion, the characteristics corresponding to the encoding process are comprised in the bitstream portion.
In another embodiment when a new parameter set is obtained, if the index of the new parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a replacement process is applied in which the current parameter set replaces the parameter set with the same index in the buffer of parameter sets.
In another embodiment when a new parameter set is obtained, if the index of the current parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a merging process is applied in which the characteristics comprised in the current parameter set replaces the corresponding characteristics stored in the parameter set with the same index in the buffer of parameter sets.
In another embodiment a syntax element comprised in parameter sets indicates which process between the replacement and the merging process has to be applied.
In another embodiment the new parameter set comprises a syntax element indicating if the buffer of parameter sets has to be emptied before storing the new parameter set.
In another embodiment the buffer of parameter sets is emptied periodically.
In another embodiment the emptying period is indicated in a sequence parameter set or a picture parameter set.
In another embodiment the emptying period depends on resynchronisation images.
In another embodiment a single identifier providing a reference on the at least one parameter set relevant for providing a set of characteristics, is used for a plurality of encoding processes of the at least two encoding processes.
In another embodiment the single identifier is associated to a syntax element indicating which encoding processes are considered in the at least one parameter set.
In another embodiment the syntax element associated with the single identifier is encoded with a variable length code depending on probabilities of changes of the characteristics values of each encoding process.
In another embodiment the at least one parameter set comprises a syntax element representing the priority of the parameter set.
In another embodiment the size of the buffer of parameters sets is fixed toagivensize.
A yet further aspect of the invention provides an encoding device for encoding a bitstream representative of video data, the device comprising: means for encoding an image to generate an encoded image using at least two different encoding processes, said encoded image comprising at least one bitstream portion corresponding to a set of coding units, means for Obtaining at least one parameter set comprising for at least one of the two different encoding processes a set of characteristics, wherein the at least one bitstream portion corresponding to a set of coding units comprises for each of the at least two different encoding processes when this encoding process is active, an identifier providing, when a parameter set is relevant for providing the characteristics, a reference on the at least one parameter set relevant for providing a set of characteristics to said encoding process.
A yet further aspect of the invention provides an decoding device for decoding a bitstream representative of video data, the device comprising: means for decoding an encoded image obtained using at least two different encoding processes, said encoded image comprising at least one bitstream portion corresponding to a set of coding units, wherein the at least one bitstream portion corresponding to a set of coding units comprises for each of the at least two different encoding processes when this encoding process is active, an identifier providing, when a parameter set is relevant for providing a set of characteristics for at least one of the two different encoding processes, a reference to at least one parameter set relevant for providing said set of characteristics, the decoding means comprising means for obtaining a set of characteristics for each active encoding process of the at least two encoding processes, and means for decoding the bitstream in function of the obtained characteristics.
A yet further aspect of the invention provides a signal carrying an information dataset comprising: an encoded image generated using at least two different encoding processes, said encoded image comprising at least one information dataset portion corresponding to a set of coding units, and at least one parameter set comprising for at least one of the two different encoding processes a set of characteristics, wherein the at least one information dataset portion corresponding to a set of coding units comprises for each of the at least two different encoding processes when this encoding process is active, an identifier providing, when a parameter set is relevant for providing the characteristics, a reference on the at least one parameter set relevant for providing a set of characteristics to said encoding process.
In another embodiment, the reference on the at least one parameter set corresponds to a parameter set index comprised in the parameter set.
In another embodiment, when no parameter set is relevant for providing the characteristics associated to an encoding process, then a given value of the identifier indicates that the characteristics are comprised in the at least one bitstream portion.
In another embodiment, any other identifier value different from the given value, indicates that the characteristics are comprised in a parameter set.
In another embodiment, for each of the at least two different encoding processes, a syntax elements indicates if the identifier is comprised in the information dataset portion.
In another embodiment, if the identifier is not comprised in the bitstream portion, the characteristics corresponding to the encoding process are comprised in the information dataset portion.
In another embodiment, a syntax element comprised in parameter sets indicates which process between a plurality of parameter sets buffering processes has to be applied.
In another embodiment, a single identifier providing a reference on the at least one parameter set relevant for providing a set of characteristics, is used for a plurality of encoding processes of the at least two encoding processes.
In another embodiment, the single identifier is associated to a syntax element indicating which encoding processes are considered in the at least one parameter set.
In another embodiment, the syntax element associated with the single identifier is encoded with a variable length code depending on probabilities of changes of the characteristics values of each encoding process.
In another embodiment, the at least one parameter set comprises a syntax element representing the priority of the parameter set.
At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system". Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.
Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.
Embodiments of the invention will now be described, by way of example only, and with reference to the following drawings in which: Figure 1 graphically illustrate video compression method of the prior art; Figure 2 is a flow chart illustrating a video encoder of the prior art; Figure 3 is a flow chart illustrating a video decoder of the prior art; Figure 4 is a flow chart illustrating the steps of a loop filtering process
of the prior art;
Figure 5 is a flow chart illustrating the steps of a deblocking filter of
the prior art;
Figure 6 is a graphical representation of the principle of the Sample Adaptive Offset (SAO) method; Figure 7 is a graphical representation of the principle of the Adaptive Loop Filter (ALF) method; Figure 8 is a block diagram schematically illustrating a data communication system in which one or more embodiments of the invention may be implemented; Figure 9 is a block diagram illustrating components of a processing device in which one or more embodiments of the invention may be implemented; Figure 10 is a block diagram illustrating a high level view of the decoding process performed by the decoder when applying the proposed APS strategy; Figure 11 is a block diagram illustrating an APS buffer management method according to the invention; Figure 12 is a graphical representation of a hierarchical organisation of a Group of Pictures; Figure 13 is a block diagram illustrating a second APS buffer management method according to the invention; Figure 8 illustrates a data communication system in which one or more embodiments of the invention may be implemented. The data communication system comprises a transmission device, in this case a server 801, which is operable to transmit data packets of a data stream to a receiving device, in this case a client terminal 802, via a data communication network 800. The data communication network 800 may be a Wide Area Network (WAN) or a Local Area Network (LAN). Such a network may be for example a wireless network (Wifi I 802.lla or b or g), an Ethernet network, an Internet network or a mixed network composed of several different networks. In a particular embodiment of the invention the data communication system may be a digital television broadcast system in which the server 801 sends the same data content to multiple clients.
The data stream (bitstream) 804 provided by the server 801 may be composed of multimedia data representing video and audio data. Audio and video data streams may, in some embodiments of the invention, be captured by the server 801 using a microphone and a camera respectively. In some embodiments data streams may be stored on the server 801 or received by the server 801 from another data provider, or generated at the server 801. The server 801 is provided with an encoder for encoding video and audio streams in particular to provide a compressed bitstream for transmission that is a more compact representation of the data presented as input to the encoder.
In order to obtain a better ratio of the quality of transmitted data to quantity of transmitted data, the compression of the video data may be for example in accordance with the HEVC format.
The client 802 receives the transmitted bitstream and decodes the reconstructed bitstream to reproduce video images on a display device and the audio data by a loud speaker.
Although a streaming scenario is considered in the example of Figure 8, it will be appreciated that in some embodiments of the invention the data communication between an encoder and a decoder may be performed using for example a media storage device such as an optical disc.
In one or more embodiments of the invention video data are transmitted in the form of VCL layer NAL units and non VCL NAL units comprising APS NAL units embedding loop filtering parameters for applicationof loop filters to reconstructed pixels of the image to provide loop filtered pixels in a final image and quantization scaling matrix parameters for application of inverse quantization on the residual block samples.
Figure 9 schematically illustrates a processing device 900 configured to implement at least one embodiment of the present invention. The processing device 900 may be a device such as a micro-computer, a workstation or a light portable device. The device 900 comprises a communication bus 913 connected to: -a central processing unit 911, such as a microprocessor, denoted CPU; -a read only memory 907, denoted ROM, for storing computer programs for implementing the invention; -a random access memory 912, denoted RAM, for storing the executable code of the method of embodiments of the invention as well as the registers adapted to record variables and parameters necessary for implementing the method of encoding a sequence of digital images and/or the method of decoding a bitstream according to embodiments of the invention; and -a communication interface 902 connected to a communication network 903 over which digital data to be processed are transmitted or received Optionally, the apparatus 900 may also include the following components: -a data storage means 904 such as a hard disk, for storing computer programs for implementing methods of one or more embodiments of the invention and data used or produced during the implementation of one or more embodiments of the invention; -a disk drive 905 for a disk 906, the disk drive being adapted to read data from the disk 906 or to write data onto said disk; -a screen 909 for displaying data and/or serving as a graphical interface with the user, by means of a keyboard 910 or any other pointing means.
The apparatus 900 can be connected to various peripherals, such as for example a digital camera 920 or a microphone 908, each being connected to an input/output card (not shown) so as to supply multimedia data to the apparatus 900.
The communication bus provides communication and interoperability between the various elements included in the apparatus 900 or connected to it.
The representation of the bus is not limiting and in particular the central processing unit is operable to communicate instructions to any element of the apparatus 900 directly or by means of another element of the apparatus 900.
The disk 906 can be replaced by any information medium such as for example a compact disk (CD-ROM), rewritable or not, a ZIP disk or a memory card and, in general terms, by an information storage means that can be read by a microcomputer or by a microprocessor, integrated or not into the apparatus, possibly removable and adapted to store one or more programs whose execution enables the method of encoding a sequence of digital images and/or the method of decoding a bitstream according to the invention to be implemented.
The executable code may be stored either in read only memory 907, on the hard disk 904 or on a removable digital medium such as for example a disk 906 as described previously. According to a variant, the executable code of the programs can be received by means of the communication network 903, via the interface 902, in order to be stored in one of the storage means of the apparatus 900 before being executed, such as the hard disk 904.
The central processing unit 911 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to the invention, instructions that are stored in one of the aforementioned storage means. On powering up, the program or programs that are stored in a non-volatile memory, for example on the hard disk 904 or in the read only memory 907, are transferred into the random access memory 912, which then contains the executable code of the program or programs, as well as registers for storing the variables and parameters necessary for implementing the invention.
In this embodiment, the apparatus is a programmable apparatus which uses software to implement the invention. However, alternatively, the present invention may be implemented in hardware (for example, in the form of an Application Specific Integrated Circuit or ASIC).
As already mentioned above, a characteristic associated with the current APS implementation is the introduction of redundancies. Indeed, all quantization and loop filtering parameters relevant for a slice should be stored in one APS NAL unit. If only one parameter changes from one slice to another, then a full APS NAL unit has to be generated, even if a majority of parameters are already present in an existing APS NAL unit.
In a first embodiment a new strategy for APS is proposed. This strategy is based on the use of several APS NAL units for decoding a slice. In the following we focus on parameters available in the current APS NAL unit syntax (the Quantization and the 3 loop filtering processes parameters). It is clear that the invention could be extended to other processes that may need a frequent adaptation of their parameters.
A first feature of the new strategy is to use a buffer of APS NAL units called in the following APS buffer. Identical APS buffers are created and maintained by the encoder and the decoder. Each APS NAL unit in this buffer is associated to an identifier aps_id. Note that the creation of an APS buffer on the encoder side is motivated only by the necessary symmetry of behavior of the encoder and the decoder.
A second feature of this new strategy consists in authorizing signaling a plurality of APS NAL units in the slice header. This feature allows associating when necessary an APS NAL unit to each quantization or loop filtering process. As a consequence new APS NAL unit are created only if none of the APS NAL units stored in the APS buffer contains the quantization or loop filtering parameters relevant for a current slice. The frequency of generation of new APS NAL units is therefore reduced. In addition, parameters are not duplicated in several APS NAL units and thus the compression rate is improved.
In figure 10 we represented a high level view of the decoding process performed by the decoder when applying the new APS strategy. Each APS NAL unit received by the decoder is decoded and then stored 1001 in the APS buffer with an index equal to the aps_id syntax element. An update mechanism 1002 of the APS buffer is then applied and will be described below with reference to the figure 11. The slice decoding process can then start by the decoding of information located in the slice header allowing retrieving the quantization and loop filtering parameters.
The Table 3 bellow represents a new slice header syntax allowing to refer to one APS for each quantization or loop filtering process. For each process, an identifier specifies which set of parameters need to be used. Here 4 process identifiers are defined: * scal ing_list_param_id * dbf_param_id * sao_param_id * alf_param_id When one encoding process is disabled the identifier syntax element may be skipped. This permits to avoid coding useless syntax elements and thus improves the compression rate.
Depending on the value of the process identifier, the decoder retrieves 1004 the set of parameters for the concerned process either in slice header or in one APS NAL unit.
If the value of the process identifier is equal to 0, it indicates that the set of parameters is defined in slice NAL unit. For example, if sao_param_id identifier of Table 3 (which is employed for SAO process) is equal to 0, LCU based approach is used for the slice. SAO parameters are interleaved in the encoded data of each LCU.
All values of process identifier higher than 0 represent an APS NAL unit in the APS buffer. The aps_id of the APS NAL unit that contains the parameters of one process is equal to the value of the process identifier minus 1. For example, if the alf_param_id syntax element is equal to 3, the aps_id of the APS NAL unit to be used is equal to 2.
In another embodiment, the identifier is preceded by a flag indicating if this identifier is present or not. If the flag indicates that the identifier is not present, this means that the set of parameters is defined in the slice NAL unit. If the flag indicates that the identifier is present, the identifier with value 0 refers to APS of id equal to 0.
Actually the preferred embodiment mentioned above corresponds to concatenating the flag and the APS identifier into one single identifier whose value 0 indicates that the set of parameters is defined in slice NAL unit. For instance, if it is considered that up to 4 APSs are supported, the codewords in the APS NAL unit for the identifier could be specified as follows: * 0' -identifier=0, the set of parameters is defined in slice NAL unit * 100' -identifier=1, the set of parameters is defined in the APS and refers to APS 0 * 101' -identifier=2, the set of parameters is defined in the APS and refers to APS 1 * 110' -identifier=3, the set of parameters is defined in the APS and refers to APS 2 * 111' -identifier=4, the set of parameters is defined in the APS and refers to APS 3 This mechanism is performed for each of the 4 processes and the decoding of the slice 1005 is done with the parameters retrieved during the processes parameters decoding stage 1004.
As can be seen in table 3 and 4, in order to suppress syntax elements become useless with the new APS strategy, modifications of the APS NAL unit and slice header syntax are proposed. Several syntax elements such as the inherit_dbl_params_from_aps_flag in the slice header (which indicates whether DBF parameters are defined in APS), slice sao interleaving flag in the slice header and aps_sao_interleaving_flag in the APS NAL unit (which both indicate whether LCIJ based representation is used for SAO parameters) and alf coef in slice flag defined in SPS (which specifies if ALF parameters are in slice orAPS NAL unit) are removed.
slice_header( ) { Descriptor first_slice_in_pic_flag u(1) if(_first_slice_in_pic_flag = = 0) ___________ slice_address u(v) slice_type ue(v) entropy_slice_flag u(1) if(_entropy_slice_flag_)_{ __________ pic_parameter_set_id ue(v) [. _____________________ if(_sample_adaptive_offset_enabled_flag_)_{ __________ slice_sample_adaptive_offset_flag u(1) [. _____________________ } ___________________ if(_scaling_list_enable_flag) __________ scaling_list_param_id ue(v) if(deblocking_filter_in_aps_enabled_flag) __________ dbf_ param_id ue(v) if(_sample_adaptive_offset_enabled_flag) __________ sao_ param_id ue(v) if(adaptive_loop filter enabled_flag_) __________ alf_ param_id ue(v) [. ___________________ } _____________________ [. _____________________ if(_entropy_slice_flag_)_{ __________ slice_qp_delta se(v) if(_deblocking_filter control present_flag_)_{ ___________ if( dbf_param_id = = 0 II deblocking filter in aps enabled flag) { __________ d isable_deblocking_filter_flag u( 1) if( !d isable_deblocking_filter_flag_)_{ ___________ beta_offset_d iv2 se(v) tc_offset_d iv2 se(v) } _____________________ } ___________________ } ___________________ [. ___________________ } _____________________ if(_adaptive_loop_filter_enabled_flag_)_{ __________ slice_adaptive_loop_filter_flag u(1) if( slice_adaptive_loop_filter_flag && alf_param_id = = 0 alf_param( __________ if(_slice_adaptive_loop_filter_flag && alf_param_id > 0) __________ alf_cu_control_param() } _____________________ if( seq_loop_filter_across_si ices_enabled_flag && (slice_adaptive_loop_filter_flag I I slice_sample_adaptive_offset_flag I I !d isable_deblocking_filter_flag_) ___________ slice_loop_filter_across_slices_enabled_flag u(1) if(_tiles_or_entropy_coding_sync_ide> 0)_{ __________ num_entry_point_offsets ue(v) if(_num_entry_point_offsets > 0)_{ __________ offset_len_minusl ue(v) for( i = 0; i C num_entry_point_offsets; i++ entry_point_offset[ i] u(v) } ___________________ } _____________________ } _____________________
Table 3
aps_rbsp( ) { Descriptor aps_id ue(v) aps_scaling_list_data_present_flag u(1) if(_aps_scaling_list_data_present_flag _________ scaling_Iist_pararn(_) __________ aps_deblocking_filter_flag u(1) if(aps_deblocking_filter_flag)_{ _________ d isable_deblocking_filter_flag u(1) if(_!disable_deblocking_filter_flag_)_{ __________ beta_offset_d iv2 se(v) tc_offset_d iv2 se(v) } __________________ } ____________________ aps_sample_adaptive_offset_flag u(1) if(_aps_sample_adaptive_offset_flag _________ aps_sao_param() _________ aps_adaptive_loop_filter_flag u(1) if(_aps_adaptive_loop_filter_flag_) _________ alf_param( _________ aps_extension_flag u(1) if(_aps_extension_flag _________ while(_more_rbsp_data(_)) _________ aps_extension_data_flag u(1) rbsp_trailing_bits(_) __________ } __________________
Table 4
Figure 11 is a flow chart representing the update mechanism of the APS buffer evocated in reference 1002 in figure 10. The stages 1101 to 1105 are successively applied to each APS NAL unit.
Firstly, the APS NAL unit is decoded during stage 1101. In particular the aps_id syntax element value is decoded and compared to the APS stored in the APS buffer. To avoid parsing APS previously stored in the buffer at each iteration, the buffer ofAPS is indexed by aps_id. Other ordering is possible (for instance, most used APS NAL units at the start of the buffer to rapidly retrieve the most used APS NAL units).
If the buffer is empty or no APS NAL unit with a same aps_id value is available in the buffer, it is considered that a new APS NAL unit is received ("yes" in test 1102). The APS NAL unit parameters are stored 1105 in the buffer and the decoding process continues with the next NAL unit. If "no" in test 1102, one APS NAL unit with the same aps_id is stored in the APS buffer. In that case, this previous APS NAL unit is updated with new parameters.
In a first embodiment of step 1104, the new APS NAL unit replaces entirely the APS NAL unit with the same aps_id: the previous APS NAL unit is invalidated and removed from the buffer and then, the new APS NAL unit is buffered 1105 at previous APS NAL unit location. This "APS replacement" embodiment provides a simple way to manage the APS update mechanism.
However, this mechanism may be improved in second embodiment which consists in not removing entirely the previous APS NAL unit. This will permit to keep previously coded parameters in the buffer while updating a part of the APS. The principle of this second method is to replace the parameters defined in the new APS NAL unit while keeping previous APS parameters that are not updated in the new APS NAL unit.
For instance, if we consider that the previous APS NAL unit defines QSM, DBF and SAD parameters and that the new APS NAL unit with same aps_id contains only SAC parameters, the update mechanism 1105 consists in generating a virtual APS NAL unit which merges parameters of new APS NAL unit and previous APS NAL unit parameters. The obtained APS NAL unit is stored 1105 in the buffer at the previous APS location.
The APS replacement method has the advantage of its simplicity but the APS merge method is more efficient in terms of amount of data to send: indeed, for the previous example the new APS NAL unit must contain parameters of QSM and DBF equal to the one in previous APS NAL unit when APS replacement is employed. Nevertheless, APS merge mode is less robust to errors since less redundancy is introduced in APS NAL units.
For this reason, in another embodiment, a new syntax element is introduced in APS NAL unit. This element (aps_reset_flag in Table 5) indicates which of the two update mechanisms should be used: when aps_reset_flag is set to false, "merge APS" method is used. On the contrary it aps_reset_flag is equal to true, "APS replacement" method is employed. This permits advantageously to select the appropriate update mechanism method according to the targeted application. In another embodiment, aps_reset_flag are defined in SPS or PPS to select the update mechanism at sequence or picture level.
aps_rbsp( ) { Descript or aps_id ue(v) aps_reset_flag u(1) aps_scaling_list_data_present_flag u( 1) it(_aps_scaling_list_data_present_flag _________ scaling_list_pararn( __________ aps_deblocking_filter_flag u(1) it(aps_deblocking_tilter_flag)_{ __________ d isable_deblocking_filter_flag u(1) if(_disable_deblocking_tilter_flag_)_{ _________ beta_offset_d iv2 se(v) tc_offset_d iv2 se(v) } __________________ } ____________________ aps_sao_interleaving_flag u(1) if(_!aps_sao_interleaving_flag_)_{ __________ aps_sample_adaptive_offset_flag u(1) if(_aps_sample_adaptive_offset_flag) _________ aps_sao_param() _________ } ____________________ aps_adaptive_loop_filter_flag u(1) it(_aps_adaptive_loop_filter_tlag _________ alf_param() _________ aps_extens ion_flag u(1) it(_aps_extension_flag _________ while(_more_rbsp_data(_)) _________ aps_extens ion_data_flag u(1) rbsp_trailing_bits( __________ } __________________
Table 5
The main advantage of the solution proposed in figure 10 is to reduce the redundancies introduced by the current APS implementation. A further improvement focusing on the reduction of bitrate related to APS coding can be obtained by minimizing the number of identifiers to be coded in the slice header.
The problem related to the reduction of the bitrate related to the APS signaling was addressed in the contribution to the HEVC standard JCTVC-H0505On APS partial update", Y. K. Wang, Y. Chen, G. Van Der Auwera, Qualcomm, contribution to the 8 JCT-VC meeting, San José, 1-10 February 2012). This solution consists in gathering some parameter set identifiers such as FF5 identifier syntax element and APS identifier syntax elements in a new intermediate NAL unit called Group Parameter Set (GPS), represented in table 11 below. Each GPS NAL unit contains and is represented by a group_parameter_set_id syntax element that is used as an identifier of the NAL unit (similarly to pps_id or aps_id). Each slice header refers to a GPS NAL unit with a gps_id syntax element defined in the slice header. APS and FF5 identifiers are thus replaced by a single GPS identifier. The following syntax elements appear in the GPS NAL unit syntax: * num_ref_aps_ids defines the number of aps_id coded in the GPS NAL unit.
* ref_aps_id is an APS identifier to be used for defining parameters of the process defined by refapsparamtype.
* ref_aps_param_type is a value defining which process is identified by ref_aps_id. ref_aps_param_type has 4 possible values, each corresponding to QSM, SAO, ALF or DBF.
Creating a new NAL header induces several issues. First, it increases the bitrate allocated to the signalization of this new NAL. Indeed, new NAL units imply new header data to insure their identification by a decoder. In addition the question of transport of these data over a network arises. Indeed, if one network packet is used for transporting a GPS NAL unit, since the size of a GPS is limited, the portion of packet header data comparing to the portion of GPS will be important, which represents a waste of bandwidth. These two limitations are contradictory with the problem the GPS NAL unit is supposed to solve, i.e. reducing the bitrate allocated to parameter set data.
group_parameter_set_rbsp( ) { group_parameter_set_id pps_id num_ref_aps_ids for( i = 0; i < num_ref_aps_ids: i) { ref_aps_id[ i] ref_aps_param_type[ i] gps_extension_flag if( gps_extension_flag) while( more rbsp data( )) gps_extension_data_flag rbsp_trailing_bits()
Table 11: GPS Syntax
A further embodiment addresses these issues. This embodiment avoids the definition of a new NAL unit by including several new syntax elements in the slice header and by providing a same codeword to identify several processes. In this solution, the slice header includes a first syntax element (num_param_id in table 6) that indicates the number of parameters identifiers (with same semantics than in figure 10) that are coded in the bitstream. Each identifier (param id) is associated with an identifier type syntax element that specifies which processes (OSM, DBE, SAO, ALE) have their parameters defined by this identifier.
This method relies on the fact that an identifier may refer to multiple processes at once. Indeed, it is known that OSM parameters are likely to be fixed for several seconds or even for the whole sequence. It is also known that DBE parameters are also likely to be fixed for similar duration. On the contrary, SAO and ALF may vary frequently. Based on these assumptions, the identifier_type value is coded with a variable length code in a first embodiment.
A possible coding of this identifier is proposed in the table 7. In this table, the values of identifier_type have been derived to use shorter code length for group of processes having frequent variation of their parameters. The invention is not limited to the coding order of table 7 for the groups of process and could be modified. In another embodiment, the variable length code is replaced by fix length coding.
It would be clear for skilled persons that the use a common code to identify several processes could have been done in the GPS NAL unit, but would not have allowed avoiding the identified issues.
slice_header( ) { Descrip tor first_slice_in_pic_flag u(1) if( first_slice_in_pic_flag = = 0) _________ slice_address u(v) slice_type ue(v) entropy_slice_flag u(1) if( entropy sliceflag) { ________ pic_parameter_set_id ue(v) [. __________________ if(_sample_adaptive_offset_enabled_flag_)_{ ________ slice_sample_adaptive_offset_flag u(1) [. __________________ } __________________ num_param_id u(v) for (i = 0; i C num_param_id; i++) { ________ param_id ue(v) identifier_type u(v) } ________________ [. .1 __________________ } __________________ [. __________________ if( entropy_slice_flag) { ________ slice_qp_delta se(v) if(_deblocking_filter_control_present_flag_)_{ _________ if( dbf_param_id = = 0 II deblocking_filter_in_aps_enabled_flag)_{ ________ disable_deblocking_filter_flag u(1) if(_disable_deblocking_filter_flag_)_{ _________ beta_offset_div2 se(v) tc_offset_d iv2 se(v) } __________________ } ________________ } __________________ } __________________ [. .1 ________________ if(_adaptive_loop_filter_enabled_flag_)_{ ________ slice_adaptive_loop_filter_flag u(1) if(_slice_adaptive_loop_filter_flag && alf_param_id = = 0) ________ alf_param( ________ if(_slice_adaptive_loop_filter_flag && alf_param_id > 0) ________ alf_cu_control_param( } __________________ if( seq_loop_filter_across_si ices_enabled_flag && (slice_adaptive_loop_filter_flag I I slice_sample_adaptive_offset_flag II disable deblocking filter flag _________ slice_loop_filter_across_slices_enabled_flag u( 1) if(_tiles_or_entropy_cod ing_sync_idc > 0_)_{ ________ num_entry_point_offsets ue(v) if( num_entry_point_offsets > 0) { ________ offset_len_minusl ue(v) for( i = 0; i < num_entry_point_offsets; i++ entry_point_offset[ i] u(v) } __________________ } __________________ } __________________
Table 6
QSM,DBF,SAOand 0
ALF ______________
SAO 10 ALF 110 QSM and DBF 1110 SAOandALF 11110 QSM 111110 DBF 1111110 QSM,DBFandSAO 11111110 QSM,DBFandALF 111111110
Table 7
Until now, in previous embodiments, it was assumed that all APS NAL units have an equal importance. However, it is know that some NAL units are more important than others. For instance SPS and PPS NAL units are particularly important since, for instance in a streaming application, when lost, the unavailability of these NAL units at the decoder side prevent from decoding.
Concerning VCL NAL units, H.264/AVC and HEVC have defined a hierarchy between images induced by temporal prediction. An image that is used as a reference image by other images has a high importance since without this image a decoder cannot decode correctly images depending on this reference image. On the contrary, an image never used as a reference has a low importance and can be removed without impacting the decoding of another image.
In a further embodiment, the notion of importance is introduced in the APS NAL unit in the form of a priority. These priorities could be used in a network application to filter packets (for instance in a router) when bandwidth conditions do not allow to transmit all the NAL units. The basic principle consists in attributing a high priority to APS NAL units that are used for or impact a large number of pictures and low priority to APS NAL units that have less impact on the decoded frames quality. Figure 12 represents a group of pictures (GOP) 10 to 18. Arrows in this figures represents temporal dependencies. For instance, in this figure, image 13 uses as reference pictures 10 and 12. In this hierarchy, APSO associated to image 10 is the most important since all images in the GOP depends on the well decoding of 10. On the contrary, APS4 associated to image 15 to 18 has a low importance, since no image uses these 4 images as reference.
Moreover, the priority of the NAL unit could be set more precisely in function of the type of parameters defined in the APS NAL unit. For instance without QSM parameters, all residuals of the slice unit cannot be decoded.
Decoding error is also very high and error propagation to other images is very huge. Since SAO and ALF are applied after DBF, if DBF parameters are not available, SAO and ALF filtering will be erroneous even if their parameters are correctly received. In addition, it is known that the visual correction obtained by ALF is less important than for SAO. Thus, the priority could be computed by summing priority values for each type of parameters defined in the APS NAL unit. For instance, QSM parameters may have a priority value equal to 4, DBF a value equal to 3, and respectively 2 and 1 for SAD and ALF.
In the preferred embodiment, the priority of APS NAL unit depends on the type of the APS parameters and on the type of the frame (reference or non-reference, INTRA or INTRA) that use the APS NAL unit. The priority of the APS NAL unit is computed with the equation below where Paps is the priority of the APS NAL unit; VP is the priority level of the p process of the K3 set of processes defined in the APS NAL unit; 1aps is the priority level defined in function of the frame type: raps = V * aps \pMaps / Process type ___________ QSM 4 DBF 3 SAO 2 ALE 1
Table 8
Intra reference 4 Inter reference 3 Intra non 2 reference ______________ Inter non 1 reference ______________
Table 9
The Table 8 and 9 indicate the value of V and 1aps When the APS NAL unit is applied to a single frame the 1aps is directly determined from the table. When APS NAL unit is valid for a set of images, the priority level of each image is determined and the final 1aps value of the APS is equal to the highest priority level of the set. In another embodiment 1aps is proportionally computed in function of the size of the set of images.
The advantage of the method above is, for example, in streaming environment when the available bandwidth is not large enough to send the whole video sequence of figure 12. In that case, the encoder may drop APS NAL units in function of their priority level while controlling the visual impact of this drop on the decoding.
All NAL unit headers in HEVC contain a one bit syntax element nal_ref_flag that specifies whether the NAL unit is a reference frame. PPS NAL units have typically a nal_ref_flag equal to 1 in HEVC standard which represent the high priority.
In one embodiment, the priority level R3FJ is indicated in the APS NAL unit syntax.
In another embodiment, the nal ref flag is used to indicate the priority of the APS NAL unit. If the priority level is above a threshold, nal ref flag is set to 1 and below the threshold nal_ref_flag is set to 0. In the preferred embodiment, the threshold value is defined such that APS NAL units used for non-reference frames have a nal_ref_flag equal to 0. However, non-reference frames can use APS that have their nal_ref_flag equal to 1. But the encoder has to take care that reference frames never refer to APS that have their nal_ref_flag equal to 0. These constraints are also mandatory at the decoder side.
The use of an APS buffer supposes that the encoder and the decoder manage symmetrically their own buffer in order to insure that the two buffers are identical when encoding/decoding a same slice. In the embodiment of figure 10, the status of the APS buffer at a given time depends of an update mechanism taking into account all APS NAL units received since the beginning of the sequence. This mechanism may induce an issue when some NAL units are lost during a transmission or when random access to an image of the video sequence is desired. Indeed, in both cases, the updating process applied during encoding cannot be applied by the decoder.
A simple method would consist in transmitting or retransmitting all APS NAL units since the beginning of the sequence, to allow the decoder to perform correctly the APS buffer updating process. However, this method is complex and time consuming if the random access point is far from the beginning of the video sequence.
In a first embodiment, the decoder relies on the APS buffer maximum size to retrieve the active APS NAL units. Indeed, if the size N of the APS buffer is predetermined, the decoder just has to retrieve the N APS NAL units immediately preceding the random access point to ensure that all active APS are available for the decoding process. In this embodiment, the buffer size is indicated in a SPS or a PPS NAL unit by a new syntax element (aps_buffer_size) which indicates the maximum number of successive APS NAL units preceding the random access point NAL unit to reconstitute the APS buffer.
In another embodiment, the buffer of APS is periodically refreshed: all active APS NAL units in the buffer are disabled and the buffer is emptied. In a first embodiment, the reset of the APS buffer is indicated by a new NAL unit type. This new NAL unit type is ABR (Adaptation parameter sets Buffer Refresh). The syntax of the ABR NAL unit is the same as the APS, only the type of NAL unit changes. When decoding this NAL unit, the updating process of stage 1002 is modified as represented in the diagram of figure 13. Two additional modules (1300, 1301) are inserted before the algorithm described with reference to the figure 11. First, NAL unit type is tested (1300). If the APS NAL unit is an ABR the APS buffer is emptied and then the algorithm of figure 11 is applied to the data contained in the ABR NAL unit. On the contrary, if the APS NAL unit is not an ABR, the APS buffer is not modified and the same algorithm applies.
In a preferred embodiment, the APS buffer refresh period is indicated in the SFS or FF5 NAL unit. A new syntax element (named aps buffer period) is inserted in the syntax of SPS or PPS. An example of modification of the SPS is given in table 10. The new syntax element is equal to the number of frames after which the buffer is refreshed or reset. In another embodiment, the APS buffer is reset systematically at each IDR or CDR frame, which is a resynchronization frame. In that case, the aps_buffer_period syntax element is conditionally added when the aps_buffer_reset_at_idr_cdr_flag flag is set to false which indicates that APS buffer is not refreshed at each IDR frame.
seq_parameter_set_rbsp( ) { Descrip tor [. . ________________ aps_buffer_reset_at_idr_cdr_flag u(1) if(aps_buffer_reset_at_idr_cdr_flag = = 0) ________ aps_buffer_period ue(v) [. .
Table 10
Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.
For example, while the previous embodiments were described independently, all embodiments could be combined in a global APS strategy.
For instance, the new APS and slice syntax with the APS buffer management strategy can be combined with the definition of APS NAL unit priority, the use of a common code for identifying several processes and/or the specific APS buffer management designed for the random access or the management of eventual transmission errors.
Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the invention, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.
In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.
Claims (63)
- CLAIMS1. A method for encoding a bitstream representative of video data comprising Encoding an image using at least two different encoding processes, to produce at least one bitstream portion corresponding to a set of coding units of the image, Obtaining at least one parameter set comprising for at least one of the two different encoding processes an associated set of characteristics, wherein, in a case where an obtained parameter set is relevant for providing a set of characteristics associated with at least one of said used encoding processes, the at least one bitstream portion is provided with an identifier providing a reference to the relevant parameter set.
- 2. A method according to claim 1, wherein the reference to the relevant parameter set corresponds to a parameter set index comprised in the parameter set.
- 3. A method according to claim 2 wherein the relevant parameter set is stored in a buffer of parameter sets, the position of the parameter set in the buffer depending on the parameter set index value.
- 4. A method according to any one of the previous claims, wherein when no parameter set is relevant for providing the characteristics associated to an encoding process, a given value of the identifier indicates that the characteristics are provided in the at least one bitstream portion.
- 5. A method according to claim 4, wherein an identifier value different from the given value indicates that the characteristics are comprised in a parameter set.
- 6. A method according to anyone of claim 1 to 3 wherein, for each of the at least two different encoding processes, a syntax element indicates if the identifier is provided in the bitstream portion.
- 7. A method according to claim 6 wherein, if the identifier is not comprised in the bitstream portion, the characteristics corresponding to the encoding process are provided in the bitstream portion.
- 8. A method according to any preceding claim wherein, the step of obtaining at least one parameter set comprises encoding a current parameter set.
- 9 A method according to claim 8 wherein the current parameter set comprises characteristics relevant for at least one bistream portion of the encoded image.
- 10. A method according to claims 8 or 9 as dependent on claim 3 wherein, when the index of the current parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a replacement process is applied in which the current parameter set replaces the parameter set with the same index in the buffer of parameter sets.
- 11. A method according to claim 8 or 9 as dependent on claim 3, wherein when the index of the current parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a merging process is applied in which the characteristics comprised in the current parameter set replaces the corresponding characteristics stored in the parameter set with the same index in the buffer of parameter sets.
- 12. A method according to claim 10 or 11 wherein, a syntax element comprised in parameter sets indicates which process between the replacement and the merging process is to be applied.
- 13. A method according to claims 8 or 9 as dependent on claim 3 wherein, the current parameter set comprises a syntax element indicating if the buffer of parameter sets have to be emptied before storing the current parameter set.
- 14. A method according to claim 3 wherein the buffer of parameter sets is emptied periodically.
- 15. A method according to claim 14 wherein the emptying period is indicated in a sequence parametersetora picture parameter set.
- 16. A method according to claims 14 or 15 wherein the emptying period depends on resynchronisation images.
- 17. A method according to any preceding claim wherein a single identifier providing a reference to the at least one parameter set relevant for providing a set of characteristics, is used for a plurality of encoding processes of the at least two encoding processes.
- 18. A method according to claim 17 wherein, the single identifier is associated to a syntax element indicating which encoding processes are considered in the at least one parameter set.
- 19. A method according to claim 18 wherein, the syntax element associated with the single identifier is encoded with a variable length code depending on probabilities of changes of the characteristics values of each encoding process.
- 20. A method according to any preceding claim wherein the at least one parameter set comprises a syntax element representing the priority of the parameter set.
- 21. A method according to claim 20 wherein the priority depends on the image containing the bitstream portion to which is associated the parameter set.
- 22. A method according to claim 20 or 21 wherein the priority depends of the type of the encoding processes for which the parameter set is relevant to provide characteristics.
- 23. A method according to any one of claims 3 to 22 wherein, the size of the buffer of parameters sets is fixed to a given size.
- 24. A method according to any preceding claim wherein the at least two encoding processes comprises, loop filtering processes and a quantization process.
- 25. A method of decoding a bitstream representative of video data comprising decoding an encoded image obtained using at least two different encoding processes, the encoded image comprising at least one bitstream portion corresponding to a set of coding units of the image, wherein, in a case where a parameter set comprising a set of characteristics of at least one of said used encoding processes is relevant, the at least one bitstream portion comprises an identifier providing a reference to the relevant parameter set, the decoding comprising obtaining a set of characteristics tar at least one used encoding process of the at least two encoding processes and, decoding the image using the obtained characteristics.
- 26. A method according to claim 25 wherein the method comprises obtaining at least one parameter set.
- 27. A method according to claim 26 wherein the reference to the relevant parameter set corresponds to a parameter set index comprised in one of the obtained at least one parameter sets.
- 28. A method according to claim 27 wherein the at least one obtained parameter set is stored in a buffer of parameter sets, the position of the parameter set in the buffer depending on the parameter set index value.
- 29. A method according to any one of claims 25 to 28, wherein when no parameter set is relevant for providing the characteristics associated to an encoding process, then a given value of the identifier indicates that the characteristics are comprised in the at least one bitstream portion.
- 30. A method according to claim 29 wherein, any other identifier value different from the given value, indicates that the characteristics are comprised in an obtained parameter set.
- 31. A method according to any one of claims 25 to 30 wherein, for each of the at least two different encoding processes, a syntax element indicates if the identifier is comprised in the bitstream portion.
- 32. A method according to claim 31 wherein, if the identifier is not comprised in the bitstream portion, the characteristics corresponding to the encoding process are comprised in the bitstream portion.
- 33. A method according to any one of claims 28 to 32 wherein, when a new parameter set is obtained, if the index of the new parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a replacement process is applied in which the current parameter set replaces the parameter set with the same index in the buffer of parameter sets.
- 34. A method according to any one of claims 28 to 32, wherein when a new parameter set is obtained, if the index of the current parameter set corresponds to the index of a parameter set stored in the buffer of parameter sets, a merging process is applied in which the characteristics comprised in the current parameter set replaces the corresponding characteristics stored in the parameter set with the same index in the buffer of parameter sets.
- 35. A method according to claim 33 or 34 wherein, a syntax element comprised in parameter sets indicates which process between the replacement and the merging process is to be applied.
- 36. A method according to claim 33 or 34 as dependent on claim 28 wherein, the new parameter set comprises a syntax element indicating if the buffer of parameter sets have to be emptied before storing the new parameter set.
- 37. A method according to claim 28 wherein the buffer of parameter sets is emptied periodically.
- 38. A method according to claim 37 wherein the emptying period is indicated in a sequence parameter set or a picture parameter set.
- 39. A method according to claims 37 or 38 wherein the emptying period depends on resynchronisation images.
- 40. A method according to any one of claims 25 to 39 wherein a single identifier providing a reference on the at least one parameter set relevant for providing a set of characteristics, is used for a plurality of encoding processes of the at least two encoding processes.
- 41. A method according to claim 40 wherein, the single identifier is associated to a syntax element indicating which encoding processes are considered in the at least one parameter set.
- 42. A method according to claim 41 wherein, the syntax element associated with the single identifier is encoded with a variable length code depending on probabilities of changes of the characteristics values of each encoding process.
- 43. A method according to any one of claims 25 to 42 wherein the at least one parameter set comprises a syntax element representing the priority of the parameter set.
- 44. A method according to any one of claims 28 to 43 wherein, the size of the buffer of parameters sets is fixed to a given size.
- 45. A device for encoding a bitstream representative of video data, the device comprising: means for encoding an image using at least two different encoding processes, to produce at least one bitstream portion corresponding to a set of coding units of the image, means for obtaining at least one parameter set comprising for at least one of the two different encoding processes an associated set of characteristics, wherein, in a case where an obtained parameter set is relevant for providing a set of characteristics associated with at least one of said used encoding processes, the at least one bitstream portion is provided with an identifier providing, a reference to the relevant parameter set.
- 46. An encoding device according to claim 45 adapted to perform a method according to any one of claims 2 to 24.
- 47. A device for decoding a bitstream representative of video data, the device comprising means for decoding an encoded image obtained using at least two different encoding processes, said encoded image comprising at least one bitstream portion corresponding to a set of coding units of the image, wherein in a case where a parameter set comprising a set of characteristics of at least one of said used encoding processes is relevant, the at least one bitstream portion comprises an identifier providing a reference to the relevant parameter set, the decoding means comprising means for obtaining a set of characteristics for at least one used encoding process of the at least two encoding processes, and means for decoding the bitstream using the obtained characteristics.
- 48. A decoding device according to claim 47 adapted to perform a method according to any one of claims 25 to 44.
- 49. A signal carrying an information dataset comprising an encoded image generated using at least two different encoding processes, said encoded image comprising at least one information dataset portion corresponding to a set of coding units, and at least one parameter set comprising for at least one of the two different encoding processes a set of characteristics, wherein the at least one information dataset portion corresponding to a set of coding units comprises for each of the at least two different encoding processes when this encoding process is active, an identifier providing, when a parameter set is relevant for providing the characteristics, a reference to the at least one parameter set relevant for providing a set of characteristics to said encoding process.
- 50. A signal according to claim 49, wherein the reference on the at least one parameter set corresponds to a parameter set index comprised in the parameter set.
- 51. A signal according to claim 49 or 50, wherein when no parameter set is relevant for providing the characteristics associated to an encoding process, then a given value of the identifier indicates that the characteristics are comprised in the at least one bitstream portion.
- 52. A signal according to claim 51 wherein, any other identifier value different from the given value, indicates that the characteristics are comprised in a parameter set.
- 53. A signal according to claim 49 or 50 wherein, for each of the at least two different encoding processes, a syntax elements indicates if the identifier is comprised in the information dataset portion.
- 54. A signal according to claim 53 wherein, if the identifier is not comprised in the information dataset portion, the characteristics corresponding to the encoding process are comprised in the information dataset portion.
- 55. A signal according to any one of claims 49 to 54 wherein, a syntax element comprised in parameter sets indicates which process between a plurality of parameter sets buffering processes has to be applied.
- 56. A signal according to any of the preceding claims 49 to 55 wherein a single identifier providing a reference to the at least one parameter set relevant for providing a set of characteristics, is used for a plurality of encoding processes of the at least two encoding processes.
- 57. A signal according to claim 56 wherein, the single identifier is associated to a syntax element indicating which encoding processes are considered in the at least one parameter set.
- 58. A signal according to claim 57 wherein, the syntax element associated with the single identifier is encoded with a variable length code depending on probabilities of changes of the characteristics values of each encoding process.
- 59. A signal according to any preceding claim from claim 49 to 58 wherein the at least one parameter set comprises a syntax element representing the priority of the parameter set.
- 60. A computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method according to any one of claims 1 to 44 when loaded into and executed by the programmable apparatus.
- 61. A computer-readable storage medium storing instructions of a computer program for implementing a method, according to any one of claims 1 to 44.
- 62. An encoding method, device or program substantially as hereinbefore described with reference to any one of Figures 8 to 13.
- 63. A decoding method, device or program substantially as hereinbefore described with reference to any one of Figures 8 to 13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB1206571.0A GB2501125B (en) | 2012-04-13 | 2012-04-13 | Method and device for providing adaptation parameters to a decoder |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB1206571.0A GB2501125B (en) | 2012-04-13 | 2012-04-13 | Method and device for providing adaptation parameters to a decoder |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| GB201206571D0 GB201206571D0 (en) | 2012-05-30 |
| GB2501125A true GB2501125A (en) | 2013-10-16 |
| GB2501125B GB2501125B (en) | 2015-08-19 |
Family
ID=46209058
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB1206571.0A Expired - Fee Related GB2501125B (en) | 2012-04-13 | 2012-04-13 | Method and device for providing adaptation parameters to a decoder |
Country Status (1)
| Country | Link |
|---|---|
| GB (1) | GB2501125B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021123357A1 (en) * | 2019-12-20 | 2021-06-24 | Canon Kabushiki Kaisha | High level syntax for video coding and decoding |
| CN113424543A (en) * | 2019-02-01 | 2021-09-21 | 北京字节跳动网络技术有限公司 | Signaling loop shaping information using parameter sets |
| CN113711606A (en) * | 2019-04-15 | 2021-11-26 | Lg 电子株式会社 | Video or image compilation based on signaling of zoom list data |
| CN113796079A (en) * | 2019-04-15 | 2021-12-14 | Lg 电子株式会社 | Video or image compilation based on scaling list parameters |
| CN113940068A (en) * | 2019-06-11 | 2022-01-14 | Lg 电子株式会社 | Video or image compilation based on zoom list parameters |
| CN114270852A (en) * | 2019-07-08 | 2022-04-01 | Lg电子株式会社 | Video or image coding based on signaling of zoom list data |
| CN114365496A (en) * | 2019-07-08 | 2022-04-15 | Lg电子株式会社 | Image or video encoding based on zoom list data |
| CN114365497A (en) * | 2019-07-08 | 2022-04-15 | Lg电子株式会社 | Video or image coding based on zoom list parameters |
| US11368684B2 (en) * | 2019-04-23 | 2022-06-21 | Qualcomm Incorporated | Adaptation parameter sets (APS) for adaptive loop filter (ALF) parameters |
| US12063362B2 (en) | 2019-03-23 | 2024-08-13 | Beijing Bytedance Network Technology Co., Ltd | Restrictions on adaptive-loop filtering parameter sets |
| US12238305B2 (en) | 2019-03-14 | 2025-02-25 | Beijing Bytedance Network Technology Co., Ltd. | Signaling and syntax for in-loop reshaping information |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| SI3603074T1 (en) * | 2017-03-20 | 2025-01-31 | Ge Video Compression, Llc | Advanced video data stream extraction and multi-resolution video transmission |
| CN113841397B (en) * | 2019-05-15 | 2024-08-16 | 现代自动车株式会社 | Image decoding method |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012109990A1 (en) * | 2011-02-16 | 2012-08-23 | Mediatek Inc. | Method and apparatus for slice common information sharing |
-
2012
- 2012-04-13 GB GB1206571.0A patent/GB2501125B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012109990A1 (en) * | 2011-02-16 | 2012-08-23 | Mediatek Inc. | Method and apparatus for slice common information sharing |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113424543A (en) * | 2019-02-01 | 2021-09-21 | 北京字节跳动网络技术有限公司 | Signaling loop shaping information using parameter sets |
| US12096021B2 (en) | 2019-02-01 | 2024-09-17 | Beijing Bytedance Network Technology Co., Ltd. | Signaling of in-loop reshaping information using parameter sets |
| CN113424543B (en) * | 2019-02-01 | 2024-03-15 | 北京字节跳动网络技术有限公司 | Use parameter set signaling to notify loop shaping information |
| US12238305B2 (en) | 2019-03-14 | 2025-02-25 | Beijing Bytedance Network Technology Co., Ltd. | Signaling and syntax for in-loop reshaping information |
| US12063362B2 (en) | 2019-03-23 | 2024-08-13 | Beijing Bytedance Network Technology Co., Ltd | Restrictions on adaptive-loop filtering parameter sets |
| CN113711606A (en) * | 2019-04-15 | 2021-11-26 | Lg 电子株式会社 | Video or image compilation based on signaling of zoom list data |
| CN113796079A (en) * | 2019-04-15 | 2021-12-14 | Lg 电子株式会社 | Video or image compilation based on scaling list parameters |
| CN113711606B (en) * | 2019-04-15 | 2025-04-01 | Lg电子株式会社 | Video or image compilation based on signaling of zoom list data |
| US12113960B2 (en) | 2019-04-15 | 2024-10-08 | Lg Electronics Inc. | Scaling list parameter-based video or image coding |
| US11368684B2 (en) * | 2019-04-23 | 2022-06-21 | Qualcomm Incorporated | Adaptation parameter sets (APS) for adaptive loop filter (ALF) parameters |
| CN113940068A (en) * | 2019-06-11 | 2022-01-14 | Lg 电子株式会社 | Video or image compilation based on zoom list parameters |
| US12262033B2 (en) | 2019-06-11 | 2025-03-25 | Lg Electronics Inc. | Scaling list parameter-based video or image coding |
| CN114270852A (en) * | 2019-07-08 | 2022-04-01 | Lg电子株式会社 | Video or image coding based on signaling of zoom list data |
| US12081802B2 (en) | 2019-07-08 | 2024-09-03 | Lg Electronics Inc. | Video or image coding based on signaling of scaling list data |
| US12113992B2 (en) | 2019-07-08 | 2024-10-08 | Lg Electronics Inc. | Scaling list data-based image or video coding |
| US12225190B2 (en) | 2019-07-08 | 2025-02-11 | Lg Electronics Inc. | Scaling list parameter-based video or image coding |
| CN114365497A (en) * | 2019-07-08 | 2022-04-15 | Lg电子株式会社 | Video or image coding based on zoom list parameters |
| CN114365496A (en) * | 2019-07-08 | 2022-04-15 | Lg电子株式会社 | Image or video encoding based on zoom list data |
| US20240040154A1 (en) * | 2019-12-20 | 2024-02-01 | Canon Kabushiki Kaisha | High level syntax for video coding and decoding |
| TWI809336B (en) * | 2019-12-20 | 2023-07-21 | 日商佳能股份有限公司 | High level syntax for video coding and decoding |
| US12238340B2 (en) * | 2019-12-20 | 2025-02-25 | Canon Kabushiki Kaisha | High level syntax for video coding and decoding |
| WO2021123357A1 (en) * | 2019-12-20 | 2021-06-24 | Canon Kabushiki Kaisha | High level syntax for video coding and decoding |
Also Published As
| Publication number | Publication date |
|---|---|
| GB201206571D0 (en) | 2012-05-30 |
| GB2501125B (en) | 2015-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI827609B (en) | Block-based adaptive loop filter (alf) design and signaling | |
| GB2501125A (en) | Providing adaptation parameters to a decoder by including an identifier to a relevant characteristic set in a bit stream portion. | |
| EP2735149B1 (en) | Adaptation parameter sets for video coding | |
| EP3058744B1 (en) | Device and method for scalable coding of video information | |
| CN107113422B (en) | A computer system for reference picture management for video encoding and decoding | |
| JP2022525470A (en) | Block-based adaptive loop filter (ALF) with adaptive parameter set (APS) in video coding | |
| CA2952497C (en) | Bitstream conformance constraints in scalable video coding | |
| EP3090551B1 (en) | Support of base layer of a different codec in multi-layer video coding | |
| US10264272B2 (en) | Device and method for scalable coding of video information | |
| US20150103887A1 (en) | Device and method for scalable coding of video information | |
| TW201313033A (en) | Buffer prediction data in video writing code | |
| WO2015103246A1 (en) | Method for coding a reference picture set (rps) in multi-layer coding | |
| GB2498225A (en) | Encoding and Decoding Information Representing Prediction Modes | |
| WO2023202956A1 (en) | Video coding and decoding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PCNP | Patent ceased through non-payment of renewal fee |
Effective date: 20240413 |