HK1244375B - Method for encoding and decoding images, encoding and decoding device, and computer program - Google Patents
Method for encoding and decoding images, encoding and decoding device, and computer program Download PDFInfo
- Publication number
- HK1244375B HK1244375B HK18102652.5A HK18102652A HK1244375B HK 1244375 B HK1244375 B HK 1244375B HK 18102652 A HK18102652 A HK 18102652A HK 1244375 B HK1244375 B HK 1244375B
- Authority
- HK
- Hong Kong
- Prior art keywords
- block
- subset
- decoding
- encoding
- blocks
- Prior art date
Links
Description
本申请是申请号为201280031335.9,申请日为2012年6月20日,题为“用于对图像编码和解码的方法、编码和解码设备以及相应的计算机程序”的中国发明专利申请的分案申请。This application is a divisional application of the Chinese invention patent application with application number 201280031335.9, application date June 20, 2012, and entitled "Method for encoding and decoding images, encoding and decoding devices and corresponding computer programs".
技术领域Technical Field
本发明一般属于图像处理的领域,且更准确地属于数字图像和数字图像序列的编码和解码。The present invention generally belongs to the field of image processing, and more precisely to the encoding and decoding of digital images and digital image sequences.
本发明由此可特别应用于在目前的数字视频编码器(MPEG,H.264等)或未来的视频编码器(ITU-T/VCEG(H.265)或ISO/MPEG(HVC))中实现的视频编码。The invention is thus particularly applicable to video coding implemented in current digital video encoders (MPEG, H.264, etc.) or future video encoders (ITU-T/VCEG (H.265) or ISO/MPEG (HVC)).
背景技术Background Art
目前的视频编码器(MPEG、H264等)使用视频序列的逐块表示。图像被分为宏块,每个宏块本身可以被分为块,并且每个块或宏块通过图像内或图像间预测来编码。于是,特定的图像通过空间预测(内预测)来编码,而其他图像在相对于一个或多个被编码-解码的参考图像的时间预测(间预测)、在本领域技术人员已知的运动补偿的帮助下进行编码。此外,针对每个块,可以对与原始块减去预测相对应的残留块进行编码。该块的系数可以在变换之后被量化,然后被熵编码器编码。Current video encoders (MPEG, H264, etc.) use a block-by-block representation of video sequences. Images are divided into macroblocks, each macroblock itself can be divided into blocks, and each block or macroblock is encoded using intra-image or inter-image prediction. Thus, certain images are encoded using spatial prediction (intra-prediction), while other images are encoded using temporal prediction (inter-prediction) relative to one or more coded-decoded reference images, with the aid of motion compensation known to those skilled in the art. Furthermore, for each block, a residual block corresponding to the original block minus the prediction can be encoded. The coefficients of this block can be quantized after transformation and then encoded by an entropy encoder.
内预测和间预测需要之前已经被编码和解码的特定块可用,从而在解码器上或在编码器上被用于预测当前块。图1示出了这样的预测编码的示例性例子,其中,图像IN被分为块,该图像的当前块MBi正在关于预订数量的之前被编码和解码的块MBr1、MBr2和MBr3进行预测编码,例如如阴影箭头所示。上述三个块特别包含紧接当前块MBi左侧的块MBr1,以及分别仅紧接当前块MBi上方和右上方的两个块MBr2和MBr3。Intra-prediction and inter-prediction require the availability of certain previously coded and decoded blocks, which are then used to predict the current block at the decoder or encoder. FIG1 shows an exemplary embodiment of such predictive coding, wherein an image IN is divided into blocks, and a current block MBi of the image is predictively coded with respect to a predetermined number of previously coded and decoded blocks MBr1 , MBr2 , and MBr3 , as indicated by the shaded arrows. These three blocks specifically include block MBr1 , which is located immediately to the left of the current block MBi , and two blocks MBr2 and MBr3, which are located immediately above and immediately to the right of the current block MBi , respectively.
熵编码器在这里更为感兴趣。熵编码器以其到达的顺序来编码信息。典型地实现“光栅扫描”类型的块的逐行遍历,如图1所示通过引用PRS,从图像左上角的块开始。对于每个块,用来表示块所必须的各个信息项(块的类型、预测模式、残留系数等)被顺序分发到熵编码器。The entropy encoder is of greater interest here. It encodes information in the order in which it arrives. This is typically done by a "raster scan" style of block traversal, starting with the block in the upper left corner of the image, as shown in Figure 1, using the PRS. For each block, the various pieces of information necessary to represent the block (block type, prediction mode, residual coefficients, etc.) are sequentially distributed to the entropy encoder.
已知在AVC压缩标准(也被称为ISO-MPEG4第10部分和ITU-T H.264)中引入的被称为“CABAC”(上下文适应二进制算术编码器)的足够复杂的有效的算术编码器。A sufficiently complex and efficient arithmetic coder called "CABAC" (Context-Adaptive Binary Arithmetic Coder) is known which was introduced in the AVC compression standard (also known as ISO-MPEG4 Part 10 and ITU-T H.264).
熵编码器实现各个概念:The entropy encoder implements various concepts:
-算术编码:编码器例如初始在文档J.Rissanen and G.G.Langdon Jr,“Universal modeling and coding,”IEEE Trans.Inform.Theory,vol.IT-27,pp.12-23,Jan.1981中描述的编码器使用符号出现概率来对该符号进行编码;Arithmetic coding: an encoder such as that originally described in the document J. Rissanen and G. G. Langdon Jr, “Universal modeling and coding,” IEEE Trans. Inform. Theory, vol. IT-27, pp. 12-23, Jan. 1981, encodes a symbol using the probability of its occurrence;
-上下文适应:在这里,这包括适应要编码的符号的出现概率。一方面,快速实现学习。另一方面,依赖于之前被编码的信息的状态,特定的上下文被用于编码。对于每个上下文,存在固有的符号出现概率与之对应。例如,上下文对应于根据给定配置来编码的符号的类型(残留系数的表示、编码模式的信号等),或者邻居的状态(例如在邻居中选择的“内”模式的数量等)。Context Adaptation: This involves adapting the probability of occurrence of the symbols to be encoded. This allows for rapid learning. Furthermore, the specific context used for encoding depends on the state of the previously encoded information. Each context has an inherent probability of occurrence associated with it. For example, the context corresponds to the type of symbol to be encoded according to a given configuration (representation of residual coefficients, signaling a coding mode, etc.) or the state of the neighborhood (e.g., the number of "inner" modes selected within the neighborhood).
-二进制化:实现要编码的符号的比特序列的成形。随后,这些各个比特被相继分发到二进制熵编码器。- Binarization: This is the shaping of the bit sequence of the symbol to be coded. These individual bits are then successively distributed to a binary entropy coder.
于是,针对使用的每个上下文,该熵编码器实现一种系统,针对所考虑的上下文来快速学习关于之前被编码的符号的概率。该学习基于对这些符号进行编码的顺序。典型地,根据如上所述的“光栅扫描”类型的顺序来遍历图像。Thus, for each context used, the entropy encoder implements a system that quickly learns the probabilities of previously encoded symbols for the context under consideration. This learning is based on the order in which these symbols are encoded. Typically, the image is traversed according to a "raster scan" type of order as described above.
在可以是0或1的给定符号b的编码期间,通过下列方式来更新当前块MBi的该符号的出现概率pi的学习:During the coding of a given symbol b, which can be 0 or 1, the learning of the probability of occurrence pi of this symbol for the current block MB i is updated in the following way:
其中,α是预定值例如0.95,pi-1是在该符号最后出现时计算的符号出现概率。Here, α is a predetermined value such as 0.95, and p i-1 is the probability of occurrence of the symbol calculated when the symbol appears last.
图1示出了这样的熵编码的示例性例子,其中,图像IN的当前块MBi被熵编码。在块MBi的熵编码开始时,所使用的符号出现概率是之前被编码和解码的块的编码之后获得的概率,根据上述“光栅扫描”类型的块的逐行遍历,该块正好在当前块MBi的前面。仅为了图的清楚,在图1中通过细箭头针对特定的块来表示这样的基于块和块的依赖性的学习。FIG1 shows an exemplary embodiment of such entropy coding, in which a current block MB i of an image I N is entropy coded. At the start of entropy coding of block MB i , the symbol occurrence probabilities used are those obtained after coding the previously coded and decoded block that immediately precedes the current block MB i , according to the aforementioned "raster scan" type of block traversal. For the sake of clarity of the figure only, such block-by-block dependency learning is indicated in FIG1 by thin arrows for specific blocks.
该类型的熵编码的缺点在于,考虑到块的“光栅扫描”遍历,在对位于一行开始的符号进行编码时,使用的概率主要对应于在前一行结束位置的符号所看到的那些概率。现在,考虑到符号概率的可能的空间变化(例如,对于和运动信息项相关的符号,位于图像右侧部分的运动可以和在左侧部分看到的不同,并且因此对于随后的局部概率来说也是类似的),可以看到概率的局部一致性的缺失,由此可能增加编码期间的效率损失。A disadvantage of this type of entropy coding is that, taking into account the "raster scan" traversal of the blocks, when coding a symbol located at the beginning of a line, the probabilities used mainly correspond to those seen for the symbol at the end of the previous line. Now, taking into account possible spatial variations in the symbol probabilities (for example, for a symbol associated with a motion information item, the motion located in the right part of the image can be different from that seen in the left part, and therefore similar for the subsequent local probabilities), a lack of local consistency in the probabilities can be observed, which can increase the efficiency losses during coding.
为了限制该现象,已经提出了对块的遍历顺序的调整,目标是确保更好的局部一致性,但编码和解码仍保持顺序。To limit this phenomenon, adjustments to the order in which blocks are traversed have been proposed, with the goal of ensuring better local consistency but still preserving sequential encoding and decoding.
该类型的熵编码器还有另一个缺点。确实,符号的编码和解码依赖于就此学习的概率,符号的解码可以仅以与在编码期间使用的顺序相同的顺序来实现。典型地,于是解码可以只是顺序的,由此阻止若干个符号的并行解码(例如从多核架构中受益)。This type of entropy coder has another drawback. Indeed, since the encoding and decoding of symbols depends on probabilities learned in this way, the decoding of symbols can only be performed in the same order as used during encoding. Typically, decoding can then only be sequential, thus preventing the parallel decoding of several symbols (which would benefit from multi-core architectures, for example).
文档Thomas Wiegand,Gary J.Sullivan,Gisle Bjontegaard,and AjayLuthra,″Overview of the H.264/AVC Video Coding Standard″,IEEE Transactions onCircuits and Systems for Video Technology,Vol.13,No.7,pp.560-576,July 2003还指出,CABAC熵编码器具有将非整数数量的比特分配给要编码的当前字母表的每个符号的特殊特征,这对于大于0.5的符号出现概率是有利的。特别地,CABAC编码器等待直到它已经读取了若干个符号,然后将预定数量的比特分配给读取的该符号集,编码器将该比特写到要发送到解码器的压缩流中。这样的规定由此使其可能使若干个符号上的比特“交互作用”,并对分数数量的比特上的符号进行编码,该数量反映了与通过符号实际发送的信息更接近的信息。与读取的符号关联的其他比特未在压缩流中发送,而是保持等候,等待被分配给CABAC编码器读取的一个或多个新的符号,使其可能再次使这些其他比特交互作用。通过已知的方式,熵编码器在给定的时刻“清空”这些未发送的比特。除非另外说明,在所述给定的时刻,编码器提取还未被发送的比特,并将它们写入到去往解码器的压缩流中。该清空例如在已经读取要编码的最后一个符号的时刻进行,以确保压缩流确实包含所有比特,该比特将允许解码器对字母表中的所有符号进行解码。通过更一般的方式,作为专用于给定编码器/解码器的性能和功能的函数来确定进行清空的时刻。The document Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra, "Overview of the H.264/AVC Video Coding Standard," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, July 2003, further states that the CABAC entropy encoder has the special feature of allocating a non-integer number of bits to each symbol of the current alphabet to be encoded, which is advantageous for symbol occurrence probabilities greater than 0.5. Specifically, the CABAC encoder waits until it has read a number of symbols before allocating a predetermined number of bits to the set of symbols read, which the encoder writes into the compressed stream to be sent to the decoder. This provision makes it possible to "interact" the bits on several symbols and encode a symbol on a fractional number of bits that reflects information that is closer to the information actually transmitted by the symbol. The other bits associated with the symbol read are not sent in the compressed stream, but remain waiting, waiting to be assigned to one or more new symbols to be read by the CABAC encoder, making it possible to interact with these other bits again. In a known manner, the entropy encoder "clears" these unsent bits at a given moment. Unless otherwise specified, at this given moment, the encoder extracts the bits that have not yet been sent and writes them to the compressed stream going to the decoder. This clearing is performed, for example, at the moment when the last symbol to be encoded has been read, to ensure that the compressed stream does contain all the bits that will allow the decoder to decode all the symbols in the alphabet. In a more general way, the moment when the clearing is performed is determined as a function of the performance and functionality specific to a given encoder/decoder.
在2011年4月15号的互联网地址http://research.microsoft.com/en-us/um/people/jinl/paper_2002/msri_jpeg.htm上可用的文档描述了一种对符合JPEG 2000压缩标准的静态图像进行编码的方法。该静态图像经过离散小波变换,然后被量化,由此使其可能获得量化的小波系数,量化索引分别与之关联。在熵编码器的帮助下对获取的量化索引进行编码。量化系数之前被分组为称为代码块的矩形块,大小典型为64x64或32x32。每个代码块然后被熵编码独立编码。于是,在对当前的代码块进行编码时,熵编码器不会使用在之前的代码块的编码期间计算的符号出现概率。熵编码器由此在每次开始代码块的编码时处于初始化状态。该方法展示了对代码块的数据进行解码而不用对相邻的代码块进行解码的好处。于是,例如,一个客户端软件可以请求一个服务器软件提供仅客户需要的压缩代码块来对图像中识别的子部分进行解码。该方法还展示了允许代码块的并行编码和/或解码的优势。于是,代码块的大小越小,并行化级别就越高。例如,对于固定为2的并行化级别,两个代码块将被并行编码和/或解码。理论上,并行化级别的值等于图像中要编码的代码块的数量。但是,考虑到该编码没有利用从当前代码块的中间环境出现的概率,该方法获得的压缩性能不是最优的。A document available on the internet at http://research.microsoft.com/en-us/um/people/jinl/paper_2002/msri_jpeg.htm, dated April 15, 2011, describes a method for encoding a still image in accordance with the JPEG 2000 compression standard. The still image undergoes a discrete wavelet transform and is then quantized, thereby obtaining quantized wavelet coefficients, each associated with a quantization index. The obtained quantization indexes are encoded with the aid of an entropy encoder. The quantized coefficients are previously grouped into rectangular blocks, typically 64x64 or 32x32 in size, called code blocks. Each code block is then independently entropy-encoded. Consequently, when encoding the current code block, the entropy encoder does not use the symbol occurrence probabilities calculated during the encoding of the previous code block. The entropy encoder is thus initialized at the start of each code block encoding. This method demonstrates the advantage of decoding the data of a code block without having to decode adjacent code blocks. Thus, for example, client software can request server software to provide only the compressed code blocks required by the client to decode an identified subsection of an image. This method also exhibits the advantage of allowing parallel encoding and/or decoding of code blocks. Thus, the smaller the code block size, the higher the level of parallelism. For example, with a fixed parallelization level of 2, two code blocks will be encoded and/or decoded in parallel. In theory, the value of the parallelization level is equal to the number of code blocks to be encoded in the image. However, given that this encoding does not exploit the probability of intermediate contexts emerging from the current code block, the compression performance achieved by this method is suboptimal.
发明内容Summary of the Invention
本发明的一个目标是修复上述现有技术的缺陷。An object of the present invention is to remedy the above-mentioned drawbacks of the prior art.
为此,本发明的主题涉及一种对至少一个图像进行编码的方法,包括下列步骤:To this end, the subject matter of the invention is a method for encoding at least one image, comprising the following steps:
-将图像分割为可以包含属于预定符号集的符号的多个块,- segmenting the image into a number of blocks that may contain symbols belonging to a predetermined set of symbols,
-将所述块分组为预定数量的块子集,- grouping said blocks into a predetermined number of block subsets,
-通过熵编码模块、通过将数字信息与所考虑的子集中的每个块的符号进行关联来对所述块子集中的每个进行编码,该编码步骤包括针对图像的第一块来初始化熵编码模块的状态变量的子步骤,- encoding each of said subsets of blocks by an entropy coding module by associating digital information with the sign of each block in the subset considered, this coding step comprising a substep of initializing the state variables of the entropy coding module for the first block of the image,
-生成表示被编码的块子集中的至少一个的至少一个数据子流,- generating at least one data substream representing at least one of the subset of blocks to be coded,
根据本发明的方法值得注意,在于:The method according to the present invention is noteworthy in that:
-在当前块是所考虑的子集中要被编码的第一块的情形下,确定该第一当前块的符号出现概率,该概率是针对至少一个其他子集的编码和解码的预定块而已经确定的那些概率,- in the case where the current block is the first block to be coded in the subset under consideration, determining for this first current block the probabilities of occurrence of symbols which are those already determined for predetermined blocks of coding and decoding of at least one other subset,
-在当前块是所考虑的子集中最后编码的块的情形下:- In case the current block is the last coded block in the considered subset:
●将在所考虑的所述子集中的块的编码期间与所述符号关联的所有数字信息写入到子流中,该子流表示所考虑的子集,writing all the digital information associated with said symbols during the encoding of the blocks in said subset under consideration into a substream representing the subset under consideration,
●实现初始化子步骤。●Implement the initialization sub-step.
上述写入步骤相当于,一旦块子集中的最后一块已被编码,清空还未被发送的数字信息(比特),如上描述所解释。The above-mentioned writing step amounts to clearing out the digital information (bits) that have not yet been sent once the last block in the block subset has been encoded, as explained in the above description.
上述写入步骤和重新初始化熵编码模块的步骤的耦合使其可能产生包含各个数据子流的被编码的数据流,该数据子流分别对应于该至少一个被编码的块子集,所述流适合根据各种并行化级别来编码,并且这与应用于块子集的编码类型不管是顺序还是并行无关。于是,在编码时可以在并行级别的选择上有很大的自由度,该并行级别作为期望的编码/解码性能的函数。解码的并行级别可变,且甚至可以与编码的并行级别不同,因为在开始块子集的解码时,解码器总是处于初始化状态。The coupling of the aforementioned writing step and the step of reinitializing the entropy coding module makes it possible to generate a coded data stream comprising individual data substreams corresponding to the at least one coded block subset, said stream being suitable for coding according to various levels of parallelism, independent of the type of coding applied to the block subset, whether sequential or parallel. Thus, a great deal of freedom can be exercised in the choice of the level of parallelism during encoding, as a function of the desired encoding/decoding performance. The level of parallelism during decoding can be varied and can even differ from the level of parallelism during encoding, since the decoder is always in an initialized state when starting decoding of the block subset.
根据第一例子,熵编码模块的状态变量是表示预定符号集的符号中的符号出现概率的区间的两个边界。According to a first example, the state variables of the entropy coding module are two boundaries of an interval representing the probability of occurrence of a symbol among the symbols of a predetermined set of symbols.
根据第二例子,熵编码模块的状态变量是本领域技术人员熟知并且在2011年6月21日在下列互联网地址http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch中描述的LZW(Lempel-Ziv-Welch)熵编码器的转换表中包含的符号的串。According to a second example, the state variable of the entropy coding module is a string of symbols contained in the conversion table of the LZW (Lempel-Ziv-Welch) entropy encoder which is well known to those skilled in the art and described on June 21, 2011 at the following Internet address http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch.
使用在所考虑的块子集的第一当前块的熵编码期间使用针对所述其他子集的第一块来确定的符号出现概率的主要好处是,通过在后者中仅存储所述符号出现概率的更新而不用考虑通过所述其他子集中的其他连续块来学习的符号出现概率,来节省编码器的缓冲存储器。The main benefit of using the symbol occurrence probabilities determined for the first block of the other subset during the entropy coding of the first current block of the block subset under consideration is that the encoder's buffer memory is saved by storing in the latter only the updates of the symbol occurrence probabilities without taking into account the symbol occurrence probabilities learned by other consecutive blocks in the other subset.
在所考虑的块子集中的第一当前块的熵编码期间使用针对所述子集中除了第一块以外的块例如第二块来确定的符号出现概率的好处是获得更准确且由此更好地学习符号出现概率,由此提升更好的视频压缩性能。The benefit of using symbol occurrence probabilities determined for blocks other than the first block in the subset of blocks under consideration, such as the second block, during entropy encoding of the first current block in the subset of blocks is to obtain more accurate and thus better learned symbol occurrence probabilities, thereby promoting better video compression performance.
在特定的实施例中,块子集被顺序或者并行编码。In certain embodiments, subsets of blocks are encoded sequentially or in parallel.
子集块被顺序编码的事实所具有的好处是展示符合H.264/MPEG-4AVC标准的根据本发明的编码方法。The fact that the subset blocks are coded sequentially has the advantage of demonstrating that the coding method according to the invention complies with the H.264/MPEG-4 AVC standard.
子集块被并行编码的事实所具有的好处是加速编码器处理时间且受益于图像编码的多平台架构。The fact that subset blocks are encoded in parallel has the benefit of speeding up encoder processing time and benefiting from a multi-platform architecture for image coding.
在另一特定实施例中,当至少两个块子集与至少一个其他的块子集并行编码时,该至少两个被编码的块子集被包含在相同的数据子流中。In another particular embodiment, when at least two block subsets are encoded in parallel with at least one other block subset, the at least two encoded block subsets are contained in the same data substream.
该规定使其特别可能节省数据子流的信令。确实,为了使解码单元能够尽可能早地对子流进行解码,需要在压缩文件中指示所考虑的子流在何处开始。当若干个块子集被包含在相同的数据子流中时,需要单个指示符,由此降低压缩文件的大小。This provision makes it particularly possible to save signaling for data substreams. Indeed, in order to enable the decoding unit to decode the substream as early as possible, it is necessary to indicate in the compressed file where the substream in question begins. When several block subsets are contained in the same data substream, a single indicator is required, thereby reducing the size of the compressed file.
在又一特定实施例中,当所述被编码的块子集想要以预定的顺序来并行编码时,在对该块子集中的每个分别编码之后给出的数据子流在为了解码而被发送之前根据该预定顺序来预先排序。In yet another particular embodiment, when said coded block subsets are intended to be coded in parallel in a predetermined order, the data substreams presented after each respective coding of the block subsets are pre-ordered according to the predetermined order before being sent for decoding.
该规定使其可能使被编码的数据子流适应特定类型的解码,而不需要对图像进行解码然后再次编码。This provision makes it possible to adapt a coded data substream to a specific type of decoding, without requiring the picture to be decoded and then encoded again.
相关地,本发明还涉及一种对至少一个图像进行编码的设备,包括:Relatedly, the present invention also relates to a device for encoding at least one image, comprising:
-将图像分割为多个块的装置,该多个块可以包含属于预定符号集的符号,- means for dividing the image into a plurality of blocks, each of which may contain symbols belonging to a predetermined set of symbols,
-用于将所述块分组为预定数量的块子集的装置,- means for grouping said blocks into a predetermined number of block subsets,
-对该块子集中的每个进行编码的装置,该编码装置包括熵编码模块,能够将数字信息与所考虑的子集中的每个块的符号相关联,该编码装置包含针对图像中的第一块来初始化熵编码模块的状态变量的子装置,means for encoding each of the subset of blocks, said encoding means comprising an entropy coding module capable of associating digital information with the sign of each block in the considered subset, said encoding means comprising sub-means for initializing, for a first block in the image, the state variables of the entropy coding module,
-用于生成数据的至少一个数据子流的装置,该数据子流表示被编码的块子集中的至少一个。- means for generating at least one data substream of data, the data substream representing at least one of the subset of blocks to be coded.
该编码装置值得注意,在于它包括:The encoding device is noteworthy in that it comprises:
-用于确定当前块的符号出现概率的装置,在当前块是所考虑的子集中要被编码的第一块的情形下,该装置确定第一块的符号出现概率为针对至少一个其他子集的编码和解码的预定块所已经确定的那些概率,means for determining the probabilities of occurrence of symbols of the current block, which, in the case where the current block is the first block to be coded in the subset under consideration, determine the probabilities of occurrence of symbols of the first block to be those already determined for predetermined blocks of the coding and decoding of at least one other subset,
-写入装置,在当前块是所考虑的子集中最后编码的块的情形下,其被激活以将在所考虑的子集中的块的编码期间已经与所述符号关联的所有数字信息写入到子流中,该子流至少表示所考虑的子集,- writing means which, in the event that the current block is the last coded block in the subset under consideration, are activated to write all the digital information that has been associated with said symbol during the coding of the blocks in the subset under consideration, into a substream representing at least the subset under consideration,
初始化子装置还被激活以重新初始化熵编码模块的状态变量。The initialization sub-device is also activated to reinitialize the state variables of the entropy coding module.
通过相应的方式,本发明还涉及一种用于对表示至少一个被编码的图像的流进行解码的方法,包括下列步骤:In a corresponding manner, the invention also relates to a method for decoding a stream representing at least one coded image, comprising the following steps:
-在所述流中识别与要解码的至少一个块子集分别对应的预定数量的数据子流,所述块能够包含属于预定符号集的符号,- identifying in said stream a predetermined number of data substreams corresponding respectively to at least a subset of blocks to be decoded, said blocks being capable of containing symbols belonging to a predetermined set of symbols,
-通过熵解码模块,通过在至少一个被识别的子流中读取与对应于该至少一个所述被识别的子流的子集中的每个块的符号相关联的数字信息,来对所述被识别的块子集进行解码,该解码步骤包括针对图像中要被解码的第一块来初始化所述熵解码模块的状态变量的子步骤,- decoding, by an entropy decoding module, the identified subset of blocks by reading, in at least one identified substream, digital information associated with a symbol corresponding to each block of the subset of at least one said identified substream, this decoding step comprising a substep of initializing state variables of said entropy decoding module for a first block to be decoded in the image,
该解码方法值得注意,在于:The decoding method is noteworthy in that:
-在当前块是所考虑的子集中要被解码的第一块的情形下,确定所考虑的子集中的第一块的符号出现概率,该概率是针对至少一个其他子集的被解码的预定块而已经确定的那些概率,- in the case where the current block is the first block to be decoded in the subset under consideration, determining the probabilities of occurrence of symbols of the first block in the subset under consideration, these probabilities being those already determined for predetermined blocks to be decoded of at least one other subset,
-在当前块是所考虑的子集中最后解码的块的情形下,实现所述初始化子步骤。- said initialization sub-step is implemented in the case where the current block is the last decoded block in the subset considered.
在特定的实施例中,所述块子集被顺序或并行地解码。In certain embodiments, the subsets of blocks are decoded sequentially or in parallel.
在另一特定的实施例中,当至少两个块子集与至少一个其他的块子集并行解码时,被识别的数据子流中的一个表示所述至少两个块子集。In another specific embodiment, one of the identified data substreams represents the at least two block subsets when the at least two block subsets are decoded in parallel with at least one other block subset.
在又一特定的实施例中,当被编码的块子集想要以预定的顺序来并行解码时,与被编码的块子集分别对应的数据子流之前在所述要解码的流中以该预定顺序来排序。In yet another specific embodiment, when the coded block subsets are intended to be decoded in parallel in a predetermined order, the data substreams respectively corresponding to the coded block subsets are previously sorted in the predetermined order in the stream to be decoded.
相关地,本发明还涉及一种用于对表示至少一个被编码的图像的流进行解码的设备,包括:Relatedly, the invention also relates to a device for decoding a stream representing at least one coded image, comprising:
-用于在所述流中识别与要解码的至少一个块子集分别对应的预定数量的数据子流的装置,所述块能够包含属于预定符号集的符号,- means for identifying in said stream a predetermined number of data substreams corresponding respectively to at least a subset of blocks to be decoded, said blocks being capable of containing symbols belonging to a predetermined set of symbols,
-用于对被识别的块子集进行解码的装置,该解码装置包括熵解码模块,其能够在至少一个所述被识别的子流中读取与对应于该至少一个被识别的子流的子集中的每个块的符号相关联的数字信息,该解码装置包括针对图像中要被解码的第一块来初始化熵解码模块的状态变量的子装置,means for decoding the identified subset of blocks, the decoding means comprising an entropy decoding module capable of reading, in at least one of said identified substreams, digital information associated with a symbol corresponding to each block of the subset of at least one identified substream, the decoding means comprising submeans for initializing, for a first block to be decoded in the image, state variables of the entropy decoding module,
该解码装置值得注意,在于它包括用于确定当前块的符号出现概率的装置,在当前块是所考虑的子集中要被解码的第一块的情形下,该装置确定该第一块的符号出现概率,作为针对至少一个其他子集的被解码的预定块而已经确定的那些概率,The decoding device is noteworthy in that it comprises means for determining the probabilities of occurrence of symbols of the current block, in the case where the current block is the first block to be decoded in the subset considered, these means determine the probabilities of occurrence of symbols of this first block as those already determined for predetermined blocks to be decoded of at least one other subset,
并且在于,在当前块是所考虑的子集中最后解码的块的情形下,初始化子装置被激活以重新初始化熵解码模块的状态变量。And it is that, in the case that the current block is the last decoded block in the subset considered, the initialization sub-means is activated to reinitialize the state variables of the entropy decoding module.
本发明的目标还在于一种包含指令的计算机程序,当程序被计算机执行时,用于执行上述编码或解码方法的步骤。The invention also relates to a computer program comprising instructions for carrying out the steps of the encoding or decoding method described above, when the program is executed by a computer.
该程序可以使用任意编程语言,并且可以采用源代码、目标代码或介于源代码和目标代码之间的中间代码的形式,例如部分编译的形式或任意其他想要的形式。The program may use any programming language and may be in the form of source code, object code or an intermediate code between source code and object code, such as a partially compiled form or any other desired form.
本发明的又一主题还在于一种记录介质,其可被计算机读取,并且包含如上所述的计算机程序指令。Yet another subject of the present invention is a recording medium that is readable by a computer and contains the computer program instructions described above.
该记录介质可以是能存储程序的实体或装置。例如,该介质可以包括存储装置例如ROM如CD ROM或微电子电路ROM,或者磁存储装置如磁盘(软盘)或硬盘。The recording medium may be an entity or device capable of storing a program. For example, the medium may include a storage device such as a ROM such as a CD ROM or a microelectronic circuit ROM, or a magnetic storage device such as a magnetic disk (floppy disk) or a hard disk.
此外,该记录介质可以是可传输介质例如电或光信号,其可以通过电或光缆、通过无线电或其他方式来传递。根据本发明的程序特别地可以从互联网类型的网络上下载。Furthermore, the recording medium may be a transmissible medium such as an electrical or optical signal, which may be transferred via electrical or optical cables, by radio or other means.The program according to the invention may in particular be downloaded from a network of the Internet type.
或者,该记录介质可以是其中包含程序的集成电路,该电路适于执行所考虑的方法或者在后者执行时使用。Alternatively, the recording medium may be an integrated circuit in which the program is embodied, the circuit being adapted to execute the method in question or to be used when the latter is executed.
上述编码装置、解码方法、解码装置和计算机程序展示了与根据本发明的编码方法所具有的相同的优势。The above-mentioned encoding device, decoding method, decoding device and computer program exhibit the same advantages as those possessed by the encoding method according to the present invention.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
阅读参考附图来描述的两个优选实施例,其他特征和优势将变得明显,在附图中:Other features and advantages will become apparent from a reading of the two preferred embodiments described with reference to the accompanying drawings, in which:
-图1表示现有技术的图像编码图,- FIG1 shows a prior art image coding diagram,
-图2A表示根据本发明的编码方法的主要步骤,- FIG2A shows the main steps of the encoding method according to the invention,
-图2B详细地表示在图2A的编码方法中实现的编码,- FIG. 2B shows in detail the encoding implemented in the encoding method of FIG. 2A ,
-图3A表示根据本发明的编码装置的第一实施例,- FIG3A shows a first embodiment of a coding device according to the invention,
-图3B表示图3A中的编码装置的编码单元,- FIG. 3B shows a coding unit of the coding device in FIG. 3A ,
-图3C表示根据本发明的编码装置的第二实施例,- FIG3C shows a second embodiment of a coding device according to the invention,
-图4A表示根据第一优选实施例的图像编码/解码图,- FIG. 4A shows an image encoding/decoding diagram according to a first preferred embodiment,
-图4B表示根据第二优选实施例的图像编码/解码图,- FIG. 4B shows an image encoding/decoding diagram according to a second preferred embodiment,
-图5A表示根据本发明的解码方法的主要步骤,- FIG5A shows the main steps of the decoding method according to the invention,
-图5B详细地表示在图5A的解码方法中实现的解码,- FIG. 5B shows in detail the decoding implemented in the decoding method of FIG. 5A ,
-图6A表示根据本发明的解码装置的实施例,- FIG. 6A shows an embodiment of a decoding device according to the invention,
-图6B表示图6A中的解码装置的解码单元,- FIG. 6B shows a decoding unit of the decoding device in FIG. 6A ,
-图7A表示实现顺序类型的编码和并行类型的解码的图像编码/解码图,- FIG. 7A shows an image encoding/decoding diagram that implements sequential type encoding and parallel type decoding,
-图7B表示以各自不同的并行级别来实现并行类型的编码/解码的图像编码/解码图。- FIG. 7B shows an image encoding/decoding diagram for implementing parallel types of encoding/decoding at different levels of parallelism.
具体实施方式DETAILED DESCRIPTION
现在将描述本发明的实施例,其中,根据例如符合H.264/MPEG-4AVC标准的编码而获得的二进制流,依据根据本发明的编码方法被用于对图像序列进行编码。在该实施例中,通过对初始符合H.264/MPEG-4AVC标准的编码器的调整,根据本发明的编码方法例如以软件或硬件的方式来实现。根据本发明的编码方法以包含步骤C1到C5的算法的形式来表示,如图2A所示。An embodiment of the present invention will now be described in which a binary stream obtained by encoding, for example, in accordance with the H.264/MPEG-4 AVC standard, is used to encode a sequence of images according to the encoding method according to the present invention. In this embodiment, the encoding method according to the present invention is implemented, for example, in software or hardware, by modifying an encoder that originally conforms to the H.264/MPEG-4 AVC standard. The encoding method according to the present invention is represented in the form of an algorithm comprising steps C1 to C5, as shown in FIG2A .
根据本发明的实施例,根据本发明的编码方法在编码装置CO中实现,其两个实施例分别在图3A和3C中表示。According to an embodiment of the present invention, the encoding method according to the present invention is implemented in an encoding device CO, two embodiments of which are respectively represented in Figures 3A and 3C.
参考图2A,第一编码步骤C1是将要编码的图像序列中的图像IE划分为多个块或宏块MB,如图4A或4B所示。所述宏块可以包含一个或多个符号,所述符号形成预定符号集的一部分。在所示例子中,所述块MB具有正方形并且都具有相同的大小。作为图像大小的函数,该图像不必是块大小的倍数,左侧的最后一块和底部的最后一块可以不是正方形。在替代的实施例中,块例如可以是长方形大小并且/或者互相不对齐。With reference to FIG2A , the first encoding step C1 consists in dividing an image IE in the sequence of images to be encoded into a plurality of blocks or macroblocks MB, as shown in FIG4A or 4B . The macroblocks may contain one or more symbols, which form part of a predetermined set of symbols. In the example shown, the blocks MB are square and all have the same size. As a function of the image size, the image does not need to be a multiple of the block size; the last block on the left and the last block at the bottom may not be square. In alternative embodiments, the blocks may, for example, be rectangular in size and/or may not be aligned with one another.
每个块或宏块自身可以进一步被分为子块,该子块本身可以再细分。Each block or macroblock may itself be further divided into sub-blocks, which may themselves be further subdivided.
该划分是通过图3A所示的划分模块PCO来执行的,其使用众所周知的划分算法。This partitioning is performed by the partitioning module PCO shown in FIG3A , which uses a well-known partitioning algorithm.
参考图2A,第二编码步骤C2是将上述块分组为要被顺序或并行编码的预订数量P的连续块的子集SE1,SE2,...,SEk,...,SEP。在图4A和4B所示的例子中,为了图的清楚,仅展示了四个子集SE1、SE2、SE3、SE4。这四个块子集每个用虚线表示,并且分别包含图像IE的前四行的块。Referring to FIG2A , the second encoding step C2 involves grouping the aforementioned blocks into a predetermined number P of consecutive block subsets SE1, SE2, ..., SEk, ..., SEP, to be encoded sequentially or in parallel. In the example shown in FIG4A and FIG4B , only four subsets SE1, SE2, SE3, and SE4 are shown for clarity. These four block subsets are each indicated by a dashed line and contain blocks from the first four rows of image IE.
该分组是通过图3A中所示的计算模块GRCO在本身众所周知的算法的辅助下执行的。This grouping is performed by the calculation module GRCO shown in FIG. 3A , with the aid of algorithms known per se.
参考图2A,第三编码步骤C3包括对所述块子集SE1到SE6中的每个进行编码,根据例如顺序类型的预定遍历次序PS来对所考虑的子集中的块进行编码。在图4A和4B所示的例子中,当前块SEk(1≤k≤P)中的块如箭头PS所示从左到右被依次编码。With reference to FIG2A , the third encoding step C3 consists in encoding each of the block subsets SE1 to SE6, encoding the blocks of the considered subset according to a predetermined traversal order PS, for example of the sequential type. In the example shown in FIG4A and 4B , the blocks of the current block SEk (1≤k≤P) are encoded successively from left to right, as indicated by the arrows PS.
根据第一变体,该编码是顺序类型,并且通过如图3A所示的单个编码单元UC来实现。通过本身已知的方式,编码器CO包括缓冲存储器MT,其被适配为包含在当前块编码时被逐渐重复更新的符号出现的概率。According to a first variant, the coding is of sequential type and is implemented by a single coding unit UC as shown in Figure 3A. In a manner known per se, the coder CO comprises a buffer memory MT adapted to contain the probabilities of the occurrence of symbols that are gradually and repeatedly updated when the current block is coded.
如图3B中更详细地展示,编码单元UC包括:As shown in more detail in FIG3B , the coding unit UC comprises:
*关于至少一个之前被编码和解码的块对当前块进行预测编码的模块,被表示为MCP;* A module that predictively encodes a current block with respect to at least one previously coded and decoded block, denoted as MCP;
*使用针对所述之前被编码和解码的块来计算的至少一个符号出现的概率来对所述当前块进行熵编码的模块,被表示为MCE。* A module for entropy encoding the current block using the probability of occurrence of at least one symbol calculated for the previously coded and decoded blocks, denoted MCE.
预测编码模块MCP是软件模块,其能够根据传统的预测技术例如以内和/或间模式对当前块进行预测编码。The predictive coding module MCP is a software module capable of predictively coding the current block according to conventional prediction techniques, for example in intra and/or inter mode.
熵编码模块MCE本身是CABAC类型,但根据本发明来调整,在描述中将进一步描述。The entropy coding module MCE is itself of CABAC type, but is adapted according to the invention, as will be described further in the description.
作为变体,熵编码模块MCE可以是本身已知的霍夫曼编码器。As a variant, the entropy coding module MCE may be a Huffman coder known per se.
在图4A和4B所示的例子中,单元UC对第一行SE1中的块从左到右进行编码。当它到达第一行SE1的最后一块时,它传递到第二行SE2的第一块。当它到达第二行SE2的最后一块时,它传递到第三行SE3的第一块。当它到达第三行SE3的最后一块时,它传递到第四行SE4的第一块,等等,直到图像IE的最后一块被编码。In the example shown in Figures 4A and 4B, the unit UC encodes the blocks in the first row SE1 from left to right. When it reaches the last block of the first row SE1, it passes to the first block of the second row SE2. When it reaches the last block of the second row SE2, it passes to the first block of the third row SE3. When it reaches the last block of the third row SE3, it passes to the first block of the fourth row SE4, and so on, until the last block of the image IE is encoded.
与以上刚才描述的不同的其他类型的遍历当然是可能的。于是,可以将图像IE划分为若干个子图像并将该类型的划分独立应用于每个子图像。编码单元还可以不和以上解释的那样相继处理各行,而是相继(处理)各列。还可以以任意方向来遍历行或列。Other types of traversal than those just described are of course possible. Thus, the image IE can be divided into several sub-images and this type of division applied independently to each sub-image. The encoding unit can also process the columns sequentially instead of the rows as explained above. It is also possible to traverse the rows or columns in any direction.
根据第二变体,该编码是并行类型,并且,仅通过由预订数量R的编码单元UCk(1≤k≤R)来实现的事实(在图3C所示的例子中R=2),它与顺序编码的第一变体相区分。已知该并行编码给编码方法带来了明显的加速。According to the second variant, the encoding is of the parallel type and is distinguished from the first variant of sequential encoding only by the fact that it is implemented by a predetermined number R of encoding units UCk (1≤k≤R) (R=2 in the example shown in FIG3C ). It is known that this parallel encoding brings a significant acceleration to the encoding method.
编码单元UCk中的每个等价于图3B所示的编码单元UC。通过相应的方式,编码单元UCk包括预测编码模块MCPk和熵编码模块MCEk。Each of the coding units UCk is equivalent to the coding unit UC shown in Figure 3B. In a corresponding manner, the coding unit UCk includes a prediction coding module MCPk and an entropy coding module MCEk.
再次参考图4A和4B,第一单元UC1对奇数排名的行中的块进行编码,而第二单元UC2对偶数排名的行中的块进行编码。更准确地说,第一单元UC1从左到右对第一行SE1中的块进行编码。当它达到第一行SE1的最后一块时,它传递第(2n+1)行即第三行SE3等的第一块。与第一单元UC1执行的处理并行,第二单元UC2从左到右对第二行SE2中的块进行编码。当它到达第二行SE2的最后一块,它传递到第(2n)行在这里是第四行SE4等的第一块。上述两个遍历被重复,直到图像IE的最后一块被编码。Referring again to Figures 4A and 4B, the first unit UC1 encodes blocks in odd-numbered rows, while the second unit UC2 encodes blocks in even-numbered rows. More precisely, the first unit UC1 encodes the blocks in the first row SE1 from left to right. When it reaches the last block of the first row SE1, it passes on the first block of the (2n+1)th row, which is the third row SE3, and so on. In parallel with the processing performed by the first unit UC1, the second unit UC2 encodes the blocks in the second row SE2 from left to right. When it reaches the last block of the second row SE2, it passes on to the first block of the (2n)th row, which is the fourth row SE4, and so on. The above two traversals are repeated until the last block of the image IE is encoded.
参考图2A,第四编码步骤C4是产生比特的L个子流F1,F2,...,Fm,...,FL(1≤m≤L≤P)来表示通过上述编码单元UC或者上述编码单元UCk中的每个来压缩的被处理的块以及每个子集SEk中被处理的块的解码版本。根据在描述中将进一步细化的同步机制,所考虑的子集中用SED1,SED2,...,SEDk,...,SEDP表示的被解码的处理的块可以被图3A所示的编码单元UC或图3C所示的编码单元UCk中的每个复用。With reference to FIG2A , the fourth encoding step C4 generates L substreams of bits F1, F2, ..., Fm, ..., FL (1≤m≤L≤P) representing the processed blocks compressed by each of the above-mentioned coding units UC or UCk, as well as decoded versions of the processed blocks in each subset SEk. According to a synchronization mechanism that will be further detailed in the description, the decoded processed blocks represented by SED1, SED2, ..., SEDk, ..., SEDP in the considered subset can be multiplexed by each of the coding units UC shown in FIG3A or UCk shown in FIG3C .
参考图3B,产生L个子流的步骤是通过流生成软件模块MGSF或MSGFk来实现的,其被适配为产生数据流例如比特。3B , the step of generating L sub-streams is implemented by a stream generation software module MGSF or MSGFk, which is adapted to generate a data stream, eg bits.
参考图2A,第五编码步骤C5包括基于上述L个子流F1,F2,...,Fm,...,FL来构造全局流F。根据一个实施例,子流F1,F2,...,Fm,...,FL被简单地并置,用额外的信息项来向解码器指示全局流F中的每个子流Fm的位置。后者之后被通信网络(未示出)发送到远程终端。后者包括图5A所示的解码器DO。根据另一实施例,这尤其有利,因为不需要对图像解码然后再次编码,在将流F发送到解码器DO之前,解码器DO之前以预定的顺序来对L个子流F1,F2,...,Fm,...,FL进行排序,该顺序对应于DO能够对子流进行解码的顺序。With reference to FIG2A , the fifth encoding step C5 comprises constructing a global stream F based on the L substreams F1, F2, ..., Fm, ..., FL described above. According to one embodiment, the substreams F1, F2, ..., Fm, ..., FL are simply concatenated, with an additional information item indicating to the decoder the position of each substream Fm within the global stream F. The latter is then transmitted via a communication network (not shown) to a remote terminal. The latter includes a decoder DO shown in FIG5A . According to another embodiment, this is particularly advantageous because, without having to decode the image and then re-encode it, the decoder DO sorts the L substreams F1, F2, ..., Fm, ..., FL in a predetermined order before transmitting the stream F to the decoder DO, which corresponds to the order in which the substreams can be decoded by the decoder DO.
于是,如描述中更详细地描述,根据本发明的解码器能够在全局流F中分离子流F1,F2,...,Fm,...,FL,并将它们分配给组成解码器的一个或多个解码单元。需要注意,全局流中的子流的该分解独立于使用单个编码单元或并行操作的若干个编码单元的选择,并且用该方法,可以只有编码器或只有解码器,其包含并行操作的单元。Thus, as described in more detail in the description, the decoder according to the present invention is able to separate the substreams F1, F2, ..., Fm, ..., FL in the global stream F and distribute them to one or more decoding units constituting the decoder. It should be noted that this decomposition of the substreams in the global stream is independent of the choice of using a single coding unit or several coding units operating in parallel, and that with this method, only the encoder or only the decoder can be provided, which includes units operating in parallel.
全局流F的该构造是在例如图3A和图3C所示的流构造模块CF中实现的。This construction of the global flow F is implemented in the flow construction module CF shown in, for example, Figures 3A and 3C.
现在将参考图2B来描述本发明的各个特定子步骤,例如在上述编码步骤C3期间在编码单元UC或UCk中实现的子步骤。Various specific sub-steps of the invention will now be described with reference to FIG. 2B , for example sub-steps implemented in the coding unit UC or UCk during the above-described coding step C3 .
在步骤C31的过程中,编码单元UC或UCk选择图4A或4B所示的当前块SEk例如第一行SE1中要编码的第一块作为当前块。During step C31 , the coding unit UC or UCk selects the current block SEk shown in FIG. 4A or 4B , for example, the first block to be coded in the first row SE1 , as the current block.
在步骤C32的过程中,单元UC或UCk测试当前块是否是图像IE的第一块(位于顶部且位于左侧),该图像在上述步骤C1中已被划分为块。During a step C32 , the unit UC or UCk tests whether the current block is the first block (at the top and on the left) of the image IE which has been divided into blocks in the above-described step C1 .
如果是这种情况,在步骤C33的过程中,熵编码模块MCE或MCEk对其状态变量进行初始化。根据所示例子,其使用如前所述的算术编码,这包含表示预定符号集中包含的符号的出现的概率的间隔的初始化。通过本身已知的方式,用两个边界L和H,分别为下边界和上边界,来初始化该间隔。下边界L的值被固定为0,而上边界的值被固定为1,由此对应于预定符号集的所有符号中的第一符号的出现的概率。该间隔的大小R因此在此刻由R=H-L=1来定义。被初始化的间隔被传统地进一步划分为多个预定的子间隔,其分别表示预定符号集中的符号的出现的概率。If this is the case, during step C33, the entropy coding module MCE or MCEk initializes its state variables. According to the example shown, arithmetic coding, as described above, is used. This involves initializing an interval representing the probability of occurrence of symbols contained in the predetermined symbol set. In a manner known per se, this interval is initialized with two boundaries, L and H, respectively, a lower boundary and an upper boundary. The value of the lower boundary, L, is fixed to 0, while the value of the upper boundary is fixed to 1, thus corresponding to the probability of occurrence of the first symbol among all symbols of the predetermined symbol set. The size R of this interval is therefore now defined by R = H - L = 1. The initialized interval is conventionally further divided into a number of predetermined subintervals, each representing the probability of occurrence of a symbol from the predetermined symbol set.
作为变体,如果使用的熵编码是LZW编码,符号的串的转换表被初始化,从而它包含所有可能的符号一次且仅一次。As a variant, if the entropy coding used is LZW coding, the conversion table of strings of symbols is initialized so that it contains all possible symbols once and only once.
如果在上述步骤C32之后,当前块不是图像IE的第一块,则在后续描述中之后被描述的步骤C40的过程中确定必要的之前被编码和解码的块的可用性。If the current block is not the first block of the image IE after the above-mentioned step C32 , the availability of necessary previously encoded and decoded blocks is determined during step C40 to be described later in the subsequent description.
在步骤C34的过程中,对图4A或4B中表示的第一行SE1中的第一当前块MB1进行编码。该步骤C34包括以下将描述的多个子步骤C341到C348。During step C34, the first current block MB1 in the first row SE1 shown in Figure 4A or 4B is encoded. This step C34 comprises a number of sub-steps C341 to C348 which will be described below.
在图2B所示的第一子步骤C341的过程中,通过内和/或间预测的已知技术对当前块MB1进行预测编码,在其过程中,关于至少一个之前被编码和解码的块来对块MB1进行预测。During a first sub-step C341 shown in FIG. 2B , the current block MB1 is predictively coded by known techniques of intra and/or inter prediction, during which the block MB1 is predicted with respect to at least one previously coded and decoded block.
毋庸置疑,例如H.264标准中提出的其他模式的内预测也是可以的。Of course, other modes of intra prediction, such as those proposed in the H.264 standard, are also possible.
还可以以内模式对当前块MB1进行预测编码,在其过程中,关于在之前被编码和解码的图像中出现的块来对当前块进行预测。其他类型的预测当然也是可能的。在对当前块的可能的预测中,根据对于本领域技术人员来说众所周知的比特率失真准则来选择最优预测。The current block MB1 can also be predictively coded in intra mode, in which the current block is predicted with respect to blocks that appeared in previously coded and decoded images. Other types of prediction are of course also possible. Among the possible predictions for the current block, the optimal prediction is selected according to a bit rate distortion criterion well known to those skilled in the art.
上述预测编码步骤使其可能构造预测的块MBp1,这是对当前块MB1的近似。与预测编码相关的信息后续将被写入到流F中,该流F被发送到解码器DO。该信息特别包括预测类型(间或内(预测)),并且如果合适,包括内预测的类型、块或宏块(如果后者已经被细分)的划分类型、在间预测模式中使用的参考图像索引和位移矢量。该信息被编码器CO压缩。The predictive coding step described above makes it possible to construct a predicted block MBp 1 , which is an approximation of the current block MB 1 . The information relating to the predictive coding is subsequently written into a stream F, which is sent to the decoder DO. This information includes in particular the prediction type (inter or intra) and, if applicable, the type of intra prediction, the type of partitioning of the block or macroblock (if the latter has been subdivided), the reference image index used in the inter prediction mode, and the displacement vector. This information is compressed by the encoder CO.
在之后的子步骤C342的过程中,从当前块MB1中减去预测块MBp1以产生残留块MBr1。During the subsequent sub-step C342 , the prediction block MBp 1 is subtracted from the current block MB 1 to generate a residual block MBr 1 .
在之后的子步骤C343的过程中,根据传统的直接变换操作例如DCT类型的离散余弦变换来对残留块MBr1进行变换,以产生变换后的块MBt1。During a subsequent sub-step C343 , the residual block MBr 1 is transformed according to a conventional direct transformation operation, for example a discrete cosine transformation of the DCT type, in order to generate a transformed block MBt 1 .
在之后的子步骤C344的过程中,根据传统的量化操作例如标量量化来对变换后的块MBt1进行量化。然后得到系数被量化的块MBq1。In the subsequent sub-step C344 , the transformed block MBt 1 is quantized according to a conventional quantization operation, such as scalar quantization, to obtain a block MBq 1 with quantized coefficients.
在之后的子步骤C345的过程中,对系数被量化的块MBq1进行熵编码。在优选的实施例中,这包括CABAC熵编码。该步骤包括:During the subsequent sub-step C345, the block MBq 1 with quantized coefficients is entropy coded. In a preferred embodiment, this involves CABAC entropy coding. This step comprises:
a)读取与所述当前块关联的符号或预定符号集中的符号,a) reading a symbol associated with said current block or a symbol from a predetermined set of symbols,
b)将数字信息例如比特与读取的符号进行关联。b) Associating digital information, such as bits, with the read symbols.
在上述变体中(根据该变体,使用的编码是LZW编码),与当前转换表中的符号的代码相对应的数字信息项与要编码的符号关联,并根据本身已知的过程来更新该转换表。In the variant described above, according to which the encoding used is the LZW encoding, an item of digital information corresponding to the code of the symbol currently in the conversion table is associated with the symbol to be encoded, and the conversion table is updated according to a procedure known per se.
在之后的子步骤C346的过程中,根据传统的反量化操作即与步骤C344中执行的量化相反的操作来对块MBq1进行反量化。然后得到系数被反量化的块MBDq1。During the subsequent sub-step C346 , the block MBq 1 is dequantized according to a conventional dequantization operation, ie, an operation opposite to the quantization performed in step C344 , thereby obtaining a block MBDq 1 with dequantized coefficients.
在之后的子步骤C347的过程中,对系数被反量化的块MBDq1进行逆变换,这是与以上步骤C343中执行的直接变换相反的操作。然后得到被解码的残留块MBDr1。In the subsequent sub-step C347 , the block MBDq 1 with inversely quantized coefficients is subjected to an inverse transform, which is the opposite operation to the direct transform performed in the above step C343 , and the decoded residual block MBDr 1 is then obtained.
在之后的子步骤C348的过程中,通过将解码的残留块MBDr1加到预测块MBp1来构造被解码的块MBD1。需要注意,后一块与在描述中将进一步描述的对图像IE进行解码的方法完成之后得到的被解码的块相同。被解码的块MBD1由此被呈现为可用,以被构成预定数量R的编码单元的一部分的编码单元UCk或任意其他编码单元使用。During the subsequent sub-step C348, the decoded block MBD 1 is constructed by adding the decoded residual block MBDr 1 to the prediction block MBp 1. It should be noted that this latter block is identical to the decoded block obtained after the method for decoding the image IE described further in the description is completed. The decoded block MBD 1 is thus rendered available for use by the coding unit UCk forming part of the predetermined number R of coding units or any other coding unit.
在完成上述编码步骤C34之后,例如如图3B所示的熵编码模块MCE或MCEk包含在第一块的编码时被逐渐重复更新的所有概率。这些概率对应于可能的语法的各个元素或者各个关联的编码上下文。After the encoding step C34, the entropy encoding module MCE or MCEk, for example as shown in FIG3B , contains all the probabilities that were updated repeatedly during the encoding of the first block. These probabilities correspond to the various elements of the possible syntax or the various associated coding contexts.
在上述编码步骤C34之后,在步骤C35的过程中执行测试以确定当前块是否是同一行的第j块,其中j是编码器CO已知的预定值,其至少等于1。After the above-mentioned encoding step C34 , a test is performed during a step C35 to determine whether the current block is the j-th block of the same row, where j is a predetermined value known to the coder CO and which is at least equal to 1.
如果是该情况,在图2B所示的步骤C36的过程中,针对第j块来计算的概率集合被存储在例如图3A或3B以及图4A和4B中所示的编码器CO的缓冲存储器MT中,所述存储器的大小被适配为存储所计算的概率数量。If this is the case, during step C36 shown in Figure 2B, the set of probabilities calculated for the jth block is stored in a buffer memory MT of the encoder CO shown, for example, in Figure 3A or 3B and Figures 4A and 4B, the size of which is adapted to store the calculated number of probabilities.
在图2B所示的步骤C37的过程中,编码单元UC或UCk测试刚才被编码的行SEk的当前块是否是图像IE中的最后一块。如果在步骤C35的过程中当前块不是行SE1的第j块,该步骤也会被实现。2B , the coding unit UC or UCk tests whether the current block of the line SEk just coded is the last block in the image IE. This step is also performed if, during step C35 , the current block is not the jth block of the line SE1 .
在当前块是图像IE的最后一块时,在步骤C38的过程中,编码方法结束。When the current block is the last block of the image IE, the coding method ends during a step C38 .
如果不是该情况,在步骤C39的过程中,根据如图4A或4B中的箭头PS所示的遍历顺序来选择要编码的下一块MBi。If this is not the case, during a step C39 , the next block MB i to be coded is selected according to the traversal order indicated by the arrow PS in FIG. 4A or 4B .
在图2B所示的步骤C40的过程中,确定对于当前块MBi的编码来说必须的之前被编码和解码的块的可用性。During step C40 shown in FIG. 2B , the availability of previously coded and decoded blocks necessary for the coding of the current block MB i is determined.
如果这是第一行SE1,该步骤包括验证位于要编码的MBi的左侧的至少一个块的可用性。但是,针对图4A或4B所示的实施例中选择的遍历顺序PS,在所考虑的行SEk中依次对各个块进行编码。结果,左侧被编码和解码的块总是可用的(除了一行的第一块)。在图4A或4B所示的例子中,这是直接位于要编码的当前块的左侧的块。If this is the first row SE1, this step involves verifying the availability of at least one block located to the left of the MB i to be coded. However, for the traversal order PS selected in the embodiment shown in FIG4A or 4B, the blocks in the row SEk under consideration are coded sequentially. As a result, a block to the left is always available (except for the first block of a row) to be coded and decoded. In the example shown in FIG4A or 4B, this is the block directly to the left of the current block to be coded.
如果这是与第一行不同的行SEk,所述确定步骤还包括验证位于前一行SEk-1中的预订数量N’的块例如分别位于当前块的上方和右上方的两个块是否可用于当前块的编码,即,它们是否已经被编码单元UC或UCk-1编码且然后被解码。If this is a row SEk different from the first row, the determining step further comprises verifying whether a predetermined number N' of blocks located in the previous row SEk-1, e.g. two blocks located above and above the right of the current block respectively, can be used for the encoding of the current block, i.e. whether they have already been encoded by the coding unit UC or UCk-1 and then decoded.
由于该测试步骤容易降低编码方法的速度,通过根据本发明的替代方式,在行的编码是并行类型的情形下,图3C所示的时钟CLK被适配为同步块编码的进展以确保分别位于当前块的上方和右上方的两个块的可用性,而不需要验证这两块的可用性。于是,编码单元UCk总是以被用于当前块的编码的前一行SEk-1中的预订数量N’(例如N’=2)的被编码和解码的块的偏移来开始第一块的编码。从软件的角度来看,这样的时钟的实现使其可能显著加速编码器CO中对图像IE中的块的处理时间。Since this test step tends to slow down the encoding method, an alternative approach according to the invention is to adapt the clock CLK shown in FIG3C to synchronize the progress of block encoding, in the case where the encoding of the lines is of the parallel type, to ensure the availability of two blocks located above and to the right of the current block, respectively, without having to verify the availability of these two blocks. Thus, the encoding unit UCk always starts the encoding of the first block with an offset of a predetermined number N' (e.g., N'=2) of coded and decoded blocks from the previous line SEk-1 used for the encoding of the current block. From a software perspective, the implementation of such a clock makes it possible to significantly speed up the processing time of the blocks in the image IE in the encoder CO.
在图2B所示的步骤C41的过程中,执行测试以确定当前块是否是所考虑的行SEk中的第一块。During step C41 shown in FIG. 2B , a test is performed to determine whether the current block is the first block in the row SEk considered.
如果是这种情况,在步骤C42的过程中,在缓冲存储器MT中仅读取在前一行SEk-1的第j块的编码期间计算的符号出现的概率。If this is the case, during a step C42 , only the probabilities of the occurrence of the symbols calculated during the coding of the j-th block of the previous row SEk-1 are read in the buffer memory MT.
根据图4A所示的第一变体,第j块是前一行SEk-1的第一块(j=1)。该读取包括用缓冲存储器MT中存在的概率来替换CABAC编码器的概率。为使其处理第二、第三和第四行SE2、SE3和SE4中的各个第一块,如细线所示的箭头在图4A中示出该读取步骤。According to a first variant shown in FIG4A , the jth block is the first block of the previous row SE k-1 (j=1). This reading involves replacing the probabilities of the CABAC coder with the probabilities present in the buffer memory MT. This reading step is illustrated in FIG4A as a thin arrow, so that it processes the first blocks of the second, third, and fourth rows SE2, SE3, and SE4, respectively.
根据图4B所示的上述步骤C43的第二变体,第j块是前一行SEk-1的第二块(j=2)。该读取包括用在缓冲存储器MT中存在的概率来替换CABAC编码器的概率。为使其处理第二、第三和第四行SE2、SE3和SE4中的各个第一块,如细虚线所示的箭头在图4B中示出该读取步骤。According to a second variant of the above-described step C43, shown in FIG4B , the jth block is the second block (j=2) of the previous row SEk-1. This reading involves replacing the probabilities of the CABAC coder with the probabilities present in the buffer memory MT. This reading step is illustrated in FIG4B as a thin dashed arrow, so that it processes the first blocks of the second, third, and fourth rows SE2, SE3, and SE4, respectively.
在步骤C42之后,通过重复上述步骤C34到C38来对当前块进行编码然后进行解码。After step C42, the current block is encoded and then decoded by repeating the above steps C34 to C38.
如果在上述步骤C41之后,当前块不是所考虑的行SEk的第一块,有利地不用读取在位于同一行SEk中的之前被编码和解码的块即在所示例子中直接位于当前块的左侧的被编码和解码的块中出现的概率。确实,考虑到如图4A或4B所示的用于读取位于同一行中的块的遍历顺序PS,在开始当前块的编码时在CABAC编码器中存在的符号出现的概率正好是在该同一行中的前一块的编码/解码后存在的那些概率。If, after the above-mentioned step C41, the current block is not the first block of the row SEk considered, it is advantageous not to read the probabilities of occurrence in the previously coded and decoded block located in the same row SEk, i.e., in the example shown, the coded and decoded block located immediately to the left of the current block. Indeed, given the traversal order PS for reading blocks located in the same row, as shown in FIG4A or 4B , the probabilities of occurrence of the symbols present in the CABAC coder at the start of the coding of the current block are exactly the same as those that existed after the coding/decoding of the previous block in the same row.
结果,在图2B所示的步骤C43的过程中,针对所述当前块的熵编码来学习符号出现的概率,这些(概率)仅对应于如图4A或4B中的双实线箭头所示的针对同一行中的所述前一块来计算的那些概率。As a result, during step C43 shown in Figure 2B, the probabilities of the occurrence of symbols are learned for the entropy coding of the current block, which (probabilities) correspond only to those probabilities calculated for the previous block in the same row as shown by the double solid arrows in Figure 4A or 4B.
在步骤C43之后,重复上述步骤C34到C38来对当前块进行编码然后进行解码。After step C43, the above steps C34 to C38 are repeated to encode and then decode the current block.
在步骤C44的过程之后执行测试以确定当前块是否是所考虑的行SEk中的最后一块。After the process of step C44 , a test is performed to determine whether the current block is the last block in the row SEk considered.
如果不是该情况,在步骤C44之后,再次实现对要编码的下一块MBi进行选择的步骤C39。If this is not the case, after step C44 , step C39 of selecting the next block MB i to be coded is implemented again.
如果当前块是所考虑的行SEk的最后一块,在步骤C45的过程中,图3A或3C的编码装置CO执行如上描述中提到的清空。为此,编码单元UCk将已经与在所考虑的所述行中的每个块的编码期间读取的符号关联的所有比特发送到相应的子流生成模块MGSFk,通过这样的方式从而模块MGSFk将所有比特写入到子流Fm中,该子流包含的二进制串表示所考虑的所述行SEk中被编码的块。这样的情况在图4A和4B中用每行SEk的末端的三角形来表示。If the current block is the last block of the row SEk under consideration, during a step C45, the coding device CO of FIG3A or 3C performs the flushing mentioned above. To this end, the coding unit UCk sends all the bits associated with the symbols read during the coding of each block of the row under consideration to the corresponding substream generation module MGSFk, in such a way that the module MGSFk writes all the bits into a substream Fm containing a binary string representing the block being coded in the row SEk under consideration. This situation is represented in FIG4A and 4B by the triangle at the end of each row SEk.
在图2B所示的步骤C46的过程中,编码单元UC或UCk执行与上述步骤C33相同的步骤,即再次初始化表示预定符号集中包含的符号的出现的概率的间隔。该重新初始化在图4A和4B中用每行SEk开始的黑点来表示。During step C46 shown in FIG2B , the coding unit UC or UCk performs the same steps as in step C33 above, namely reinitializing the intervals representing the probability of occurrence of the symbols contained in the predetermined symbol set. This reinitialization is indicated in FIG4A and 4B by the black dots at the beginning of each row SEk.
在该编码级别上执行步骤C45和C46的好处是,在编码单元UC或编码单元UCk处理的下一块的编码期间,编码器CO处于初始化状态。于是,如下将在描述中进一步描述,并行工作的解码单元可能从该点开始对压缩过的流F进行直接解码,因为它满足处于初始化状态。The benefit of performing steps C45 and C46 at this coding level is that during the coding of the next block processed by coding unit UC or coding unit UCk, the encoder CO is in an initialized state. Thus, as will be described further below, the decoding units operating in parallel may start decoding the compressed stream F directly from this point on, since it satisfies the initialized state.
解码部分的实施例的详细描述Detailed description of an embodiment of the decoding part
现在将描述根据本发明的解码方法的实施例,其中,通过对初始符合H.264/MPEG-4AVC标准的编码器的调整,根据本发明的解码方法例如以软件或硬件的方式来实现。An embodiment of a decoding method according to the invention will now be described, wherein the decoding method according to the invention is implemented, for example, in software or hardware by adapting an encoder originally compliant with the H.264/MPEG-4 AVC standard.
根据本发明的编码方法以包含步骤D1到D4的算法的形式来表示,如图5A所示。The encoding method according to the present invention is represented in the form of an algorithm comprising steps D1 to D4, as shown in FIG5A.
根据本发明的实施例,根据本发明的解码方法在图6A所示的解码装置DO中实现。According to an embodiment of the present invention, the decoding method according to the present invention is implemented in a decoding device DO shown in FIG. 6A .
参考图5A,第一解码步骤是在所述流F中识别L个子流F1,F2,...,Fm,...,FL,该子流分别包含如图4A或4B所示之前被编码的块或宏块MB的P个子集SE1,SE2,...,SEk,...,SEP。为此,流F中的每个子流与指示符关联,该指示符旨在允许解码器DO确定每个子流Fm在流F中的位置。作为变体,在完成上述编码步骤C3时,解码器CO以解码器DO期望的顺序对流F中的子流F1,F2,...,Fm,...,FL进行排序,由此避免在流F中插入子流标识符。该规定由此使其可能降低与数据流F的比特率相关的成本。With reference to FIG5A , the first decoding step consists in identifying, in the stream F, L substreams F1, F2, ..., Fm, ..., FL, each containing P subsets SE1, SE2, ..., SEk, ..., SEP of previously coded blocks or macroblocks MB, as shown in FIG4A or 4B . To this end, each substream in the stream F is associated with an indicator intended to allow the decoder DO to determine the position of each substream Fm in the stream F. As a variant, upon completion of the encoding step C3 described above, the decoder CO sorts the substreams F1, F2, ..., Fm, ..., FL of the stream F in the order desired by the decoder DO, thereby avoiding the need to insert substream identifiers in the stream F. This provision thus makes it possible to reduce the costs associated with the bit rate of the data stream F.
在图4A或4B所示的例子中,所述块MB具有正方形并且都具有相同的大小。取决于图像大小,该图像不必是块大小的倍数,左侧的最后一块和底部的最后一块可以不是正方形。在替代的实施例中,块例如可以是长方形大小并且/或者互相不对齐。In the examples shown in FIG4A or 4B , the blocks MB are square and all have the same size. Depending on the image size, the image does not have to be a multiple of the block size; the last block on the left and the last block at the bottom may not be square. In alternative embodiments, the blocks may be rectangular in size and/or may not be aligned with each other.
每个块或宏块自身可以进一步被分为子块,该子块本身可以再细分。Each block or macroblock may itself be further divided into sub-blocks, which may themselves be further subdivided.
该识别是通过例如图6A所示的流提取模块EXDO来执行的。This identification is performed by, for example, the flow extraction module EXDO shown in FIG. 6A .
在图4A或4B所示的例子中,预定数量P等于6,并且为了图的清楚,仅用虚线示出了四个子集SE1、SE2、SE3、SE4。In the example shown in FIG. 4A or 4B , the predetermined number P is equal to 6, and for clarity of the drawing, only four subsets SE1 , SE2 , SE3 , SE4 are shown with dotted lines.
参考图5A,第二解码步骤D2是对所述块子集SE1、SE2、SE3和SE4中的每个进行解码,所考虑的子集中的块是根据预定的遍历顺序PS来编码的。在图4A或4B所示的例子中,当前子集SEk(1≤k≤P)中的块如箭头PS所示从左到右被依次被解码。在步骤D2完成时,得到被解码的块子集SED1,SED2,SED3,...,SEDk,...,SEDP。Referring to FIG5A , the second decoding step D2 decodes each of the block subsets SE1, SE2, SE3, and SE4, where the blocks in the considered subset are encoded according to a predetermined traversal order PS. In the example shown in FIG4A or 4B , the blocks in the current subset SEk (1≤k≤P) are decoded sequentially from left to right as indicated by the arrow PS. Upon completion of step D2, decoded block subsets SED1, SED2, SED3, ..., SEDk, ..., SEDP are obtained.
这样的解码可以是顺序类型的,且因此在单个解码单元的辅助下执行。Such decoding may be of sequential type and therefore performed with the aid of a single decoding unit.
但是,为了能够受益于多平台解码架构,块子集的解码是并行类型并且通过数量R的解码单元UDk(1≤k≤R)来实现,例如如图6A所示R=4。该规定由此允许解码方法的明显加速。通过本身已知的方式,解码器DO包括缓冲存储器MT,其被适配为包含在当前块解码时被逐渐重复更新的符号出现的概率。However, in order to be able to benefit from a multi-platform decoding architecture, the decoding of the block subset is of a parallel type and is carried out by a number R of decoding units UDk (1≤k≤R), for example, R=4 as shown in FIG6A . This provision thus allows a significant acceleration of the decoding method. In a manner known per se, the decoder DO includes a buffer memory MT adapted to contain the probability of the occurrence of symbols, which is gradually and repeatedly updated during the decoding of the current block.
如图6B中更详细的表示,每个解码单元UDk包括:As shown in more detail in FIG. 6B , each decoding unit UDk comprises:
●通过学习针对至少一个之前被解码的块来计算的至少一个符号出现的概率来对所述当前块进行熵解码的模块,用MDEk来表示,a module for entropy decoding the current block by learning the probability of occurrence of at least one symbol calculated for at least one previously decoded block, denoted MDEk,
●关于所述之前被解码的块来对当前块进行预测解码的模块,用MDPk来表示。The module that performs predictive decoding on the current block with respect to the previously decoded block is denoted by MDPk.
预测解码模块SUDPk能够根据传统的预测技术例如内和/或间模式来对当前块进行解码。The predictive decoding module SUDPk is capable of decoding the current block according to conventional prediction techniques such as intra and/or inter modes.
熵解码模块MDEk本身是CABAC类型,但根据本发明来调整,如下在描述中将进一步描述。The entropy decoding module MDEk is itself of CABAC type, but adapted according to the invention, as will be further described below in the description.
作为变体,熵解码模块MDEk可以是本身已知的霍夫曼解码器。As a variant, the entropy decoding module MDEk may be a Huffman decoder known per se.
在图4A或4B所示的例子中,第一单元UD1从左到右对第一行SE1中的块进行解码。当它到达第一行SE1的最后一块,它传递到第(n+1)行在这里是第5行等的第一块。第二单元UC从左到右对第二行SE2中的块进行解码。当它到达第二行SE2的最后一块,它传递到第(n+2)行这里是第6行等的第一块。该遍历重复直到单元UD4,其从左到右对第四行进行解码。当它到达第一行的最后一块,它传递到第(n+4)行在这里是第8行的第一块,等待,直到最后被识别的子流中的最后一块被解码。In the example shown in FIG4A or 4B, the first unit UD1 decodes the blocks in the first row SE1 from left to right. When it reaches the last block of the first row SE1, it passes on to the first block of the (n+1)th row, here, row 5, and so on. The second unit UC decodes the blocks in the second row SE2 from left to right. When it reaches the last block of the second row SE2, it passes on to the first block of the (n+2)th row, here, row 6, and so on. This process repeats until the unit UD4 decodes the fourth row from left to right. When it reaches the last block of the first row, it passes on to the first block of the (n+4)th row, here, row 8, and so on until the last block of the last identified substream is decoded.
与上述刚才描述的不同的遍历类型当然是可能的。例如,每个解码单元可以不处理如上解释的嵌入的行而是嵌入的列。还可以以任意方向来遍历行或列。Different traversal types than those just described are of course possible. For example, each decoding unit may process not the embedded rows as explained above but the embedded columns. It is also possible to traverse the rows or columns in any direction.
参考图5A,第三解码步骤D3是基于在解码步骤D2中得到的每个被解码的子集SED1,SED2,...,SEDk,...,SEDP来重构被解码的图像ID。更准确地说,每个被解码的子集SED1,SED2,...,SEDk,...,SEDP中被解码的块被发送到例如如图6A所示的图像重构单元URI。在步骤D3的过程中,当这些变得可用时,单元URI将被解码的块写入到被解码的图像。Referring to FIG5A , the third decoding step D3 reconstructs the decoded picture ID based on each decoded subset SED1, SED2, ..., SEDk, ..., SEDP obtained in decoding step D2. More specifically, the decoded blocks in each decoded subset SED1, SED2, ..., SEDk, ..., SEDP are sent to a picture reconstruction unit URI, such as that shown in FIG6A . During step D3 , the unit URI writes the decoded blocks to the decoded picture when it becomes available.
在图5A所示的第四解码步骤D4的过程中,图6A所示的单元URI给出被完全解码的图像ID。During the fourth decoding step D4 shown in FIG. 5A , the unit URI shown in FIG. 6A gives the fully decoded image ID.
现在将参考图5B来描述本发明的各个特定子步骤例如在并行解码的上述步骤D2期间在解码单元UDk中实现的子步骤。Various specific sub-steps of the invention will now be described with reference to FIG. 5B , for example sub-steps implemented in a decoding unit UDk during the above-mentioned step D2 of parallel decoding.
在步骤D21的过程中,解码单元UDk选择图4A或4B所示的当前块SEk中要解码的第一块作为当前块。During step D21 , the decoding unit UDk selects the first block to be decoded among the current blocks SEk shown in FIG. 4A or 4B as the current block.
在步骤D22的过程中,解码单元UDk测试当前块是否是被解码的图像在的第一块,在该情形下,是否是子流F1中的第一块。During a step D22 , the decoding unit UDk tests whether the current block is the first block of the image to be decoded, in this case the first block of the sub-stream F1 .
如果是该情况,在步骤D23的过程中,熵解码模块MDE或MDEk初始化其状态变量。根据所示例子,这需要对表示预定符号集中包含的符号的出现的概率的间隔进行初始化。If this is the case, during a step D23 , the entropy decoding module MDE or MDEk initializes its state variables. According to the example shown, this entails initializing the intervals representing the probabilities of the occurrence of the symbols contained in the predetermined set of symbols.
作为变体,如果使用的熵解码是LZW解码,符号的串的转换表被初始化,从而它包含所有可能的符号一次且仅一次。步骤D23等同于上述编码步骤C33,后面不会再被描述。As a variant, if the entropy decoding used is LZW decoding, the conversion table of strings of symbols is initialized so that it contains all possible symbols once and only once. Step D23 is identical to the encoding step C33 described above and will not be described again.
如果在上述步骤D22之后,当前块不是被解码的图像ID中的第一块,在后续描述中之后被描述的步骤D30的过程中确定必要的之前被解码的块的可用性。If the current block is not the first block in the decoded picture ID after the above-mentioned step D22, the availability of a necessary previously decoded block is determined during step D30 to be described later in the subsequent description.
在步骤D24的过程中,对图4A或4B所示的第一行SE1中的第一当前块MB1进行解码。该步骤D24包括将在下面描述的多个子步骤D241到D246。During step D24, the first current block MB1 in the first row SE1 shown in Figure 4A or 4B is decoded. This step D24 includes a plurality of sub-steps D241 to D246 which will be described below.
在第一子步骤D241的过程中,对与当前块相关的语法元素进行熵解码。该步骤主要包括:In the first sub-step D241, entropy decoding is performed on the syntax elements related to the current block. This step mainly includes:
a)读取与所述第一行SE1关联的子流中包含的比特,a) reading the bits contained in the substream associated with said first row SE1,
b)基于读取的比特来重构符号。b) Reconstruct the symbol based on the read bits.
在上述变体中(根据该变体,使用的解码是LZW解码),读取与当前转换表中的符号代码相对应的数字信息项,基于读取的代码来重构符号,并根据本身已知的过程来更新转换表。In the variant described above, according to which the decoding used is LZW decoding, the digital information item corresponding to the symbol code currently in the conversion table is read, the symbol is reconstructed based on the code read and the conversion table is updated according to a procedure known per se.
更准确地说,与当前块相关的符号元素被例如如图6B所示的CABAC熵解码模块MDE1解码。后者对压缩文件中的比特子流F1进行解码以产生符号元素,并且同时以该方式来重新更新其概率,从而在后者对符号进行解码时,该符号出现的概率等于在上述熵解码步骤C345期间对该相同符号进行编码时得到的那些概率。More precisely, the symbol elements associated with the current block are decoded by a CABAC entropy decoding module MDE1, such as that shown in FIG6B . The latter decodes the bit substream F1 of the compressed file to generate the symbol elements and simultaneously re-updates their probabilities in such a way that, when the latter decodes a symbol, the probability of its occurrence is equal to the probabilities obtained when encoding the same symbol during the aforementioned entropy decoding step C345.
在之后的子步骤D242的过程中,通过内和/或间预测的已知技术来对当前块MB1进行预测解码,在其过程中,关于至少一个之前被解码的块来预测块MB1。During the subsequent sub-step D242 , the current block MB1 is predictively decoded by known techniques of intra and/or inter prediction, during which the block MB1 is predicted with respect to at least one previously decoded block.
毋庸置疑,例如在H.264标准中提出的其他内预测模式也是可以的。Of course, other intra prediction modes, such as those proposed in the H.264 standard, are also possible.
在该步骤的过程中,在前一步骤中解码的语法元素的辅助下执行预测解码,该语法元素特别包括(间或内)预测的类型,并且如果合适,包括内预测的模式、块或宏块(如果后者已经被细分)的划分类型、在间预测模式中使用的参考图像索引和位移矢量。During this step, predictive decoding is performed with the help of syntax elements decoded in the previous step, which include in particular the type of (inter or intra) prediction and, if applicable, the mode of intra prediction, the partitioning type of the block or macroblock (if the latter has been subdivided), the reference image index used in the inter prediction mode and the displacement vector.
上述预测解码步骤使其可能重构预测的块MBp1。The predictive decoding step described above makes it possible to reconstruct the predicted block MBp 1 .
在之后的子步骤D243的过程中,在之前被解码的语法元素的辅助下重构量化的残留块MBq1。During the subsequent sub-step D243 , the quantized residual block MBq 1 is reconstructed with the help of previously decoded syntax elements.
在之后的子步骤D244的过程中,根据传统的反量化操作即与上述步骤C344中执行的量化相反的操作来对被量化的残留块MBq1进行反量化,以产生被解码的反量化的块MBDt1。During the subsequent sub-step D244 , the quantized residual block MBq 1 is dequantized according to a conventional dequantization operation, ie an operation opposite to the quantization performed in the above-mentioned step C344 , to generate a decoded dequantized block MBDt 1 .
在之后的子步骤D245的过程中,对被反量化的块MBDt1进行逆变换即与上述步骤C343中执行的直接变换相反的操作。于是得到被解码的残留块MBDr1。In the subsequent sub-step D245 , the inverse transform is performed on the dequantized block MBDt 1 , which is an operation opposite to the direct transform performed in the above step C343 , thereby obtaining the decoded residual block MBDr 1 .
在之后的子步骤D246的过程中,通过将被解码的残留块MBDr1加到预测块MBp1来构造被解码的块MBD1。被解码的块MBD1由此被呈现为可用,以被解码单元UD1或构成预订数量N的解码单元的一部分的任意其他解码单元使用。During a subsequent sub-step D246 , the decoded block MBD 1 is constructed by adding the decoded residual block MBDr 1 to the prediction block MBp 1. The decoded block MBD 1 is thus rendered available for use by the decoding unit UD1 or any other decoding unit forming part of the predetermined number N of decoding units.
在完成上述解码步骤D246时,例如如图6B所示的熵解码模块MDE1包含在第一块的解码的同时被逐渐重复更新的所有概率。这些概率对应于可能的语法的各个元素并且对应于各个关联的解码上下文。Upon completion of the above decoding step D246 , the entropy decoding module MDE1 , for example as shown in FIG6B , contains all the probabilities that are gradually and repeatedly updated while decoding the first block. These probabilities correspond to the various elements of the possible syntax and to the various associated decoding contexts.
在上述解码步骤D24之后,在步骤D25的过程中执行测试来确定当前块是否是同一行的第j块,其中j是解码器DO已知的预定值,其至少等于1。After the above-mentioned decoding step D24 , a test is performed during a step D25 to determine whether the current block is the jth block of the same row, where j is a predetermined value known to the decoder DO and at least equal to 1.
如果是这种情况,在步骤D26的过程中,针对第j块计算的概率集合被存储在例如如图6A所示以及图4A或4B中的解码器DO中的缓冲存储器MT,所述存储器的大小被适配为存储所计算的概率数量。If this is the case, during step D26 , the set of probabilities calculated for the jth block is stored in a buffer memory MT, for example as shown in FIG. 6A and in the decoder DO in FIG. 4A or 4B , the size of said memory being adapted to store the number of calculated probabilities.
在步骤D27的过程中,单元UDk测试刚才被解码的当前块是否是最后的子流中的最后一块。During a step D27 , the unit UDk tests whether the current block that has just been decoded is the last block of the last sub-stream.
如果是这种情况,在步骤D28的过程中,解码方法结束。If this is the case, the decoding method ends during a step D28 .
如果不是该情况,在步骤D29的过程中根据图4A或4B中的箭头PS所示的遍历顺序来选择要解码的下一块MBi。If this is not the case, the next block MB i to be decoded is selected during a step D29 according to the traversal order indicated by the arrows PS in FIG. 4A or 4B .
如果在上述步骤D25的过程中,当前块不是所考虑的行SEDk的第j块,则执行以上步骤D27。If, during the above step D25 , the current block is not the j-th block of the row SEDk considered, the above step D27 is executed.
在上述步骤D29之后的步骤D30的过程中,确定对于当前块MBi的解码来说必要的之前被解码的块的可用性。考虑到这需要通过不同解码单元UDk来对块进行并行解码的事实,可能出现这些块没有被指定对这些块进行解码的解码单元来解码,且因此它们还不可用。所述确定步骤包括验证位于前一行SEk-1中的预订数量N’的块例如分别位于当前块的上方和右上方的两个块是否可用于当前块的解码,即,它们是否已经被指定对其进行解码的解码单元UDk-1解码。所述确定步骤还包括确定位于要编码的当前块MBi的左侧的至少一块的可用性。但是,考虑到图4A或4B所示的实施例中选择的遍历顺序PS,所考虑的行SEk中的块被依次解码。结果,左侧被解码的块总是可用的(除了一行中的第一块)。在图4A或4B所示的例子中,这需要直接位于当前块的左侧的块被解码。为此,只有分别位于当前块的上方和右上方的两个块的可用性被测试。During step D30, following step D29, the availability of previously decoded blocks necessary for decoding the current block MB i is determined. Given the fact that this requires parallel decoding of blocks by different decoding units UDk, it may happen that these blocks have not yet been decoded by the decoding units designated to decode them and are therefore not yet available. This determination step involves verifying whether a predetermined number N' of blocks located in the previous row SEk-1, for example, two blocks located above and to the right of the current block, are available for decoding the current block, i.e., whether they have already been decoded by the decoding unit UDk-1 designated to decode it. This determination step also includes determining the availability of at least one block located to the left of the current block MB i to be encoded. However, given the selected traversal order PS in the embodiment shown in FIG. 4A or 4B , the blocks in the row SEk considered are decoded sequentially. As a result, the blocks decoded to the left are always available (except for the first block in a row). In the example shown in FIG. 4A or 4B , this requires decoding the block directly to the left of the current block. To this end, only the availability of the two blocks located above and to the right of the current block, respectively, is tested.
由于该测试步骤容易降低解码方法的速度,通过根据本发明的替代方式,图6A所示的时钟CLK被适配为同步块解码的进展以确保分别位于当前块的上方和右上方的两个块的可用性,而不需要验证这两块的可用性。于是,如图4A或4B所示,解码单元UDk总是以被用于当前块的解码的前一行SEk-1中的预订数量N’(这里N’=2)的被解码的块的偏移来开始第一块的解码。从软件的角度来看,这样的时钟的实现使其可能显著加速解码器DO中对每个子集SEk中的块进行处理的时间。Since this test step tends to slow down the decoding method, an alternative approach according to the invention is to adapt the clock CLK shown in FIG6A to synchronize the progress of block decoding to ensure the availability of two blocks located above and to the right of the current block, respectively, without verifying the availability of these two blocks. Thus, as shown in FIG4A or 4B , the decoding unit UDk always starts decoding the first block with an offset of a predetermined number N′ (where N′=2) of decoded blocks in the previous row SEk-1 used for decoding the current block. From a software perspective, such a clock implementation makes it possible to significantly speed up the processing time of each block in the decoder DO for each subset SEk.
在步骤D31的过程中,执行测试以确定当前块是否是所考虑的行SEk中的第一块。During a step D31 , a test is performed to determine whether the current block is the first block in the row SEk considered.
如果是这种情况,在步骤D32的过程中,在缓冲存储器MT中仅读取在前一行SEk-1的第j块的解码期间计算的符号出现的概率。If this is the case, during a step D32 , only the probabilities of the occurrence of the symbols calculated during the decoding of the j-th block of the previous row SEk-1 are read in the buffer memory MT.
根据图4A所示的第一变体,第j块是前一行SEk-1的第一块(j=1)。该读取包括用缓冲存储器MT中存在的概率来替换CABAC解码器的概率。为使其处理第二、第三和第四行SE2、SE3和SE4中的各个第一块,如细线所示的箭头在图4A中示出该读取步骤。According to a first variant shown in FIG4A , the jth block is the first block of the previous row SE k-1 (j=1). This reading involves replacing the probabilities of the CABAC decoder with the probabilities present in the buffer memory MT. This reading step is illustrated in FIG4A as a thin arrow so that it processes the first blocks of the second, third, and fourth rows SE2, SE3, and SE4, respectively.
根据图4B所示的上述步骤D32的第二变体,第j块是前一行SEk-1的第二块(j=2)。该读取包括用在缓冲存储器MT中存在的概率来替换CABAC解码器的概率。为使其处理第二、第三和第四行SE2、SE3和SE4中的各个第一块,如细虚线所示的箭头在图4B中示出该读取步骤。According to a second variant of the above-described step D32, shown in FIG4B , the jth block is the second block (j=2) of the previous row SEk-1. This reading involves replacing the probabilities of the CABAC decoder with the probabilities present in the buffer memory MT. This reading step is illustrated in FIG4B as a thin dashed arrow, so that it processes the first blocks of the second, third, and fourth rows SE2, SE3, and SE4, respectively.
在步骤D32之后,通过重复上述步骤D24到D28来对当前块进行解码。After step D32, the current block is decoded by repeating the above steps D24 to D28.
如果在上述步骤D31之后,当前块不是所考虑的行SEk的第一块,有利地不用读取在位于同一行SEk中的之前被解码的块即在所示例子中直接位于当前块的左侧的被解码的块中出现的概率。确实,考虑到如图4A或4B所示的用于读取位于同一行中的块的遍历顺序PS,在开始当前块的解码时在CABAC解码器中存在的符号出现的概率正好是在该同一行中的前一块的解码后存在的那些概率。If, after the above-mentioned step D31, the current block is not the first block of the row SEk considered, it is advantageous not to read the probabilities present in the previously decoded block located in the same row SEk, i.e., in the example shown, the block immediately to the left of the current block. Indeed, given the traversal order PS for reading blocks located in the same row, as shown in FIG4A or 4B , the probabilities of the symbols present in the CABAC decoder at the start of decoding the current block are exactly the same as those present after the decoding of the previous block in the same row.
结果,在步骤D33的过程中,针对所述当前块的熵解码来学习符号出现的概率,所述概率仅对应于如图4A或4B中的双实线箭头所示的针对同一行中的所述前一块来计算的那些概率。As a result, during step D33 , the probabilities of symbol occurrence are learned for entropy decoding of the current block, which probabilities correspond only to those calculated for the previous block in the same row as indicated by the double solid arrows in FIG. 4A or 4B .
在步骤D33之后,重复上述步骤D24到D28来对当前块进行解码。After step D33, the above steps D24 to D28 are repeated to decode the current block.
在步骤D34的过程之后执行测试以确定当前块是否是所考虑的行SEk中的最后一块。After the process of step D34 a test is performed to determine whether the current block is the last block in the row SEk considered.
如果不是该情况,在步骤D34之后,再次实现对要解码的下一块MBi进行选择的步骤D29。If this is not the case, after step D34 , step D29 of selecting the next block MB i to be decoded is implemented again.
如果当前块是所考虑的行SEk的最后一块,在步骤D35的过程中,解码装置UDk执行与上述步骤D23相同的步骤,即再次初始化表示预定符号集中包含的符号的出现的概率的间隔。该重新初始化在图4A和4B中用每行SEk开始的黑点来表示。If the current block is the last block of the row SEk considered, during a step D35 the decoding means UDk perform the same steps as those of the above-described step D23, namely reinitializing the intervals representing the probabilities of occurrence of the symbols contained in the predetermined set of symbols. This reinitialization is represented in Figures 4A and 4B by the black dots at the beginning of each row SEk.
于是,解码器DO在每行开始时处于初始化状态,由此从选择解码的并行级别以及优化解码的处理时间的角度来看允许更大的灵活性。The decoder DO is then in an initialized state at the beginning of each line, thereby allowing greater flexibility from the point of view of choosing the level of parallelism of the decoding and optimizing the processing time of the decoding.
在图7A所示的示例性编码/解码图中,编码器CO包括单个编码单元UC,如图3A所示,而解码器DO包括六个解码单元。In the exemplary encoding/decoding diagram shown in FIG. 7A , the encoder CO includes a single encoding unit UC, as shown in FIG. 3A , while the decoder DO includes six decoding units.
解码单元对行SE1、SE2、SE3、SE4、SE5和SE6顺序进行编码。在所示例子中,行SE1到SE4被完全编码,行SE5正在被编码的过程中且行SE6还没有被编码。考虑到编码的顺序性,编码单元UC被适配为给出流F,该流包含依次排序的子流F1、F2、F3、F4,以对行SE1、SE2、SE3和SE4进行编码。为此,子流F1、F2、F3和F4用与分别表示被编码的行SE1、SE2、SE3、SE4的相同的阴影来表示。通过在所述被编码的行的编码结束时的清空步骤以及在开始对要编码/解码的下一行的编码或解码时重新初始化概率的间隔,当解码器DO每次读取子流来解码时,解码器处于初始化状态,并且可以以优化的方式用例如可被安装在四个不同平台上的解码单元UD1、UD2、UD3和UD4对四个子流F1、F2、F3和F4进行并行解码。The decoding unit sequentially encodes lines SE1, SE2, SE3, SE4, SE5, and SE6. In the example shown, lines SE1 through SE4 are fully encoded, line SE5 is in the process of being encoded, and line SE6 has not yet been encoded. To account for the sequential nature of encoding, the encoding unit UC is adapted to provide a stream F containing sequentially ordered substreams F1, F2, F3, and F4 to encode lines SE1, SE2, SE3, and SE4. To this end, substreams F1, F2, F3, and F4 are represented using the same shading as the encoded lines SE1, SE2, SE3, and SE4, respectively. By clearing the substreams at the end of encoding each encoded line and reinitializing the probability interval at the start of encoding or decoding the next line to be encoded/decoded, the decoder DO is in an initialized state each time it reads a substream for decoding. The four substreams F1, F2, F3, and F4 can be decoded in parallel in an optimized manner using decoding units UD1, UD2, UD3, and UD4, which can be installed on, for example, four different platforms.
在图7B所示的示例性编码/解码图中,编码器CO包括如图3C所示的两个编码单元UC1和UC2,而解码器DO包括六个解码单元。In the exemplary encoding/decoding diagram shown in FIG. 7B , the encoder CO includes two encoding units UC1 and UC2 as shown in FIG. 3C , while the decoder DO includes six decoding units.
编码单元UC1对奇数排名的行SE1、SE3和SE5进行顺序编码,而编码单元UC2对偶数排名的行SE2、SE4和SE6进行顺序编码。为此,行SE1、SE3和SE5表现为白色背景,而行SE2、SE4和SE6表现为带点的背景。在所示例子中,行SE1到SE4被完全编码,行SE5正在被编码的过程中而行SE6还没有被编码。考虑到所执行的编码的级别为2的并行类型的事实,编码单元UC1被适配为给出子流F2n,该子流可被分解为在对行SE2和SE4分别解码之后得到的两个部分F2和F4。解码器CO因此被适配为向解码器DO发送流F,该流包含两个子流F2n+1和F2n的并置,以及因此与图7A所示不同的子流排序F1、F3、F2、F4。为此,子流F1、F2、F3和F4用与分别表示被编码的行SE1、SE2、SE3、SE4的相同的阴影来表示,子流F1和F3表现为白色背景(奇数排序的行的编码),且子流F2和F4表现为带点的背景(偶数排序的行的编码)。Coding unit UC1 sequentially encodes the odd-ranked lines SE1, SE3, and SE5, while coding unit UC2 sequentially encodes the even-ranked lines SE2, SE4, and SE6. To this end, lines SE1, SE3, and SE5 appear with a white background, while lines SE2, SE4, and SE6 appear with a dotted background. In the example shown, lines SE1 through SE4 are fully encoded, line SE5 is in the process of being encoded, and line SE6 has not yet been encoded. Taking into account the fact that the level of parallelism performed is 2, coding unit UC1 is adapted to provide a substream F 2n , which can be decomposed into two parts, F2 and F4, obtained after decoding lines SE2 and SE4, respectively. Decoder CO is therefore adapted to send a stream F to decoder DO, which contains the concatenation of the two substreams F 2n+1 and F 2n , and thus a substream ordering F1, F3, F2, F4 that is different from that shown in FIG. 7A . To this end, substreams F1, F2, F3 and F4 are represented with the same shading as the coded lines SE1, SE2, SE3, SE4 respectively, substreams F1 and F3 appearing with a white background (coding of odd-numbered lines) and substreams F2 and F4 appearing with a dotted background (coding of even-numbered lines).
关于结合图7A所述的优势,该编码/解码图还展示了能够清空解码器的优势,该解码器的解码并行级别完全独立于编码的并行级别,由此使其可能进一步优化编码器/解码器的操作。As for the advantages described in connection with FIG. 7A , this encoding/decoding diagram also demonstrates the advantage of being able to flush out a decoder whose decoding parallelism level is completely independent of the encoding parallelism level, thereby making it possible to further optimize the operation of the encoder/decoder.
Claims (13)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| FR1155606A FR2977111A1 (en) | 2011-06-24 | 2011-06-24 | METHOD FOR ENCODING AND DECODING IMAGES, CORRESPONDING ENCODING AND DECODING DEVICE AND COMPUTER PROGRAMS |
| FR1155606 | 2011-06-24 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| HK1244375A1 HK1244375A1 (en) | 2018-08-03 |
| HK1244375B true HK1244375B (en) | 2021-01-29 |
Family
ID=
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7751720B2 (en) | Image decoding method, image decoding system, and computer-readable medium for image decoding | |
| HK1244375B (en) | Method for encoding and decoding images, encoding and decoding device, and computer program | |
| HK1244374B (en) | Method for encoding and decoding images, encoding and decoding device, and computer program | |
| HK1244377B (en) | Method for encoding and decoding images, encoding and decoding device, and computer program | |
| HK1244376B (en) | Method for encoding and decoding images, encoding and decoding device, and computer program | |
| HK1244373B (en) | Method for encoding and decoding images, encoding and decoding device, and computer program | |
| HK1243849B (en) | Method for encoding and decoding images, encoding and decoding apparatus, and computer readable storage medium | |
| HK1243849A1 (en) | Method for encoding and decoding images, encoding and decoding apparatus, and computer readable storage medium |