TWI870841B - Method and apparatus for video coding - Google Patents

Abstract

A method and apparatus for Overlapped Boundary Motion Compensation (OBMC). A current MV (Motion Vector) for the current block/subblock is determined and a neighbouring MV for a neighbouring block/subblock associated with the current block/subblock is determined. A number of lines to be applied for OBMC (Overlapped Block Motion Compensation), one or more weightings associated with the OMBC for the number of lines, or both are determined according to one or more boundary matching costs between the current block/subblock and the neighbouring block/subblock, MV difference between the current MV and the neighbouring MV, or predicted luminance data associated with the current block/subblock. The OBMC is applied to at least one of the current block/subblock and the neighbouring block/subblock according to OBMC parameters comprising the number of lines, said one or more weightings, or both.

Description

Video encoding and decoding method and related device

The present invention relates to video coding and decoding. More specifically, the present invention relates to the adaptive design of the number of lines and weightings for the lines in Overlapped Block Motion Compensation (OBMC) processing in a video coding and decoding system.

Versatile video coding (VVC) is the latest international video coding standard jointly developed by the ITU-T Video Coding Experts Group (VCEG) and the Joint Video Experts Group (JVET) of the ISO/IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO/IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile Video Coding, published in February 2021. VVC improves the coding efficiency by adding more coding tools on the basis of its predecessor HEVC (High Efficiency Video Coding), and can also process various types of video sources, including 3-dimensional (3D) video signals.

FIG. 1A illustrates an exemplary adaptive Inter/Intra video coding system including loop processing. For intra prediction, prediction data is derived based on previously encoded and decoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also known as residual. The prediction error is then processed by a transform (T) 118 and a subsequent quantization (Q) 120. The transform and quantization residues are then encoded by an entropy coder 122 for inclusion in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with side information such as motion and codec modes associated with intra-frame prediction and inter-frame prediction, as well as other information such as parameters associated with a loop filter applied to the underlying image area. The side information associated with intra-frame prediction 110, inter-frame prediction 112, and the intra-loop filter 130 is provided to the entropy coder 122 as shown in FIG. 1A. When using the inter-frame prediction mode, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residues are processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to restore the residues. The residues are then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer 134 and used to predict other frames.

As shown in FIG. 1A , the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC 128 may be subject to various impairments. Therefore, a loop filter 130 is often applied to the reconstructed video data before it is stored in a reference picture buffer 134 to improve the video quality. For example, a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF) may be used. It may be necessary to merge the loop filter information into the bitstream so that the decoder can correctly restore the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be incorporated into the bitstream. In FIG. 1A , the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference picture buffer 134. The system in FIG. 1A is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, H.264, or VVC.

As shown in FIG. 1B , the decoder may use similar or identical functional blocks as the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bit stream into quantized transform coefficients and required coding and decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). The intra-frame prediction 150 on the decoder side does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-frame prediction based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) 140 based on the inter-frame prediction information received from the entropy decoder without motion estimation.

According to VVC, similar to HEVC, the input picture is partitioned into non-overlapped square block regions called CTUs (Codec Tree Units). Each CTU can be divided into one or more smaller-sized Codec Units (CUs). The resulting CU partitions can be square or rectangular. In addition, VVC divides CTUs into Prediction Units (PUs) as units for applying prediction processing (e.g., inter-frame prediction, intra-frame prediction, etc.).

The VVC standard incorporates various new codec tools to further improve codec efficiency beyond the HEVC standard. In addition, various new codec tools have been proposed for consideration in the development of new codec standards other than VVC. Among the various new codec tools, a brief description of some codec tools related to the present invention is as follows.

Overlapping Block Motion Compensation ( Overlapped Block Motion Compensation , abbreviated as OBMC )

Overlapping Block Motion Compensation (OBMC) finds the Linear Minimum Mean Squared Error (LMMSE) estimate of a pixel's intensity value based on a motion-compensated signal derived from its nearby block motion vectors (MVs). From an estimation-theoretic point of view, these MVs are viewed as different plausible hypotheses of their true motion, and to maximize codec efficiency, their weights should minimize the mean squared prediction error subject to a unity gain constraint.

When High Efficiency Video Codec (HEVC) was developed, several proposals were made to use OBMC to provide coding gains. Some of them are described below.

In JCTVC-C251 (Peisong Chen et al., “Overlapped block motion compensation in TMuC”, Joint Collaboration Group on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 3rd Meeting: Guangzhou, China, October 7-15, 2010, document: JCTVC-C251), OBMC is applied to geometric partitions. In geometric partitions, a transformed block is likely to contain pixels belonging to different partitions. In geometric partitions, since two different motion vectors are used for motion compensation, pixels at the partition boundary may have large discontinuities, resulting in visual artifacts similar to blocking effects. This in turn reduces the transform efficiency. The two regions created by the geometric partitioning are denoted by Region 1 and Region 2. A pixel from Region 1 (2) is defined as a boundary pixel if any of its four connected neighbors (left, top, right, and bottom) belongs to Region 2 (1). Figure 2 shows an example where the gray pixel belongs to the boundary of Region 1 (gray region) and the white pixel belongs to the boundary of Region 2 (white region). If the pixel is a boundary pixel, motion compensation is performed using a weighted sum of the motion predictions from the two motion vectors. The weight is 3/4 for the prediction using the motion vector of the region containing the boundary pixel and 1/4 for the prediction using the motion vector of the other regions. Overlapping borders improves the visual quality of the reconstructed video while also providing BD rate gains.

In JCTVC-F299 (Liwei Guo et al., "CE2: Overlapped Block Motion Compensation for 2NxN and Nx2N Motion Partitions", ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 Joint Working Group on Video Coding (JCT-VC), 6th Meeting: Torino, July 14-22, 2011, document: JCTVC-F299), OBMC is applied to symmetrical motion partitions. If the coding unit (CU) is divided into two 2NxN or Nx2N prediction units (PUs), OBMC is applied to the horizontal boundary of the two 2NxN prediction blocks and the vertical boundary of the two Nx2N prediction blocks. Since these partitions may have different motion vectors, the pixels at the partition boundaries may have large discontinuities, which may produce visual artifacts and also reduce transform/encodec efficiency. In JCTVC-F299, OBMC is introduced to smooth the boundaries of motion partitions.

Figures 3A-B illustrate examples of OBMC for 2NxN (Figure 3A) and Nx2N blocks (Figure 3B). Gray pixels are pixels belonging to partition 0 and white pixels are pixels belonging to partition 1. The overlap area in the luminance component is defined as 2 rows (columns) of pixels on either side of the horizontal (vertical) boundary. For pixels that are 1 row (column) away from the partition boundary, i.e., pixels marked as A in Figures 3A-B, the OBMC weighting factors are (3/4, 1/4). For pixels that are 2 rows (columns) away from the partition boundary, i.e., pixels marked as B in Figures 3A-B, the OBMC weighting factors are (7/8, 1/8). For chroma components, the overlap area is defined as 1 row (column) of pixels on each side of the horizontal (vertical) boundary, with weight factors of (3/4, 1/4).

Currently, OBMC is performed after normal MC, and BIO is also applied in these two MC processes separately. That is, the MC result of the overlapping area between two CUs or PUs is generated by another process instead of in the normal MC process. Bi-Directional Optical Flow (BIO) is then applied to improve these two MC results. This helps skip redundant OBMC and BIO processes when two adjacent MVs are the same. However, compared with integrating the OBMC process into the normal MC process, the bandwidth and MC operations required for the overlapping area are increased. For example, the current PU size is 16x8, the overlapping area is 16x2, and the interpolation filter in MC is 8-tap. If OBMC is executed after normal MC, then we need (16+7)x(8+7) + (16+7)x(2+7) = 552 reference pixels per reference list for the current PU and the associated OBMC. If the OBMC operation is merged into one phase with normal MC, then there are only (16+7)x(8+2+7) = 391 reference pixels per reference list for the current PU and the associated OBMC. Therefore, in the following, several methods are proposed to reduce the computational complexity or memory bandwidth of BIO when BIO and OBMC are enabled simultaneously.

In JEM (Joint Exploration Model), OBMC is also applied. In JEM, unlike H.263, OBMC can be turned on and off using CU-level syntax. When OBMC is used in JEM, OBMC is performed on all motion compensation (MC) block boundaries except the right and bottom boundaries of the CU. In addition, it is applicable to both luminance and chrominance components. In JEM, one MC block corresponds to one codec block. When the CU is encoded and decoded in sub-CU mode (including sub-CU merging, affine, and FRUC modes), each sub-block of the CU is an MC block. In order to handle CU boundaries in a unified manner, OBMC is performed on all MC block boundaries at the sub-block level, where the sub-block size is set to 4×4, as shown in Figures 4A and 4B.

When OBMC is applied to the current sub-block, in addition to the current motion vector, the motion vectors of four connected neighboring sub-blocks (if available and different from the current motion vector) are also used to derive the prediction block of the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal of the current sub-block. The prediction block based on the motion vector of the neighboring sub-block is denoted as PN, N represents the index of the neighboring upper, lower, left and right sub-blocks, and the prediction block based on the motion vector of the current sub-block is denoted as PC. FIG. 4A shows an example of OBMC of a sub-block of the current CU 410, which uses the adjacent upper sub-block (i.e., when the current sub-block is _PN1 ), the left adjacent sub-block (i.e., when the current sub-block is _PN2 ), and the left and upper sub-blocks (i.e., when the current sub-block is _PN3 ). FIG. 4B shows an example of OBMC of the current CU 420 for ATMVP mode, where the block PN uses MVs from four adjacent sub-blocks for OBMC. When the PN is based on the motion information of the adjacent sub-block containing the same motion information as the current sub-block, OBMC is not performed from the PN. Otherwise, each sample of the PN is added to the same sample in the PC, i.e., four columns/columns of the PN are added to the PC. Weight factors {1/4, 1/8, 1/16, 1/32} are used for PN, and weight factors {3/4, 7/8, 15/16, 31/32} are used for PC. Small MC blocks (i.e., when the height or width of the codec block is equal to 4 or the CU is coded in sub-CU mode) are an exception, and only two columns/columns of the PN are added to the PC. In this case, weight factors {1/4, 1/8} are used for PN, and weight factors {3/4, 7/8} are used for PC. For PN generated based on motion vectors of vertically (horizontally) neighboring sub-blocks, samples in the same column (column) of the PN are added to the PC with the same weight factor.

In JEM, for CUs with a size less than or equal to 256 luma samples, a CU level flag is sent to indicate whether OBMC is applied to the current CU. For CUs with a size greater than 256 luma samples or not encoded or decoded using AMVP mode, OBMC is applied by default. In the encoder, when OBMC is applied to a CU, its impact is taken into account in the motion estimation stage. OBMC uses the prediction signal formed by the motion information of the top neighboring blocks and the left neighboring blocks to compensate the top and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

OBMC is applied in the Joint Exploration Model for VVC development (JEM). For example, as shown in FIG. 5 , for the current block 510, if the upper block and the left block are coded and decoded in inter-frame mode, the MV of the upper block is taken to generate OBMC block A, and the MV of the left block is taken to generate OBMC block L. The predictors of OBMC block A and OBMC block L are blended with the current predictor. In order to reduce the memory bandwidth of OBMC, it is recommended to perform MC of the upper 4 rows, MC of the left 4 columns, and neighboring blocks. For example, when executing the MC of the upper block, 4 additional columns are taken out to generate a block of (above block + OBMC block A). The prediction sub-data of OBMC block A is stored in the buffer and used to encode and decode the current block. When executing the MC of the left block, 4 additional columns are taken out to generate a block of (left block + OBMC block L). The prediction sub-data of OBMC block L is stored in the buffer and used to encode and decode the current block. Therefore, when performing MC on the current block, four additional columns and four columns of reference pixels are taken out to generate the predictor, OBMC block B and OBMC block R of the current block, as shown in FIG. 6A (OBMC block BR as shown in FIG. 6B may also be generated). OBMC block B and OBMC block R are stored in a buffer for OBMC processing of the bottom neighboring block and the right neighboring block.

For MxN blocks, if MV is not an integer and an 8-tap interpolation filter is applied, a reference block of size (M+7)x(N+7) is used for motion compensation. However, if BIO and OBMC are applied, additional reference pixels are required, which increases the worst-case memory bandwidth.

There are two different schemes to implement OBMC. In the first scheme, OBMC blocks are generated in advance when performing motion compensation for each block. These OBMC blocks will be stored in a local buffer for neighboring blocks. In the second scheme, OBMC blocks are generated before the blending process for each block when performing OBMC.

In both schemes, several methods are proposed to reduce the computational complexity, especially for interpolation filtering, and the additional bandwidth requirement of OBMC.

Template matching based OBMC （ Template Matching based OBMC ）

Recently, a template matching based OBMC scheme was proposed for the emerging international coding standard (JVET-Y0076). As shown in Figure 7, for each top block of size 4×4 at the top CU boundary, the above template size is equal to 4×1. In Figure 7, box 710 corresponds to the CU. If N adjacent blocks have the same motion information, the above template size is expanded to 4N×1 because the MC operation can be processed at one time, which is the same as in ECM-OBMC. For each left block of size 4×4 at the left CU boundary, the left template size is equal to 1×4 or 1×4N.

For each 4×4 top block (or N 4×4 block groups), the predicted values of the boundary samples are derived according to the following steps: – Take block A as the current block and its upper neighbor block AboveNeighbour_A as an example. The operation of the left block is performed in the same way. – First, three template matching costs (Cost1, Cost2, Cost3) are measured between the reconstructed sample of the template and its corresponding reference sample derived from the MC process based on the following three types of motion information by SAD: i. Cost1 is calculated based on the motion information of A. ii. Cost2 is calculated based on the motion information of AboveNeighbour_A. iii. Cost3 is calculated based on the weighted prediction of the motion information of A and AboveNeighbour_A, with weighting factors of 3/4 and 1/4 respectively. - Secondly, choose one of the three methods to calculate the final prediction result of the boundary sample by comparing Cost1, Cost2 and Cost 3.

The original MC result using the motion information of the current block is represented as Pixel1, and the MC result using the motion information of the neighboring block is represented as Pixel2. The final prediction result is represented as NewPixel. ․ If Cost1 is the smallest, then . ․ If Cost2 is the smallest, use blending mode 1. For the luminance block, the number of blending pixel rows is 4. – – – – For chroma blocks, the number of mixed pixel columns is 1. – ․ If Cost3 is minimum, use blending mode 2. For luma blocks, the number of blended pixel columns is 2. – – For chroma blocks, the number of mixed pixel rows/columns is 1. –

In this application, an adaptive OBMC method is disclosed to improve encoding and decoding efficiency.

The present invention provides a video encoding and decoding method and related devices.

The present invention provides a video encoding and decoding method, comprising receiving input data associated with a current block/subblock, wherein the input data includes pixel data of the current block/subblock to be encoded on the encoder side or coded data associated with the current block/subblock to be decoded on the decoder side; determining a current motion vector of the current block/subblock; determining a neighboring motion vector of a neighboring block/subblock associated with the current block/subblock; determining a current motion vector of the current block/subblock according to one or more of the current block/subblock and the neighboring ... The method comprises: determining a number of rows to be applied for overlapped block motion compensation, one or more weights associated with overlapped block motion compensation for the number of rows, or both based on a plurality of boundary matching costs, a motion vector difference between a current motion vector and a neighboring motion vector, or predicted luminance data associated with a current block/subblock; and applying overlapped block motion compensation to at least one of the current block/subblock and the neighboring block/subblock according to overlapped block motion compensation parameters including the number of rows, the one or more weights, or both.

The present invention also provides a device for video encoding and decoding, the device includes one or more electronic devices or processors, and is used to: receive input data associated with a current block/subblock, wherein the input data includes pixel data of the current block/subblock to be encoded on the encoder side or coded data associated with the current block/subblock to be decoded on the decoder side; determine a current motion vector of the current block/subblock; determine a neighboring motion vector of a neighboring block/subblock associated with the current block/subblock; and determine a current motion vector of a neighboring block/subblock associated with the current block/subblock according to the pixel data of the current block/subblock and the neighboring block/subblock. The method comprises: determining a number of rows to be applied for overlapped block motion compensation, one or more weights associated with overlapped block motion compensation for the number of rows, or both based on one or more boundary matching costs between neighboring blocks/subblocks, a motion vector difference between a current motion vector and a neighboring motion vector, or predicted luminance data associated with the current block/subblock; and applying overlapped block motion compensation to at least one of the current block/subblock and the neighboring block/subblock according to overlapped block motion compensation parameters including the number of rows, the one or more weights, or both.

The video encoding and decoding method and related devices of the present invention can improve the encoding and decoding efficiency.

These and other objects of the present invention will no doubt become apparent to those having ordinary skill in the art after reading the following detailed description of the preferred embodiments as illustrated in the various figures and drawings.

It will be readily understood that the components of the present invention as generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention as illustrated is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References throughout this specification to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in conjunction with that embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily all refer to the same embodiment.

In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more of the specific details, or using other methods, components, etc. In other cases, well-known structures or operational details are not shown or are not shown to avoid obscuring aspects of the present invention. The illustrated embodiments of the present invention will be best understood with reference to the accompanying drawings, in which like parts are represented by like numbers throughout. The following description is intended to be exemplary only and simply illustrates certain selected embodiments of the apparatus and methods consistent with the present invention as claimed herein.

A new OBMC hybrid adaptive weighting method is disclosed. In the method, when OBMC is applied to the current block, the number of rows or the weight of the rows in the OBMC can be adaptively changed, for example, according to its block width or height, aspect ratio, block shape, MV of the current block/subblock, MV of neighboring blocks/subblocks, weighted or unweighted MV difference between the current subblock and neighboring blocks, boundary similarity, predictor luminance signal, MV gradient at the boundary, cost metric or a combination thereof. The number of rows or the weight of the rows in the OBMC can be changed according to some specific characteristics in the CU, or it can always remain unchanged regardless of the CU characteristics.

In one embodiment, when the current block area (i.e., block width×block height) is less than or greater than a threshold, the number of rows or the weight of the rows in the OBMC will decrease or increase accordingly.

In another embodiment, when the aspect ratio of the current block is less than or greater than a threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly. For example, if the MV difference is less than the threshold, the first weighting can be used, and if the MV difference is greater than the threshold, the second weighting can be used. In one example, the first weight is greater than the second weight and in another example, the second weight is greater than the first weight.

In another embodiment, the MV of the current sub-block is compared with the MV of the adjacent block to obtain the MV difference. If the MV difference is less than or greater than the threshold, the number of rows or the weight of the row in the OBMC will be reduced or increased accordingly. In another example, after the MV difference is obtained, the MV difference is weighted by multiplying the MV difference by another value to obtain a weighted MV difference, and then the weighted MV difference is compared with the threshold.

In another embodiment, in order to measure the boundary similarity, when it is determined to perform OBMC on the current sub-block, the boundary matching cost is first calculated by comparing the current sample with the neighboring samples. The current sample can be a predicted sample or a reconstructed sample. Similarly, the neighboring samples can be neighboring predicted samples or neighboring reconstructed samples. To determine the number of rows or the weight of rows in OBMC, the boundary matching cost can be used as follows: Cost 1 = BoundaryMatchingCost(Neighboring Reconstruction, Current Predictor or Current Reconstruction) Cost 2 = BoundaryMatchingCost(Neighboring Reconstruction, Neighboring Predictor) For Neighboring Predictor: Weighting = Cost 1 / (Cost 1 + Cost 2) For Current Predictor: Weighting = Cost 2 / (Cost 1 + Cost 2)

In the above, the predicted or reconstructed sample of the current block or sub-block is derived based on the MV of the current block or sub-block. The neighbor predictor refers to the predictor generated based on the MV of the neighboring block or sub-block. It can be seen that the ratio of the weight of the neighbor predictor to the weight of the current predictor corresponds to the ratio of the boundary matching cost 1 to the boundary matching cost 2.

In another example, instead of always using weighting based on cost 1 and cost 2, one cost can be selected from cost 1 and cost 2 and compared to a threshold. If the selected cost is less than or greater than the threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly. In another embodiment, a boundary matching cost or a template matching cost is calculated. Different comparisons can be applied to costs and thresholds. For example, the comparison may correspond to "less than or equal to", "less than", "greater than or equal to", or "greater than". Depending on the comparison results, the blending line number or blending weighting can be reduced or increased. The threshold can depend on block width or height, aspect ratio, block shape, MV of the current block/subblock, MV of neighboring blocks/subblocks, weighted or unweighted MV difference between current subblock and neighboring blocks, boundary similarity, predicted sub-luminance signal, MV gradient at boundaries, cost metric, or a combination thereof.

In another embodiment, when it is determined to perform OBMC on the current sub-block, the luminance signal of the current predictor is first calculated. If the luminance value (or the average value of the current block/sub-block) is less than or greater than a threshold, the number of rows in OBMC or the weight of the row will be reduced or increased accordingly.

In another embodiment, when it is determined to perform OBMC on the current sub-block, the sample level offset is first calculated using the spatial gradient to obtain the MV gradient. If the MV gradient is less than or greater than the threshold, the number of rows in the OBMC or the weight of the row will be reduced or increased accordingly. In another example, when the comparison with the threshold is performed, optical flow derivation will be applied based on the MV gradient.

Any of the aforementioned proposed OBMC methods may be implemented in an encoder and/or a decoder. For example, at the encoder end, the required processing may be implemented in a prediction sub-derivation module (e.g., a portion of an inter-frame prediction unit as shown in FIG. 1A ). However, the encoder may also use additional processing units to implement the required processing. For the decoder end, the required processing may be implemented in a prediction sub-derivation module, such as a portion of the MC unit 152 shown in FIG. 1B . However, the decoder may also use additional processing units to implement the required processing. The inter-frame prediction 112 and MC 152 are shown as separate processing units, which may correspond to executable software or firmware code stored on a medium (e.g., hard disk or flash memory) for use in a CPU (central processing unit) or a programmable device (e.g., a DSP (digital signal processor) or an FPGA (field programmable gate array)). Alternatively, any of the proposed methods can be implemented as a circuit coupled to a predictor derivation module of an encoder and/or a predictor derivation module of a decoder in order to provide the information required by the predictor derivation module.

FIG. 8 shows a flow chart of an exemplary overlapping block motion compensation (OBMC) process in a video coding and decoding system according to an embodiment of the present invention. The steps shown in the flow chart can be implemented as program codes that can be executed on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flow chart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to execute the steps in the flow chart. According to the method, input data associated with the current block is received in step 810, wherein the input data includes pixel data of the current block to be encoded on the encoder side or coded data associated with the current block to be decoded on the decoder side. In step 820, the current MV (motion vector) of the current block/subblock is determined. In step 830, the neighboring MVs of the neighboring blocks/subblocks associated with the current block/subblock are determined. In step 840, the number of rows to be applied for OBMC (Overlapping Block Motion Compensation), one or more weights associated with the OMBC of the number of rows, or both are determined based on one or more boundary matching costs between the current block/subblock and the neighboring blocks/subblocks, the MV difference between the current MV and the neighboring MVs, or the predicted luminance data associated with the current block/subblock. In step 850, OBMC is applied to at least one of the current block/sub-block and the neighboring blocks/sub-blocks based on OBMC parameters including the number of rows, the one or more weights, or both.

The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. Without departing from the spirit of the present invention, a person skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention. In this disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, a person skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics.

The above description is provided to enable one having ordinary knowledge in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to one having ordinary knowledge in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but rather to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are illustrated to provide a thorough understanding of the present invention. However, one having ordinary knowledge in the art will understand that the present invention may be practiced.

The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, an embodiment of the present invention can be one or more circuits integrated into a video compression chip or a program code integrated into the video compression software to perform the processing described herein. The embodiments of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also involve many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines the specific methods embodied by the present invention. Software code or firmware code can be developed in different programming languages and in different formats or styles. Software code can also be compiled for different target platforms. However, different code formats, styles and languages of software code and other ways of configuring code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the invention is indicated by the appended patent application rather than by the foregoing description. All changes that fall within the meaning and range of equivalents of the patent application should be included within its scope.

110: Intra-frame prediction 112: Inter-frame prediction 114: Switch 116: Adder 118: Transform 120: Quantization 122: Entropy encoder 130: Loop filter 124: Inverse quantization 126: Inverse transform 128: Reconstruction 134: Reference picture buffer 136: Prediction data 140: Entropy decoder 150: Intra-frame prediction 152: MC 410, 420, 710: CU 510: Current block 810~850: Steps

FIG. 1A illustrates an exemplary adaptive Inter/Intra video coding system including loop processing. FIG. 1B shows a corresponding decoder for the encoder in FIG. 1A. FIG. 2 illustrates an example of overlapping motion compensation for geometric partitions. FIG. 3A-B illustrate examples of OBMC for 2NxN (FIG. 3A) and Nx2N blocks (FIG. 3B). FIG. 4A illustrates an example of a sub-block to which OBMC is applied, wherein the example includes a sub-block at a CU/PU boundary. FIG. 4B illustrates an example of a sub-block to which OBMC is applied, wherein the example includes a sub-block encoded and decoded in AMVP mode. FIG. 5 illustrates an example of OBMC processing using upper and left neighboring blocks for the current block. FIG. 6A shows an example of OBMC processing of the right and bottom portions of the current block using neighboring blocks from the right and bottom. FIG. 6B shows an example of OBMC processing of the right and bottom portions of the current block using neighboring blocks from the right, bottom, and bottom-right. FIG. 7 shows an example of OBMC based on template matching, where the template size is equal to 4×1 for each top block having a size of 4×4 at the top CU boundary. FIG. 8 shows a flowchart of an exemplary overlapping block motion compensation (OBMC) process in a video coding and decoding system according to an embodiment of the present invention.

810~850: Steps

Claims (14)

A video encoding and decoding method, the method comprising: Receiving input data associated with a current block/subblock, wherein the input data comprises pixel data of the current block/subblock to be encoded on the encoder side or coded data associated with the current block/subblock to be decoded on the decoder side; Determining a current motion vector of the current block/subblock; Determining a neighboring motion vector of a neighboring block/subblock associated with the current block/subblock; Determine the number of rows to be applied for overlapped block motion compensation, one or more weights associated with the overlapped block motion compensation for the number of rows, or both based on one or more boundary matching costs between the current block/subblock and the neighboring block/subblock, a motion vector difference between the current motion vector and the neighboring motion vector, or predicted luminance data associated with the current block/subblock; and Apply the overlapped block motion compensation to at least one of the current block/subblock and the neighboring block/subblock based on overlapped block motion compensation parameters including the number of rows, the one or more weights, or both. A video encoding and decoding method as described in claim 1, wherein the number of rows, the one or more weights, or both are determined by comparing the motion vector difference with a threshold. A video encoding and decoding method as described in claim 2, wherein if the motion vector difference is less than the threshold, a first weight is used, and if the motion vector difference is greater than the threshold, a second weight is used. A video encoding and decoding method as described in claim 3, wherein the first weight is greater than the second weight. A video encoding and decoding method as described in claim 3, wherein the second weight is greater than the first weight. The video encoding and decoding method as described in claim 1, wherein the number of rows, the one or more weights, or both are determined based on the one or more boundary matching costs including a first boundary matching cost and a second boundary matching cost, wherein the first boundary matching cost is calculated between one or more first predicted or reconstructed samples of the current block/sub-block and one or more reconstructed samples of the neighboring block/sub-block, The second boundary matching cost is calculated between one or more second predicted or reconstructed samples of the previous block/sub-block and one or more reconstructed samples of the neighboring block/sub-block, and the one or more first predicted or reconstructed samples of the current block/sub-block are determined using the current motion vector, and the one or more second predicted or reconstructed samples of the current block/sub-block are determined using the neighboring motion vector. A video coding and decoding method as described in claim 6, wherein the one or more weights include the one or more first predicted or reconstructed samples of the current block/sub-block and the one or more second predicted or reconstructed samples of the current block/sub-block, respectively, to form a first weight and a second weight of one or more final predicted or reconstructed samples of the current block/sub-block, wherein the ratio of the first weight to the second weight corresponds to the ratio of the second boundary matching cost to the first boundary matching cost. A video encoding and decoding method as described in claim 6, wherein the number of rows, the one or more weights, or both are determined in response to comparing one of the first boundary matching cost and the second boundary matching cost with a threshold. A video encoding and decoding method as described in claim 8, wherein if one of the first boundary matching cost and the second boundary matching cost is less than the threshold, the number of rows, the one or more weights, or both are reduced or increased accordingly. A video encoding and decoding method as described in claim 8, wherein if one of the first boundary matching cost and the second boundary matching cost is greater than the threshold, the number of rows, the one or more weights, or both are reduced or increased accordingly. A video encoding and decoding method as described in claim 8, wherein the threshold depends on the block width or height, aspect ratio or block shape of the current block/sub-block, the current motion vector, the neighboring motion vector, weighted or unweighted motion vector difference, boundary similarity, predicted sub-luminance signal strength, motion vector gradient at one or more boundaries between the current block/sub-block and the neighboring block/sub-block, cost metric or a combination thereof. A video encoding and decoding method as described in claim 1, wherein the number of rows, the one or more weights, or both are determined based on the predicted brightness data associated with the current block/sub-block. A video encoding and decoding method as described in claim 12, wherein the predicted brightness data corresponds to an average value of the current block/sub-block. A device for video encoding and decoding, the device includes one or more electronic devices or processors, which are arranged to: Receive input data associated with a current block/subblock, wherein the input data includes pixel data of the current block/subblock to be encoded on the encoder side or coded data associated with the current block/subblock to be decoded on the decoder side; Determine a current motion vector of the current block/subblock; Determine a neighboring motion vector of a neighboring block/subblock associated with the current block/subblock; Determine the number of rows to be applied for overlapped block motion compensation, one or more weights associated with the overlapped block motion compensation for the number of rows, or both based on one or more boundary matching costs between the current block/subblock and the neighboring block/subblock, a motion vector difference between the current motion vector and the neighboring motion vector, or predicted luminance data associated with the current block/subblock; and Apply the overlapped block motion compensation to at least one of the current block/subblock and the neighboring block/subblock based on overlapped block motion compensation parameters including the number of rows, the one or more weights, or both.