CN1263309C - Motion vector prediction method used for video coding - Google Patents

Abstract

The invention discloses a motion vector prediction method for video coding. The method can be used in the direct coding mode in B frames to obtain more accurate motion vector prediction, so as to more truly reflect the motion of objects in the video. The principle of linear interpolation realizes the motion vector prediction of macroblocks in B frames. Specifically: first obtain the motion vector of each block of the backward reference frame; for each block’s motion vector, obtain the current motion track from the forward reference frame to the backward reference frame; For the most matching block on the corresponding trajectory, after completing the motion estimation process of all blocks in the backward reference frame, the corresponding motion vector is obtained from the current image. The present invention can more truly embody the movement of the object in the video and obtain more accurate motion vector prediction; it can realize a new type of predictive coding by combining with forward predictive coding and backward predictive coding.

Description

Motion Vector Prediction Method for Video Coding

technical field

The present invention relates to a motion vector prediction method for video coding, in particular to a direct coding mode that can be used in B frames to obtain more accurate motion vector prediction, so as to more truly reflect the motion of objects in the video . A method for realizing motion vector prediction of a macroblock in a B frame by using the principle of linear interpolation, belonging to the technical field of digital audio and video coding.

Background technique

Efficient video codec technology is the key to realize high-quality, low-cost multimedia data storage and transmission. Currently commonly used coding methods include predictive coding, orthogonal transform coding, vector quantization coding, and so on. These methods are based on signal processing theory and are often called first-generation coding techniques. The more popular international standards for image coding are based on this coding theory, using a coding method based on block matching motion compensation, discrete cosine transform and quantization. Typically, there are international standards such as MPEG-1, MPEG-2 and MPEG-4 introduced by the International Organization for Standardization/International Electrotechnical Commission First Joint Technical Group (ISO/IEC JTC1), as well as international standards proposed by the International Telecommunication Union (ITU-T). H.26x series recommended. These video coding standards are widely used in the industry.

The above-mentioned video coding standards all adopt the hybrid video coding (Hybrid Video Coding) strategy, which usually includes four main modules: prediction, transformation, quantization and information entropy coding. The main function of the prediction module is to use the encoded and reconstructed image to predict the current image to be encoded (inter-frame prediction), or use the encoded and reconstructed image block (or macroblock) in the image to predict the current image block to be encoded (or macroblock) for prediction (intra prediction). The main function of the transformation module is to transform the input image block to another space, so that the energy of the input signal can be concentrated on the low-frequency transformation coefficients as much as possible, thereby reducing the correlation between elements in the image block and facilitating compression. The main function of the quantization module is to map the transformed coefficients to a finite element set that is beneficial to encoding; the main function of the information entropy coding module is to express the quantized transformation coefficients with variable-length codes according to statistical laws.

The video decoding system includes modules similar to video encoding, and mainly reconstructs the decoded image by entropy decoding, inverse quantization, and inverse transformation of the input code stream. In addition to the above modules, the video codec system usually includes some auxiliary coding tools, and these tools will also contribute to the coding performance (compression ratio) of the entire system.

The main function of prediction based on motion compensation is to eliminate the redundancy in time of the video sequence. Most of the coding efficiency in video coding comes from the prediction module. The video encoding process is the process of encoding each frame of the video sequence. Commonly used video coding systems encode each frame of image with a macroblock as the basic unit. When coding each frame of image, it can be divided into intra-frame (I frame) coding, predictive (P frame) coding and bidirectional predictive (B frame) coding, etc. Generally speaking, during encoding, I frame, P frame and B frame are encoded interspersedly, for example, in the order of IBBPBBP.

The introduction of B-frames can effectively solve the "occlusion problem" caused by different motion directions or speeds between moving objects or between objects and the background. The coding of the B frame can make the coding compression efficiency reach a code rate of more than 200:1. The coding of the macroblock in the B frame includes four modes: Direct, Forward Prediction, Backward Prediction and Bi-directional Prediction. Since the B-frame technology needs to perform forward and backward motion estimation at the same time, the computational complexity is relatively high; at the same time, in order to distinguish the forward and backward motion vectors, additional identification information must be introduced. However, the direct encoding mode may not encode the motion vector information, therefore, the bits for encoding the motion vector information may be reduced, and the encoding efficiency may be improved effectively.

In coding, the key to whether a macroblock adopts the Direct coding mode is whether the prediction of the motion vector is accurate. The direct mode in traditional B-frame coding uses adjacent macroblocks in time domain to realize motion vector prediction. In other words, when encoding a macroblock in a B frame, it must be derived from the motion vector of the corresponding macroblock in the first reference frame. Therefore, this prediction method is based on the fact that adjacent macroblocks have similar motion trajectories. on a hypothetical basis. However, since the motion of the current macroblock does not completely depend on the motion of adjacent macroblocks in the time domain, especially at the edge of objects, this kind of prediction often leads to inaccurate prediction results due to the relative offset of motion between objects. . This will be more pronounced when the block size for motion compensated prediction has been reduced to 4x4.

The image between two reference frames in the video can be partially restored by the method of linear interpolation; therefore, this method can be applied to B frame coding. Referring to Figure 7, the traditional linear interpolation technology is implemented based on the pixel level, where B is the frame that needs to be interpolated, and the two P frames before and after are formed by normal encoding; when decoding, you can simply use the two before and after P frames are used to interpolate B frames, that is, the position of each block corresponds to the position of the P frame, and then the pixels of the corresponding blocks of the previous and subsequent P frames are linearly weighted. Although this technology can effectively realize static scene encoding, it often cannot achieve good results for objects with complex motion.

In view of this, the method of motion compensation interpolation can be introduced to solve the coding of linear and uniform moving objects by interpolating motion vectors. However, the moving object does not always move along a straight line, and the motion vector prediction is often inaccurate due to noise or brightness changes, especially the discontinuity formed on the moving object (such as the edge part of the object).

Contents of the invention

The main purpose of the present invention is to provide a kind of motion vector prediction method for video coding, utilize the linear interpolation principle to realize the motion vector prediction of macroblock in B frame, can be used in the Direct encoding mode in B frame, obtain more accurate Motion vector prediction, so as to reflect the motion of objects in the video more realistically; at the same time, it is combined with forward predictive coding and backward predictive coding to realize a new type of predictive coding.

The purpose of the present invention is achieved like this:

The motion vector of each macroblock in the B frame to be encoded is predicted by the motion vector between two adjacent reference frames, and the prediction of the motion vector of a macroblock in the B frame does not come from the first reference frame behind The motion vector of the macroblock at the corresponding position, if the macroblock is on the motion trajectory of a certain macroblock in the first reference frame later, its motion vector is the same as the motion trajectory of the macroblock.

In order to estimate the motion vector of each macroblock in the B frame more accurately, the motion vector of each motion estimation block in each macroblock is predicted by a method similar to linear interpolation. When encoding a B frame, the prediction direction of each macroblock in the backward reference frame must first be preprocessed, and the corresponding B frame can be found through the motion trajectory of the backward reference frame relative to the forward reference frame. macroblock. The specific prediction methods are:

1. Obtain the motion vector of each block of the backward reference frame;

2. For each fast motion vector, obtain the motion trajectory from the forward reference frame to the backward reference frame;

3. Using the motion trajectory, calculate the most matching block falling on this trajectory in the current image;

4. After completing the motion estimation process of all blocks in the backward reference frame, the corresponding motion vector will be obtained in the current image.

Due to image noise and irregular motion of different objects, some blocks in the current image may "pass through" more than one motion vector. This situation can be resolved in one of the following ways:

The first method is: adopt the "proximity principle", the object motion for continuous images in the time domain is basically linear, and the deviation from the corresponding position of another frame will not be too large, so it can be considered that the distance from the current position in the space domain is The motion vector obtained from the nearest block is used as the candidate vector.

The second method is: use the "average value" method to obtain the candidate vector; since in many cases, the edge blocks of the object often cause the "occlusion problem" of the motion, so the method of "passing through" the corresponding block is adopted. Several different motion vectors are averaged, and the averaged motion vector is used as a candidate vector.

In addition, when the motion vector "passed" by some blocks is zero, it can be assumed that this block has no direct mode; or use the spatial prediction method to predict the motion vector; or use the forward motion vector method to realize the prediction of the motion vector ; Or use the method of zero motion vector to predict the motion vector.

According to the above-mentioned technical scheme, it can be seen that the present invention can more truly embody the motion of objects in the video and obtain more accurate motion vector prediction; it can be combined with forward predictive coding and backward predictive coding to realize a new type of predictive coding , and its effect is similar to traditional B-frame coding.

Description of drawings

FIG. 1 is a schematic diagram of a motion vector passing through a current coding block;

FIG. 2 is a schematic diagram of multiple motion vectors passing through a current coding block;

Fig. 3 is the schematic diagram that the present invention decides to select the real motion vector from a plurality of vectors through adjacent blocks;

FIG. 4 is a schematic diagram of predicting motion vectors through adjacent blocks in the present invention;

Fig. 5 is the schematic diagram that all adjacent blocks have no motion vector situation;

FIG. 6 is a schematic diagram of predicting motion vectors using the extension of forward motion vectors in the present invention;

FIG. 7 is a schematic diagram of an existing frame interpolation method;

Fig. 8 is a flowchart of an embodiment of the present invention.

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

Please refer to Fig. 1 and Fig. 8, the present invention calculates the step of motion vector for current encoding block:

1. Perform motion estimation on the backward reference frame to obtain the best motion vector for each n×n block;

2. Preprocessing the candidate motion vectors of each n×n block of the current frame;

Suppose the width and height of the image are width and height respectively. This has a total of width/n x height/n blocks of motion estimation. For each block of size n×n in the backward reference frame, assuming the motion vector of its central point is its estimated motion vector, the trajectory of motion is obtained along the direction of this motion vector, and found in the current frame best matching block.

When the block motion vector of each n×n motion block of the backward reference frame (assuming starting from the middle of this n×n block) passes through the current image, the intersection point of the current image and this vector can be obtained, and then the intersection point will be calculated Which n×n block falls in, the forward motion vector of this block is in the same direction as the motion vector of the backward reference frame, the modulus is 1/2, and the backward motion vector of the current block is in the opposite direction, and the modulus is 1 /2.

For each block, do motion estimation relative to the reference frame. For different matching blocks, you can use a given criterion (such as SAD, SSD, etc.) to find the best matching block, and get the corresponding motion vector from the current image.

If the position of the upper left corner of a certain motion block in the current and backward reference frames is BACK_POS(x, y), its motion vector is (mvx, mvy) (where mvx and mvy are motion vectors for x, y on the two-dimensional coordinate axis axis direction), then the position FOR_POS(x, y) in the forward reference frame is:

FOR_POS (x, y) = BACK_POS (x-n/2, y-n/2) + (mvx, mvy);

The corresponding current block position CURR_POS(x, y) is:

CURR_POS (x, y) = BACK_POS (x-n/2, y-n/2) + (mvx/2, mvy/2);

Thus, the corresponding block position can be obtained as CURR_BLOCK=CURR_POS(X, Y)/n, the forward motion vectors of this block are respectively (mvx/2, mvy/2), and the backward motion vectors are respectively (- mvx/2, -mvy/2).

3. Referring to FIG. 2, when multiple motion vectors point to the same estimated block, after the above processing, two or more motion vectors may be obtained on some estimated blocks. At this time, the method of spatial domain prediction is used to obtain the real motion vector. Specifically, the best matching motion vector is selected according to the adjacent block with the obtained motion vector, that is, a candidate motion vector is obtained by taking the average value of the motion vectors of adjacent blocks, and then the candidate vector is used as the reference of the current block Motion vector. As shown in Figure 3, a candidate motion vector can be obtained by taking the average value of the motion vectors of adjacent blocks, and then compare multiple motion vectors of the current block with the candidate vectors, and finally select the block with the closest direction . Wherein, a square with a dotted line represents an adjacent block, and a square with a solid line represents a current block, where n1 and n2 are motion vectors of adjacent blocks. Since there are two motion vectors passing through the current block, the motion vector closest to the direction of the n1 and n2 vectors can be found.

4. When there is no motion vector passing through, but there is a motion vector in the adjacent block, the corresponding motion vector can be obtained by means of spatial prediction; A candidate motion vector can be obtained, and then the candidate vector is used as the reference motion vector of the current block. That is, the forward and backward motion vector of the current block is in the same direction as the reference motion vector, and the modulus is also the same. Referring to Fig. 4, the square of dashed line represents the adjacent block, the square of solid line represents the current block, wherein n1, n2 are the motion vectors of adjacent blocks, and the square containing character A represents that no motion vector passes through the current block (real The square of the line); since no motion vector passes through the current block, the vector average of n1 and n2 can be used as the motion vector of the current block. Referring to FIG. 5 , the dotted squares represent adjacent blocks, the solid squares represent the current block, and the squares containing the character A represent no motion vectors passing through the current block (solid squares).

5. When no motion vector passes through the current block and there is no motion vector in the adjacent block at the same time, the forward motion vector can be used for processing; as shown in Figure 6, the motion vector of the corresponding position block of the forward reference frame is used as The reference motion vector for the current block. That is, the direction of the forward motion vector of the current block is opposite to that of the reference motion vector, exactly the same. The direction of the backward motion vector is the same as that of the reference vector, and the magnitude is the same. Referring to FIG. 6 , it is a schematic diagram of the present invention using the extension of the forward motion vector to predict the motion vector; wherein, the square with a dotted line represents an adjacent block, the square with a solid line represents the current block, and the square containing the character A represents no block. The motion vector passes through the current block (the square with the solid line), and F represents the situation where the motion vector of the forward reference frame passes through the current block.

6. If the extension of the forward motion vector also does not pass through the motion block, then set its motion vector to zero.

Referring to Fig. 8, a specific processing flow of the present invention is:

Step a: Carry out motion estimation of the backward reference frame, and obtain its motion trajectory for each 4×4 block of the backward reference frame;

Step b: According to the motion track, find the 4×4 block of the current frame falling into the motion track;

Step c: if the motion vector of each macroblock has been calculated, end;

Step d: Count the number of motion vectors passing through the current 4×4 block;

Step e: if the number is greater than 1, then adopt the method of spatial domain prediction to obtain the real motion vector, and execute step d;

Step f: If the number is equal to 1, set the motion vector as the motion vector of the current block, and execute step d;

Step g: if the number is equal to 0, then further judge whether there is a motion vector in the adjacent block;

Step h: if there is, then use the adjacent block to perform spatial prediction to obtain the motion vector; perform step d;

Step i: If not, further judge whether the extension of the forward motion vector passes through the current block;

Step j: If yes, use the motion vector of the forward block as the motion vector of the current block; perform step d;

Step k: If not, set the motion vector of the current block to zero; execute step d.

Finally, it should be noted that: the above embodiments are only used to illustrate the present invention rather than limit the technical solutions described in the present invention; Those of ordinary skill in the art should understand that the present invention can still be modified or equivalently replaced; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered by the claims of the present invention.

Claims (5)

1. A motion vector prediction method for video coding, characterized in that: the method at least includes: Step 1: Obtain the motion vectors of each block of the backward reference frame; Step 2: For the motion vector of each block, obtain the motion trajectory from the forward reference frame to the backward reference frame; Step 3: According to the motion trajectory obtained in step 2, obtain the most matching block falling on the corresponding trajectory in the current image; the most matching block is specifically obtained by the following method: Step 31: When the motion vector of the motion block in the backward reference frame passes through the current image, calculate the block where the intersection falls into by using the obtained intersection point between the current image and the motion vector; Step 32: Set the forward motion vector of the current block to the same direction as the motion vector of the backward reference frame, modulo 1/2; at the same time set the backward motion vector of the current block to its forward motion vector The direction of is opposite, the modulus is 1/2; For the motion vector intersecting with multiple motion trajectories in the current image, select the most matching motion vector according to the blocks of the adjacent motion vectors that have been obtained; specifically, it is obtained by weighting the motion vectors of the adjacent blocks Get a candidate motion vector, and then compare multiple motion vectors of the current block with the candidate vector; finally, take the motion vector of the closest block in its direction; Step 4: complete the motion estimation process of all blocks in the backward reference frame, and obtain the corresponding motion vector from the current image; If the motion vector intersected by the motion track and the current image is zero, the motion vector is predicted by spatial domain prediction, or the motion vector is predicted by forward motion vector prediction, or the motion vector is predicted by zero motion vector. 2. The motion vector prediction method for video coding according to claim 1, wherein the weighted value is obtained by mean weighting, linear weighting or non-linear weighting. 3. The motion vector prediction method for video coding according to claim 1, characterized in that: the spatial domain prediction is to obtain a candidate motion vector by taking the average value of the motion vectors of adjacent blocks, and then use the The candidate vector is the reference motion vector of the current block. 4. The motion vector prediction method for video coding according to claim 1, characterized in that: the forward motion vector prediction method is to use the motion vector of the corresponding block in the forward reference frame as the reference of the current block The motion vector, that is, the direction of the forward motion vector of the current block is opposite to that of the reference motion vector, and the direction is exactly the same; the direction of the backward motion vector is the same as that of the reference vector, and the direction is also the same. 5. The motion vector prediction method for video coding according to claim 1, characterized in that: said zero motion vector prediction method is to determine the motion vector of the current block through intra-frame prediction.

Images