CN1568015A - Multiple visual-angle video coding-decoding prediction compensation method and apparatus - Google Patents

Abstract

The invention discloses a multi-view video encoding and decoding prediction and compensation method. The encoding is as follows: between two views, use one as a reference view and the other as a target view, perform global motion prediction on the reference view after deformation, and obtain Global motion prediction parameters; then perform global motion prediction compensation and motion vector prediction compensation for each macroblock; obtain the residual image of the macroblock according to the selected prediction compensation; transform, quantize, and entropy encode the residual image, and finally output the macro Block coding code stream; the decoding first obtains the global motion prediction parameters of the current frame in the target view to the reference view, and enters the decoding process of each macroblock; performs entropy decoding, inverse quantization and inverse transformation, and obtains the prediction compensation method adopted by the macroblock , residual image and other information. The present invention adopts global motion prediction and compensation and motion vector prediction and compensation methods between two related view sequences, can make more full use of the correlation between views in multi-view video, and can effectively encode and decode multi-view videos.

Description

Multi-view video coding and decoding prediction compensation method and device

technical field

The present invention relates to a digital image processing technology, in particular to a digital image coding and decoding technology, in particular to a multi-view video coding and decoding method.

Background technique

With the rapid development of network and multimedia technology, more and more images and videos appear and transmit in the form of digital media, and efficient video codec technology is the key to realize digital media storage and transmission. At present, a new digital media form, that is, three-dimensional video, has been widely used in multimedia application systems. The so-called 3D video refers to a collection of several video sequences that are synchronized in time and correlated in space. Compared with traditional two-dimensional video, three-dimensional video can provide richer visual information, so it can provide users with higher-quality subjective visual enjoyment, and can be applied to applications such as video conferencing and digital entertainment. Multiview Video is a common 3D video. The so-called multi-view video refers to a group of video sequences that are synchronously collected by multiple cameras at different viewing angles. Through some synthesis technology, multi-view video can provide users with scenes with three-dimensional visual sense. Furthermore, a key feature of multi-view video is the interactivity in the scene, where the user is also able to choose his own viewpoint.

Compared with the traditional two-dimensional video, the multi-view video has a larger amount of data, and one of the costs of realizing its function is that the video data representing the scene grows at a geometric speed with the increase of the viewpoint (that is, the number of cameras). Therefore, for efficient The processing method of multi-view video data is very important. In order to realize the storage and transmission of multi-view video, it is necessary to efficiently encode and decode the multi-view video. At present, advanced video coding and decoding technologies usually exist in the form of standards. Typical video compression standards include the MPEG series of international standards MPEG developed by the Moving Picture Expert Group (MPEG) under the International Organization for Standardization (ISO). -1, MPEG-2, MPEG-4, etc., and the H.26x series of video compression recommendations proposed by the International Telecommunication Union (ITU). At present, the MPEG organization is working on the standardization of three-dimensional audio and video (3DAV), but no specific standard has been formed yet.

One of the simplest methods for encoding and decoding multi-view video data is to use existing codec standards, such as MPEG-1, MPEG-2, MPEG-4, JVT or H.26x, etc., to convert the video data of each view It is regarded as an ordinary video sequence and is coded and decoded separately. However, this method does not take advantage of the correlation between views, and the coding efficiency is not very high. Although some technologies in the MPEG standard can be used to some extent to use the correlation between various views for encoding and decoding, the original intention of these technologies is not specifically for multi-view video encoding and decoding, and does not fully consider the multi-view video. characteristics, so the coding efficiency is not optimal. There are also some codec systems that utilize the spatial correlation information between images of different perspectives at the same time in multi-view video, but because they are mainly proposed for the coding of single-view (2D) video, they do not consider multi-view video The special relationship between views, for example, the scene content of the corresponding images between adjacent views is mostly the same, and the motion trend of the scene is exactly the same, but the angle changes, which can be seen in many instances ( Except for fast motion instances), the similarity between frames before and after a view is higher than the similarity between different views, so motion compensated inter prediction is likely to replace disparity compensation between view channels (the motion prediction compensation) prediction, and the disparity compensation between viewing channels is only in some places superior to encoding each viewing separately, and its coding efficiency is not very ideal.

Contents of the invention

The technical problem to be solved by the present invention is to propose a novel multi-view video coding and decoding prediction compensation method and device based on global motion prediction compensation and motion vector prediction compensation, so as to improve the coding efficiency of multi-view video coding and decoding.

The method for predictive compensation of multi-view video encoding and decoding in the present invention uses global motion prediction compensation between two views to extract the spatial correlation of corresponding images between each view, thereby improving coding efficiency, and then realizes global motion prediction compensation On the basis of , the motion vector between the frames before and after another view (target view) is predicted by the motion vector between the frames before and after one view (reference view). Specifically:

When encoding, use one of the views as the reference frame, and the other adjacent view as the target view, and the target view image refers to the reference view image after deformation (translation, rotation, scaling, etc.) for global motion prediction compensation to obtain global motion prediction parameters, and then use these parameters to calculate the global motion prediction cost of each macroblock (generally measured by the absolute difference between each pixel of the original macroblock and the reconstructed macroblock obtained by decoding the coded macroblock), Get the global motion vector for this macroblock. Motion vector prediction and compensation utilizes the obtained global motion vector and the coded motion vector between frames before and after the reference view to predict the motion vector between the frames before and after the target view, and calculates the motion prediction cost of this method. According to the motion prediction cost, select the encoding prediction compensation method used by the current macroblock, and then perform global motion compensation of the macroblock (if the global motion prediction compensation method is selected) or local motion compensation of the macroblock according to the selected prediction compensation method (if the motion vector prediction and compensation method is selected), and then obtain the residual image of the macroblock. Finally, transform, quantize, and entropy code the residual image, and finally output the coded stream.

When decoding, first obtain the global motion prediction parameters of the current frame in the target view to the reference view, and then use these parameters to calculate the global motion vector of the currently decoded macroblock and save it for later use. Re-decoding to obtain the prediction compensation method used by the macroblock and the residual image. According to different prediction and compensation methods, the image of the macroblock is reconstructed according to the motion compensation method during encoding, and combined with the decoded residual image to form the final macroblock decoded image; each macroblock is decoded using the above method, that is A decoded image of the frame is available.

The present invention also proposes a multi-view video coding prediction and compensation device, including a reference view image encoding unit and a target view image encoding unit. The reference view image encoding unit performs encoding through local motion prediction and compensation, and finally outputs the encoded reference view code stream ; The target-view image coding unit encodes through global motion prediction compensation and motion vector prediction compensation, and finally outputs the coded target-view code stream.

The present invention utilizes the global motion prediction and compensation and motion vector prediction and compensation methods used between two related view sequences. Compared with the existing coding methods, it can make full use of the correlation between each view in multi-view video, so it can Effectively encode and decode multi-view video.

The global motion prediction compensation acts on multiple images collected by multiple cameras at the same time. Its purpose is to extract the spatial correlation of these images, fully considering the characteristics between adjacent views of multi-view videos, especially most of the scene content. The same, the same motion trend, the difference is that either the angle is different, or there are characteristics such as relative displacement between the entire scene, these characteristics make the global motion prediction compensation can obtain more accurate motion prediction than the direct local motion prediction compensation, and It does not need to code the motion vector, and reduces the bits for coding the motion vector information, thereby improving the coding efficiency.

The motion vector prediction and compensation method is based on the realization of the global motion prediction and compensation method. Its purpose is to use the motion information of the sequence captured by a certain camera that has been encoded to predict the motion information in the sequence captured by other cameras. The same method can be used to implement in the encoder and the decoder respectively, so there is no need to encode the motion vector, and the bits for encoding the motion vector information are reduced, thereby improving the encoding efficiency.

Description of drawings

Fig. 1 is a schematic diagram of motion vector relationship;

Fig. 2 is the motion vector central point prediction flow chart that the present invention proposes;

Fig. 3 is a flow chart of obtaining the global motion vector from the reference view to the target view through iterative matching;

Fig. 4 is a schematic diagram of a multi-view video coding prediction compensation device;

Fig. 5 is a schematic diagram of a second multi-view video coding prediction compensation device;

Fig. 6 is a schematic diagram of the prediction principle based on the MPEG-4 time-scalable coding method.

Detailed ways

In multi-view video sequences, there is a considerable correlation between the corresponding frames of two video sequences of adjacent viewing angles, that is, the scene content of the corresponding frames of two video sequences of adjacent viewing angles is mostly the same, and the motion trend is the same. The place where the angle is different, or there is a relative displacement between the entire scene, which also implies that the motion vector between the frames before and after a view (target view) can be predicted by the motion vector between the frames before and after another view (reference view) to get. After they undergo simple deformations (translation, rotation, affine transformation, etc.), most of the images are very similar. If these characteristics can be used, the global motion prediction and compensation method is adopted between two corresponding frames, and the motion vector prediction and compensation method is adopted between the frames before and after the target view, the coding efficiency of multi-view video can be improved. Including the following steps:

The encoding steps are as follows:

First, one of the views is used as a reference view, and the other adjacent view (hereinafter referred to as the target view) is deformed to perform global motion prediction on the reference view to obtain global motion prediction parameters. Then enter the encoding process of each macroblock global motion prediction compensation and motion vector prediction compensation, the steps are as follows:

Step 1. Global motion prediction compensation encoding process:

Step 1.1, use the obtained global motion prediction parameters to calculate the global motion prediction cost of the macroblock (generally measured by the sum of absolute differences between the pixels of the original macroblock and the reconstructed macroblock obtained by decoding the encoded macroblock );

Step 1.2, calculating and obtaining the global motion vector of the macroblock, saving it for future use;

Step 2, motion vector prediction compensation encoding process:

Step 2.1, using the obtained global motion vector of the macroblock, the saved global motion vector of the previous frame and the coded motion vector prediction between the frames before and after the reference view to obtain the motion vector between the frames before and after the target view;

Step 2.2, using the predicted motion vector to calculate the local motion prediction cost of the macroblock;

Step 3, selecting the smaller prediction compensation among the calculated global motion prediction cost and the local motion prediction cost as the prediction compensation of the macroblock;

Step 4, according to the selected prediction compensation or carry out the global motion compensation of macroblock (if the global motion prediction compensation method is selected) or carry out the local motion compensation of macroblock (if the motion vector prediction compensation method is selected), and then with the original macro Block difference to obtain macroblock residual image;

Step 5: Perform transformation, quantization, and entropy coding on the residual image, and finally output a coded stream of macroblocks.

The decoding steps are as follows:

First decode to obtain the global motion prediction parameters of the current frame in the target view to the reference view, and then enter the decoding process of each macroblock, the steps are as follows:

Step 1. Perform entropy decoding, inverse quantization and inverse transformation to obtain information such as the prediction compensation method and residual image adopted by the macroblock;

Step 2, using the obtained global motion prediction parameters to calculate and obtain the global motion vector of the currently decoded macroblock, and save it for later use;

Step 3, if the macroblock adopts global motion prediction compensation, then decode according to the following steps:

Step 3.1, deforming the corresponding reference frame in the reference view according to the obtained global motion prediction parameters;

Step 3.2, using the obtained global motion vector to refer to the deformed reference frame to perform global motion compensation on the currently decoded macroblock to obtain a macroblock compensated image;

In step 3.3, the macroblock compensation image is combined with the decoded residual image to obtain a decoded macroblock image.

Step 4, if the macroblock adopts motion prediction compensation, then decode according to the following steps:

Step 4.1, using the saved decoded global motion vector and the decoded motion vector between frames before and after the reference view, to calculate and obtain the motion vector between frames before and after the target view;

Step 4.2, using the calculated motion vector to refer to the previous decoded frame of the target view to perform motion compensation on the currently decoded macroblock to obtain a macroblock compensated image;

In step 4.3, the macroblock compensation image is combined with the decoded residual image to obtain a decoded macroblock image.

Figure 1 is a schematic diagram of the relationship between motion vectors. In Figure 1, the target view and the reference view are two views in a multi-view video sequence, one is used as the reference view, and the other is used as the target view. It represents the relationship between the reference view and the frame before and after the target view at a certain encoding time (the i-1th frame represents the previous frame, and the i-th frame represents the current frame), and describes the global relationship between the reference view and the target view. The relationship between motion vectors (indicated by GMC0 and GMC1 in Figure 1) and motion vectors in the same view (indicated by MV0 and MV1 in Figure 1), where:

The upper two grids represent the two frames of images before and after the target view at a certain moment, the two lower grids represent the two frames of images before and after the reference view at a certain moment, and each small square in the grid represents a macroblock (image pixel block), the gray square in the target view grid represents the current coded macroblock.

GMC0 represents the global motion vector between the previous coded target frame and the previous coded reference frame;

GMC1 represents the global motion vector between the current encoding target frame and the corresponding encoded reference frame;

MV0 refers to the motion vector between frames before and after the reference view;

MV1 represents the motion vector between frames before and after the target view, and it is also the motion vector that needs to be obtained by motion vector prediction in the present invention.

Fig. 2 is a flow chart of motion vector central point prediction proposed by the present invention. It describes a method of predictive coding in which the target view sequence predicts the motion vectors of the previous and subsequent frames corresponding to the target view by referring to the previous and subsequent frame motion vectors of the view and the obtained global motion vector.

Specifically, when encoding a certain macroblock of the target view (indicated by a gray square in Figure 1), complete the following steps:

Step 1. Obtain global motion vectors GMC0 and GMC1 (see Fig. 1 for explanation, they have been obtained in the process of global motion prediction and compensation, see encoding implementation step 1.2) and local motion vector MV0 (generated during encoding reference view);

Step 2, calculate and obtain the center point (CurXcent, CurYcent) of current macroblock, (represent by CurCentPos in Fig. 1, promptly the small black dot in the gray square);

Step 3, calculate and obtain the position (CurRefX, CurRefY) of point (CurXcent, CurYcent) in the reference view image by GMC1 motion vector and position (CurXcent, CurYcent), (represented by CurPosRef among Fig. 1);

Step 4, use MV0 to calculate the corresponding position (PreRefX, PreRefY) of the obtained point (CurRefX, CurRefY) in its previous frame reference view image (represented by PrePosRef in Figure 1);

Step 5. Using GMC0, obtain the global motion vector (GMVX, GMVY) from the reference view to the target view through iterative matching, (reverse the GMC0 in Fig. 1), and the detailed process will be described in Fig. 3;

Step 6, reverse the obtained global motion vector (GMVX, GMVY), and then use it to calculate the corresponding position (PreXcent, PreYcent) of the previous frame image of the target view (PreRefX, PreRefY), (use in Fig. 1 PrePosCur to represent);

Step 7. Calculate and obtain the motion vector MV1, MV1X=PreXcent-CurXcent, MV1Y=PreXcent-CurYcent; (the vector represented by the thick dotted line in Fig. 1 ).

It is worth pointing out that, in order to pursue higher prediction accuracy, in addition to the central point prediction method proposed by the present invention, multi-point prediction and then calculating the average value can also be used, but this still belongs to the processing idea of the present invention.

Fig. 3 is a method of iterative matching to obtain the global motion vector from the reference view to the target view. It describes the implementation of the bold functional box in Figure 2. Its input is the calculated corresponding position point (PreRefX, PreRefY) in the reference view, the global motion vector GMC0 from the target view to the reference view, and finally outputs the corresponding global motion vector from the reference view to the target view (GMVX, GMVY). in,

MBSIZE represents the size of the macroblock, which can take different values according to different environments;

N controls the maximum number of iterations, which can be set according to actual needs. It is used to ensure that when an exact match cannot be obtained through iteration, the control iteration ends. Its size controls the scope of the iterative search;

When Minerr is initialized, it is MAXERROR, and MAXERROR can take any maximum threshold according to the actual situation. Minerr records the smallest iteration error in the iterative process, and finally uses the global motion vector of the target view macroblock with the smallest Minerr (iteration error) to represent the global motion vector (GMVX, GMVY) from the reference view to the target view.

Specifically, Minerr is initialized, making iterative error Minerr=MAXERROR;

Calculate the macroblock position (PreMBX, PreMBY) where the obtained point (PreRefX, PreRefY) is located, and obtain the global motion vector (GMVX, GMVY) through GMC0 and the macroblock position (PreMBX, PreMBY);

Subtract the global motion vector (GMVX, GMVY) from the point (PreRefX, PreRefY) to obtain a new position (PreCurX, PreCurY); then calculate the macroblock position (PreCurMBX, PreCurMBY) where the obtained point (PreCurX, PreCurY) is located.

The iterative process is as follows: compare the macroblock position (PreCurMBX, PreCurMBY) obtained after iteration with the macroblock position (PreMBX, PreMBY) before iteration, if they are equal, it means that it is completely obtained, the iteration error Minerr is 0, and exit the iteration directly Process, if not equal, if it is on the left side of the macroblock position before iteration, then obtain the absolute difference Xerr between the X-direction position on the left side of the macroblock and the X-direction position of the corresponding position point after iteration, otherwise obtain the X-direction position on the right side of the macroblock and the absolute difference Xerr of the X-direction position of the corresponding position point after iteration, similar to this, if it is above, then obtain the absolute difference Yerr of the Y-direction position on the upper side of the macroblock and the Y-direction position of the corresponding position point after iteration, otherwise obtain The absolute difference Yerr between the position in the Y direction of the lower side of the macroblock and the position in the Y direction of the corresponding point after iteration, and finally the sum of Xerr and Yerr is compared with Minerr, and the minimum value is given to Minerr.

FIG. 4 shows a multi-view video coding prediction compensation device 1 as a specific embodiment of the present invention. The input of the device is the original video stream of any two views in the multi-view video (adjacent views are better), one of which is called the reference view, and the other is called the target view. The reference view is coded by local motion prediction and compensation method, and finally the coded reference view code stream is output; the target view image is coded by global motion prediction compensation and motion vector prediction compensation, and finally the coded target view code stream is output. in,

The reference view image and the target view image are the input reference view original video image and the target view original video image respectively;

The reference view reconstructed image buffer and the target view reconstructed image buffer respectively store the reconstructed reference view image and target view image after encoding of the previous frame.

Its working sequence is to first encode a frame of reference view image, and then encode a frame of target view image. The following are the processes of encoding the reference view image and encoding the target view image respectively:

1. Reference view image coding unit

It implements prediction compensation coding of a frame of reference view. It uses local motion estimation, and thus obtains motion vectors, which are entropy coded. Save the motion vector in the motion vector buffer for use when encoding the target view image. The motion vector cooperates with the reconstructed reference view image in the previous frame to complete the reference view motion compensation, and obtain the compensated reference image, that is, the reference view prediction image in Figure 4, which is compared with the original reference view image to obtain the residual image . The residual image is transformed, quantized and entropy coded to finally form a reference video stream. The quantized residual image is dequantized and inversely transformed, and accumulated with the reference view prediction image to obtain the reference view reconstruction image, which is put into the reference view reconstruction buffer for the next frame of reference view Used when encoding images and target views.

2. Target view image coding unit

It realizes the predictive compensation encoding of a frame of target image.

Perform global motion estimation between the input original image of the target view and the reconstructed image of the reference view, obtain the global motion parameters, and thus obtain the global motion prediction cost and the global motion vector, and save the global motion vector in the global motion vector buffer in, for later use;

Use the global motion vector obtained during the encoding of the target view image in the previous frame, the currently obtained global motion vector and the motion vector saved during the encoding of the corresponding reference view image to perform target view motion vector prediction, and obtain the target view image with reference to the previous frame reconstruction The local motion vector of the target view;

Calculate the local motion prediction cost of the target view image, and select the smaller prediction compensation method in it and the global motion prediction cost by the selection controller to perform the following work;

According to the selected prediction compensation method, either perform global motion compensation (if the global motion prediction compensation method is selected) or perform local motion compensation (if the motion vector prediction compensation method is selected), and obtain the compensated target image, that is, the target in Figure 4 Viewing the predicted image, the image is compared with the original target viewing image to obtain the residual image;

The residual image is transformed, quantized and entropy coded to finally form the target video stream. The quantized residual image is dequantized and inversely transformed, and accumulated with the target-view prediction image to obtain the target-view reconstruction image, which is put into the target-view reconstruction buffer for the next frame of target-view reconstruction. Used when encoding images.

FIG. 5 is another embodiment of the present invention, showing a multi-view video coding prediction compensation device 2 . The difference between the device 2 and the device 1 is that when encoding the target view image, the global motion prediction compensation and the motion vector prediction compensation are used as two prediction modes to complete the coding work together with other prediction modes. That is, in addition to coding the target view image using global motion prediction compensation and motion vector prediction compensation, other prediction compensation methods (such as direct prediction compensation methods) that have been used can also be introduced at present, which contribute to the efficiency of multi-view video coding improvement. The input of the device 2 is the original video streams of any two views in the multi-view video (adjacent views are better), wherein one view is called a reference view, and the other view is called a target view.

The upper part of Figure 5 (the part enclosed by the upper rectangle) is the encoding process of the reference view image, and the lower half (the part enclosed by the lower rectangle box) is the encoding process of the target view image; Motion prediction compensation module and motion vector prediction module, which are added to the whole device as two modes;

The reference view image and the target view image are the input reference view original video image and the target view original video image respectively;

The reference frame buffer and the target frame buffer respectively store the reconstructed reference view image and target view image after encoding of the previous frame.

Its working sequence is to first encode a frame of reference view image, and then encode a frame of target view image. The following are the processes of encoding the reference view image and encoding the target view image respectively:

1. Reference view image coding unit

This process is the same as that in device 1. For details, refer to the reference view image coding description in FIG. 4 .

2. Target view image coding unit

It realizes the predictive compensation encoding of a frame of target image.

The global motion estimation is performed between the input original image of the target view and the reference view to obtain the global motion parameters, and thus obtain the global motion prediction cost and the global motion vector, and save the global motion vector for later use. done in the global motion estimation module;

Use the global motion vector obtained during the encoding of the target view image in the previous frame, the currently obtained global motion vector and the motion vector saved during the encoding of the corresponding reference view image to perform target view motion vector prediction, and obtain the target view image with reference to the previous frame reconstruction The local motion vector of the target view, and calculate the local motion prediction cost of the target view image. done in the motion vector prediction module;

Perform other prediction and compensation methods, and thus obtain motion vectors, which are entropy coded. Calculate the prediction cost of this prediction compensation method. Done in the target frame motion estimation module;

According to the principle of minimum prediction cost, choose the appropriate prediction compensation method. According to the selected prediction compensation method, either perform global motion compensation (if the global motion prediction compensation method is selected, it will be completed in the global motion compensation module), or perform local motion compensation (if the motion vector prediction compensation method is selected, it will be completed in the target frame motion Completing in the compensation module), or performing compensation in other ways (finishing in the target frame motion compensation module) to obtain a compensated target view image, which is compared with the original target view image to obtain a residual image;

The residual image is transformed, quantized and entropy coded to finally form the target video stream. The quantized residual image is dequantized and inversely transformed, and accumulated with the compensated target view image to obtain the target view reconstructed image, which is put into the target frame buffer for the next frame of target view. Used when encoding images.

Figure 6 describes the prediction principle based on the MPEG-4 time scalable coding method:

Compared with the existing coding method, the method of the present invention adopting global motion estimation prediction and motion vector prediction between two related view sequences can make full use of the correlation between each view in the multi-view video, so it can effectively analyze the multi-view video Codec. They can be used in combination with the MPEG-4 coding standard, and the following steps are used to realize global motion estimation prediction and motion vector prediction processing (take the stereoscopic view sequence as an example, it has only two views, which are called left view and right view respectively, so it is the most simple multi-view video sequence).

1. Encoding end processing:

In Figure 6, the left-view image is used as the reference view sequence, and the right-view image is used as the current view sequence. GMC0 represents the global motion estimation and prediction between the left and right images of the previous frame, GMC1 represents the global motion estimation and prediction between the left and right images of the current frame, and MV0 represents the reference MV1 represents the motion prediction vector between frames before and after the current view (that is, the left-view image), and MV1 needs to be obtained by using GMC0, GMC1 and MV0 for motion vector prediction. I represents intra-frame coding, P represents predictive coding, and B represents bidirectional predictive coding. In order to be compatible with the MPEG-4 standard, the present invention embeds global motion estimation prediction and motion vector prediction by adding a macroblock prediction mode. Therefore, in addition to the original prediction mode, two prediction modes are added to the B picture, which are global motion prediction (represented by GME) and motion vector prediction (represented by MVP). Its specific implementation process is as follows:

1. Reference video encoding: This step is the same as the original step of MPEG-4, and thus obtains the motion vector MV0 between the preceding and following frames.

2. The image of the current view can not only refer to the image of the reference view, but also refer to the image of the previous frame of the current view. Therefore, except for the first frame, the B frame image in MPEG-4 is used for encoding. Here, the local motion estimation prediction method is the same as the original steps of MPEG-4.

3. Perform global motion estimation and compensation between the current view image and the reference view image, and obtain the global motion parameter GMC1 and the global motion vector of the macroblock.

4. Use the encoded MV0, GMC0 and GMC1 to perform motion vector prediction between the frames before and after the current view. The method of central point prediction is adopted, and its prediction method is shown in Figure 2.

5. Mode selection method:

1) select the best among the original modes of the B frame in MPEG-4, adopt the original selection strategy of MPEG-4;

2) Select the optimal prediction mode through the rate-distortion optimization (RDO-rate-distortion optimization) strategy in the selected original mode, GME and MVP mode with the best B frame. The selection method is to select the mode with the smallest MSE+λRate, where MSE is the mean square error between the original image of the macroblock and the reconstructed image of the macroblock, and Rate is the encoded code bit of the macroblock. λ is a Lagrangian operator, where λ=(0.85×2QP/3)1/2, and QP is a quantization coefficient.

6. Perform motion compensation. The GME mode adopts the global motion estimation compensation, and other modes adopt the original compensation method of MPEG-4.

7. Form a macroblock code stream. Macroblocks using GME and MVP modes do not need to encode motion vectors, and others use the original method of MPEG-4.

2. Decoder processing:

1. To decode the reference video image, the original decoding method of MPEG-4 is adopted.

2. To decode the current view image, the following steps are adopted:

1) decoding the global motion estimation parameters of the current frame;

2) If the current macroblock is in the original prediction mode of MPEG-4, then decode it according to the original decoding method of MPEG-4, otherwise, if it is the GME type, perform global motion compensation to reconstruct the macroblock, otherwise, if it is the MVP type, then follow the encoding method The method in step 4 is used to predict and obtain the motion vector, and then perform motion compensation using the original compensation method of MPEG-4.

3) The residual image is decoded and combined with the compensated image to generate a decoded image.

The above embodiments are only used to illustrate rather than limit the technical solution of the present invention. Those skilled in the art should understand that: the present invention can be modified or equivalently replaced without departing from any modification or partial replacement of the spirit and scope of the present invention. All of them should be included in the scope of the claims of the present invention.

Claims (14)

1. A method for multi-view video codec prediction compensation, the encoding step comprising:

between two views, one view is used as a reference view, the other adjacent view is used as a target view, and global motion prediction is carried out on the reference view after deformation to obtain global motion prediction parameters; then, global motion prediction compensation and motion vector prediction compensation of each macro block are carried out;

selecting the smaller prediction compensation in the global motion prediction cost and the local motion prediction cost obtained by calculation as the prediction compensation of the macro block;

according to the selected prediction compensation, global motion prediction compensation of the macro block is carried out, or local motion compensation of the macro block is carried out, and then difference operation is carried out with the original macro block, so that a macro block residual image is obtained;

transforming, quantizing and entropy coding the residual image, and finally outputting a macro block coding code stream;

the decoding steps are as follows:

firstly, decoding to obtain global motion prediction parameters of a current frame in a target view to a reference view, and then entering a decoding process of each macro block;

entropy decoding, inverse quantization and inverse transformation are carried out to obtain information such as a prediction compensation mode and a residual image adopted by a macro block;

calculating and obtaining the global motion vector of the current decoding macro block by using the obtained global motion prediction parameters, and storing the global motion vector;

and combining the obtained macroblock compensation image and the decoded residual image by adopting a global motion prediction compensation mode or motion prediction compensation for the macroblock to obtain a decoded macroblock image.

2. The method of claim 1, wherein the global motion prediction compensation coding process comprises:

calculating the global motion prediction cost of the macro block by using the obtained global motion prediction parameters;

and calculating to obtain the global motion vector of the macro block, and storing the global motion vector for later use.

3. The method of claim 2, wherein the global motion prediction cost is measured by an absolute difference between pixels of an original macroblock and pixels of a reconstructed macroblock obtained by decoding the encoded macroblock.

4. The method of claim 1, wherein the MVP coding process comprises:

predicting and obtaining a motion vector between the front frame and the rear frame of the target view by using the obtained global motion vector of the macro block, the saved global motion vector of the previous frame and the motion vector between the front frame and the rear frame of the coded reference view;

and calculating the local motion prediction cost of the macro block by using the motion vector obtained by prediction.

5. The method of claim 1, wherein the global motion prediction compensation of the macroblock in the decoding comprises:

deforming the corresponding reference frame in the reference view according to the obtained global motion prediction parameters;

and performing global motion compensation on the current decoding macro block by using the obtained global motion vector to refer to the deformed reference frame to obtain a macro block compensation image.

6. The method of claim 1, wherein the motion prediction compensation of the macroblock in the decoding comprises:

calculating to obtain a motion vector between the front frame and the rear frame of the target view by using the saved decoded global motion vector and the decoded motion vector between the front frame and the rear frame of the reference view;

and performing motion compensation on the current decoding macro block by using the motion vector obtained by calculation and referring to the previous decoding frame of the target view to obtain a macro block compensation image.

7. A motion vector center point prediction method is characterized by comprising the following steps:

step 1, obtaining global motion vectors GMC0 and GMC1 in the global motion prediction compensation process, and obtaining local motion vectors MV0 when encoding a reference view;

step 2, calculating to obtain a central point (CurXcent, CurYcent) of the current macro block;

step 3, calculating the position (CurRefX, CurRefY) of the obtained point (CurXcent, CurYcent) in the reference view image through the GMC1 motion vector and the position (CurXcent, CurYcent);

step 4, calculating the corresponding position (PreRefX, PreRefY) of the obtained point (CurRefX, CurRefY) in the reference view image of the previous frame by using MV 0;

step 5, obtaining a global motion vector (GMVX, GMVY) from a reference view to a target view by utilizing GMC0 through iterative matching;

step 6, inverting the obtained global motion vector (GMVX, GMVY), and further utilizing the obtained global motion vector to calculate the corresponding position (PreRefX, PreRefY) of the position (PreXcent, PreYcent) in the image of the previous frame of the target view;

and 7, calculating to obtain a motion vector MV1, wherein MV1X is PreXcent-CurXcent, and MV1Y is PreXcent-CurYcent.

8. The method of claim 7, wherein the step 5 obtains the global motion vector from the reference view to the target view by iterative matching, and comprises:

initializing Minerr, and making iteration error Minerr be MAXERROR;

calculating a macroblock position (PreMBX, PreMBY) where the acquisition point (PreRefX, PreRefY) is located, and acquiring a global motion vector (GMVX, GMVY) through GMC0 and the macroblock position (PreMBX, PreMBY);

subtracting the global motion vector (GMVX, GMVY) from the point (PreRefX, PreRefY) to obtain a new position (PreCurX, PreCurY); calculating the position (PreCurMBX, PreCurMBY) of the macro block where the acquisition point (PreCurX, PreCurY) is located;

comparing the positions of the macro blocks (PreCurMBX, PreCurMBY) obtained after iteration with the positions of the macro blocks (PreMBX, PreMBY) before iteration, and entering an iteration process.

9. The method of claim 7, wherein the macroblock position obtained after the iteration is compared with the macroblock position before the iteration, and if the two macroblock positions are equal, the macroblock position is completely obtained, the iteration error Minerr is 0, and the iteration process is directly exited.

10. The method according to claim 7, wherein the macroblock position obtained after the iteration is compared with the macroblock position before the iteration, if the two are not equal, if the two are on the left side of the macroblock position before the iteration, the absolute difference Xerr between the left X-direction position of the macroblock and the X-direction position of the corresponding position point after the iteration is obtained, otherwise, the absolute difference Xerr between the right X-direction position of the macroblock and the X-direction position of the corresponding position point after the iteration is obtained; if the absolute difference is above, the absolute difference Yerr between the Y-direction position of the upper side of the macro block and the Y-direction position of the corresponding position point after iteration is obtained, otherwise, the absolute difference Yerr between the Y-direction position of the lower side of the macro block and the Y-direction position of the corresponding position point after iteration is obtained, and finally, the sum of Xerr and Yerr is compared with Minerr, and the minimum value of the sum is taken to Minerr.

11. The method of predicting a motion vector center point of claim 7, further comprising setting a control maximum number of iterations N, and controlling the iteration to end when a complete match cannot be obtained by iteration.

12. A multi-view video coding prediction compensation apparatus comprising a reference view coding unit and a target view coding unit, characterized in that:

the reference view image coding unit obtains a motion vector through the processing of a local motion estimation module and carries out entropy coding on the motion vector; meanwhile, the motion vector cooperates with the reconstructed reference view image of the previous frame to complete the motion compensation of the reference view, and a compensated reference image is obtained, and the image is subtracted from the original reference view image to obtain a residual image; the residual image is subjected to a transformation, quantization and entropy coding module, and finally a reference video stream is output;

in the target view image coding unit, a global motion estimation module receives a target view original image and a reference view reconstructed image as input, obtains global motion parameters, obtains global motion prediction cost and a global motion vector from the global motion parameters, and stores the global motion vector in a global motion vector buffer area;

the global motion vector obtained when the previous frame of target view image stored in the buffer area is coded, the currently obtained global motion vector and the motion vector stored when the corresponding reference view image is coded are used for predicting through a target view motion vector predicting module, the local motion vector of the target view image, which is obtained by referring to the previous frame and reconstructing the target view, is output to a local motion predicting cost module to calculate the local motion predicting cost of the target view image, and a predicting compensation mode with smaller global motion predicting cost is selected through a selection controller;

obtaining a compensated target image according to the selected prediction compensation mode, and obtaining a residual image by subtracting the image from the original target video image;

and the residual image is processed by a transformation, quantization and entropy coding module, and finally a target video stream is output.

13. The multiview video coding prediction compensation device of claim 12, wherein: and the residual image processed by the quantization module is input into an inverse quantization and inverse transformation module, and is accumulated with the reference view predicted image, and the obtained reference view reconstructed image is stored in a reference view reconstruction buffer area.

14. The multiview video coding prediction compensation device of claim 12, wherein:

and the residual image processed by the quantization module is input into an inverse quantization and inverse transformation module, is accumulated with the target view predicted image, and stores the obtained target view reconstructed image into a target view reconstructed buffer area.

Images