CN1194544C - Video Coding Method Based on Time-Space Domain Correlation Motion Vector Prediction - Google Patents

Abstract

The invention relates to a video coding method based on time-space domain correlation motion vector prediction, which belongs to the technical field of computer digital video coding. The present invention is characterized in that it proposes a motion vector prediction method combining motion vector correlation in time domain and space domain, and a description method of motion correlation between adjacent macroblocks. The steps of the method are that the camera converts the state of the target object into a video signal, and the video signal is converted into a digital video sequence through the acquisition card, and stored in the video buffer, and used as a system input for compression; the computer enters and completes the motion compensation based on the original video frame /DCT's hybrid encoding subroutine, which performs video compression and generates code stream files. The present invention focuses on the motion vector prediction subroutine, and its prediction method mainly includes: firstly, judge the motion correlation between adjacent macroblocks, and then start to execute the macroblock motion vector prediction program according to the judgment result. The system effectively improves the motion vector prediction accuracy and video coding efficiency.

Description

Video Coding Method Based on Time-Space Domain Correlation Motion Vector Prediction

Technical field:

The invention relates to the technical field of computer digital video coding, and is aimed at video coding systems, such as HDTV, HD-DVD, streaming media servers and other applications.

Background technique:

Video information has a large amount of data. In order to effectively utilize bandwidth, video information must be compressed efficiently before storage or transmission. Coding efficiency is one of the important indicators to characterize the coding system. In the sense of rate distortion, coding efficiency refers to the bit rate output by the coding system under the condition of certain reconstructed video quality; or under the condition of certain output bit rate, The quality of the reconstructed video obtained.

In the hybrid coding method based on motion compensation/DCT, the motion vector obtained by motion estimation needs to be encoded, and together with the texture information, a compressed code stream is formed, so that the video image can be correctly reconstructed during decoding. According to statistics, the code stream output by the motion vector can usually account for about 10-40% of the entire output code stream. Motion vector coding usually includes two parts: motion vector prediction and entropy coding. After the motion vector is predicted by a certain method, the entropy can be effectively reduced, thereby improving its coding efficiency. In addition, when using the coding framework based on H.264, the prediction of the motion vector will also affect the final selection of the motion vector and the reference frame. Therefore, motion vector prediction is of great significance to improve video coding efficiency.

In the existing video coding methods, such as H.264, H.263 and MPEG-4, etc., the motion vector coding adopts the method of median prediction. That is, as shown in Figure 1, E represents the current coded macroblock, A, B, C and D represent its adjacent macroblocks on the left, upper, upper right and upper left in the space domain respectively, and the motion vector of macroblock E The predictor takes the median value of the motion vectors of the three macroblocks A, B, and C. It should be said that the median prediction method is a simple and efficient motion vector prediction method, and its main disadvantages are: (1) It does not utilize the correlation of the motion vector in the time domain. (2) For video sequences with complex motion, the coding efficiency is usually greatly reduced.

Invention content:

The purpose of the present invention is to overcome the deficiencies of the motion vector prediction method in the current coding system, effectively improve the coding efficiency of the motion vector, and design a motion vector prediction method that combines the correlation of the motion vector in the time domain and the space domain. Its technical thinking is characterized by:

1. The correlation of the motion vector in the time domain and the space domain is comprehensively utilized, and different prediction methods are adopted according to the motion correlation between macroblocks, which improves the prediction accuracy of the motion vector;

2. A description method suitable for the motion correlation between adjacent macroblocks in the video sequence in the space-time domain is proposed. In this method, the motion correlation of adjacent macroblocks is described by the absolute difference of the motion vectors of adjacent macroblocks in time and space.

Refer to Fig. 3-Fig. 6 for the technical solution of the present invention. This video encoding method based on temporal-space domain correlation motion vector prediction is to convert the state of the target object into a video signal by the camera (1) and place it in the acquisition card (2); the acquisition card converts the video signal into a digital video sequence, And stored in the video cache, these digital video sequences or the video data stored in the computer hard disk in the form of video sequence files are called original video sequences, which are used as the input of the system for compression; the computer (3) stores the original Video sequence and carry out video encoding subroutine, and generate the code stream file after the compression, the present invention is characterized in that:

The computer first reads a frame of video data from the video memory of the acquisition card or the video sequence file stored on the computer hard disk into the buffer area of the computer, and encodes the frame by executing the video encoding subroutine; the time for the computer to read the video data The interval is usually 1/30 second.

The video encoding subroutine adopts the hybrid encoding method based on motion compensation/DCT, except for the motion vector prediction subroutine, the encoding framework of the international video encoding standard H.264 JM6.0 is adopted.

Video encoding subroutine During video encoding, an input video sequence frame is divided into 16×16 macroblocks, and the encoding process is carried out in units of macroblocks; after the computer starts to execute the video encoding subroutine, it first initializes the encoding , these initialization tasks mainly include the setting of encoding parameters and buffers.

Then the computer performs motion estimation and compensation on the first macroblock in the order from top to bottom and from left to right; after motion estimation and compensation, the motion vector and residual image of the macroblock are obtained, and for the motion vector, First execute the motion vector subroutine to predict, and then perform context-adaptive variable length coding (CAVLC) on the prediction difference; and first perform DCT transformation on the residual image, and then quantize and CAVLC the DCT coefficients; then output the The compressed code stream of the macroblock is put into the buffer of the computer; at last, it is decoded and the corresponding reference macroblock is generated; after the encoding of the macroblock is completed, the above encoding process is executed cyclically for all the macroblocks of the current coded frame, and a video sequence is completed. Frame encoding; after one frame is encoded, the next frame is cyclically encoded until the last frame of the video sequence is encoded, and a compressed code stream file is generated, and the system execution program ends.

In the above-mentioned video encoding subroutine, the prediction method used in the motion vector prediction subroutine is based on the difference in motion correlation between adjacent macroblocks, and comprehensively utilizes the correlation in the time and space domains to perform predictive encoding of.

In its described motion vector prediction subroutine, its prediction method is as follows:

(1) First, judge the motion correlation between adjacent macroblocks; for the adjacent relationship between the macroblocks shown in Figure 2 in the time domain and the spatial domain, let E _i be the coded macroblock of the current frame i, A _i , B _i , C _i , D _i , G _i , H _i are the adjacent macroblocks on the left, upper, upper right, upper left, right and lower respectively, in the previous frame (i-1) The macroblocks at corresponding positions are denoted as E _i-1 , A _i-1 , B _i-1 , C _i-1 , D _i-1 , G _i-1 and H _i-1 respectively.

Here, let B={C ₁ C ₂ C ₃ C ₄ C ₅ } be a set composed of motion vectors of E _i neighboring macroblocks A _i , B _i , E _i-1 , G _i-1 and H _i-1 .

(a) If for i, i∈[1, 5], all satisfy

|C _i -C ₃ |≤TH

Then the motion correlation between adjacent macroblocks of E _i is high.

(b) If for i, i∈[1, 5], all satisfy

|C _i -C ₃ |＞TH

Then the motion correlation between adjacent macroblocks of E _i is low.

(2) According to the judgment result of (1), the prediction method of the motion vector of the current coded macroblock is as follows:

In order to utilize the correlation in the time and space domains, the present invention selects the motion vectors of A _i , B _i and C _i as candidate predictors in the space domain, and the motion vectors of E _i-1 , G _i-1 and H _i-1 The motion vector is used as a candidate predictor in the time domain, and the specific algorithm for motion vector prediction is as follows:

(1) When E _i-1 , G _i-1 and H _i-1 do not exist, use A _i , B _i , and C _i as candidate predictors;

(2) When the motion correlation between adjacent macroblocks is high, select A _i , B _i and E _i-1 as candidate predictors;

(3) When the motion correlation between adjacent macroblocks is low, select A _i , B _i , G _i-1 and H _i-1 as candidate predictors;

(4) When any one of A _i , B _i , C _i and D _i adopts intra-frame coding, if the motion vector of the macroblock at the (i-1) frame and its corresponding position exists, then the motion vector of the macroblock is selected at The (i-1) frame has the same motion vector and reference frame as its corresponding position block, otherwise the motion vector of the block is set to (0, 0);

(5) If there is a macroblock with the same reference frame as E _i in A _i , B _i , C _i , E _i-1 , G _i-1 and H _i-1 , the predictor selects the same macroblock motion vector, otherwise the predictor chooses the median of the candidate predictors;

The prediction factor is the prediction value of the motion vector of the currently coded macroblock; the above-mentioned prediction methods are used for the components of the motion vector in the horizontal and vertical directions.

According to a large number of experimental test results, we choose TH=8.

The video coding system of the present invention designs a motion vector prediction method that combines the correlation of the motion vector in the time domain and the space domain. According to the difference of motion correlation between macroblocks, this prediction method comprehensively utilizes the correlation in time and space domains, effectively improves the accuracy of motion vector prediction, and finally improves the coding efficiency of the system.

The specific implementation manner will be described in detail below in conjunction with the accompanying drawings.

Description of drawings:

Figure 1 is the adjacent relationship of macroblocks in the spatial domain;

Fig. 2 is the adjacent relationship of macroblocks on the time domain and the space domain;

Fig. 3 is a block diagram of a video coding system;

Fig. 4 is the main program block diagram of video encoding system;

Fig. 5 is a subroutine block diagram of video encoding;

Fig. 6 is a subroutine diagram of the motion vector prediction method.

Detailed ways:

The purpose of the video coding system of the present invention is to compress the data volume of the video sequence as much as possible on the premise of obtaining a certain video reconstruction quality, so as to facilitate the storage or transmission of video information. In the block diagram of the video encoding system shown in Figure 3, cameras and capture cards are commercially available to convert the target object into a digital video sequence that can be processed by a computer. The video sequence file in the figure indicates that the input of the system can also be a digital video sequence obtained through other equipment and stored in the computer hard disk in advance. These digital video sequences are the raw video sequences that are used as input to the system for compression. Execute the video coding subroutine on the computer to compress the original video sequence, and the compressed result is stored on the hard disk of the computer in the form of a code stream file. Compared with the input original video sequence, the data volume of the generated code stream file is very small, so the system achieves the purpose of compressing video information.

Figure 4 shows the overall block diagram of the system to execute the program. In Figure 4, the video encoding subroutine adopts a hybrid encoding method based on motion compensation/DCT. Except this part of motion vector prediction subroutine, the present invention adopts the basic coding framework of international video coding standard H.264 JM6.0, and its program flow is as shown in Figure 5. It should be pointed out that during video encoding, an input video sequence frame is divided into 16×16 macroblocks, and the encoding process is performed in units of macroblocks. After the computer starts to execute the video encoding subroutine, it first initializes the encoding, which includes the video sequence format, frame rate, quantization parameters, search range of motion vectors, number of reference frames, buffers, and output stream file names etc. settings. Then the computer first reads the data of the first macroblock into the buffer according to the order from top to bottom and from left to right, performs motion estimation and compensation on the macroblock, and obtains the motion vector of the macroblock after motion estimation and compensation And the residual image, for the motion vector therein, first execute the motion vector subroutine shown in Figure 6 to predict, then carry out context-adaptive variable length coding (CAVLC) to the prediction difference; and for the residual image, first Carry out DCT transformation, then quantize and CAVLC code the DCT coefficients, and then output the compressed code stream of the macroblock and store it in the buffer. In order to meet the needs of the next frame encoding, it is finally decoded to generate a corresponding reference macroblock, which is stored in the computer memory. After the encoding of the macroblock is completed, the above encoding process is cyclically performed on all the macroblocks of the current encoding frame, and the encoding of a video sequence frame is completed.

For the video encoding subroutine shown in Figure 5, its program block diagram of the motion vector prediction subroutine part wherein is as shown in Figure 6, this motion vector prediction method is the core of the present invention, and motion vector prediction method in the current international video coding standard different. According to the different degree of motion correlation between adjacent macroblocks, this prediction method comprehensively utilizes the correlation in time and space domains to predict the motion vector. Its specific implementation process is as follows. In the initial process shown in Figure 5, set both lengths to:

$\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; 8</span>}}$

where W is the width of the image frame and H is the height of the image frame, which are used to store the motion vectors of all 4×4 sub-blocks of the previous frame and the current frame.

After the computer enters the motion vector prediction subroutine in FIG. 6 , it firstly judges the motion correlation between adjacent macroblocks E _i for the current coded macroblock E i , and the algorithm program executed is as described above. After the judgment of the motion correlation between adjacent macroblocks E _i is completed, according to the judgment result, the motion vector prediction procedure of the macroblock E _i starts to be executed, and the executed program steps are also as described above.

Finally, the motion vector of the currently coded macroblock is subtracted from the predictor to obtain the prediction difference.

It should be noted that the computer needs to separately execute the above program on the components of the motion vector in the horizontal and vertical directions to complete the prediction of the motion vector.

In order to verify the actual coding efficiency of the present invention, the following comparative experiments were carried out. For the same video sequence, take the Bus (352 * 288, 150 frames, 30fps) sequence as an example here, carry out (1) the coding program of the present invention on the coding system of the present invention; (2) international coding standard H.264 JM6 .0 basic encoding procedure, Table 2 is the comparison of the present invention and H.264 encoding efficiency. The only difference between the encoding method of the present invention and H.264 is the difference in the selected motion vector prediction method, wherein the present invention adopts the motion vector prediction method based on temporal-space domain correlation. Both encoding parameters are set according to the encoding configuration file shown in Table 1, and the quantization parameter QP takes four values of 20, 28, 34 and 40 respectively for experiments.

Table 1 is the configuration file of the encoding system. The configuration file is used to set the coding parameters of the system, which mainly include: the precision of motion estimation is 1/4 pixel, the search range is ±32 pixels; the number of reference frames is 2 frames; the entropy coding adopts context-adaptive variable-length coding ( CAVLC); block size adaptive transformation (ABT) is prohibited; Hadamard transformation and R-D optimization must be performed; the structure of the picture group is that the first frame is an I frame, and all subsequent frames are P frames;

PSNR in Table 2 represents the peak signal-to-noise ratio of the reconstructed video image. Bit rate refers to the number of bits output per second. The saved code rate is defined as the percentage of the difference between the output code rate of H.264 encoding and the encoding output of the present invention and the output code rate of H.264 encoding when the quantization parameter is constant. The average saved code rate is defined as the average saved code rate under different quantization parameter conditions.

The comparison of the coding efficiency results in Table 2 is the comparison of the output bit rate results under the condition of the same reconstruction video quality (PSNR) when the two use the same quantization parameter. The smaller the output bit rate, the higher the encoding efficiency. It can be seen that the present invention is superior to the encoding efficiency of the H.264 encoding system.

Table 1 is the configuration file of the coding system: encoding parameters state Motion Estimation Accuracy 1/4 pixel range of motion estimation ±32 reference frame 2 entropy coding CAVLC ABT 0ff Hadamard transformation 1 RD optimization 1 GOP structure IPP

Table 2 is an example of encoding results: test sequence this invention H.264 Saving code rate (%) PSNR(db) bit rate (kbps) PSNR(db) bit rate (kbps) bus 20 41.06 3222.31 41.06 3227.51 0.16 28 34.75 1134.94 34.75 1142.13 0.63 34 30.30 464.99 30.30 472.58 1.61 40 26.42 191.84 26.42 197.46 2.85 average save rate 1.31

Claims (3)

1. The video coding method based on temporal-space domain correlation motion vector prediction is that the camera converts the state of the target object into a video signal and places it in the capture card; the capture card converts the video signal into a digital video sequence and stores it in the video buffer , these digital video sequences or video data stored in the computer hard disk in the form of video sequence files are used as the input of the system for compression; the computer stores the original video sequences and executes the video encoding program, and generates compressed code stream files, The present invention is characterized in that: The computer first reads a frame of video data from the video memory of the acquisition card or the video sequence file stored on the computer hard disk into the buffer of the computer, and encodes the frame by executing the video encoding subroutine; Video encoding subroutine During video encoding, an input video sequence frame is divided into 16×16 macroblocks, and the encoding process is carried out in units of macroblocks; after the computer starts to execute the video encoding subroutine, it first initializes the encoding ; Then the computer performs motion estimation and compensation on the first macroblock in the order from top to bottom and from left to right; after motion estimation and compensation, the motion vector and residual image of the macroblock are obtained, and the motion vector , first execute the motion vector subroutine to predict, and then perform context-adaptive variable length coding (CAVLC) on the prediction difference; and first perform DCT transformation on the residual image, and then quantize and CAVLC the DCT coefficients; then output The compressed code stream of the macroblock is put into the buffer of the computer; finally, it is decoded to generate the corresponding reference macroblock; after the encoding of the macroblock is completed, the above encoding process is executed cyclically for all the macroblocks of the current coded frame to complete the encoding of a video Encoding of sequence frames; after one frame is encoded, the next frame is cyclically encoded until the last frame of the video sequence is encoded, and a compressed code stream file is generated, and the system execution program ends; In the video coding subroutine mentioned above, the motion vector prediction subroutine adopts a motion vector prediction method based on temporal and spatial domain correlation. This method comprehensively utilizes the correlation of motion vector in time domain and space domain, and adopts different prediction methods according to the difference of motion correlation between adjacent macroblocks. 2. the video coding method based on time-space domain correlation motion vector prediction according to claim 1, is characterized in that, video coding subroutine has adopted the hybrid coding method based on motion compensation/DCT, except this part of motion vector prediction subroutine In addition, the coding framework of the international video coding standard H.264 JM6.0 is adopted. 3. The video coding method based on time-space domain correlation motion vector prediction according to claim 1, characterized in that according to the difference in motion correlation between adjacent macroblocks, the comprehensive The motion vector prediction method using the correlation of the motion vector in the time domain and the space domain, the prediction method is as follows: (1) Firstly, the judgment of motion correlation between adjacent macroblocks is carried out; for any current coded macroblock, the description method of motion correlation between its adjacent macroblocks is: Let E _i be the coded macroblock of the current frame i, A _i , B _i , C _i , D _i , G _i , H _i are the left, upper, right upper, left upper, right and lower sides respectively Adjacent macroblocks, the macroblocks corresponding to i-1 in the previous frame are denoted as E _i-1 , A _i-1 , B _i-1 , C _i-1 , D _i-1 , g _i-1 and Hi _-1 ; Let B={C ₁ C ₂ C ₃ C ₄ C ₅ } be the set of motion vectors of E _i adjacent macroblocks A _i , B _i , E _i-1 , G _i-1 and H _i-1 ; (a) If for i, i∈[1, 5], all satisfy |C _i -C ₃ |≤TH Then the motion correlation between adjacent macroblocks of E _i is high; (b) If for i, i∈[1, 5], all satisfy |C _i -C ₃ |＞TH Then the motion correlation between adjacent macroblocks of E _i is low; (2) According to the judgment result of (1), the prediction method of the motion vector of the current coded macroblock is as follows: In order to utilize the correlation in the time and space domains, the present invention selects the motion vectors of A _i , B _i and C _i as candidate predictors in the space domain, and the motion vectors of E _i-1 , G _i-1 and H _i-1 The motion vector is used as a candidate predictor in the time domain, and the specific algorithm for motion vector prediction is as follows: (a) When E _i-1 , G _i-1 and H _i-1 do not exist, use A _i , B _i , and C _i as candidate predictors; (b) When the motion correlation between adjacent macroblocks is high, select A _i , B _i and E _i-1 as candidate predictors; (c) When the motion correlation between adjacent macroblocks is low, select A _i , B _i , G _i-1 and H _i-1 as candidate predictors; (d) When any one of A _i , B _i , C _i and D _i adopts intra-frame coding, if the motion vector of the macroblock at the i-1 frame and its corresponding position exists, then the motion vector of the macroblock is selected at i- 1 frame has the same motion vector and reference frame as its corresponding position block, otherwise the motion vector of this block is set to (0, 0); (e) If there is a macroblock with the same reference frame as E _i in A _i , B _i , C _i , E _i-1 , G _i-1 and H _i-1 , the predictor selects the same macroblock motion vector, otherwise the predictor chooses the median of the candidate predictors; The predictor is the predicted value of the motion vector of the currently coded macroblock; the above-mentioned prediction method is used for the components of the motion vector in the horizontal and vertical directions; the computer needs to execute the above-mentioned program separately for the components of the motion vector in the horizontal and vertical directions, and the Motion Vector Prediction.

Images