CN1843040A - Scalable video coding and decoding methods, and scalable video encoder and decoder - Google Patents

Abstract

Scalable video coding and decoding methods, a scalable video encoder, and a scalable video decoder. The scalable video coding method includes receiving a GOP, performing temporal filtering and spatial transformation thereon, quantizing and generating a bitstream. The scalable video encoder for performing the scalable video coding method includes a weight determination block which determines a weight for scaling. The scalable video decoding method includes dequantizing the coded image information obtained from a received bitstream, performing descaling, inverse spatial transformation, and inverse temporal filtering on the scaled transform coefficients, thereby recovering video frames. The scalable video decoder for performing the scalable video decoding method includes an inverse weighting block. The standard deviation of Peak Signal to Noise Ratios (PSNRs) of frames included in a group of pictures (GOP) is reduced so that video coding performance can be increased.

Description

Scalable video encoding and decoding method and scalable video encoder and decoder

technical field

The present invention relates to video compression and in particular to scalable video encoding and decoding methods using weights and encoders and decoders respectively using said methods.

Background technique

With the development of information communication technology including the Internet, video communication as well as text and voice communication have increased. Conventional text communication cannot satisfy various needs of users, and therefore, demand for multimedia services that can provide various types of information such as text, pictures, and music has increased. Multimedia data requires a large-capacity storage medium and a wide frequency band for transmission because the amount of multimedia data is usually large. For example, a 24-bit true-color image with a resolution of 640*480 requires a capacity of 640*480*24 bits per frame, that is, about 7.37M bits of data. When sending this image at 30 frames per second, a bandwidth of 221 Mbit/s is required. When storing a 90-minute movie based on such images, a storage space of about 1200 Gbit is required. Therefore, a compression encoding method is necessary to transmit multimedia data including text, video and audio.

The basic principle of data compression is to remove data redundancy. Data can be compressed by removing spatial redundancy where the same color or object is in an image, temporal redundancy, or memory visual redundancy taking into account human vision and limited perception of high-frequency signals Repetition, in which temporal redundancy occurs when there is a small change between adjacent frames in a moving picture, or the same sound is repeated audibly. Data compression can be classified as lossy/lossless by whether or not the source data is lost, intra/inter by whether or not each frame is compressed independently, and by the time required for compression versus the time required for recovery The same and are classified as symmetric/asymmetric compression. Data compression is defined as real-time compression where the compression/recovery time delay does not exceed 50 ms and scalable compression where frames have different resolutions. For text or medical data, lossless compression is usually used. For multimedia data, lossy compression is usually used. Meanwhile, intra-frame compression is generally used to remove spatial redundancy, and inter-frame compression is generally used to remove temporal redundancy.

Different types of transmission media for multimedia have different properties. Currently used transmission media have various transmission rates. For example, an ultra-high-speed communication network can transmit data of tens of megabits per second, while a mobile communication network has a transmission rate of 384 kbits per second. In traditional video coding methods such as Moving Picture Experts Group (MPEG)-1, MPEG-2, H.263, and H.264, temporal redundancy is removed by motion compensation based on motion estimation and compensation, and by transform encoding to remove spatial redundancy. These methods have satisfactory compression ratios, but they do not have the flexibility of true scalable bitstreams because they use reflexive methods in the main algorithm. Therefore, in order to support transmission media with various speeds or transmit multimedia at a data rate suitable for the transmission environment, data coding methods with scalability, such as wavelet video coding and sub-band video coding, may be suitable for the multimedia environment. Scalability indicates the ability to partially decode a single compressed bitstream. Scalability includes: spatial scalability, indicating video resolution; signal-to-noise ratio (SNR) scalability, indicating video quality level; and temporal scalability, indicating frame rate. Scalable video encoders encode a single stream and can send portions of the encoded stream at different quality levels, resolutions, or frame rates to accommodate constraints such as bit rate, error, and resources. A scalable video decoder can decode the transmitted video stream while changing the quality level, resolution or frame rate.

Inter-frame wavelet video coding (IWVC) can provide a very flexible, scalable bit stream. However, conventional IWVC has lower performance than encoding methods such as H.264. Due to this lower performance, although IWVC has good scalability, it is only used in very limited applications. Therefore, improving the performance of data encoding methods with scalability is a problem.

Figure 1 is a flowchart of IWVC.

In step S1, an image is received in units of a group of pictures (GOP) including a plurality of frames. Preferably, for temporal scalability, said GOP comprises ²ⁿ (n=1, 2, 3, . . . ) frames. In the embodiment of the present invention, the GOP includes 16 frames, and various operations are performed in units of GOP.

Next, in step S2, motion estimation is performed using Hierarchical Variable Size Block Matching (HVSBM). When the original image size is N*N, wavelet transform is used to obtain images of level 0 (N*N), level 1 (N/2*N/2), and level 2 (N/4*N/4). For class 2 images, the motion estimation block size is changed from 16*16 to 8*8 and 4*4, motion estimation is performed for each block, and the magnitude of absolute distortion (MAD) is obtained with respect to each block. Similarly, for a class 1 image, the motion estimation block size is changed from 32*32 to 16*16, 8*8, and 4*4, motion estimation is performed for each block, and MAD is obtained with respect to each block. For images of level 0, the motion estimation block size is changed from 64*64 to 32*32, 16*16, 8*8, and 4*4, motion estimation is performed for each block, and MAD is obtained with respect to each block.

Next, at step S3, the motion estimation tree is prune to minimize MAD.

Then, at step S4, motion compensated temporal filtering (MCTF) is performed using the pruned best motion estimation tree, which will be explained with reference to FIG. 2 . Referring to FIG. 2 , the numbers written in each frame indicate the position of the frame in the time series, and Wn (where n=1, 2, . . . , 15) indicate subbands obtained after MCTF. In other words, fr0-fr15 represent 16 frames included in a single GOP before MCTF is performed on them.

First, in temporal level 0, MCTF is performed forward on 16 image frames, thereby obtaining 8 low frequency frames and 8 high frequency subbands W8, W9, W10, W11, W12, W13, W14 and W15 . In temporal level 1, MCTF is performed forward for 8 low frequency frames, thereby obtaining 4 low frequency frames and 4 high frequency subbands W4, W5, W6 and W7. In temporal level 2, MCTF is performed forward on the 4 low frequency frames obtained in temporal level 1, thereby obtaining 2 low frequency frames and 2 high frequency subbands W2 and W3. Finally, in temporal level 3, MCTF is performed forward on the 2 low frequency frames obtained in temporal level 2, thereby obtaining a single low frequency subband W0 and a single high frequency subband W1. Thus, as a result of MCTF, a total of 16 subbands W0-W15 are obtained, including 15 high frequency subbands and a single low frequency subband at the last level. After obtaining the 16 subbands, spatial transformation and quantization are performed on the 16 subbands in step S5 of FIG. 1 . Thereafter, a bit stream is generated at step S6, which includes data obtained from spatial transformation and quantization and motion vector data obtained from motion estimation.

Contents of the invention

technical problem

Although traditional IWVC has good scalability, it still has disadvantages. Typically, to quantitatively measure the performance of video coding, a peak signal-to-noise ratio (PSNR) is used. When the difference between the original image and the encoded image is small, the PSNR value is large. When the difference between the original image and the encoded image is large, the PSNR value is small. When two images are exactly the same, the PSNR value is infinite. Figure 3 shows the distribution of average PSNR values with respect to frame index in conventional IWVC. As shown in Figure 3, the PSNR value varies greatly with respect to the frame index within the GOP. At positions such as fr0, fr4, fr8, fr12, and fr16 (ie, fr0 in another GOP), PSNR values become smaller than at their adjacent positions. When the PSNR value varies greatly with respect to the frame index, the video picture quality varies greatly over time. When the picture quality is temporarily and greatly changed, people perceive that the picture quality is deteriorated. As described above, the difference in picture quality hinders commercial services such as streaming services. Therefore, reducing the variation of PSNR values is critical for wavelet-based scalable video coding. At the same time, reducing the amount of variation in PSNR values between frames within a GOP is important in scalable video coding using wavelet-based spatial transforms, and in scalable video coding using other types such as discrete cosine transform (DCT) Spatial transforms are also important in scalable video coding.

Technical solutions

The present invention provides scalable video encoding and decoding methods and scalable video encoders and decoders thereof, which make it possible to reduce changes in peak signal-to-noise ratio (PSNR).

According to an aspect of the present invention, there is provided a scalable video coding method, comprising: (a) receiving a plurality of video frames, and performing motion compensated temporal filtering (MCTF) on the plurality of video frames to remove temporal redundancy; and (b) obtaining scaled transform coefficients from the video frame from which temporal redundancy has been removed, quantizing the scaled transform coefficients, and generating a bitstream.

The video frame received in step (a) has been wavelet transformed in order to remove spatial redundancy from the video frame, and can be obtained by applying predetermined weights to some subbands in the video frame from which temporal redundancy has been removed The scaled transform coefficients.

The scaled transform coefficients may also be obtained in step (b) by applying predetermined weights to some subbands in the video frame from which temporal redundancy has been removed, and performing spatial transformation on the weighted subbands.

Preferably, the step ( In b), the scaled transform coefficients are obtained. In this case, a predetermined weight is determined for each group of pictures (GOP). The predetermined weight has a single value for a single GOP, and is preferably determined according to the magnitude of the absolute distortion of the GOP. Here, it is preferable that the transform coefficients scaled using the predetermined weights are obtained from subbands that have a higher peak value than the low PSNR frame in the subbands used to construct the low PSNR frame The signal-to-noise ratio (PSNR) frame exerts substantially little impact.

The bitstream generated in step (b) includes information on weights used to obtain the scaled transform coefficients.

According to another aspect of the present invention, a scalable video encoder receives a plurality of video frames and generates a bitstream. The scalable video encoder includes a temporal filtering block that performs MCTF on video frames to remove temporal redundancy from the video frames, a spatial transformation block that performs spatial transformation on video frames to remove spatial redundancy from the video frames A weight determination block that determines weights to be used for scaling transform coefficients obtained from some subbands among transform coefficients obtained as a result of removing temporal redundancy and spatial redundancy from a video frame; a quantization block, which quantizes the scaled transform coefficients; and a bitstream generation block, which uses the quantized transform coefficients to generate a bitstream.

The spatial transform block may perform wavelet transform on the video frame to remove spatial redundancy from the video frame, the temporal filter block may generate transform coefficients using subbands obtained by performing MCTF on the wavelet transformed video frame, and the The weight determination block may determine weights using wavelet-transformed frames, and multiply the determined weights by transform coefficients obtained from some subbands, thereby obtaining scaled transform coefficients.

The temporal filtering block may obtain subbands by performing MCTF on video frames, the weight determination block may use video frames to determine weights, and multiply the determined weights by some subbands to obtain scaled subbands, And the spatial transformation block may perform spatial transformation on the scaled subbands, thereby obtaining scaled transformation coefficients.

Also, the temporal filtering block may obtain subbands by performing MCTF on video frames, and the spatial transformation block may generate transform coefficients by performing spatial transformation on subbands, and the weight determination block may determine weights using video frames, And the determined weight is multiplied by the transform coefficient obtained from the predetermined subband, thereby obtaining the scaled transform coefficient.

Here, it is preferable to determine the predetermined weight for each group of pictures (GOP) according to the magnitude of the absolute distortion of the GOP. Preferably, the transform coefficients scaled with predetermined weights are obtained from the subbands used to construct the low PSNR frame for a high peak signal-to-noise ratio (PSNR ) frame exerts substantially little influence.

The bitstream generation block may include information on weights used to obtain the scaled transform coefficients.

According to another aspect of the present invention, a scalable video decoding method is provided, including: extracting encoded image information, encoding order information and weight information from a bit stream; dequantizing (dequantize) the encoded image information obtaining scaled transform coefficients; and performing descaling, inverse spatial transformation, and inverse temporal filtering on the scaled transform coefficients in a decoding order opposite to an encoding order indicated by the encoding order information, thereby Restore video frames.

The decoding order is eg descaling, inverse temporal filtering and inverse spatial transformation. In addition, the decoding order may be inverse spatial transformation, descaling and inverse temporal filtering, or may be descaling, inverse spatial transformation and inverse temporal filtering.

For each group of pictures (GOP), eg predetermined weights are extracted from the bitstream. Here, the number of frames constituting the GOP is 2 ^k (where k=1, 2, 3, . . . ).

The transform coefficients to be descaled using said predetermined weights are obtained, for example, from the subbands W4, W6, W8, W10, W12 and W14 that have been generated during encoding.

According to another aspect of the present invention there is provided a scalable video decoder comprising: a bitstream analysis block which analyzes a received bitstream to extract encoded image information, encoding order information and weights from said bitstream information; an inverse quantization block, which dequantizes the encoded image to obtain scaled transform coefficients; an inverse weighting block, which performs descaling; an inverse spatial transformation block, which performs an inverse spatial transformation; and an inverse temporal filtering block, which performing inverse temporal filtering, the scalable video decoder performs descaling, inverse spatial transformation, and inverse temporal filtering on the scaled transform coefficients in an order opposite to the encoding order indicated by the encoding order information, by This restores video frames.

In a non-limiting example, the decoder performs decoding in the order descaling, inverse temporal filtering and inverse spatial transformation. Also, the decoder may perform decoding in the order of inverse spatial transformation, descaling and inverse temporal filtering or in the order of descaling, inverse spatial transformation and inverse temporal filtering.

In another non-limiting example, the bitstream analysis block extracts predetermined weights from the bitstream for each group of pictures (GOP). Here, the number of frames constituting the GOP is 2 ^k (where k=1, 2, 3, . . . ).

According to one embodiment of the invention, said inverse weighting block performs inverse scaling on transform coefficients scaled from subbands W4, W6, W8, W10, W12 and W14 that have been generated during encoding.

Description of drawings

The above and other features and advantages of the present invention will become more apparent by the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings, in which:

Fig. 1 is the flowchart of traditional interframe wavelet video coding (IWVC);

Figure 2 illustrates conventional motion compensated temporal filtering (MCTF);

3 is a graph showing the peak signal-to-noise ratio (PSNR) that occurs when a Foreman (Foreman) sequence of two groups of pictures (GOP) is performed at a speed of 512 Kbps for conventional IWVC;

Fig. 4 is a flowchart of a scalable video coding method according to an embodiment of the present invention;

Figure 5 illustrates a procedure for determining subbands to scale according to one embodiment of the present invention;

Figure 6 illustrates a schematic diagram of optimal scaling factors in terms of magnitude of absolute distortion (MAD);

7 is a graph for comparing average PSNR values obtained in the present invention with those obtained in conventional techniques;

Figure 8 illustrates MCTF using different time directions according to one embodiment of the present invention;

Figure 9 is a functional block diagram of a scalable video encoder according to one embodiment of the present invention;

Figure 10 is a functional block diagram of a scalable video encoder according to another embodiment of the present invention; and

FIG. 11 is a functional block diagram of a scalable video decoder according to one embodiment of the present invention.

Detailed ways

Exemplary, non-limiting embodiments of the invention will now be described in detail with reference to the accompanying drawings.

FIG. 4 is a flowchart of a scalable video coding method according to an embodiment of the present invention.

First, in step S10, an image is received in units of a group of pictures (GOP) including a plurality of frames. In one embodiment of the present invention, a single GOP includes 16 frames, and all operations are performed in GOP units.

After receiving the image, the weight, ie the scaling factor, is calculated in step S20. The calculation of the scaling factor will be described below.

Thereafter, at step S30, motion estimation is performed using Hierarchical Variable Size Block Matching (HVSBM). After said motion estimation, at step S40, the motion estimation tree is pruned so as to minimize the magnitude of absolute distortion (MAD).

Next, in step S50, motion compensated temporal filtering (MCTF) is performed using the pruned best motion estimation tree. As a result of MCTF, a total of 16 subbands were obtained, including 15 high frequency subbands and a single low frequency subband. In step S60, the 16 sub-bands undergo spatial transformation. A discrete cosine transform (DCT) can be used as the spatial transform, but it is preferable to use a wavelet transform. Thereafter, at step S70, frame scaling is performed using the scaling factor obtained at step S20. The frame scaling is described below. After frame scaling, embedded quantization is performed in step S80 and then a bitstream is generated in step S90. The bitstream includes encoded image information, motion vector information, and scaling factor information. During encoding, time transformation can be performed after space transformation, and scaling can be performed after time transformation. Information about the encoding order can be included in the bitstream, so a decoder can recognize different encoding orders. However, the bitstream does not necessarily include encoding order information. When encoding order information is not included in the bitstream, encoding may be recognized as being performed in a predetermined order. In an embodiment of the invention, the high frequency subband indicates the result of comparing two image frames ('a' and 'b') ((a-b)/2), and the low frequency subband indicates the average value of the two image frames ( (a+b)/2). However, the present invention is not limited thereto. For example, a high frequency subband may indicate the difference between two frames (a-b), and a low frequency subband may indicate one of the two compared frames (a).

Figure 5 illustrates a procedure for determining subbands to scale according to one embodiment of the present invention. Subbands indicate multiple high frequency frames and a single low frequency frame obtained as a result of temporal filtering. High frequency frames are called high frequency subbands, and low frequency frames are called low frequency subbands. In scalable video coding, MCTF is used as a temporal filter. When using MCTF, temporal redundancy can be removed and temporal scalability can be obtained.

The relationship between the video frames fr0-fr15 and the subbands W0-W15 generated from the MCTF and the method of restoring the time frame will be explained with reference to FIG. 5 . The relationship between video frames fr0-fr15 and subbands W0-W15 can be defined as follows:

fr15=W0+W1+W3+W7+W15

fr14=W0+W1+W3+W7-W15

fr13=W0+W1+W3-W7+W14

fr12=W0+W1+W3-W7-W14

fr11=W0+W1-W3+W6+W13

fr10=W0+W1-W3+W6-W13

fr9=W0+W1-W3-W6+W12

fr8=W0+W1-W3-W6-W12

fr7=W0-W1+W2+W5+W11

fr6=W0-W1+W2+W5-W11

fr5=W0-W1+W2-W5+W10

fr4=W0-W1+W2-W5-W10

fr3=W0-W1-W2+W4+W9

fr2=W0-W1-W2+W4-W9

fr1=W0-W1-W2-W4+W8

fr0=W0-W1-W2-W4-W8.

As shown in FIG. 3, frames fr0, fr4, fr8 and fr12 have particularly low peak signal-to-noise ratio (PSNR) compared to adjacent frames, and they are called low PSNR frames. The reason why the low PSNR frame period occurs is related to the MCTF order. In other words, motion estimation errors occur during MCTF and are accumulated with increasing temporal levels. The degree of accumulation is determined by the MCTF structure. The degree of accumulation is high for frames replaced by high frequency subbands at low temporal levels. On the contrary, a frame replaced by a high frequency subband at a high temporal level and a frame replaced by a low frequency subband at the highest temporal level have high PSNR values, and these frames are referred to as high PSNR frames.

Thus, the filtered subbands to be multiplied by the scaling factor can be selected from the subbands needed to reconstruct the low PSNR frame. Multiplied by the scaling factor means that more bits are allocated. In other words, multiplying the subbands by the scaling factor represents, when considering the preferential allocation of bits to the larger transform coefficients during embedded quantization, towards the obtained from the selected subbands compared to the other transform coefficients The transform coefficients allocate more bits. Allocating more bits to low PSNR frames in a GOP encoded with a predetermined number of bits means allocating fewer bits to frames other than low PSNR frames in the GOP. Likewise, when the PSNR value of the low PSNR frame increases, the PSNR value of the high PSNR frame decreases. The subbands that are needed to reconstruct low PSNR frames and exert less influence on high PSNR frames are selected to be multiplied by the scaling factor. In other words, the subbands that are least used to reconstruct high PSNR frames (hereinafter referred to as least changed subbands) should be selected. Accordingly, subbands W8, W10, W12 and W14 are selected first. However, since frames fr0 and fr8 have particularly lower PSNR values than other frames, special compensation is required for frames fr0 and fr8. To this end, in the described embodiment of the invention, the subbands W4 and W6 are additionally selected as the least changed subbands to be multiplied by the scaling factor in order to greatly reduce the change in PSNR values.

Likewise, as shown in FIG. 5, among the subbands W0-W15 obtained using MCTF, the minimally changed subbands W4, W6, W8, W10, W12, and W14 are multiplied by the scaling factor 'a'. In order to reduce the calculation amount of video encoding, it is preferable to calculate the scaling factor of each GOP, instead of calculating the scaling factor for all frames in the video one at a time. In the above embodiments of the present invention, the same scaling factor is used for the minimum change subbands W4, W6, W8, W10, W12 and W14 in order to reduce the amount of computation, but the spirit of the present invention is not limited to the above embodiments. It should be construed as including in the spirit of the present invention video encoding and decoding techniques in which subbands obtained through MCTF operations are weighted so as to reduce changes in PSNR values. Therefore, the case of multiplying the subbands by different scaling factors is also included in the scope of the present invention.

Various methods can be used to determine the scaling factor by which the subbands are multiplied. In one embodiment of the invention, the scaling factor is obtained for each GOP in terms of MAD. In said embodiment of the invention, MAD is defined by formula (1).

$\mathrm{<span\; class="notranslate">\; MAD</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&times;</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\frac{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Here, 'i' indicates a frame index, 'n' indicates a last frame index in a GOP, T(x,y) indicates a picture value at a position (x,y) in a T frame, and the size of a single frame is p*q.

To implement the invention, the scaling factor is multiplied by the subbands according to the MAD. Next, the PSNR value of each frame is obtained. Next, an optimal scaling factor 'a' is obtained as shown in FIG. 6 .

Fig. 6 illustrates a schematic diagram of optimal scaling factors according to MAD. In FIG. 6 , the solid line is a graph of values obtained in actual experiments, and the broken line is a graph obtained by approximating the values with a linear formula. Use formula (2) to obtain the scaling factor 'a'.

a=1.3 (if MAD<30)

a=1.4-0.0033MAD (if 30<MAD<140)

a=1 (if MAD>140) (2)

After obtaining the scaling factor 'a', scaling is performed on the subbands. In other words, among the subbands W0-W15 obtained using MCTF, the minimum change subbands W4, W6, W8, W10, W12, and W14 are scaled according to formula (3).

W4=a*W4, W6=a*W6

W8=a*W8, W10=a*W10

W12=a*W12, W14=a*W14 (use formula (2) to get "a") (3)

FIG. 7 is a graph comparing the average PSNR value obtained in one embodiment of the present invention with that obtained in the case of using a conventional MCTF.

Referring to FIG. 7, the change in PSNR value in the embodiment of the present invention is smaller than in the case of using the conventional MCTF. In addition, it can be found that the low PSNR value in the conventional case is improved in the present invention, while the high PSNR value in the conventional case is reduced in the present invention.

In addition to the method of weighting some frames in the GOP during the traditional MCTF performed only in the forward direction, the PSNR value can also be improved by combining forward temporal filtering and inverse temporal filtering according to predetermined rules during MCTF. An example of combined forward and inverse temporal filtering is shown in Table 1.

Table 1 mode flag Level 0 Grade 1 Level 2 Level 3 Forward (F=0) ++++++++ ++++ ++ + Inverse (F=1) -------- ---- -- - Combined forward and inverse (F=2) case (a) case (b) case (c) case (d) +-+-+-+-+-+-+-+-++++++++++++---- ++--+-+-++--++-- +-+-+-+- +(-)+(-)--

The cases (c) and (d) are characterized in that the low-frequency frame at the last level (hereinafter referred to as the reference frame) is located at the center of the 1st to 16th frames (ie, the 8th frame). A reference frame is the most essential frame in video encoding. Restoring other frames from the base frame. Restoration performance degrades when the temporal distance between a frame and the reference frame increases. Therefore, in cases (c) and (d), a combination of forward temporal filtering and inverse temporal filtering is done so that the reference frame is at the center, frame 8, to minimize the distance between the reference frame and every other frame time distance.

In cases (a) and (b), the average temporal distance (ATD) is minimized. To calculate ATD, time distance is calculated. Temporal distance is defined as the position difference between two frames. Referring to FIG. 3 , the time distance between the first frame and the second frame is defined as 1, and the time distance between frame 2 and frame 4 is defined as 2. ATD is obtained by dividing the sum of temporal distances between frames on which motion estimation operates in pairs by the number of frame pairs defined for motion estimation.

In case (a),

$\mathrm{<span\; class="notranslate">\; ATD</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; 15</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1.53</span>}$

In case (b),

$\mathrm{<span\; class="notranslate">\; ATD</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 5</span>}}{\mathrm{<span\; class="notranslate">\; 15</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1.53</span>}$

In the forward mode and inverse mode shown in Table 1,

$\mathrm{<span\; class="notranslate">\; ATD</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 8</span>}}{\mathrm{<span\; class="notranslate">\; 15</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 2.13</span>}$

In case (c),

$\mathrm{<span\; class="notranslate">\; ATD</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; 15</span>}}\mathrm{<span\; class="notranslate">\; 1.73</span>}$

In case (d),

$\mathrm{<span\; class="notranslate">\; ATD</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 15</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 67</span>}$

In actual simulation, as the ATD decreases, the PSNR value increases in order to improve the performance of video coding.

Fig. 8 illustrates MCTF performed in different time directions shown in case (a). Solid lines indicate forward temporal filtering and dashed lines indicate inverse temporal filtering. When MCTF is performed as shown in FIG. 8, the relationship between frames fr0-fr15 and subbands W0-W15 is defined as follows:

fr15=W0+W1-W3-W7-W15

fr14=W0+W1-W3-W7+W15

fr13=W0+W1-W3+W7+W14

fr12=W0+W1-W3+W7-W14

fr11=W0+W1+W3-W6-W13

fr10=W0+W1+W3-W6+W13

fr9=W0+W1+W3+W6+W12

fr8=W0+W1+W3+W6-W12

fr7=W0-W1+W2+W5-W11

fr6=W0-W1+W2+W5+W11

fr5=W0-W1+W2-W5+W10

fr4=W0-W1+W2-W5-W10

fr3=W0-W1-W2+W4-W9

fr2=W0-W1-W2+W4+W9

fr1=W0-W1-W2-W4+W8

fr0=W0-W1-W2-W4-W8.

In case (a) in Table 1, the PSNR value also changes according to the frame index. A frame index having a low PSNR value is determined, and a minimum change subband exerting a small influence on frames other than the frame corresponding to the determined frame index is also determined. After computing the MAD, the least altered subband is multiplied by the appropriate scaling factor. In the direction of temporal filtering during MCTF, a frame in a GOP corresponding to a certain index has good performance, while a frame in a GOP corresponding to another specific index has poor performance. The present invention is characterized by the operation of determining a frame index with a low PSNR value when determining the temporal filtering order, and then determining, among the subbands used to reconstruct the frame corresponding to the determined frame index, for Frames other than the frame corresponding to the frame index apply the least affected sub-bands, which are then multiplied by the scaling factor. In one embodiment of the invention, a single scaling factor is used for the subbands in a GOP and is determined by MAD.

In addition, even when MCTF is performed using a plurality of reference frames unlike conventional MCTF, multiplication of scaling factors can be performed using the relationship between frames and subbands in the same manner as described above.

FIG. 9 is a functional block diagram of a scalable video encoder according to one embodiment of the present invention.

The scalable video encoder includes a motion estimation block 110 , a motion vector encoding block 120 , a bitstream generation block 130 , a temporal filtering block 140 , a spatial transformation block 150 , an embedded quantization block 160 and a weight determination block 170 .

The motion estimation block 110 obtains a motion vector of a block in each frame to be encoded from a matching block in a reference frame. The temporal filtering block 140 also uses the frames. Motion vectors can be obtained using hierarchical methods such as Hierarchical Variable Size Block Matching (HVSBM). The motion vectors obtained by the motion estimation block 110 are provided to the temporal filtering block 140 so that MCTF can be performed. The motion vectors are also encoded by the motion vector encoding block 120 and then included in the bitstream by the bitstream generation block 130 .

The temporal filtering block 140 performs temporal filtering of video frames with reference to motion vectors received from the motion estimation block 110 . Temporal filtering is performed using MCTF and is not limited to conventional MCTF. For example, the temporal filtering order can be changed, or multiple reference frames can be used.

Meanwhile, the weight determination block 170 calculates the MAD with respect to the video frame using formula (1), and obtains the weight using the calculated MAD according to formula (2). The obtained weights can be multiplied by the subbands according to formula (3). In an exemplary embodiment, the weights are multiplied by transform coefficients resulting from the spatial transform performed by the spatial transform block 150 . In other words, transform coefficients are obtained by spatially transforming subbands to be multiplied by weights in formula (3), and then multiplied by weights. It is obvious that the multiplication of weights can be performed after temporal filtering, after which a spatial transformation can be performed.

The transformed coefficients scaled according to weights are sent to the embedded quantization block 160 . The embedded quantization block 160 performs embedded quantization of the scaled transform coefficients, thereby generating encoded image information. The encoded image information and encoded motion vectors are sent to the bitstream generation block 130 . The bitstream generation block 130 generates a bitstream including the encoded image information, the encoded motion vector and weight information. The bitstream is sent over a channel.

According to an exemplary embodiment, the spatial transformation block 150 uses wavelet transformation to remove spatial redundancy with respect to video frames for spatial scalability. Alternatively, spatial transformation block 150 may use DCT to remove spatial redundancy with respect to video frames.

Also, when wavelet transform is used, unlike conventional video coding, spatial transformation can be performed before temporal filtering. This operation is explained with reference to FIG. 10 .

FIG. 10 is a functional block diagram of a scalable video encoder according to another embodiment of the present invention.

Referring to FIG. 10 , a video frame is wavelet transformed by a spatial transformation block 210 . According to the well-known method of wavelet transformation, a single frame is divided into four, and one quadrant of the frame is replaced by a reduced image (referred to as L image) and replace the other three quadrants of the frame with a type of information (called the H image) from which the entire image can be recovered from the L image. In the same manner, an L image frame can be replaced with an LL image having 1/4 the area of the L image frame and information from which the L image can be restored. A compression method called JPEG2000 uses image compression using this wavelet method. Unlike a DCT image, a wavelet-transformed image includes original image information, and enables video coding with spatial scalability using downscaled images.

The motion estimation block 220 obtains motion vectors with respect to the spatially transformed frame. The motion vectors are used for temporal filtering by temporal filtering block 240 . The motion vectors are also encoded by the motion vector encoding block 230 and then included in the bitstream generated by the bitstream generation block 270 .

Weight determination block 260 determines weights from the spatially transformed frames. The determined weight is multiplied by the transform coefficient obtained from the least changed subband among the subbands resulting from temporal filtering. The scaled transform coefficients are quantized by the embedded quantization block 250 and thus converted into coded images. The encoded images are used by the bitstream generation block 270 together with motion vectors and weights to generate a bitstream.

Meanwhile, the video encoder may include two video encoders shown in FIGS. 9 and 10 for performing two types of video encoding, and the video encoder may generate a bitstream using encoded Images are obtained using the encoding order that provides better performance among the encoding orders shown in FIGS. 9 and 10 with respect to each GOP. In such a video encoder, information on the encoding order is included in the bitstream to be transmitted. In the embodiment shown in Figures 9 and 10, information about the encoding order is also included in the bitstream so that a decoder can decode all pictures that have been encoded in a different order.

When temporal filtering is performed before spatial transformation in conventional video compression, a transformation coefficient indicates a value generated by spatial transformation. In other words, a transform coefficient is called a DCT coefficient when it is generated by DCT, or a wavelet coefficient when it is generated by wavelet transform.

In an embodiment of the present invention, the term 'transform coefficient' is intended to mean a value obtained by removing spatial redundancy and temporal redundancy from a frame before performing quantization (ie embedded quantization). In other words, in the embodiment shown in FIG. 9 , the transform coefficients indicate coefficients generated by spatial transformation as in conventional video compression. However, in the embodiment shown in FIG. 10, the transform coefficients indicate coefficients resulting from temporal filtering.

The term 'scaled transform coefficients' as used in the present invention is intended to include those produced by scaling transform coefficients using weights, or by using weights to perform a spatial transformation on the result of scaling subbands obtained by temporal filtering value. Meanwhile, it may be considered to multiply transform coefficients not scaled with weights by 1, and thus scaled transform coefficients may include transform coefficients that have not been scaled and transform coefficients that have been scaled using weights.

FIG. 11 is a functional block diagram of a scalable video decoder according to one embodiment of the present invention.

The scalable video decoder includes: a bitstream analysis block 310, which analyzes the input bitstream, thereby extracting encoded image information, encoded motion vector information and weight information; encoded image information extracted by stream analysis block 310, thereby obtaining scaled transform coefficients; inverse weighting block 370, which uses said weight information to descale the scaled transform coefficients; inverse spatial transform blocks 330 and 360, performing inverse spatial transformation; and, inverse temporal filtering blocks 340 and 350, performing inverse temporal filtering.

The scalable video decoder shown in Fig. 11 includes two inverse temporal filtering blocks 340 and 350 and two inverse spatial transformation blocks 330 and 360, so that it can recover all pictures encoded in different orders. However, in an actual implementation, temporal filtering and spatial transformation may be performed on a computing device using software. In this case, only a single software module for temporal filtering and only a single software module for spatial transformation may be provided together with the option to select the order of operations.

The bitstream analysis block 310 extracts encoded image information from the bitstream, and sends the encoded image information to the inverse embedding quantization block 320 . The inverse embedding quantization block 320 then performs inverse embedding quantization on the encoded image information, thereby obtaining scaled transform coefficients. Bitstream analysis block 310 also sends weight information to inverse weighting block 370 .

The inverse weighting block 370 descales the scaled transform coefficients according to the weight information to obtain transform coefficients. The descaling is associated with a coding order. When encoding has been performed in the order of temporal filtering, spatial transformation, and scaling, the inverse weighting block 370 descales the scaled transform coefficients before the inverse spatial transformation block 330 . Next, the inverse spatial transformation block 330 performs an inverse spatial transformation. Thereafter, the inverse temporal filtering block 340 restores the video frames by inverse temporal filtering.

When encoding has been performed in the order of temporal filtering, scaling, and spatial transformation, the inverse spatial transformation block 330 performs inverse spatial transformation on the scaled transform coefficients, and then the inverse weighting block 370 descales the Block 330 processes the scaled transform coefficients. Thereafter, the inverse temporal filtering block 340 restores the video frames by inverse temporal filtering.

When encoding has been performed in the order of spatial transformation, temporal filtering, and scaling, the inverse weighting block 370 descales the scaled transform coefficients, thereby obtaining transform coefficients. Next, an inverse temporal filtering block 350 constructs an image using the transform coefficients and performs inverse temporal filtering on the image. Next, an inverse spatial transformation block 360 performs an inverse spatial transformation on the image, thereby restoring a video frame. The encoding order can be changed according to GOP. In this case, the bitstream analysis block 310 obtains encoding order information from the GOP header of the bitstream. Meanwhile, the basic encoding order may be predetermined, and the bitstream may not include encoding order information. In this case, decoding may be performed in the reverse order of the basic encoding order. For example, when the basic coding sequence is temporal filtering, spatial transformation and scaling, if the bitstream does not include coding sequence information, descaling, inverse spatial transformation and inverse temporal filtering are performed sequentially on the bitstream (i.e., as used in Figure 11 The inverse spatial transformation block 330 and the inverse temporal filtering block 340 in the lower dashed box in ) for decoding).

In the above-described embodiments, it has been described that the scalable video encoder transmits a bitstream including weights, and the scalable video decoder restores video images using the weights. The present invention is not limited thereto. For example, a scalable video encoder can transform information (ie MAD information) from which a scalable video decoder can derive weights.

Video encoders and video decoders can be implemented in hardware. Alternatively, they can be realized using a general-purpose computer including a central processing unit capable of calculation and a memory, and software for executing encoding and decoding methods. Such software can be recorded on a recording medium such as a compact disc read only memory (CD-ROM) or a hard disk so that the software can realize a video encoder and a video decoder together using a computer.

Accordingly, it will be apparent to those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as defined by the appended claims. In the embodiments described above, MCTF has been used, but any type of cycle time filtering is to be construed as being included within the scope of the present invention.

The values obtained by the experiments of the present invention are shown in Tables 2-7. In the present invention, the average PSNR is not much different from that obtained by conventional MCTF. However, the present invention reduces the standard deviation compared to conventional MCTF.

Table 2: Average PSNR in Foreman sequence bit rate this invention Traditional MCTF (forward filtering) 128 30.88 30.91 256 35.66 35.68 512 39.19 39.23 1024 43.65 43.71

Table 3: Standard Deviation in Foreman Sequence bit rate this invention Traditional MCTF (forward filtering) 128 1.22 1.23 256 0.89 0.94 512 0.75 0.84 1024 0.62 0.74

Table 4: Average PSNR in Canoa sequence bit rate this invention Traditional MCTF (forward filtering) 128 28.46 28.45 256 32.58 32.58 512 37.76 37.76 1024 45.36 45.43

Table 5: Standard Deviations in Canoa Sequences bit rate this invention Traditional MCTF (forward filtering) 128 0.859 0.861 256 1.004 1.007 512 1.000 1.020 1024 1.070 1.090

Table 6: Average PSNR in Tempete sequence bit rate this invention Traditional MCTF (forward filtering) 128 27.98 27.99 256 32.2 32.28 512 35.42 35.5 1024 37.78 37.82

Table 7: Standard Deviation in Tempete Sequence bit rate this invention Traditional MCTF (forward filtering) 128 0.348 0.350 256 0.591 0.670 512 0.555 0.682 1024 0.564 0.654

Industrial Applicability

The present invention provides a model capable of reducing changes in PSNR values between frame indices in scalable video coding. In other words, according to the present invention, high PSNR values of frames in a single GOP are reduced, while low PSNR values of other frames in the GOP are increased, so that video coding performance can be improved.

Therefore, it should be understood that the above-mentioned embodiments are only for illustration, and should not be construed as limiting the present invention. The scope of the present invention is given by the appended claims rather than the foregoing description, and all changes and equivalents which fall within the scope of the claims are intended to be embraced in the present invention.

Claims (33)

1. scalable method for video coding comprises:

(a) receive a plurality of frame of video, and a plurality of frame of video are carried out motion compensated temporal filtering (MCTF), from frame of video, to remove time redundancy; And

(b) from the frame of video that is removed time redundancy, obtain scalable conversion coefficient, quantize scalable conversion coefficient, and produce bit stream.

2. according to the described scalable method for video coding of claim 1, wherein, the frame of video that receives in step (a) has been passed through wavelet transformation, thereby removed spatial redundancy from frame of video, and by use to some subbands in the frame of video that is removed time redundancy predetermined weight obtain scalable conversion coefficient.

3. according to the described scalable method for video coding of claim 1, wherein, by using predetermined weight to some subbands in the frame of video that is removed time redundancy, then the subband of weighting is carried out spatial alternation, come in step (b) acquisition scalable conversion coefficient.

4. according to the described scalable method for video coding of claim 1, wherein, by the frame of video that is removed time redundancy is carried out spatial alternation, use predetermined weight to conversion coefficient in the conversion coefficient that produces by spatial alternation, that obtain from some subbands then, come in step (b) acquisition scalable conversion coefficient.

5. according to the described scalable method for video coding of claim 4, wherein, determine predefined weight for each picture group (GOP), for single GOP, described predefined weight has single and identical value.

6. according to the described scalable method for video coding of claim 5, wherein, determine described predefined weight according to the amplitude of the absolute distortion of GOP.

7. according to the described scalable method for video coding of claim 6, wherein, obtain to use described predefined weight and scalable conversion coefficient from subband, described subband is at the subband that is used for constructing low PSNR frame, compare with low PSNR frame, high Y-PSNR (PSNR) frame is applied the essence slight influence.

8. according to the described scalable method for video coding of claim 7, wherein, each GOP comprises 16 frames; On single direction, carry out MCTF; Calculate the amplitude (MAD) of absolute distortion by following formula,

$\mathrm{MAD}=8\&times;\underset{i=0}{\overset{\frac{n-1}{2}}{\mathrm{\&Sigma;}}}\underset{x=0}{\overset{p-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{q-1}{\mathrm{\&Sigma;}}}|T_{2i+1}(x,y)-T_{2i}(x,y)|$

Wherein, ' i ' indicates frame index, the last frame index of ' n ' indication in GOP, T (x, y) position of indication in the T frame (x, picture value y), and the size of single frame is p*q; According to the following predefined weight ' a ' that calculates, a=1.3 (if MAD＜30), a=1.4-0.0033MAD (if 30＜MAD＜140), and a=1 (if MAD＞140); And obtain to use predefined weight and scalable conversion coefficient from subband W4, W6, W8, W10, W12 and W14.

9. according to the described scalable method for video coding of claim 1, wherein, the bit stream that in step (b), produces comprise about be used to obtain the information of weight of scalable conversion coefficient.

10. a scalable video encoder receives a plurality of frame of video, and produces bit stream, and described scalable video encoder comprises:

The temporal filtering piece is carried out motion compensated temporal filtering (MCTF) to frame of video, to remove time redundancy from frame of video;

The spatial alternation piece is carried out spatial alternation to frame of video, to remove spatial redundancy from frame of video;

Weight is determined piece, determines weight, and described weight will be used for scalable at the conversion coefficient conversion coefficient that obtains as removing the result of time redundancy and spatial redundancy from frame of video, that obtain from some subbands;

Quantize block, quantize scalable conversion coefficient; And

Bit stream produces piece, uses quantized transform coefficients to produce bit stream.

11. according to the described scalable video encoder of claim 10, wherein, described spatial alternation piece is carried out wavelet transformation to remove spatial redundancy from frame of video to frame of video, described temporal filtering piece uses the subband that obtains by the frame of video execution MCTF to wavelet transformation to produce conversion coefficient, and described weight determines that piece uses the frame of wavelet transformation to determine weight, and determined weight be multiply by the conversion coefficient that obtains from some subbands, obtain thus scalable conversion coefficient.

12. according to the described scalable video encoder of claim 10, wherein, described temporal filtering piece obtains subband by frame of video is carried out MCTF, described weight determines that piece uses frame of video to determine weight, and with determined weight multiply by some subbands with obtain scalable subband, and described spatial alternation piece to scalable subband carry out spatial alternation, obtain thus scalable conversion coefficient.

13. according to the described scalable video encoder of claim 10, wherein, described temporal filtering piece obtains subband by frame of video is carried out MCTF, described spatial alternation piece produces conversion coefficient by subband is carried out spatial alternation, and described weight determines that piece uses frame of video to determine weight, and determined weight be multiply by the conversion coefficient that obtains from predetermined sub-band, obtain thus scalable conversion coefficient.

14. according to the described scalable video encoder of claim 13, wherein, determine predefined weight for each picture group (GOP), for single GOP, described predefined weight has single and identical value.

15., wherein, determine predefined weight according to the amplitude of the absolute distortion of GOP according to the described scalable video encoder of claim 14.

16. according to the described scalable video encoder of claim 15, wherein, obtain to use predefined weight and scalable conversion coefficient from subband, described subband is at the subband that is used for constructing low PSNR frame, compare with low PSNR frame, high Y-PSNR (PSNR) frame is applied the essence slight influence.

17. according to the described scalable video encoder of claim 16, wherein, each GOP comprises 16 frames; On single direction, carry out MCTF; Calculate the amplitude (MAD) of absolute distortion by following formula,

$\mathrm{MAD}=8\&times;\underset{i=0}{\overset{\frac{n-1}{2}}{\mathrm{\&Sigma;}}}\underset{x=0}{\overset{p-1}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{q-1}{\mathrm{\&Sigma;}}}|T_{2i+1}(x,y)-T_{2i}(x,y)|$

Wherein, ' i ' indicates frame index, the last frame index of ' n ' indication in GOP, T (x, y) position of indication in the T frame (x, picture value y), and the size of single frame is p*q; According to the following predefined weight ' a ' that calculates, a=1.3 (if MAD＜30), a=1.4-0.0033MAD (if 30＜MAD＜140), and a=1 (if MAD＞140); And obtain to use predefined weight and scalable conversion coefficient from subband W4, W6, W8, W10, W12 and W14.

18. according to the described scalable video encoder of claim 10, wherein, described bit stream produce piece comprise about be used to obtain the information of weight of scalable conversion coefficient.

19. a scalable video encoding/decoding method comprises:

From bitstream extraction image encoded information, coded sequence information and weight information;

By coded image information is gone to quantize to obtain scalable conversion coefficient; And

With with come by the opposite decoding order of the coded sequence of described coded sequence information indication to scalable conversion coefficient carry out and remove scalable, inverse spatial transform and filter between the inverse time, thereby recover frame of video.

20. according to the described scalable video encoding/decoding method of claim 19, wherein, described decoding order is to filter and inverse spatial transform between scalable, inverse time.

21. according to the described scalable video encoding/decoding method of claim 19, wherein, described decoding order is inverse spatial transform, go scalable and filter between the inverse time.

22. according to the described scalable video encoding/decoding method of claim 19, wherein, described decoding order is scalable, inverse spatial transform and filters between the inverse time.

23. according to the described scalable video encoding/decoding method of claim 22, wherein, for each picture group (GOP), from the bitstream extraction predefined weight.

24. according to the described scalable video encoding/decoding method of claim 23, wherein, the number that constitutes the frame of GOP is 2 ^k(k=1 wherein, 2,3 ...).

25., wherein, obtain to use predefined weight and contrary scalable conversion coefficient from subband W4, W6, W8, W10, W12 and the W14 that during encoding, produces according to the described scalable video encoding/decoding method of claim 23.

26. a scalable Video Decoder comprises:

The bit stream analysis piece is analyzed the bit stream received with from described bitstream extraction image encoded information, coded sequence information and weight information;

Inverse quantization block, with coded image go to quantize with obtain scalable conversion coefficient;

Contrary weighting block, execution is gone scalable;

The inverse spatial transform piece is carried out inverse spatial transform; And

Filter block between the inverse time is carried out between the inverse time and is filtered,

Described scalable Video Decoder with come by the opposite order of the coded sequence of described coded sequence information indication to scalable conversion coefficient carry out and remove scalable, inverse spatial transform and filter between the inverse time, recover frame of video thus.

27. according to the described scalable Video Decoder of claim 26, wherein, described decoding order is scalable for going, filtration and inverse spatial transform between the inverse time.

28. according to the described scalable Video Decoder of claim 26, wherein, described decoding order is inverse spatial transform, go scalable and filter between the inverse time.

29. according to the described scalable Video Decoder of claim 26, wherein, described decoding order is scalable for going, inverse spatial transform and filtering between the inverse time.

30. according to the described scalable Video Decoder of claim 29, wherein, the weight that described bit stream analysis piece is scheduled to from described bitstream extraction for each picture group (GOP).

31. according to the described scalable Video Decoder of claim 30, wherein, the number that constitutes the frame of GOP is 2 ^k(k=1 wherein, 2,3 ...).

32. according to the described scalable Video Decoder of claim 26, wherein, described contrary weighting block is for carrying out contrary scalable from the scalable conversion coefficient of subband W4, the W6, W8, W10, W12 and the W14 that have produced during encoding.

33. a recording medium has computer-readable code, is used for carrying out the step according to the method for any one claim of claim 1-9 and 19-25.

Images