JP2000023158A - Video decoder - Google Patents

Abstract

(57) [Problem] To provide a moving picture decoder for decoding and displaying coded moving picture data, in order to reduce both display delay and unnatural motion, a virtual reception buffer model is set to a variable frame rate. An extended decoder with new frame interval control means is provided. SOLUTION: There is provided means for controlling a display interval of a decoded image frame so that a sum of squares of a transmission delay amount and a frame display interval error is minimized. The sum of squares is a weighted sum of squares, and the means changes the virtual reception buffer delay amount. Further, the virtual reception buffer delay amount is determined by a function equivalent to the minimum value of the sum of squares.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a moving picture decoder for decoding and displaying coded moving picture data.
More specifically, the present invention relates to a variable frame rate moving image decoder having a frame display interval control unit corresponding to a variable frame rate method for changing a frame interval according to an effective amount of generated information when transmitting at a low bit rate.

[0002]

2. Description of the Related Art In recent years, with the spread of the Internet and mobile digital communications, image compression coding at a low bit rate has attracted attention as a technique applicable to various applications. In particular, the conventional variable frame rate moving image decoder displays a decoded image by skipping frames in order to reduce the amount of generated information.
Frame thinning is performed according to frame interval information (frame skip information) described in the bit stream.

FIG. 1 is a configuration diagram of a conventional variable frame rate moving image decoder. The decoder comprises an image decoder 1, a frame memory 2, a display frame memory 3, and a delay switch 4. The frame memory 2
This temporarily stores the image data output from the image decoder 1 until it becomes displayable. The display frame memory 3 is a memory immediately before display on the screen,
It is accumulated for each moving image to be displayed. The delay switch 4
The switching control for sending the image data stored in the frame memory 2 to the display frame memory 3 is performed. The delay switch 4 receives a VBV (video buffering verifie) received from the image decoder 1.
r) Display timing is controlled based on the delay value.

FIG. 2 is a timing chart from encoding to display of moving image data in a conventional system. In figure 2, P _n: n-th coded image fe _n: Encoding frame from the image P _n to the image P _{n + 1} intervals c _n: image P _n required to transmit the time fd _n: image P display frame interval from _n until the image P _{n + 1} vd _n: means the time (corresponding to the VBV delay value) to stream the first image P _n is actually displayed after being input to the reception buffer.

FIG. 2A shows image data units P _{1 to} P _7.
Indicates that the data is encoded at the timing intervals fe _{1 to} fe ₆ . FIG. 2B shows the remaining data amount of the transmission buffer for transmitting the image data encoded at the timing of FIG. 2A. FIG. 2C shows FIG.
The remaining amount of data of the VBV that receives the image data transmitted at the timing (b) is shown. FIG. 2D shows the timing at which the image data received at the timing of FIG. 2C is displayed.

For example, in FIG. 2A, when image data P ₁ is encoded, P ₂ is encoded at a frame interval of fe ₁ . When P ₁ is encoded, it is stored in the transmission buffer as shown in FIG. 2B, and indicates that the transmission is completed when the time c ₁ has elapsed. Then, as shown in FIG. 2 (c), P ₁ sent is stored in the VBV buffer. Second display timing described above is intended to be displayed when the time c ₁ after P ₁ is encoded has elapsed, 2
It is shown in broken lines in (c) and (d). P ₂ is displayed when the transmission of the output buffer has been completed.
Therefore, frame interval fe ₁ of P ₁ and P ₂ is not guaranteed. On the other hand, the first display timing described above is intended to be displayed when the time vd ₁ after P ₁ is encoded has elapsed. P ₂ is displayed when the time fd ₁ after P ₁ is encoded has elapsed. fd ₁ = fe ₁
So, the frame interval fe ₁ of P ₁ and P ₂ is guaranteed.

[0007] Dly _n is the amount of delay from the moment of encoded image data P _n until decoding and displaying at the receiving side, when _{n ≧ 2, Dly n = Dly} n-1 + fd n-1 -fe n-1 ( 1) When n = 1, Dly ₁ = vd ₁ (2)

As a result, the following equation (3) is obtained.

(Equation 1)

A display time error Derr _n which occurs when the display interval between the decoding side and the encoding side is different.
Is given as follows:

(Equation 2)

[0010] On the other hand, with respect to vd _n, represented by the following equation from its definition.

(Equation 3)

[0011] However, an overflow of the VBV buffer, taking into account the underflow _{prevention, c n ≦ vd n ≦ B} (B is the buffer size) (9) is a condition.

FIG. 2D shows two conventional display timings. The first display timing is displayed in accordance with the frame interval at the time of encoding, that is, at the timing of _Pn in FIG. The second display timing is displayed simultaneously with the arrival of the encoded stream, that is, at the position indicated by the broken line from _{Pn in} FIG.

[0013] In the first display timing, the display interval of the images on the receiving side, in the case of performing the control to follow during encoding, since the fd _n = fe _n, from the equation _{(3), Dly n = vd} 1 (10), and the display delay is always constant. The first display timing has a maximum display delay and no display error.

From the buffer underflow condition, the following equations (11) and (1) are obtained from equations (8) and (9).
2) is required.

(Equation 4)

[0015] The total time required to transmit the information amount generated in the encoder, it is considered to be approximately equal to the sum of the frame interval, and Σfe _n ≒ Σc _n, also fe
Since _n = an fd _n become the following equation (13). c _n ≦ vd ₁ (13)

The condition must be satisfied for all n, and the following equation (14) is a condition in consideration of the overflow condition.

(Equation 5)

From the equations (6) and (7), the display time error is Derr _n = 0. Therefore, the delay amount is always vd ₁ , but the display interval error is always 0.

In the first display timing, when the image display on the receiving side is kept the same as the frame interval at the time of encoding on the transmitting side in the variable frame rate system, the display timing of all frames is set to at least the maximum generation information. It must be delayed by a time corresponding to the amount. Accordingly, a very low rate at which the time required to transmit the amount of generated information of one frame tends to be large causes a large delay.

At the second display timing, vd _n =
c _n, and from equation (8), the following equation (16) (1
7) (18) is required.

(Equation 6) The second display timing has a minimum display delay and has a display error.

As described above, Dly _n = c _n (19) is obtained as Σfe ≒ Σc _n . This satisfies the buffer condition of equation (9). The delay amount is the minimum, which is the lower limit of the buffer underflow condition. Also, the delay amount is clearly smaller than the delay amount of the first display timing.

With respect to the display error, from the equation (6), Derr _n = | fe _n-1 -c _n | (20), and the display error is expressed by the frame at the first display timing and the frame at the second display timing. It occurs depending on the difference in the amount of generated information from the frame. Therefore, when the present display method is used, the amount of delay is immediately after transmission and becomes minimum, but a display error occurs.

In the second display timing, decoding and display can be performed simultaneously with the arrival of image data without being aware of the frame interval on the receiving side. In this case, the display delay is the size of frame skip. , The average delay amount is small, but the frame skipping time during encoding and the frame display interval on the receiving side are different, so that the error of the display time becomes large, resulting in an unnaturally reproduced image.

Therefore, in the case of real-time transmission, the decoder first decodes the first received image immediately after arrival,
Store the frame memory. Thereafter, each frame is decoded each time it is received, and is stored in the frame memory. As for the frame display, the first frame is displayed with the display time delayed by the time corresponding to the transmission time of the maximum possible amount of generated information, and the subsequent frames are successively displayed according to the frame display interval information. And the contents of the frame memory were displayed.

[0024]

However, in the conventional decoder, the frame display interval is not controlled on the receiving side in accordance with the display timing determined at the time of encoding. The following problems have occurred in the coded transmission.

At the first display timing, in order for the decoder to perform display in accordance with the frame display interval at the time of encoding, the image information must arrive at the receiving side by the specified display time. In addition, it is necessary to delay the display time by the amount (vd ₁ ) corresponding to the transmission time of the maximum amount of generated information. Therefore, if there is an image with a small amount of generated information that is rare but exists with a small amount of average generated information, it is necessary to delay the display by the amount of generated information, resulting in a large transmission delay. .

On the other hand, at the second display timing,
In order to reduce the amount of transmission delay, it is possible to ignore the frame display interval during encoding altogether and perform decoding and display simultaneously with the arrival of the encoded information at the decoder. Is greatly different between the encoder side and the decoder side, which causes a problem that the motion of an object in an image becomes unnatural.

Accordingly, the present invention provides a moving picture decoder having a new frame interval control means, in which the virtual reception buffer model is extended to a variable frame rate in order to reduce both display delay and unnatural motion. The purpose is to:

[0028]

SUMMARY OF THE INVENTION Accordingly, a moving picture decoder according to the present invention has means for controlling a display interval of a decoded image frame so that a sum of squares of a transmission delay amount and a frame display interval error is minimized. It is. By having such a means, between the first display timing (broken line) and the second display timing (solid line) shown in FIG. It is possible to display at the optimal display timing.

According to another embodiment of the present invention, the sum of squares is a weighted sum of squares. Thereby, the minimum value of the sum of squares is obtained according to the usage mode, and it is possible to perform display at more optimal display timing.

According to another embodiment of the present invention, the means changes a virtual reception buffer delay amount.

According to another embodiment of the present invention, the virtual receive buffer delay is determined by a function equivalent to the minimum value of the sum of squares.

[0032]

FIG. 3 is a block diagram of a moving picture decoder according to the present invention. As compared with FIG. 1, a frame interval control device 5 is provided. The frame interval control device 5
Is for controlling the VBV delay value Vd _n from the image decoder 1, a frame interval fe _n the encoding frame, receives the time and c _n required for image transmission, the delay switch 4 frame interval control signal .

Regarding the conventional control method, the first display timing with the maximum display delay and no display error, and the second timing with the minimum display delay and the display error,
Frame display interval control apparatus according to the present invention, by controlling the receive buffer delay value vd _n by using the evaluation function can be displayed by keeping small the amount of delay while suppressing an increase in display errors.

According to the present invention, the sum of the squares of the average delay amount Dly _N and the average display error Derr _{N in} the processing frame number N is used as an evaluation function, and vd _N minimizing the sum is sequentially obtained to obtain the frame display timing. Perform control.
This is represented by the following equation:

[0035]

(Equation 7)

Thus, vd _N gives the minimum value of E. Regarding η, when η ≧ 0.5, delay amount priority type control is performed, and when η <0.5, display error priority type control is performed.

FIG. 4 is an internal block diagram of the frame interval control means according to the equation (23).

FIG. 5 is a diagram showing the relationship between the frame processing number and the delay with respect to the priority parameter η. FIG. 6 is a diagram showing the relationship between the frame processing number and the timing error with respect to the priority parameter η.

When η = 0, control is performed with priority given to the display error, and the delay amount is constant at the maximum value and there is no display error. As η is increased, both the delay and the error become smaller with respect to a frame in which information generation is small. In the case of η = 1, the display is completed immediately after arrival, the delay becomes the minimum necessary for transmission, and the display error is not controlled at all.

In actual moving image transmission, while setting a small delay amount in a scene having a stationary characteristic,
Display errors can also be managed accurately. For a sudden scene change or the like, this means that a slight display error is allowed and displayed immediately after arrival as much as possible.

In the above description, one embodiment of the present invention has been described. However, for various applications, various changes, modifications and omissions in the scope of the technical idea and viewpoint of the present invention can be easily made by those skilled in the art. It can be carried out. Therefore, the above description is merely an example and is not intended to be limiting. The invention is limited only as defined by the following claims and equivalents thereof.

[0042]

As described in detail above, the moving picture decoder of the present invention is capable of completely following the frame interval at the time of encoding on the decoder side or completely ignoring it. Instead of displaying an image, it is possible to control the frame display interval in consideration of both the delay amount and the display error.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a conventional video decoder.

FIG. 2 is a timing chart of each stage of a conventional video decoder. (A) is an encoding timing diagram,
(B) is a diagram showing the remaining data amount of the sending buffer;
(C) is a diagram showing the remaining amount of data in the virtual reception buffer of the decoder, and (d) is a display timing diagram.

FIG. 3 is a configuration diagram of a moving image decoder according to the present invention.

FIG. 4 is an internal configuration diagram of a frame interval control unit according to the present invention.

FIG. 5 is a diagram illustrating a relationship between a frame processing number and a delay with respect to a parameter η according to the present invention.

FIG. 6 is a diagram illustrating a relationship between a frame processing number and a timing error with respect to a parameter η according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image decoder 2 Frame memory 3 Display frame memory 4 Delay switch 5 Frame interval control means

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK34 LB07 PP04 TA00 TA07 TB04 TC16 TC36 TD06 UA05 UA34 UA38 5C063 AB03 AC01 CA05 CA16 CA36 CA38

Claims (4)

[Claims] 1. A moving image decoder for decoding and displaying encoded moving image data, wherein a display interval of a decoded image frame is controlled such that a sum of squares of a transmission delay amount and a frame display interval error is minimized. A decoder comprising: 2. The decoder according to claim 1, wherein the sum of squares is a weighted sum of squares. 3. The decoder according to claim 1, wherein said means changes a virtual reception buffer delay amount. 4. The decoder according to claim 3, wherein the virtual reception buffer delay amount is determined by a function equivalent to a minimum value of the sum of squares.