JP2013038688A - Moving picture encoding and distributing system - Google Patents

Abstract

A moving image coding and distribution system capable of predicting the occurrence of frame skip and frame repeat in advance and performing frame interpolation in advance is obtained.
A decoder board 6 decodes moving image encoded data in accordance with a first clock. The graphic board 7 processes the decoded moving image data according to the second clock to generate a moving image to be displayed on the monitor 11. The frame skip / repeat occurrence prediction device 8 predicts the timing at which moving image data is frame skipped or frame repeated based on the phase difference between the first clock and the second clock. The frame interpolation device 9 performs an interpolation process on the frame of the moving image data at the predicted timing.
[Selection] Figure 1

Description

  The present invention relates to a moving image coding and distribution system that can predict the occurrence of frame skip and frame repeat in advance and perform frame interpolation in advance.

  In the moving image encoding / delivery system, moving image data is encoded and transmitted by an encoding device, which is received by a decoding device and decoded by a decoder board. The decoded moving image data is processed by the graphic board and displayed on the monitor.

  However, frame skipping (frame skip) or frame repetition (frame repeat) occurs in the moving image data, and there is a case where the image that visually gives a bad impression is generated on the moving image displayed on the monitor. This is because the internal clock cycles of the decoder board and the graphic board are different, so that they cannot be synchronized with each other. On the other hand, after a frame skip occurs, a method for reducing the visual bad impression given by the occurrence of a click due to the frame skip has been proposed by detecting it and interpolating the missing frame (for example, , See Patent Document 1).

JP-A-10-215458

  However, in the conventional method, it is not possible to predict the occurrence of frame skip or frame repeat in advance and perform frame interpolation in advance. For this reason, in a monitoring system or a conference system that distributes moving images in real time, it has not been possible to reduce the visual negative impression caused by the occurrence of a click due to frame skip or frame repeat.

  The present invention has been made to solve the above-described problems, and its purpose is to predict the occurrence of frame skip and frame repeat in advance and to perform frame interpolation in advance. Is what you get.

  A moving image coding and distribution system according to the present invention includes a decoder board that decodes moving image encoded data in accordance with a first clock, and processes the decoded moving image data in accordance with a second clock and displays the result on a monitor. A graphic board for generating a moving image to be generated, and a frame skip / repeat occurrence prediction device for predicting a timing at which the moving image data is frame skipped or frame repeated based on a phase difference between the first clock and the second clock And a frame interpolation device that performs interpolation processing on the frame of the moving image data at the predicted timing.

  According to the present invention, it is possible to predict the occurrence of frame skip and frame repeat in advance and perform frame interpolation in advance.

It is a figure which shows the moving image coding delivery system which concerns on Embodiment 1 of this invention. It is a figure which shows the frame skip and repeat generation | occurrence | production prediction apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the output frame of a decoder board and a graphic board in case frame skip generate | occur | produces. It is a figure which shows the output frame of a decoder board and a graphic board in case frame repeat generate | occur | produces. It is a figure for demonstrating the interpolation process with respect to a frame skip. It is a figure for demonstrating the interpolation process with respect to a frame repeat. It is a figure which shows the moving image coding delivery system which concerns on Embodiment 2 of this invention.

  A video encoding / delivery system according to an embodiment of the present invention will be described with reference to the drawings. The same or corresponding components are denoted by the same reference numerals, and repeated description may be omitted.

Embodiment 1 FIG.
FIG. 1 is a diagram showing a moving image coding and delivery system according to Embodiment 1 of the present invention. In the encoding device 1, an encoder board 2 and a transmission device 3 are provided. In the decoding device 4, a receiving device 5, a decoder board 6, a graphic board 7, a frame skip / repeat occurrence prediction device 8, and a frame interpolation device 9 are provided.

  The moving image data input from the camera 10 is encoded by the encoder board 2 of the encoding device 1. The transmission apparatus 3 transmits the encoded moving image encoded data to the network. The receiving device 5 of the decoding device 4 receives the transmitted moving image encoded data. The decoder board 6 decodes the moving image encoded data according to the first clock. The graphic board 7 processes the decoded moving image data in accordance with the second clock, and generates and outputs a moving image to be displayed on the monitor 11.

  The frame skip / repeat occurrence prediction device 8 predicts the timing at which moving image data is frame skipped or frame repeated based on the phase difference between the first clock and the second clock. Specifically, the frame skip / repeat occurrence prediction device 8 compares the accumulated time of the frame outputs of the decoder board 6 and the graphic board 7 with reference to the time when the phases of the first clock and the second clock are aligned. Thus, the timing at which the moving image data is skipped or repeated is predicted. In the first embodiment of the present invention, the frame interpolation device 9 is provided in the subsequent stage of the decoder board 6 in the decoding device 4.

  The frame interpolation device 9 performs an interpolation process on the frame of the moving image data at the predicted timing. Specifically, in the case of frame skip, the frame interpolation device 9 performs an interpolation process on frames before and after skip or on a plurality of frames before and after skip, and in the case of frame repeat, the frame to be repeated or the frame to be repeated Interpolation processing is performed on a plurality of frames before and after.

  FIG. 2 is a diagram showing the frame skip / repeat occurrence prediction apparatus 8 according to Embodiment 1 of the present invention. The frame skip / repeat occurrence prediction device 8 includes a determination device 12, a first calculation device 13, and a second calculation device 14. The determination device 12 compares the period T1 of the first clock with the period T2 of the second clock, and if frame skip occurs when T1 is smaller than T2, and frame repeat occurs when T1 is larger than T2. judge.

The first computing device 13 sets the number of the frame in which the nth frame skip occurs with the minimum integer X satisfying Equation 1 below, based on the time when the phases of the first clock and the second clock are aligned. Calculate as
X ≧ T2 × n / (T2-T1) (1)

The second computing device 14 sets the number of the frame in which the mth frame repeat is generated as the reference when the phases of the first clock and the second clock are aligned, and is the smallest integer Y that satisfies the following Equation 2. Calculate as
Y> T2 * (m-1) / (T1-T2) (2)

  Here, the mechanism for predicting the timing at which moving image data is frame skipped or frame repeated by Equations (1) and (2) will be described separately for the case where frame skip occurs and the case where frame repeat occurs.

  FIG. 3 is a diagram illustrating output frames of the decoder board and the graphic board when frame skip occurs. The fifth frame on the decoder board 6 side is a frame skip. The magnitude relationship (T1 × 5 <T2 × 4) between the cumulative time up to the fifth frame on the decoder board 6 side and the cumulative time up to the fourth frame on the graphic board 7 side is the magnitude relationship before that (eg, T1 × 4> T2 × 3). This inversion means the occurrence of frame skip. Therefore, if the accumulated time of the output frames of the decoder board 6 and the graphic board 7 is compared with the time when the phases of the first clock and the second clock are aligned, the timing at which frame skip occurs can be predicted. Can do.

When the frame number where the frame skip occurs is X, the following formula (3) is established.
T1 × X ≦ T2 × (X−n) (3)
From the equation (3), the above equation (1) is derived. In addition, Formula (1) can also be expressed as the following Formula (4) using the Gauss symbol [].
X = − [− T2 × n / (T2−T1)] (4)

  Here, assuming that the period T1 of the first clock is 10 and the period T2 of the second clock is 13, frame skip occurs because T1 is smaller than T2. In this case, when Equation (1) is calculated, the frame numbers X at which the first, second and third (n = 1, 2, 3) frame skips are generated are 5, 9, and 13, respectively.

  FIG. 4 is a diagram showing output frames of the decoder board and graphic board when frame repeat occurs. The first frame on the decoder board 6 side is frame repeat, and the fourth frame is also frame repeat. The frame repeat of the first frame is self-evident, and the next fourth frame is considered. The magnitude relationship (T1 × 4> T2 × 5) between the cumulative time up to the fourth frame on the decoder board 6 side and the cumulative time up to the fourth frame on the graphic board 7 side (T1 × 4> T2 × 5) It is reversed compared to 3 <T2 × 4). This inversion means the occurrence of frame repeat. Therefore, if the accumulated times of the output frames of the decoder board 6 and the graphic board 7 are compared on the basis of the time when the phases of the first clock and the second clock are aligned, the timing at which frame repeat occurs can be predicted. Can do.

When the frame number where frame repeat occurs is Y, the following formula (5) is established.
T1 * Y> T2 * (Y + m-1) (5)
From the equation (5), the above equation (2) is derived. In addition, Formula (2) can also be expressed as the following Formula (6) using the Gauss symbol [].
Y = [1 + T2 × (m−1) / (T1−T2)] (6)

  Here, assuming that the period T1 of the first clock is 13 and the period T2 of the second clock is 10, frame repeat occurs because T1 is greater than T2. In this case, when Equation (2) is calculated, the frame numbers Y at which the second, third and fourth (n = 2, 3, 4) frame repeats are generated are 4, 7, and 11, respectively.

  FIG. 5 is a diagram for explaining an interpolation process for frame skip. When a frame skip is predicted to occur in the frame D2 of the moving image data, interpolation processing is performed on the frames D1 and D3 before and after the frame skip. If this interpolation process is not performed, the monitor display will be clogged, but if the interpolation process is performed, the monitor display will be less clogged and the movement will be smooth. Note that interpolation processing may be performed on a plurality of frames before and after skipping.

  FIG. 6 is a diagram for explaining an interpolation process for frame repeat. When it is predicted that a frame repeat will occur in the frame D2 of the moving image data, an interpolation process is performed on the repeated frame D2 ′. If this interpolation process is not performed, the monitor display will be clogged, but if the interpolation process is performed, the monitor display will be less clogged and the movement will be smooth. Note that interpolation processing may be performed on a frame to be repeated and a plurality of frames before and after the frame.

  As described above, in Embodiment 1 of the present invention, frame interpolation is performed in advance by predicting in advance the timing at which moving image data is frame skipped or frame repeated. As a result, even in a monitoring system or a conference system that distributes moving images in real time, it is possible to reduce the visual bad impression caused by the occurrence of scratches due to frame skip or frame repeat.

Embodiment 2. FIG.
FIG. 7 is a diagram showing a moving image coding and delivery system according to Embodiment 2 of the present invention. A frame interpolation device 9 is provided in front of the encoder board 2 in the encoding device 1. The information predicted by the frame skip / repeat occurrence prediction device 8 in the decoding device 4 is sent online or offline to the frame interpolation device 9 in the coding device 1. The internal configuration of the frame skip / repeat occurrence prediction apparatus 8 is the same as that of the first embodiment shown in FIG. Other configurations are the same as those in the first embodiment. Thus, even when frame interpolation is performed in advance by the encoding apparatus 1, the same effect as in the first embodiment can be obtained.

  It should be noted that the influence of the FIFO in the decoding device 4 is not taken into consideration in the equations (1) and (2). When the influence of the FIFO is taken into consideration, the values obtained by the equations (1) and (2) may be corrected according to the number of FIFO stages.

DESCRIPTION OF SYMBOLS 1 Encoding apparatus 2 Encoder board 4 Decoding apparatus 6 Decoder board 7 Graphic board 8 Frame skip and repeat generation prediction apparatus 9 Frame interpolation apparatus 11 Monitor 12 Determination apparatus 13 First calculation apparatus 14 Second calculation apparatus

Claims (5)

A decoder board for decoding moving image encoded data according to a first clock;
A graphic board for processing the decoded moving image data in accordance with the second clock and generating a moving image to be displayed on the monitor;
A frame skip / repeat occurrence prediction device that predicts a timing at which the moving image data is frame skipped or frame repeated based on a phase difference between the first clock and the second clock;
A moving image coding and distribution system comprising: a frame interpolation device that performs an interpolation process on a frame of the moving image data at the predicted timing.   The frame skip / repeat occurrence prediction apparatus compares the accumulated time of the frame outputs of the decoder board and the graphic board with reference to the time point when the phases of the first clock and the second clock are aligned. The moving image coding and delivery system according to claim 1, wherein a timing at which the moving image data is skipped or repeated is predicted. The frame skip / repeat occurrence predicting device comprises:
Judgment by comparing the period T1 of the first clock with the period T2 of the second clock and determining that frame skip occurs when T1 is smaller than T2, and frame repeat occurs when T1 is larger than T2. Equipment,
With reference to the time when the phases of the first clock and the second clock are aligned, the frame number where the nth frame skip occurs is the smallest integer satisfying X ≧ T2 × n / (T2−T1) A first computing device that computes as X;
With reference to the point in time when the phases of the first clock and the second clock are aligned, the frame number in which the mth frame repeat is generated is expressed as Y> T2 × (m−1) / (T1−T2). 3. The moving image coding and delivery system according to claim 1, further comprising a second calculation device that calculates a minimum integer Y that satisfies the second calculation device.   In the case of frame skip, the frame interpolation device performs an interpolation process on frames before and after skipping or on a plurality of frames before and after skipping, and in the case of frame repeat, repeats frames or frames to be repeated and a plurality of frames before and after that. Interpolation processing is performed with respect to this, The moving image encoding delivery system of any one of Claims 1-3 characterized by the above-mentioned. An encoder board that encodes moving image data, and further includes an encoding device that transmits the encoded moving image encoded data;
The decoder board and the graphic board are provided in a decoding device,
5. The frame interpolation device is provided in a preceding stage of the encoder board in the encoding device, or is provided in a subsequent stage of the decoder board in the decoding device. The moving image coding distribution system according to any one of the above.

Images