JP2009194474A - Video encoding device - Google Patents

Abstract

【Task】
The present invention provides a moving picture coding apparatus that can set a threshold value that correlates with the complexity of motion in consideration of direct mode performance.
[Solution]
The evaluation value calculation unit 202 calculates the absolute value sum of the motion prediction residuals in the direct mode of the block of the encoding target image. An absolute value sum of prediction residuals with reference frames that have already been subjected to coding processing is calculated as an evaluation value in the direct mode. The threshold value setting unit 204 calculates a threshold value for determining the motion prediction method of the encoding target image from the average value of the prediction code amount of the block of the reference frame. The comparison unit 205 compares the evaluation value calculated by the evaluation value calculation unit 202 with the threshold calculated by the threshold setting unit 204. The determination unit 206 determines whether or not to select the direct mode as the motion prediction method of the encoding target image from the result of the comparison unit 205. When the evaluation value is smaller than the threshold value, direct mode selection is set.

[Selection] Figure 2

Description

  The present invention relates to a moving image encoding apparatus.

  When the encoding target frame is a bi-predictive picture, motion prediction is performed from a plurality of reference images and encoded. There are a plurality of motion prediction modes, and an optimum method is selected from these modes. However, if the amount of codes generated in all methods is calculated and one method is determined from the result, calculation for motion prediction methods other than the adopted method is wasted.

There has been proposed a technique for early determination of a motion prediction mode by comparing a function of a quantization parameter of an encoding target block with an evaluation value based on a motion prediction residual in a spatial direct mode (for example, Non-Patent Document 1).
Xiaoan Lu, “Fast Mode Decision and Motion Estimation for H.264 with a focus on Mpeg-2 / H.264 transcoding,” ISCAS, 2005 ITU-T Recommendation H.264, “Advanced Video Coding for generic audiovisual services.” (ISO / IEC 14496-10 | ITU-T Rec. H.264)

  In the above method, there is a problem that an appropriate motion prediction method cannot be determined because early determination is performed based on a threshold value that does not reflect the complexity of the motion of the encoding target image. Since the spatial direct mode predicts motion from surrounding motion vectors, if the motion of the encoding target image is common between adjacent blocks, a highly accurate motion vector can be calculated. However, when the direction and magnitude of the motion are different for each block, the accuracy of the motion vector calculated in the spatial direct mode is low. That is, the evaluation value based on the motion prediction residual in the direct mode varies depending on the motion complexity of the encoding target image. For this reason, there is a problem that the early determination rate in the direct mode greatly fluctuates at a threshold value that does not reflect the complexity of motion.

  The present invention has been made to solve the above-described problem, and an object of the present invention is to provide a moving picture coding apparatus that performs early determination of the direct mode based on a threshold value that reflects the complexity of motion.

In order to achieve the above object, in an image encoding apparatus for image information that performs encoding of image information by selecting a motion compensation mode for each encoding target block of an encoding target frame from a plurality of motion compensation schemes, A first calculation unit that calculates a code amount average value of a code amount of a motion vector for each block of the reference frame of the reference frame, and a threshold value based on the code amount average value calculated by the first calculation unit A motion vector for obtaining a motion vector of a coding target block based on a first motion prediction method using a second calculation unit that calculates for each frame and a motion vector of a plurality of coded blocks adjacent to the coding target block And a third calculation for calculating an evaluation value based on a motion prediction residual of the motion vector obtained by the calculation unit and the motion vector calculation unit of the encoding target block A comparison unit that compares the threshold value and the evaluation value, and a determination unit that determines a motion vector of the coding target block as a motion vector of the motion prediction method when the evaluation value is smaller than the threshold value. A motion prediction calculation unit that performs motion prediction of the coding target block based on a second motion prediction method different from the first motion prediction method when the evaluation value is equal to or greater than the threshold; There is provided a moving image encoding apparatus comprising: an encoding unit that encodes an encoding target block based on a prediction method.

ADVANTAGE OF THE INVENTION According to this invention, the efficiency of the early determination of the motion estimation system of an encoding object frame can be improved.

  Hereinafter, embodiments of the present invention will be described.

(First embodiment)
FIG. 1 is a block diagram showing a moving picture encoding apparatus according to the first embodiment. Performs video encoding processing compliant with H.264 / MPEG-4 AVC. The moving image coding apparatus according to the present embodiment includes a difference value calculation unit 101, an orthogonal transformation unit 102, a quantization unit 103, a motion prediction unit 104A, an inverse quantization unit 105, an inverse orthogonal transformation unit 106, a frame memory as main components. 107, an adder 108, and an entropy encoder 109.

  The frame memory 107 stores decoded images of encoded frames in units of frames. The motion prediction unit 104A detects the motion of each block of the encoding target image frame from the already encoded reference image frame stored in the frame memory 107, and obtains a motion vector and a motion prediction signal. The difference value calculation unit 101 subtracts the motion prediction signal calculated by the motion prediction unit 104A from the video signal to be encoded, and outputs a motion compensation prediction error signal that is the obtained difference signal.

  The orthogonal transform unit 102 performs a discrete cosine transform on the difference value (motion compensation prediction error signal) output from the difference value calculation unit 101. The quantization unit 103 quantizes the transform coefficient obtained by the discrete cosine transform and outputs it. Discrete cosine transform is one type of orthogonal transform that decomposes an image into spatial frequency components, and various other orthogonal transform methods are known. Other orthogonal transforming means may be used as necessary, such as when applying to other systems. The entropy encoding unit 109 performs variable length encoding on the transform coefficient quantized by the quantization unit 103 and output.

  The inverse quantization unit 105 inversely quantizes the transform coefficient quantized and output by the quantization unit 103 and transforms it into a discrete cosine transform coefficient. The inverse orthogonal transform unit 106 performs inverse discrete cosine transform on the discrete cosine transform coefficient output from the inverse quantization unit 105 to generate a decoded image of the motion prediction residual. The addition unit 108 adds the prediction residual value to the prediction value, obtains a decoded image of the prediction residual that reproduces the pixel value that is the result of encoding, and causes the frame memory 107 to hold the decoded image.

  FIG. 2 is a block diagram showing the motion prediction unit 104A.

  The motion prediction unit 104A includes a motion vector calculation unit 201, an evaluation value calculation unit 202, a code amount calculation unit 203, a threshold setting unit 204, a comparison unit 205, a determination unit 206, a motion prediction calculation unit 207, and a method selection unit 208.

  The motion vector calculation unit 201 calculates a motion vector by median prediction obtained from the median of motion vectors of surrounding blocks encoded in the past of the encoding target image frame.

  The evaluation value calculation unit 202 calculates the absolute value sum of the motion prediction residuals in the direct mode of the block of the encoding target image. Directly the sum of absolute differences for each pixel between the block portion indicated by the motion vector in the decoded image of the reference frame that has already been encoded and held in the encoding target frame memory 107, and the moving image signal to be encoded Calculate as the mode evaluation value.

  The code amount calculation unit 203 calculates a prediction code amount of a motion vector obtained based on the motion prediction method selected by the method selection unit 208 when the reference frame is encoded.

  The threshold value setting unit 204 calculates a threshold value for early determination of the motion prediction method of the encoding target block as the direct mode. First, the code amount of the motion vector is calculated for each block of the reference frame, and the average value is calculated from the calculated code amount of the motion vector. Based on the calculated average value, a threshold value for determining the motion prediction method of the encoding target image is calculated. The calculation method will be described later.

  The comparison unit 205 compares the evaluation value calculated by the evaluation value calculation unit 202 with the threshold calculated by the threshold setting unit 204. The comparison result is output to the determination unit 206.

  The determination unit 206 determines whether or not to select the direct mode as the motion prediction method of the encoding target image from the result of the comparison unit 205. When the evaluation value is smaller than the threshold value, direct mode selection is set in the method selection unit 208. If the evaluation value is larger than the threshold value, the prediction calculation unit 207 is requested to select another motion prediction method.

  The motion prediction calculation unit 207 evaluates and determines a motion prediction method other than the direct mode when the direct mode is not selected. The evaluation values of the 16x16, 8x16, and 16x8 size motion prediction methods are output to the method selection unit 208. Note that the type of motion prediction method other than the direct mode calculated here may be any of the methods described in Non-Patent Document 2 according to the standard.

When the direct mode is selected by the determination unit 206, the method selection unit 208 determines the motion prediction method of the encoding target frame as the direct mode. In other cases, a motion prediction method that minimizes the evaluation value obtained by the evaluation value calculation unit 202 and the motion prediction calculation unit 207 is adopted, and a motion vector and a difference image signal of the corresponding method are output.
Next, the operation of the moving picture encoding apparatus of this embodiment will be described.

  FIG. 3 is a flowchart showing the processing contents of threshold setting of the motion prediction unit 104A. Of the frames that are referred to by the encoding target frame for which early determination is performed, this processing is executed after the encoding of the immediately preceding frame in the encoding order as shown in the lower part of FIG.

(Step S101)
In step S101, the prediction code amount of the motion vector of any one of the reference frames of the encoding target frame is calculated. The prediction code amount cost of the motion vector for each macroblock of the reference frame is obtained. The prediction code amount cost of the motion vector is calculated by equation (1).

  Where λ is a predetermined coefficient depending on the quantization parameter, mvx and mvy are the x-direction and y-direction components of the motion vector, respectively, and mvpredx and mvpredy are the x of the motion prediction vector calculated by one of the prediction methods. It is a component of direction and y direction.

  An average value cost_ave is calculated by taking the sum of the costs of all the blocks in the reference frame and dividing the sum by the number of macroblocks in one frame.

(Step S102)
Next, the threshold calculation unit 204 calculates thres_med based on Equation (2) based on the average value cost_ave of the prediction code amount of the motion vector of the reference frame.

(Step S203)
Next, the threshold value calculation unit 204 sets a threshold value thres for determining the direct mode. The threshold value thres is set based on equation (3).

Where THRES_MIN is a predetermined constant,
MAX (x, y) is a function that selects the larger value of x and y. Therefore, the larger one of thres_med calculated in step S102 and THRES_MIN set in advance as the lower limit value of the threshold value thres is selected and set as the threshold value thres.

  As shown in equation (2), thres_med is calculated based on cost_ave, which is an evaluation value indicating the complexity of the motion of the encoding target image.

  Therefore, the more complex the motion, the lower the performance of motion prediction, and the value of cost_ave, which is an evaluation value, increases. Accordingly, as the value of cost_ave increases, the value of thres_med also increases. Conversely, when the complexity of motion is low, the value of cost_ave, which is the evaluation value, decreases, and the value of the threshold thres_med also decreases accordingly. In that case, it can be considered that the reliability of the motion vector predicted by the direct mode is high. Since it is known that there is a logarithmic relationship between cost_ave and the evaluation value based on the residual in the direct mode, the logarithm is used as a calculation formula for setting the threshold in the formula (2).

  Examining the relationship between the image degradation and the threshold value thres_med for determining the direct mode at an early stage, it is known that image degradation does not occur much even if a certain value is set as the threshold value regardless of the frame. Therefore, in equation (3), when thres_med is smaller than a predetermined value THRES_MIN, the threshold is set to THRES_MIN in order to forcibly determine the motion prediction method in the direct mode. As a result, when the reliability of the prediction method in the direct mode is high, the threshold thres is reduced, so that evaluation calculation of other prediction methods that are not finally adopted can be prevented.

  FIG. 4 is a flowchart illustrating a method of performing the direct mode early determination of the motion prediction unit 104A.

(Step S201)
First, the motion vector calculation unit 201 predicts a motion vector of an encoding target image in the direct mode. The evaluation value calculation unit 202 calculates a difference absolute value sum BD_SAD from the decoded image block of the reference frame corresponding to the motion vector calculated in the direct mode of the block of the encoding target image. BD_SAD is calculated based on equation (4).

  Here, (x, y) corresponds to the pixel position of the frame, and Cur (x, y) is a pixel value at (x, y) of the encoding target image. mvx and mvy indicate the x-direction and y-direction components of the direct mode motion vector, respectively. Ref (x + mvx, y + mvy) is a pixel value at the position (x + mvx, y + mvy) of the decoded image of the reference image. This is executed for each macroblock of the encoding target frame. The calculated sum of absolute differences BD_SAD is used as the evaluation value of the direct mode of the block of the encoding target frame.

  Here, the absolute value sum of the prediction residual is used as the evaluation value, but the Hadamard transform is further applied to the prediction residual and the absolute value sum is calculated (SATD) or the square error sum (SSD). It may be used.

(Step S202)
Next, the comparison unit 205 compares the threshold value thres calculated in step S202 with the evaluation value BD_SAD. If the evaluation value BD_SAD is smaller than the threshold value thres, the process proceeds to step S203. If BD_SAD is larger than the threshold value thres, the process proceeds to step S204.

(Step S203)
When the evaluation value BD_SAD is smaller than the threshold value thres, the early determination unit 206 sets the motion prediction method of the block to the direct mode. The system selection unit 208 is notified that the motion prediction system is set to the direct mode.

(Step S204)
On the other hand, the motion prediction calculation unit 207 continues to calculate motion prediction methods other than the direct mode, and outputs an evaluation value calculated from each method to the method selection unit 208. Based on these evaluation values, the method selection unit 208 adopts a motion prediction method that minimizes the evaluation value. Then, the motion vector and the difference image signal of the corresponding method are output.

  FIG. 7 is a diagram illustrating an example of an H.264 prediction structure. The upper diagram shows the display order during reproduction along the time series. The figure below shows the order in which frames are encoded.

  Before the frame B1 is encoded, the frame P3 that is the reference frame is encoded. When determining the motion prediction method of the frame B1, the prediction code amount of the motion vector of the frame P3 is calculated in step S101. In step S201, the absolute value sum of the prediction residuals of the frame P3 and the frame B1 is obtained.

  In the moving picture coding apparatus according to the present embodiment, threshold calculation is performed from the average value of the prediction code amounts of the motion vectors of the reference frame, which is correlated with the motion complexity of the coding target picture. By using the average value of the code amount of the motion vector of the reference frame, the motion complexity at the macro level of the screen is evaluated. Thereby, a threshold value reflecting the performance of motion prediction can be set for the encoding target image.

  In addition, by quickly determining the direct mode, which is one of the motion prediction methods, by comparing the evaluation value of the direct mode with a threshold value, other motion prediction calculations can be omitted with high efficiency.

(Second Embodiment)
A moving image encoding apparatus according to the second embodiment will be described in detail. Here, a duplicate description of the same parts as those in FIG. 2 is omitted.

  FIG. 5 is a block diagram showing the motion prediction unit 104A extracted from the video encoding apparatus according to the second embodiment.

  In this embodiment, the motion prediction unit 104A in FIG. 2 further includes a threshold calculation formula setting unit 309.

  The threshold calculation formula setting unit 309 recalculates the parameters (a, b) of the calculation formula shown in the formula (2) of the threshold calculation unit 304 every time encoding of the encoding target frame is completed. Specifically, the evaluation value BD_SAD calculated by the evaluation value calculation unit 302 is summed for all the blocks in the frame, and the value obtained by dividing the sum by the number of macroblocks is input, and the coefficient of the calculation formula of the threshold value calculation unit 304 To reset. The threshold calculation formula setting unit 309 also has a function of detecting a scene change.

At this time, the coefficients a and b of the threshold value thres_med are set by Expression (5).

  BD_SAD_AVG is an average of the evaluation values BD_SAD of the frames subjected to the encoding process. BD_SAD_AVG is calculated by taking the sum of the evaluation value BD_SAD for each block for one frame and dividing it by the number of blocks in the frame. Further, count is the number of times that the threshold calculation formula based on the formula (5) is reset for the sequence. Each time the reset count count is increased, the range of change of a and b in the calculation formula for calculating the threshold shown in equation (2) is gradually narrowed.

  Since the relationship between the motion vector code amount and the direct mode performance hardly varies within the same scene, the threshold is stabilized by gradually narrowing the change width for the same scene. Since the nature of the direct mode may change greatly depending on the scene change, it is preferable to initialize count to 0 at the scene change point.

  FIG. 6 is a flowchart showing the processing details of resetting the threshold calculation formula of the motion prediction unit 104A, which is executed after encoding the encoding target frame.

(Step S301)
First, the average value BD_SAD_AVG is calculated by dividing the sum of the evaluation values BD_SAD of the direct mode for each block calculated by the evaluation value calculation unit 202 at the time of encoding by the number of macroblocks.

(Step S302)
Next, in the threshold calculation formula setting unit 310, based on BD_SAD_AVG, thres_med described in formula (2) and cost_ave which is the average value of cost described in formula (1), Perform resetting. At this time, resetting of the threshold calculation formula is set by formula (5).

  In the moving picture coding apparatus according to the present embodiment, the threshold calculation formula (2) is reset using the average value of the direct mode evaluation values actually calculated by motion prediction. The relationship between the average value of the motion vector prediction code amount and the performance of the direct mode varies greatly even within the sequence. A motion prediction method with higher efficiency by dynamically reflecting the relationship between the average value of the prediction code amount of the motion vector of the frame referenced every time the scene changes and the direct mode of the encoding target frame in the threshold. Can be determined early.

  Each embodiment is not limited to the above example, and various modifications can be made. For example, the value calculated by the code amount calculation unit 203 may be substituted with the amount of encoded data of motion vector information output from the entropy encoding unit 109. The evaluation value calculated by the evaluation value calculation unit 202 and other motion prediction method calculation units may be SATD that is the sum of absolute values of frequency component images obtained by frequency conversion by Hadamard transform.

  In addition, the motion vector calculated by the motion vector calculation unit 201 is not the motion vector of the direct mode described in Non-Patent Document 2, which is the median value of the motion vectors of the surrounding blocks, but the motion vector of the surrounding blocks. It may be an average value.

In addition, the reference frame for calculating the average value of the prediction code amount cost of the motion vector represented by the expression (1) is not the reference frame immediately before in the coding order shown in the lower part of FIG. 7 but the upper part of FIG. It may be the closest frame in the displayed order.

The functional block diagram which shows the moving image encoder of 1st Embodiment. The functional block diagram which shows 104 A of motion estimation parts of the moving image encoder of 1st Embodiment. The flowchart which shows the operation | movement of the threshold value calculation of 1st Embodiment. The flowchart which shows the operation | movement of the early determination of the mode of 1st Embodiment. The block diagram which shows 104 A of motion estimation parts of the moving image encoder of 2nd Embodiment. 10 is a flowchart illustrating an operation of resetting a threshold value calculation formula according to the second embodiment. The figure which shows the example of the prediction structure of H.264 / AVC.

Explanation of symbols

101 ... difference value calculation unit 102 ... direct transformation unit 103 ... quantization unit 104 ... motion prediction unit 105 ... inverse quantization unit 106 ... inverse orthogonal transformation unit 107 ... frame memory 108 ... addition unit 109 ... entropy coding unit 201, 301 ... motion vector calculation unit 202, 302 ... evaluation value calculation unit 203,303 ... code amount calculation unit 204,304 ... threshold setting unit 205,305 ... comparison unit 206,306 ... determination unit 207,307 ... motion prediction calculation unit 208, 308... Method selection unit 309... Threshold calculation formula setting unit

Claims (10)

In an image encoding apparatus for image information that performs encoding of image information by selecting a motion compensation mode for each encoding target block of an encoding target frame from a plurality of motion compensation methods,
A first calculation unit that calculates a code amount average value of a code amount of a motion vector for each block of a reference frame of the encoding target frame;
A second calculation unit that calculates a threshold value based on the average code amount calculated by the first calculation unit for each reference frame;
A motion vector calculation unit for obtaining a motion vector of an encoding target block based on a first motion prediction method using a motion vector of a plurality of encoded blocks adjacent to the encoding target block;
A third calculation unit that calculates an evaluation value based on a motion prediction residual of a motion vector obtained by the motion vector calculation unit of the encoding target block;
A comparison unit for comparing the threshold value and the evaluation value;
A determination unit that determines a motion vector of the encoding target block as a motion vector of the first motion prediction method when the evaluation value is smaller than the threshold;
A motion prediction calculation unit that performs motion prediction of the coding target block based on a second motion prediction method different from the first motion prediction method when the evaluation value is equal to or greater than the threshold;
A method selection unit for selecting a motion prediction method from the obtained motion prediction method;
A moving picture encoding apparatus comprising: an encoding unit that encodes an encoding target block based on a selected motion prediction method.
The moving image encoding apparatus according to claim 1, wherein the second calculation unit calculates the threshold value such that the threshold value increases as the code amount average value increases.
The code based on a value calculated from an evaluation value average value that is an average value of the evaluation values of encoded frames that have been subjected to encoding processing, and the code amount average value of a reference frame of the encoded frame A calculation formula setting unit that sets, as a calculation formula, a linear function related to the average code amount of the reference frame of the frame to be converted;
The moving image encoding apparatus according to claim 2, wherein the second calculation unit calculates the threshold based on the calculation formula.
4. The moving picture encoding apparatus according to claim 3, wherein the calculation formula set by the calculation formula setting unit is a linear function relating to a logarithm of the code amount average value.
5. The moving picture encoding apparatus according to claim 4, wherein the calculation formula setting unit reduces the change in the slope and intercept of the linear function as the number of times that the linear function is set increases.
It further comprises a scene detection means for detecting a scene change from the input encoding target frame group,
6. The moving picture coding apparatus according to claim 5, wherein the calculation formula setting unit initializes the number of times the coefficient is set when the scene detection unit detects a scene change.
The third calculation unit calculates the sum of absolute differences between the block of the reference frame indicated by the motion vector of the motion prediction method and the encoding target block as the evaluation value. Video encoding device.
The moving image encoding apparatus according to claim 1, wherein the second calculation unit uses the lower limit value as a threshold value when the threshold value is smaller than a predetermined lower limit value.
2. The moving picture encoding apparatus according to claim 1, wherein the motion vector calculation unit is calculated by median prediction from motion vectors of a plurality of encoded blocks adjacent to the encoding target block.
2. The moving picture encoding apparatus according to claim 1, wherein the motion vector calculation unit calculates an average of motion vectors of a plurality of encoded blocks adjacent to the encoding target block.

Images