JP2012175548A - Moving image encoding apparatus and moving image encoding method - Google Patents

Abstract

A moving picture coding apparatus and a moving picture coding method capable of processing a high-quality image while suppressing a code amount with a small amount of calculation are provided. In the moving picture encoding apparatus in which the difference between the predicted encoding target image and the original image is encoded to obtain the encoded stream 311, the encoding target image is referred to by referring to the same image as the encoding target image. An intra-screen prediction unit 305 that performs prediction, an inter-screen prediction unit 306 that predicts an encoding target image with reference to an image different from the encoding target image, and an intra-screen prediction unit 305 and an inter-screen when encoding the difference A mode selection unit 307 that determines which prediction result of the prediction unit 306 is used is provided, and the mode selection unit 307 performs the determination based on a parameter value at the time of inter-screen prediction.
[Selection] Figure 3

Description

  The present invention relates to a moving picture coding technique for coding a moving picture, and particularly to a moving picture coding apparatus and a moving picture coding method that switch between intra prediction and inter prediction.

As a technique for recording and transmitting large-capacity moving picture information as digital data, encoding methods such as MPEG (Moving Picture Experts Group) have been conventionally established, and these are further encoded by MPEG-1, MPEG-2, MPEG- 4. H. H.264 / AVC (Advanced Video Coding) and other international standardized encoding methods.
These encoding methods have been adopted as encoding methods for video content in digital satellite broadcasting, DVDs, Blu-ray recorders, mobile phones, digital cameras, terrestrial digital broadcasting, etc. The range is expanding and becoming familiar.

  By the way, in these standards, the image information (decoded image) for which the encoding process has been completed is used, the image to be encoded is predicted in units of blocks including a plurality of pixels, and a difference (prediction difference) from the original image is calculated. Encoding, thereby eliminating the redundancy of moving images and reducing the amount of code, but refer to the peripheral area in the same screen for prediction in block units of the encoding target image at this time In-screen prediction (hereinafter referred to as intra prediction) and inter-screen prediction (hereinafter referred to as inter prediction) performed with reference to an image different from the target image. It can be switched accordingly.

  The switching of the prediction means at this time is, for example, H.264. H.264 / AVC, MPEG-4, and the like are performed in units of macroblocks configured of 16 × 16 pixels, but the switching method in this case is not particularly defined in the standard. However, in many cases, an evaluation method using a cost function based on the sum of pixel value differences SAD (Sum of Absolute Differences) between the original image and the predicted image is generally used.

  However, in the case of an encoder having a pipeline structure that performs parallel processing in units of macroblocks, for example, encoding of macroblocks in the vicinity is not yet completed at the time of determining the target macroblock prediction means. In many cases, a prediction image of intra prediction using a decoded image of a peripheral block cannot be generated, and therefore SAD cannot be calculated.

  Therefore, when determining the prediction means, instead of using the “decoded image” of the neighboring block as described above, the SAD obtained by so-called pseudo intra prediction in which the “original image” of the neighboring block is used for intra prediction. A method of performing cost calculation using (referred to as pseudo SAD) is known as a prior art (see, for example, Patent Documents 1, 2, and 3).

JP 2007-288473 A JP 2008-252346 A JP 2009-81830 A

The above-mentioned prior art does not take into consideration that the pseudo SAD value may be far from the SAD value, and there is a problem that the encoding efficiency is greatly reduced due to erroneous selection of the prediction means. It was.
In the case of the prior art, particularly in the low bit rate band, the values of SAD and pseudo-SAD are far apart due to the influence of quantization error, and therefore, the determination necessary for selecting the prediction means, so-called intra / inter determination (intra / inter determination) An error occurs in (also referred to as inter determination), and as a result, the coding efficiency is greatly reduced.

More specifically, when using a rate control function that controls the quantization parameter so that the encoded stream has a desired bit rate, the value of the quantization parameter in the target macroblock is determined at the time of performing the intra / inter determination. Often not.
On the other hand, since the optimum prediction means often varies depending on the bit rate, it is usual to include the quantization parameter item in the cost function used for intra / inter determination.

However, as a result, a tentative value must be set for the quantization parameter when calculating the cost. However, if this tentative value is significantly different from the actual value, an appropriate judgment result is obtained. As a result, the encoding efficiency is lowered.
However, the provisional value here is, as the name suggests, "provisional", so it is only guessed, and the correct setting is relied upon. As a result, the conventional technology is said to reduce the encoding efficiency. Problems will arise.

  On the other hand, some of the standards related to the above-mentioned encoding schemes perform prediction by dividing a macroblock into smaller blocks. In this case, there are a wide variety of division patterns, and optimal prediction means and division patterns are used. The determination of the combination (encoding mode) requires an enormous amount of calculation and does not solve the problem.

  In addition, in the case of a video encoder that performs parallel processing in units of macroblocks, the completion of encoding of adjacent blocks cannot be obtained when the encoding mode of the target block is selected, so that an intra-predicted image cannot be generated. There is no high intra / inter determination.

In addition, in the case of the prior art that controls the quantization parameter using rate control, the provisional quantization parameter is set and the determination is made. At this time, the quality of the intra / inter determination depends greatly on the quantization parameter. To do.
Therefore, in this case, if the set temporary quantization parameter is different from the actual quantization parameter, an error occurs in the determination result, resulting in a problem that the image quality is deteriorated.

At this time, H.C. In new standards such as H.264 / AVC, the block size can be changed according to the design.
However, on the other hand, since the types of encoding modes that are candidates increase, an enormous amount of calculation is required to determine the optimum mode.
An object of the present invention is to provide a moving picture coding apparatus and a moving picture coding method capable of processing a high-quality image while suppressing a coding amount under a small calculation amount.

The above object is to provide a video encoding apparatus that encodes a difference between a predicted encoding target image and an original image, and refers to an image different from the encoding target image for prediction of the encoding target image. Inter-screen prediction means, prediction of the encoding target image with reference to the same image as the encoding target image, intra-screen prediction means when encoding the difference, Mode selection means for determining which of the intra-screen prediction means uses the prediction result is provided, and the mode selection means is achieved by performing the determination based on a parameter value at the time of inter-screen prediction.
Similarly, the above object is to encode a difference between the predicted encoding target image and the original image in a moving image encoding method that encodes a difference between the predicted encoding target image and the original image. The prediction of the encoding target image is performed with reference to an image different from the encoding target image, and the prediction of the encoding target image is performed with reference to the same image as the encoding target image. This is also achieved by selecting the case based on the threshold value set as the parameter value at the time of inter-screen prediction.

The threshold value introduced by the present invention has very low dependency on the picture and bit rate, and can be set as a constant, so there is no problem, so even if the quantization parameter is not yet determined, such as when using rate control, it is objective. This makes it possible to determine the optimal intra / inter determination. In addition, by performing the intra / inter determination before determining the block size, it is possible to narrow down the encoding mode, and the processing amount required for the encoding mode selection process can be reduced.
Therefore, according to the present invention, it is possible to provide a moving picture coding technique capable of providing a high-quality image with a small calculation amount and a small code amount.

It is a block diagram which shows Embodiment 1 of the image coding apparatus by this invention. It is a block diagram of the mode selection part in Embodiment 1 of this invention. It is a block diagram which shows Embodiment 2 of the image coding apparatus by this invention. It is a block diagram of the mode selection part in Embodiment 2 of this invention. It is explanatory drawing of a moving image encoding process. It is explanatory drawing of the prediction process between screens. It is explanatory drawing of the prediction process in a screen. It is explanatory drawing of the category classification | category in the prediction in a screen in this invention, and the prediction between screens. It is explanatory drawing regarding the performance of the prediction in a screen, and the prediction between screens. It is a characteristic view which shows an example of the actual data showing the property of the prediction in a screen, and the prediction between screens. It is a characteristic view which shows another example of the actual data showing the property of the prediction in a screen, and the prediction between screens. It is explanatory drawing of the intra / inter determination in this invention. It is explanatory drawing of the problem which generate | occur | produces when performing parallel processing. It is explanatory drawing regarding the pixel position of a differential filter. It is explanatory drawing which shows an example of a differential filter. It is explanatory drawing which shows another example of a differential filter. It is a flowchart of the mode selection process by Embodiment 1 of this invention. It is explanatory drawing of the mode selection process by a general method and a present Example. It is a flowchart of the mode selection process by Embodiment 2 of this invention.

Hereinafter, a moving picture coding apparatus and a moving picture coding method according to the present invention will be described with reference to illustrated embodiments.
At this time, in the present invention, for example, the above-described H.264. In a moving image encoding method such as H.264 / AVC, in order to suppress redundancy of a moving image and reduce a code amount, an encoding target image is predicted using image information for which encoding processing has been completed, and an original image The difference (referred to as prediction difference) is encoded.

An outline of the encoding process in the present invention at this time will be described.
First, here, a case will be described in which prediction is performed in units of blocks obtained by finely dividing an image in order to use the local properties of a moving image. That is, as shown in FIG. 5, prediction is performed on the target image 501 in units of macroblocks 502 composed of 16 × 16 pixels according to the raster scan order.

As shown in the figure, the prediction means is roughly divided into two types, inter prediction (inter-screen prediction) 503 and intra prediction (intra-screen prediction) 504.
At this time, as shown in the figure, this macro block is divided into smaller blocks and predicted.
At the time of encoding, in addition to the prediction means, which division pattern is to be used is determined for each macroblock, the combination of the prediction means and the division pattern is set to “encoding mode”, and the identifier is encoded as header information. .

This inter prediction is further divided into inter prediction P (Predictive) predicted from one reference image and inter prediction B (Bi-predictive) predictable from two reference images, as indicated by 505, and This inter prediction includes a skip mode in which no motion vector or prediction difference is encoded, and a direct mode.
In any mode, intra / inter determination can be performed. Therefore, these are not particularly distinguished here and are collectively referred to as inter prediction.

Next, FIG. 264 / AVC conceptually shows the operation of inter-screen prediction processing. As shown in the figure, when performing inter-screen prediction, an encoded image included in the same video 601 as the encoding target image 603 is shown. The decoded image is a reference image 602, and a block (predicted image) 605 having a high correlation with the target block 604 in the target image is searched from the reference image (referred to as motion search).
At this time, in addition to the prediction difference calculated as the difference between both blocks, the motion vector 606 expressed as the difference between the coordinate values of both blocks is also encoded as header information necessary for prediction.

Next, (a) in FIG. 264 / AVC conceptually shows intra-frame prediction (intra prediction) processing. In this intra-screen prediction, encoded blocks B, C adjacent to the left, upper left, upper, upper right of the encoding target block A are shown. , D and E are used for prediction.
That is, for the prediction, 13 decoded pixels 701 that are included in these blocks B to E and are in contact with the encoding target block A are used. At this time, all pixels on the same straight line with the prediction direction vector 702 as an inclination are predicted from the same reference pixel.

Here, H. In the case of H.264 / AVC, an optimal one can be selected in units of blocks from eight types of prediction directions such as vertical, horizontal, and diagonal.
At this time, in addition to the prediction mode based on the directionality as described above, DC prediction for predicting all the pixels included in the block to be encoded based on the average value of the reference pixels is also prepared as the prediction mode 2, and eight types of prediction are performed. It has been added to the direction.
Information indicating which mode is selected from the nine types of prediction modes 703 is encoded together with the prediction difference as header information.

  Incidentally, in the case of pseudo intra prediction, it becomes as shown in FIG. 7B, and in this case, the reference image and the target pixel are derived from the original image. This point is different from the case of intra prediction in FIG.

  Next, FIG. 8 is a graph conceptually showing important properties related to intra prediction and inter prediction. Here, intra / inter determination is performed by an ideal coding mode selection method capable of realizing high coding efficiency. In this case, the SAD value that is increased when the reverse determination (incorrect determination) is performed with respect to the SAD value in the case of SAD, and the horizontal axis indicates the Inter SAD 801 indicating the SAD at the time of inter prediction, On the vertical axis, the average value 802 of the increase in SAD when the determination is wrong is shown as Y.

First, the intra prediction characteristic 803 represents the SAD increase when the inter prediction is erroneously selected when the intra prediction is to be selected, and the inter prediction characteristic 804 is related to the case where the inter prediction is to be selected. The SAD increase when the intra prediction is erroneously selected is shown.
Therefore, it can be seen from the graph of FIG. 8 that each macroblock is divided into three categories according to the value of InterSAD given as an inter prediction parameter.

First, in the category 1 region 805 having a small Inter SAD 801, the SAD increase 802 is smaller in the intra prediction characteristics 803.
Therefore, it can be seen that, in the category 805, the intra prediction characteristic 803 is more advantageous in that the accuracy of the Inter prediction is higher.
Next, in the category 2 region 806 in which the InterSAD 801 is increased to some extent, one intra prediction characteristic 803 is increased, and the other inter prediction characteristic 804 is decreased and intersects in the middle. It is an area that cannot be generally said.

In the category 3 area 807 in which the Inter SAD 801 is considerably large, the SAD increase 802 is smaller in the inter prediction characteristic 804.
Therefore, it can be seen that the category 3 region 807 is more advantageous when the inter prediction characteristic 804 is selected because the accuracy of the inter prediction is higher.
Therefore, points where a significant difference appears between the intra prediction characteristics 803 and the inter prediction characteristics 804 on both sides of the point X where the intra prediction characteristics 803 and the inter prediction characteristics 804 intersect with each other are respectively selected as threshold values T1 and T1. Set to threshold T2.

In this case, the particularly important point here is that these threshold values T1 and T2 have extremely low dependence on the picture and QP (Quantization Parameter) of the target moving image, and are constant under any circumstances. It is that there is no problem even if treated as.
Here, the reason why the threshold value T1 and the threshold value T2 can be treated as constants will be conceptually described with reference to FIG.

First, as shown in FIG. 9 (a), a category 1 is generally classified as a macro that is often used for inter prediction when the object is stationary or moved in parallel. It is a block (901).
In this case, the InterSAD value range has a considerably large width depending on the QP value (902), but the InterSAD value is generally small in any band.

Next, as shown in FIG. 9B, in the category 2 (903), a macroblock whose inter prediction performance is deteriorated due to a change between screens of a complicated pattern such as when there is a three-dimensional movement or a lighting change. Are classified.
In this case as well, the width of the range of InterSAD is generated by QP (904), but the value of InterSAD is generally a large value.

On the other hand, as shown in FIG. 9 (c), in category 3 (905), occlusion, scene change, etc. occur and the target object does not exist in the reference image, and inter prediction is almost functioning. Macroblocks that are not to be classified are classified.
In this case, the value of InterSAD is somewhat different (906), but the value is no longer dependent on QP.

  From the results of the category classification as described above, due to the difference in prediction accuracy, the difference in SAD between categories is generally larger than the difference in SAD within the category, and the maximum value of Category 1 Inter SAD is the category. 2 is smaller than the minimum value of InterSAD, and at this time, the maximum value of InterSAD in category 2 is smaller than the minimum value of category 3 (907).

At this time, the Inter SAD is generally as follows.
SAD1 (QP: large) <SAD2 (QP: small)
And SAD2 (QP: large) <SAD3 (QP: minimum value)
Accordingly, in any of FIGS. 10 (a), (b), and (c), they can be distinguished from each other regardless of the value of QP. As a result, the boundary of the category is fixed under any circumstances. Therefore, it can be seen that the threshold values T1 and T2 can be treated as constants.

Here, FIG. 10 shows data when an actual video (Seq1: 1080i, 4: 2: 2, 10 bits) is encoded. FIG. 10 (a) shows QP = 12 in Seq1. In the case of FIG. 10B, FIG. 10B is also the case of QP = 27, and FIG. 10C is the case of QP = 42. Therefore, even if the QP is changed, the distribution of the graph is almost changed. I understand that I don't.
Next, FIG. 11 shows data when a completely different video (Seq2: 1080i, 4: 2: 2, 10 bits) is encoded. FIG. 11 (a) shows a QP in Seq1. FIG. 11B shows the case where QP = 27, and FIG. 11C shows the case where QP = 42.

In the case of FIG. 11, although the distribution of the graph is different from that in FIG. 10, the properties of each category are the same as in FIG. 10, and therefore the threshold values T1 and T2 for dividing the categories are also in the case of FIG. It turns out that the same value as is good.
However, when the dependency on the QP of the intra / inter determination result does not matter, such as when the rate control function is not used, it is not always necessary to fix the threshold T1 and the threshold T2. For example, the threshold is set according to the QP. If changed, a slight increase in coding efficiency has been confirmed.

As a result, when each macroblock is divided into three types of categories with respect to SAD for inter prediction, there is no inconvenience even if division is performed using threshold values T1 and T2 which are constants, and each category is advantageous. It can be seen that the prediction means to be can be selected significantly.
Therefore, a method for performing intra / inter determination based on these threshold values T1 and T2 will be described with reference to FIG.

First, when the target macroblock belongs to category 1, inter prediction is advantageous as shown in the figure. In this case, inter prediction is selected as “Case 1”.
Next, when it belongs to category 2, it cannot be generally said which prediction means is advantageous. In this case, more detailed intra / inter determination is performed as “Case 2”.
In addition, since intra prediction is advantageous when belonging to category 3 (case 3), in this case, intra prediction is selected as “case 3”.

By the way, in order to perform this intra prediction, it is necessary that encoding of neighboring macroblocks is completed and a decoded image that can be used as a reference pixel is acquired.
Here, first, the positional relationship between neighboring macroblocks referred to when performing intra prediction will be described with reference to FIG.
In the target image of FIG. 13, for example, when intra prediction is performed in the target block (1301), refer to the four macro blocks (1302) located on the left, upper left, upper, and upper right of the target block. become.

For this reason, if the encoding is not completed for these four macroblocks (1302), intra prediction cannot be performed.
Therefore, since the encoding at this time is performed according to the order of the raster scan, for example, when encoding is performed sequentially, the encoding for these neighboring macro blocks is surely completed when the target block is encoded. Therefore, intra prediction can be executed using decoded images of neighboring macroblocks.

  If both intra prediction and inter prediction are possible when the encoding mode is selected, generally, prediction processing is performed once in all candidate encoding modes, and the cost value is calculated by the following equations 1 and 2. Cost is calculated and the coding mode that gives the smallest cost value is selected.

(Equation 1)

(Equation 2)

Here, Dist is the prediction error, Rate is the code amount of the header accompanying the prediction, B is the original image of the target block, and B ′ is the predicted image of the target block.
At this time, weight is a coefficient value for adjusting the ratio of the prediction error and the code amount contributing to the cost value, and is statistically determined according to the type of the encoding mode and the value of the quantization parameter.
On the other hand, SAD is defined by the following (Equation 3).

(Equation 3)

However, p [i, j] indicates the pixel value of the coordinate (i, j) in the target block B, and q [i, j] indicates the pixel value of the coordinate (i, j) in the predicted image B ′ of the target block. ing.
As the pixel value at this time, only the value of the luminance component may be used, or a value may be combined with the value of the luminance component and the color difference component.
Here, it is said that the use of the prediction error function given by the following equation (4) instead of that given by the equation (2) is more effective.

(Equation 4)

  Here, SATD represents the sum of Hadamard Absolute Transformed Differences, and the Hadamard transform, which is a type of frequency transform method, is applied to the difference value between the original image and the predicted image of the target block. After that, the absolute value sum of each coefficient value is calculated and defined by the following equation (5).

(Equation 5)

Here, Tr represents a function for performing a Hadamard transform on the block, and Tr (B) [a, b] represents a transform coefficient component (a, b) after the Hadamard transform is performed on the target block B.
For the cost function at this time, any index other than the above may be used as long as it can reflect the similarity between the original image of the target block and the predicted image, such as sum of squared differences (SSD).

Thus, for example, in an encoder having a pipeline structure that performs parallel processing in units of macroblocks, the encoding of these peripheral macroblocks may not be completed at the time of mode selection of the target macroblock. Many.
In this case, intra prediction cannot be performed using decoded images of neighboring blocks, and prediction errors such as SAD cannot be calculated.

  Therefore, in this case, when the encoding mode is selected, a pseudo intra prediction is performed using the original image of the peripheral block instead of the decoded image of the peripheral block, and the mode is determined based on the SAD (pseudo SAD) at that time. A method of generating a prediction image again and encoding it in a newly determined mode after encoding of neighboring macroblocks is often used.

However, in this case, a difference occurs between the original image and the decoded image of the reference pixel due to the influence of the quantization error, and particularly at a low bit rate, the SAD and the pseudo SAD are separated from each other. There was a problem that the image quality deteriorated significantly.
On the other hand, in the case of inter prediction, since another image that has already been encoded is referred to, even if parallel processing is performed in units of macroblocks, it can be executed in any block in the target image and is always accurate. SAD can be calculated.

Under such circumstances, as described with reference to FIG. 12, by using the present invention that can make a determination based on the result of inter prediction that allows accurate SAD to be always calculated, Even if intra prediction cannot be performed, highly accurate intra / inter determination is possible.
In this case, the further determination method when the target macroblock is classified into category 2 is not particularly limited, but for example, the following method is effective.

  That is, first, a SAD (Inter SAD) which is a representative mode (for example, a mode having the smallest block size or a mode having the smallest SAD) in the inter prediction mode and a typical mode (for example, the intra prediction mode) If the absolute value of the difference in pseudo SAD (referred to as intrapseud SAD) of the smallest block size mode or the smallest SAD mode is less than a preset threshold value T3 of the desired width, it is clear due to the influence of the quantization error. Since it is difficult to perform intra / inter determination, generally, inter prediction that is more effective than intra prediction is selected.

Then, under the circumstances other than the above, assuming that the superiority and inferiority of intra prediction and inter prediction can be sufficiently determined, cost calculation is performed using the above equation (1) using InterSAD and Intra pseudSAD as prediction errors, The prediction means with the smaller cost value is selected.
At this time, if the value of the threshold value T3 is changed according to the value of the quantization parameter, a particularly high effect can be exhibited. However, even if a fixed value is used to satisfy the requirements such as rate control, there is a sufficient effect. .

Further, the cost may be calculated using SATD (pseudo-SATD) or SSD (pseudo-SSD) calculated using a pseudo-intra prediction image without using pseudo-SAD as a prediction error of intra prediction. The determination may be made by comparing prediction errors without performing calculation.
In addition, as the cost value of intra prediction at this time, for example, the variance value calculated using the original image of the target block or each pixel (original source) of the target block can be used without using the pseudo intra prediction error as described above. A result obtained by applying a differential filter to (image) may be used, or a value obtained by combining these may be used.

Here, when using a differential filter, it is effective to use peripheral pixels (original image) of the target block as shown in FIG. 14, for example.
As the differential filter at this time, it is effective to use, for example, the Sobel filter shown in FIG. 15 or the pre-witt filter shown in FIG. 16, and these filters are used to change the angle, for example, in the target block. It is preferable to calculate the strength of the edge and calculate the cost value based on the largest value.

Next, an embodiment of the present invention in which encoding is performed using the intra / inter determination method described above will be described as Embodiment 1.
FIG. 1 shows a moving image encoding apparatus 100 according to the first embodiment. An original image 101 is input to the moving image encoding device 100, and an encoded stream of the input original image 101 is generated.

  For this reason, the moving picture encoding apparatus 100 includes an input image memory 102 that holds the input original image 101, a block dividing unit 103 that divides the input image into small regions, and pseudo intra prediction in units of blocks. The intra-prediction prediction unit 105 that performs the inter-screen prediction unit 106 that performs inter-screen prediction based on the amount of motion detected by the motion search unit 104, and a prediction mode (prediction means and block size) that matches the nature of the image. And a mode selection unit 107 that determines the in-screen prediction according to the result of the mode selection unit 107.

  Further, a subtraction unit 109 for generating a prediction difference, a frequency conversion unit 110 and a quantization processing unit 111 that perform encoding on the prediction difference, and an adaptive encoding according to the occurrence probability of the code Variable length encoding unit 112, inverse quantization processing unit 113 and inverse frequency transform unit 114 for decoding the prediction difference once encoded, addition for generating a decoded image using the decoded prediction difference The unit 115 includes a reference image memory 116 for holding the decoded image and using it for subsequent prediction.

Next, the operation of the moving picture coding apparatus 100 will be described.
Here, it is assumed that the original image 101 is input to the moving image encoding apparatus 100.
Then, first, the input image memory 102 holds one image from the original image 101 as an encoding target image, and divides the image into fine blocks by the block dividing unit 103, and the motion search unit 104, pseudo screen To the intra prediction unit 105, the inter-screen prediction unit 106, the intra-screen prediction unit 108, and the subtraction unit 109.

Therefore, the motion search unit 104 calculates the motion amount of the corresponding block using the decoded image stored in the reference image memory 116, and passes the motion vector to the inter-screen prediction unit 106.
As a result, each of the pseudo intra-screen prediction unit 105 and the inter-screen prediction unit 106 performs a plurality of pseudo intra-screen prediction processes using the original images of neighboring blocks and an inter-screen prediction process referring to another encoded image. The mode selection unit 107 selects an optimal prediction mode.

First, when the mode selection result is the intra mode, when the encoding of the surrounding macroblock is completed, the in-screen prediction unit 108 performs accurate in-screen prediction using the decoded image of the surrounding block in the corresponding mode. The prediction image is sent to the subtraction unit 109 and the addition unit 115.
On the other hand, when the mode selection result is the inter mode, the predicted image of the corresponding mode already created in the inter-screen prediction unit 106 is sent to the subtraction unit 109 and the addition unit 115.

  Therefore, the subtraction unit 109 takes the difference (prediction difference) between the original image of the target block and the prediction image created by the mode selected by the mode selection unit 107, and the generated prediction difference is sent to the frequency conversion unit 110. As a result, in the frequency conversion unit 110 and the quantization processing unit 111, frequency conversion and quantization processing such as DCT are performed in units of blocks of a specified size with respect to the transmitted prediction difference, respectively. The data is passed to the variable length coding unit 112 and the inverse quantization unit 113.

  As a result, first, the variable-length encoding processing unit 112 performs encoding based on the probability of symbol generation for the quantized frequency transform coefficient and header information, and generates an encoded stream of the input original image 101 As a result, the original operation as the moving picture encoding apparatus 100 can be performed.

On the other hand, the inverse quantization processing unit 113 performs inverse frequency transformation processing such as inverse quantization processing and inverse DCT on the frequency transformation coefficient after quantization together with the inverse frequency transformation unit 114 to obtain a prediction difference. The data is sent to the adder 115.
As a result, the adding unit 115 adds the predicted image and the decoded prediction difference to generate a decoded image, and the decoded image is stored in the reference image memory 116.

Next, details of the mode selection unit 107 at this time will be described in detail with reference to FIG.
As shown in the figure, the mode selection unit 107 calculates the cost for the intra mode and selects an optimal encoding mode, and calculates the cost for the inter mode and the optimal encoding mode. And an intra / inter determination unit 203 that determines prediction means (intra prediction or inter prediction).

Then, the intra mode determination unit 201 calculates an intra pseudo SAD from both the original image sent from the block division unit 103 and the intra pseudo prediction image calculated by the intra-prediction screen prediction unit 105, and calculates a cost value. .
On the other hand, the inter mode determination unit 202 calculates an inter SAD from both the original image sent from the block division unit 103 and the inter prediction image calculated by the inter-screen prediction unit 106, and calculates a cost value.
Then, the intra / inter determination unit 203 selects a prediction unit for the target macroblock using the intra-pseudo SAD and the inter SAD in the representative mode using each prediction unit.

Next, the mode selection process at this time will be described with reference to the flowchart of FIG.
When the mode selection process is started, first, inter prediction between all modes is executed, cost calculation is performed for each mode (1701), and the one that gives the smallest SAD among the inter modes is inter representative. Mode, and the SAD is calculated and set to Inter SAD (1702).
Next, pseudo prediction is performed in all modes for the intra mode, cost calculation is performed (1703), and the mode in which the pseudo SAD is minimized is set as the intra representative mode, and the pseudo SAD is set as Intra SAD (1704).

Next, the currently calculated InterSAD is compared with the threshold value T1, and it is determined whether or not the value of the InterSAD is less than the threshold value T1 (1705).
First, the determination result is YES, that is,
InterSAD <T1
In this case, since it corresponds to “Case 1” in FIG. 12, that is, category 1, the inter prediction is selected as the prediction means, and the mode having the smallest cost value in the inter mode is set as the optimum encoding mode. Select (1706).

If the determination result is NO, the process proceeds to processing (1707). This time, InterSAD is compared with the threshold value T2, and it is determined whether or not the value of InterSAD is equal to or greater than the threshold value T2.
First, the determination result is YES, that is,
InterSAD ≧ T2
In this case, since it corresponds to “Case 3” in FIG. 12, that is, category 3, intra prediction is selected as the prediction means, and the mode having the smallest cost value in the intra mode is set as the optimum encoding mode. Select (1708).

Therefore, if the determination result is NO, it corresponds to “Case 2” of FIG. 12, that is, category 2, and therefore, more detailed intra / inter determination is performed, and therefore the process proceeds to determination processing (1709). Thus, it is determined whether or not the absolute value of the difference between Inter SAD and Intra SAD is less than a preset threshold value T3.
Here, as shown in FIG. 12, this threshold value T3 is arbitrarily set in advance around a point X at which the intra prediction characteristic 803 and the inter prediction characteristic 804 for the Inter SAD (801) intersect and become equal. This is the desired range.

First, the determination result is YES, that is,
｜ InterSAD-IntraSAD | <T3
In the case of, the mode with the smallest cost value is selected from the inter modes (1710). This is because, as described above, inter prediction is generally more effective than intra prediction.
On the other hand, if the determination result is NO, the cost value calculation using intra pseudo SAD and inter SAD is performed for all coding modes, and the mode with the smallest cost value is selected as the optimum coding mode. Yes (1711).

  As described above, when any one of the processing (1706), the processing (1708), the processing (1710), and the processing (1711) is completed, the mode selection processing ends (1712). When the mode selection process for one macroblock is completed, the generation of the encoded stream of the input original image 101 can be obtained under the selection of the optimal prediction mode. Will be performed.

By the way, in Embodiment 1 demonstrated above, as shown in FIG. 17, before the intra / inter determination (1705), prediction by all the modes is performed (1701)-(1703).
However, here, particularly when a software encoder or the like that performs parallel processing in units of macroblocks is targeted, only prediction of the inter representative mode is performed before intra / inter determination, and thereafter, processing (1705) to (1707) is performed. If a predicted image is generated in another mode according to a determination result such as), unnecessary prediction processing can be omitted, and the amount of calculation can be further reduced.

In this case, in order to determine the inter prediction mode, prediction is executed in the inter mode other than the representative mode in the processing (1706) and the processing (1710), while all processing is performed by the processing (1708) to determine the intra prediction mode. The pseudo prediction may be executed in the intra mode.
In the determination using the intra representative mode (1709), the pseudo prediction may be performed only in the intra representative mode.

Next, Embodiment 2 of the present invention will be described.
The above-described first embodiment considers application in a situation where intra prediction cannot be performed at the time of mode selection of the target macroblock because the encoding processing of neighboring macroblocks is incomplete due to parallel processing or the like. The second embodiment described below is intended for cases where encoding of neighboring macroblocks has already been completed at the time of mode selection, such as when encoding processing is performed sequentially, and intra prediction can always be performed correctly.
Therefore, the second embodiment is highly effective particularly when an encoder is realized by software.

Here, first, FIGS. 18 (a) and 18 (b) are diagrams respectively showing the flow of mode selection processing by a general encoder, for example, an encoder by a general method such as the above-described conventional technique, and the encoder according to the present invention. It is shown as an example.
First, in the case of a general encoder, as shown in FIG. 18 (a), prediction processing is executed once for all macroblocks in all encoding modes, and the above-described equation (1) is used. The cost is calculated by selecting the mode with the lowest cost.
However, in this case, as described above, a very large amount of calculation is required for the prediction process when the mode is selected.

On the other hand, in the case of the present invention, by first performing the intra / inter determination already described with reference to FIG. 12, it is possible to narrow down the mode and reduce the number of generations of predicted images, thereby reducing the processing amount.
That is, in the embodiment of the present invention, as shown in FIG. 18B, first, prediction is performed in an inter representative mode (for example, an inter mode having a block size of 8 × 8) (1802), and the SAD value at that time is used. Can narrow down candidates.
First, when the SAD value is smaller than the threshold value T1, a mode is selected from the inter modes (Case 1).

Therefore, in the present invention, the predicted image generation process using the inter mode can be omitted.
Next, when the SAD value is equal to or greater than the threshold value T2, a mode is selected from the intra modes (Case 3).
Therefore, in this embodiment, the predicted image generation process using the inter mode excluding the representative mode can be omitted.
When the SAD value is other than the above, for example, as in the general method, prediction processing is performed in all encoding modes, and a mode that minimizes the cost may be selected (case 2).

FIG. 3 shows a moving image encoding apparatus 300 according to Embodiment 2 of the present invention. When an original image 301 is input to this apparatus, an encoded stream 311 of the input original image 301 is generated. It is like that.
For this reason, the moving image encoding apparatus 300 includes an input image memory 302 that holds the input original image 301, a block dividing unit 303 that divides the input image into small regions, and a screen that performs intra-screen prediction in units of blocks. An inter-screen prediction unit 306 that performs inter-screen prediction on a block basis based on the motion amount detected by the inner prediction unit 305 and the motion search unit 304 is provided.

  Further, a mode selection unit 307 that determines a prediction mode (prediction means and block size) that matches the nature of the image, a subtraction unit 308 for generating a prediction difference, and a frequency conversion unit that encodes the prediction difference 309 and the quantization processing unit 310, a variable length coding unit 311 for performing adaptive coding according to the occurrence probability of the code, an inverse quantization processing unit 312 for decoding the prediction difference once coded, and the inverse A frequency conversion unit 313, an addition unit 314 for generating a decoded image using the decoded prediction difference, and a reference image memory 315 for holding the decoded image and using it for later prediction are provided. Yes.

Next, the operation of the moving picture coding apparatus 300 will be described.
Here, it is assumed that the original image 301 is input to the moving image encoding apparatus 300.
Then, first, the input image memory 302 holds one image from the original image 301 as an encoding target image, and divides the image into fine blocks by the block dividing unit 303, and the motion search unit 304, The data is passed to the prediction unit 305 and the inter-screen prediction unit 306.
Therefore, first, the motion search unit 304 calculates the motion amount of the corresponding block using the decoded image stored in the reference image memory 315, and passes the motion vector to the inter-screen prediction unit 306.

At this time, the intra-screen prediction unit 305 and the inter-screen prediction unit 306 each perform intra-screen prediction processing with reference to the encoded peripheral blocks and inter-screen prediction processing with reference to another encoded image with a plurality of block sizes. Execute.
Then, an optimal prediction mode is selected by the mode selection unit 307, and a prediction image created in the selected mode is sent to the subtraction unit 308 and the addition unit 314.
Therefore, the subtraction unit 308 generates a difference (prediction difference) between the original image of the target block and the prediction image created in the mode selected by the mode selection unit 307 and passes it to the frequency conversion unit 309.

The frequency conversion unit 309 and the quantization processing unit 310 perform frequency conversion and quantization processing such as DCT on a block basis having a designated size for the transmitted prediction difference, and the variable length encoding unit 311. To the inverse quantization unit 312.
Therefore, first, the variable length encoding processing unit 311 encodes the quantized frequency transform coefficient and header information based on the occurrence probability of the symbol, thereby generating an encoded stream 311.

In addition, the inverse quantization processing unit 312 and the inverse frequency transform unit 313 perform inverse frequency transform such as inverse quantization and inverse DCT on the quantized frequency transform coefficient, respectively, obtain a prediction difference, and add the addition unit 314. Send to.
Therefore, the adding unit 314 generates a decoded image by adding the predicted image and the decoded prediction difference, and stores the decoded image in the reference image memory 315.

Next, details of the mode selection unit 307 in the moving picture coding apparatus 300 will be described with reference to FIG.
The mode selection unit 307 includes an intra / inter determination unit 401 that determines a prediction unit, an intra mode determination unit 402 that calculates a cost for the intra mode and selects an optimal encoding mode, and a cost for the inter mode. An inter-mode determination unit 403 that calculates and selects an optimal encoding mode.

At this time, first, the intra / inter determination unit 401 calculates the SAD of the representative mode from the predicted image of the inter representative mode created by the inter-screen prediction unit 306 and the original image sent from the block dividing unit 303, Subsequently, in accordance with the SAD value, the intra-screen prediction unit 305 and the inter-screen prediction unit 306 generate a prediction image in an encoding mode necessary for the next determination.
Then, the prediction image of this encoding mode is sent to the intra mode determination unit 402 and the inter mode determination unit 403, and the final mode is determined.

Next, the procedure of mode selection processing by the mode selection unit 307 at this time will be described with reference to FIG.
When this mode selection process is started, first, inter-screen prediction is executed in the representative mode of the inter mode (1901), and the SAD in this mode is calculated to be Inter SAD (1902).
Next, it is checked whether or not the InterSAD value at this time is smaller than the threshold value T1 (1903).
When the result is YES, inter prediction is selected as a prediction means. First, after performing inter-screen prediction in an inter mode other than the representative mode (1904), the mode with the lowest cost is selected. (1905).

If the result of the determination process (1903) is NO, that is, if the value of InterSAD is greater than or equal to the threshold value T1, then it is checked whether or not the value of InterSAD at this time is greater than or equal to the threshold value T2 (1906).
When the result is YES, intra prediction is selected as the prediction means, intra prediction is executed in all intra modes (1907), and then the mode with the lowest cost is selected among them (1907). 1908).
Therefore, if the result of the determination process (1906) is NO, that is, if the value of InterSAD is not less than the threshold value T1 and less than the threshold value T2, prediction is executed in the inter mode other than the representative mode (1909), and then all intra The prediction is executed in the mode (1910), and then the mode having the smallest cost value among all the modes is selected (1911).

  As described above, when one of the processing (1905), processing (1908), and processing (1911) is completed, the mode selection processing ends (1912). Mode selection processing is completed, and the generation of the encoded stream of the input original image 301 can be obtained under the selection of the optimal prediction mode, and the moving image encoding apparatus 300 exhibits its original performance. Can be made.

Here, in the present invention, since only the intra / inter determination unit is targeted, the mode determination method in the intra prediction mode and the inter prediction mode is not particularly limited.
Accordingly, as in the above-described embodiment, prediction may be performed in all modes to perform cost calculation, or determination may be performed by another method, for example, considering the directionality of the edge.
At this time, it is more effective to save the calculation amount by narrowing down the mode by some method.

In the above embodiment, the mode with the smallest SAD is set as the representative mode. However, there are various ways to select the mode, such as the mode with the smallest block size and the combination of the mode with the largest block size and a plurality of modes. Yes, the selection method of the representative mode does not matter.
Similarly, in the above embodiment, macroblocks are classified into one of three types of patterns by SAD in inter-representation mode, but prediction is performed in inter-representation mode, such as SATD, SSD, motion vector, and header code amount. Any value may be used as long as it is a feature quantity that can be acquired by performing the above, and the number of patterns to be classified does not matter.

In the above-described embodiment, prediction and frequency conversion are performed in units of blocks. However, other than this, for example, calculation may be performed in units of objects separated from the background of the image.
Similarly, DCT is cited as an example for frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation), KLT ( Any orthogonal transformation used for removing correlation between pixels, such as Karhunen-Loeve Transformation, may be used.

At this time, in the intra mode, it is not necessary to perform intra prediction, and the original image may be directly subjected to frequency conversion as in the case of MPEG-1 or MPEG-2 intra coding. There is no need to make it special.
Further, the present invention relates to H.264. 261, MPEG-1, H.264. 262 / MPEG-2, MPEG-4, H.264. 263, H.M. The present invention is applicable not only to H.264 / AVC but also to any moving picture coding system such as a next generation standard that will be established in the future.

100 moving picture coding apparatus (moving picture coding apparatus according to Embodiment 1)
300 Moving Image Encoding Device (Moving Image Encoding Device According to Embodiment 2)

Claims (3)

In a moving image encoding apparatus that predicts an encoding target image and encodes a difference between the predicted encoding target image and an original image,
Inter-screen prediction means for performing prediction of the encoding target image with reference to an image different from the encoding target image;
In-screen prediction means for performing prediction of the encoding target image with reference to the same image as the encoding target image;
A mode selection unit for determining which of the inter-screen prediction unit and the intra-screen prediction unit to use a prediction result when encoding the difference;
The moving picture encoding apparatus, wherein the mode selection means performs the determination based on a threshold set as a parameter value at the time of inter-screen prediction. The moving image encoding device according to claim 1,
The mode selection means selects the inter-screen prediction means in a region where the prediction error of inter-screen prediction is small, and selects the intra-screen prediction means in a region where the prediction error of inter-screen prediction is large. Encoding device. In a moving image encoding method of a method of encoding a difference between a predicted encoding target image and an original image,
When encoding the difference between the predicted encoding target image and the original image,
The prediction of the encoding target image is performed with reference to an image different from the encoding target image, and the prediction of the encoding target image is performed with reference to the same image as the encoding target image. With the case
A moving picture encoding method, wherein selection is made based on a threshold value set as a parameter value at the time of inter-screen prediction.