JP2868160B2 - Video decoding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video decoding method for decoding a codeword obtained by a video coding method for orthogonally transforming a prediction error of motion compensation inter-frame prediction for each block.

[0002]

2. Description of the Related Art As a video encoding method, there is a method of orthogonally transforming a prediction error of motion compensation inter-frame prediction (MC prediction) (references: "Okubo, Hatori," Standardization of low bit rate video encoding method ", Proceedings of the National Meeting of the Institute of Image Electronics Engineers of Japan, pp.106-111, 1989 ”). FIG.
FIG. 3 is a diagram for explaining the video encoding method. In the figure, reference numeral 1401 denotes a difference calculation unit; 1402, an orthogonal transformation unit;
403 is a quantization unit, 1404 is an inverse quantization unit, 1405 is an orthogonal inverse transformation unit, 1406 is an addition operation unit, and 1407 is MC
A prediction unit 1408 is an encoding control unit.

In this video coding method, the prediction error of the motion compensation inter-frame prediction is divided into blocks and processed on a block basis. The MC prediction unit 1407 calculates a motion vector and an M for each block from the input image and the image of the previous frame.
A C prediction image is generated. The difference calculation unit 1401 obtains a difference between the MC predicted image and the input image, that is, a prediction error. The encoding control unit 1408 determines whether each block is a valid block or an invalid block based on the magnitude of the prediction error, and controls the encoding process as follows. Note that an effective block is a block whose prediction error has an effective value, and an invalid block is a block whose prediction error has no effective value.

[0004] For an effective block, a prediction error is orthogonally transformed by an orthogonal transformation unit 1402, an orthogonal transformation coefficient is quantized by a quantization unit 1403, and a quantization index is output together with a motion vector and a valid flag. For invalid blocks, no quantization index is output.
The motion vector and the invalid flag are output. For the effective block, the quantization index is equal to that of the inverse quantization unit 140.
4, the orthogonal transform coefficients are restored by orthogonal quantization, the orthogonal transform coefficients are orthogonally inverse transformed by the orthogonal inverse transform unit 1405 to restore the prediction error, and the MC prediction image and the prediction error are added by the addition operation unit 1406. Are added to generate a locally decoded image. For an invalid block, the MC prediction image is used as it is as a locally decoded image. The locally decoded image thus generated is referred to when encoding the next frame.

FIG. 15 is a diagram for explaining a video decoding method for a video coded by the video coding method. In the figure, 1501 is an inverse quantization unit, 1502 is an orthogonal inverse transformation unit, 1503 is an addition operation unit, and 1504 is MC
A prediction unit 1505 is a decoding control unit. For a block having a valid flag (valid block), the inverse quantization unit 1501 restores the orthogonal transform coefficient by inversely quantizing the quantization index.
In addition, the prediction error is restored by performing the orthogonal inverse transformation at 02, and the addition calculation unit 1503 adds the prediction error and the MC prediction image generated by the MC prediction unit 1504 to obtain a decoded image. For a block having an invalid flag (invalid block), the MC prediction image generated by the MC prediction unit 1504 is used as a decoded image as it is. The MC prediction image is M
The C prediction unit 1504 obtains the image at the relative position indicated by the motion vector by cutting out the image of the previous frame.

[0006]

In the video decoding method, orthogonal inverse transformation is always performed on all effective blocks. Therefore, the processing speed of the orthogonal inverse transform unit is
It had to be fast enough to avoid congestion. However, orthogonal inverse transformation is a complicated process consisting of a large number of operations, and a high-speed technique requires advanced technology.

Therefore, the number of blocks to which the orthogonal inverse transform is applied is reduced by utilizing information on the number of input blocks per unit time (the number of effective blocks input per unit time) and the screen display state. Techniques have been developed to prevent the orthogonal inverse transform unit from being congested even when the processing speed of the unit is low. Japanese Patent Application No. 2-40709 (Title of Invention: Image Decoding Method), Japanese Patent Application No. 2-46661 (Title of Invention: Image Decoding Method), Japanese Patent Application No. 2-216664.
The invention described in the specification of the issue (title of the invention: image decoding method) corresponds to this. In each of these inventions, the orthogonal inverse transform can be omitted by accumulating the inter-frame difference information in the form of orthogonal transform coefficients, thereby reducing the number of blocks to which the orthogonal inverse transform is applied. However, these inventions are based on the premise that a simple inter-frame preceding value prediction is used as an inter-frame prediction method, and when MC prediction is used as in the video encoding method, There was a problem that it was not supported.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to reduce the number of blocks to which orthogonal inverse transform is applied when MC prediction is used, and to provide an orthogonal inverse transform unit. It is an object of the present invention to provide a video decoding method in which the orthogonal inverse transform unit is not congested even if the processing speed is low.

[0009]

In order to achieve the above object, according to the present invention, an image of a previous frame is obtained by combining a base pixel value and an orthogonal transform coefficient of a difference between the pixel value and a true pixel value. Information is stored. However, depending on the processing applied to the previous frame, the image information of the previous frame may be expressed only by pixel values.

When an MC predicted image is generated by MC prediction, a block to be referred to in the previous frame is derived based on a motion vector, and an image that extends over a plurality of blocks in the previous frame is used as the MC predicted image. (See Fig. 16)
In, a predicted image is generated by MC prediction after the image information of the plurality of blocks is represented only by pixel values. That is, the orthogonal transform coefficient of the preceding frame image information of the plurality of blocks is subjected to the orthogonal inverse transform, the numerical value after the orthogonal inverse transform is added to the pixel value, and MC prediction is performed. In this case, the MC prediction image is represented only by pixel values. On the other hand, when the MC prediction image includes only the pixels of one block (see FIG. 17), the image information of the previous frame represented by a set of the pixel value and the orthogonal transformation coefficient is used as the MC prediction image.

For the processing on the orthogonal transform coefficient obtained by inversely quantizing the quantization index from the encoding side, one of the following two is selected by an external control signal. First, the orthogonal transform coefficient obtained by the inverse quantization is represented by M
This is a process of adding to the orthogonal transformation coefficient of the C prediction image and storing the image information of the current frame by a set of the orthogonal transformation coefficient after the addition operation and the pixel value of the MC prediction image. The other is to add the orthogonal transform coefficient obtained by inverse quantization to the orthogonal transform coefficient of the MC prediction image, perform orthogonal inverse transform on the orthogonal transform coefficient after the addition operation, and calculate the value of the orthogonal inverse transform by MC prediction. This is a process of adding the pixel value to the image pixel value and storing the pixel value after the addition operation as image information of the current frame. In this case, the image information of the current frame is represented only by pixel values. However, in any of the above processes, when the MC predicted image is represented only by pixel values, the operation of adding the orthogonal transform coefficient obtained by the inverse quantization to the orthogonal transform coefficient of the MC predicted image is omitted. Is done.

[0012]

According to the above-mentioned means, in the MC prediction, when an image extending over a plurality of blocks in the previous frame is an MC prediction image, the image information of the plurality of blocks referred to is represented by only pixel values. Therefore, in the MC prediction, the MC prediction image can be generated only by reading the pixel value of the previous frame at the relative position shifted by the motion vector.
Also, in the case where an MC prediction image is generated by referring to only one block of the previous frame in MC prediction, the pixel value of the previous frame of the block at the relative position shifted by the motion vector and the orthogonal transform coefficient are simply read. Can generate an MC prediction image.

The prediction error sent from the encoding side is:
There are cases where the process is added to the MC predicted image in the orthogonal transform coefficient space and cases where the process is added to the MC predicted image in the pixel value space, and which process is performed is switched by an external control signal. Therefore, it is possible to perform control so that the former process is performed when it is desired to reduce the number of applications of the orthogonal inverse transform, and that the latter process is performed when the number of applications of the orthogonal inverse transform does not need to be reduced. It is possible.

As described above, in the present invention, the number of orthogonal inverse transforms can be adjusted even when MC prediction is used.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be specifically described below with reference to the drawings. In the description of the present embodiment, the case of decoding the i-th frame is taken as an example. As shown in FIG. 12, one frame image is composed of a total of twelve blocks, three vertically and four horizontally, and each block is composed of eight vertically and horizontally as shown in FIG. An example is shown in which eight pixels have a total of 64 pixels. However, the present invention can be similarly performed when decoding a frame other than the i-th frame, and can be similarly performed when the block configuration of the frame and the pixel configuration of the block are different.

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of a video decoding method according to the present invention. In the figure, 101 is an image storage unit, 102 is a codeword storage unit, 103 is a previous frame pre-processing unit, 104 is an MC prediction unit, 105 is an effective block processing unit, and 106 is a decoding control unit.

In order to decode an image encoded using MC prediction, an image of the previous frame is required. The image storage unit 101 stores the previous frame (the (i-1) th frame)
The image and the processed current frame (i-th frame) image are stored. When the processing of the i-th frame is completed,
For the processing of the next (i + 1) th frame, the current frame image, that is, the i-th frame image is set as the previous frame image.

FIG. 2 shows the configuration of the image storage unit 101. In the figure, 201 is a pixel value memory of the previous frame, 202
Is an orthogonal transformation coefficient memory of the previous frame, 203 is a mode management table of the previous frame, 204 is a pixel value memory of the current frame, 205 is an orthogonal transformation coefficient memory of the current frame, and 2
06 is a mode management table of the current frame.

When processing the i-th frame, the image information of the (i-1) -th frame is stored in the pixel value memory 201 and the orthogonal transform coefficient memory 202. The pixel value memory 204 and the orthogonal transform coefficient memory 205 are units that store the processed image information of the i-th frame. The mode management table 203 and the mode management table 206 are tables in which storage modes of image information are described. Mode management
The table 203 stores the image information of the (i-1) th frame.
This is a table in which modes are described.
The file 206 stores the storage mode of the image information of the i-th frame.
This is the table described. Of the mode management table 203
An example is shown in FIG. In FIG. 3, the value 1 is written
The image information of the block that is
Combination of values and orthogonal transform coefficients of orthogonal transform coefficient memory 202
Stored in the orthogonal transform coefficient memory 202.
Data is valid. On the other hand, the block in which the value 0 is written
Image information is stored only in the pixel values of the pixel value memory 201.
The contents of the orthogonal transform coefficient memory 202 are invalid
(For example, in the case of FIG. 3, the image of the block (1, 1)
Information is stored as a combination of pixel values and orthogonal transform coefficients.
Therefore, the image information of the block (1, 2) is stored only with the pixel value.
Has been). Mode management in mode management table 206
The pixel value memory 204 is configured similarly to the table 203.
And the image of the i-th frame comprising the orthogonal transform coefficient memory 205
Expresses the storage mode of image information.

When the processing of the i-th frame is completed, the contents stored in the pixel value memory 204 are stored in the pixel value memory 2.
01, the storage contents of the orthogonal transformation coefficient memory 205 are copied to the orthogonal transformation coefficient memory 202, and the storage contents of the mode management table 206 are copied to the mode management table 203, respectively.

The pre-frame pre-processing unit 103 pre-processes the image information of the (i-1) -th frame stored in the image storage unit 101, and only reads out the stored values of the pixel value memory 201 and the orthogonal transform coefficient memory 202. To generate an MC prediction image. 4 and 5 are flowcharts of the processing in the previous frame preprocessing unit 103.

As shown in FIG. 4 and FIG. 5, the processing procedure in the previous frame pre-processing section 103 first clears the internal table to 0 (step 401). As shown in FIG. 6 (a diagram showing an example of the internal table of the previous frame pre-processing unit 103), the internal table has one column for each block. Next, based on the motion vector, it is checked for each block whether or not the MC predicted image extends over a plurality of blocks of the (i-1) th frame (step 402). Then, if the data spans a plurality of blocks (YES), the value 1 is written in the column of the plurality of blocks in the internal table (step 403). When each block of the i-th frame has a motion vector shown in FIG. 7 (a diagram showing an example of a motion vector of each block), for example, the block (1, 1) corresponds to this. In the case of the block (1, 1), the motion vector is (2, 1).
-1), the block (1,1), the block (1,2), the block (2,1), and the block (2, 2)
The (i-1) th frame image over the four blocks is M
It becomes a C prediction image, and blocks (1,
The value 1 is written in the columns of 1), block (1, 2), block (2, 1), and block (2, 2). On the other hand, when the MC prediction image is composed of only one block of pixels as in the block (3, 3) (NO), nothing is written in the internal table. If each block of the i-th frame has the motion vector shown in FIG. 7, these operations are performed on all blocks to create the internal table shown in FIG. 6 (step 404).

Next, the mode management table 203 is checked for the block whose internal table value is 1 (steps 405 and 406). If the value of the mode management table 203 is 1 (YES), the orthogonal transform coefficient memory 202
The above orthogonal transform coefficient is subjected to orthogonal inverse transform (step 407),
The value after the orthogonal inverse transformation and the value on the pixel value memory 201 are added, and the value after the addition is overwritten on the area of the block in the pixel value memory 201 (step 408), and the mode management table 203 is set to the value 0. (Step 409). If the value of the mode management table 203 is 0, nothing is performed. The operation is performed on all blocks whose value in the internal table is 1.

The codeword storage unit 102 stores codewords such as a quantization index and a motion vector until the preprocessing of the (i-1) th frame is completed.

The MC prediction section 104 generates an MC prediction image based on the motion vector. When the motion vector is (x,
If y), the i-th frame image at a position shifted by x pixels in the horizontal direction and y pixels in the vertical direction is set as the MC prediction image (see FIG. 8). FIG. 9 shows the flow of processing of MC prediction section 104.
This is shown in (flowchart of processing for one block).

The processing procedure of the MC prediction unit 104 is shown in FIG.
As shown in (1), first, it is checked whether or not the MC predicted image extends over a plurality of blocks of the (i-1) th frame based on the motion vector (step 901). If the MC prediction image is composed of only one block of pixels (NO) (in this embodiment, both x and y are divisible by 8), the mode management table 203 of the referred block is checked (step S1). 902). And
The value of the mode management table 203 is 1 (YE
In the case of S), a set of the pixel value of the block in the pixel value memory 201 and the orthogonal transform coefficient of the block in the orthogonal transform coefficient memory 202 is output as an MC prediction image (step 903). In the example of FIG. 7, the block (3, 3) corresponds to this case, and the block (3, 3) of the i-th frame has M
As the C prediction image, the pixel values and the orthogonal transform coefficients of the block (3, 4) of the (i-1) th frame are output as the MC prediction image. If the value of the mode management table 203 is 0, the pixel value of the block in the pixel value memory 201 is output as the MC prediction image (step 9).
04). On the other hand, when the MC prediction image extends over a plurality of blocks (YES) (x is not divisible by 8, or
When y is not divisible by 8, the i-1 frame image information of the plurality of blocks is already obtained only from the pixel value memory 201 by the operation of the previous frame preprocessing unit 103. Therefore, the value of the pixel value memory 201 at a position shifted by x pixels in the horizontal direction and y pixels in the vertical direction is read out and output as an MC prediction image (step 9).
05). The MC prediction image thus generated is passed to the effective block processing unit 105 in the case of an effective block (steps 906 and 907). On the other hand, in the case of an invalid block, it is stored in the pixel value memory 204 and the orthogonal transform coefficient memory 205 of the image storage unit 101 (steps 906, 9).
08). At this time, when the MC prediction image is represented by a set of a pixel value and an orthogonal transformation coefficient, the value 1 is written to the mode management table 206 (steps 909 and 91).
0). If the MC prediction image is represented only by pixel values, the value 0 is written to the mode management table 206 (90
9, 911).

The effective block processing unit 105 dequantizes the quantization index of the effective block to restore the orthogonal transform coefficient of the prediction error, and adds the prediction error to the MC prediction image. The processing flow of the effective block processing unit 105 is shown in FIG.
This is shown in FIG. 11 (flowchart of processing for one block).

The processing procedure of the effective block processing unit 105 is as follows.
As shown in FIG. 10, first, the quantization index of the effective block is inversely quantized to restore the orthogonal transform coefficient of the prediction error (step 1001). Then, the following processing is performed on the orthogonal transform coefficients.

(1) When an orthogonal inverse transform execution command is given from outside (YES in step 1002) (a) When an MC predicted image is represented by a set of pixel values and orthogonal transform coefficients (step 1003) YES) The orthogonal transformation coefficient is added to the orthogonal transformation coefficient of the MC prediction image (step 1004), and the numerical value after the addition is subjected to orthogonal inverse transformation (step 1005), and the converted numerical value is used as the pixel value of the MC prediction image. The values are added (step 1006), and the numerical value after the addition is stored in the pixel value memory 204 (step 100).
7). Then, the value 0 is written to the mode management table 206 (step 1008).

(B) When the MC predicted image is represented only by pixel values (NO in step 1003) The orthogonal transform coefficient of the prediction error is subjected to orthogonal inverse transform (step 1009), and the converted value is used as the MC predicted image. (Step 1)
010), the numerical value after the addition is stored in the pixel value memory 204 (step 1011). Then, the value 0 is written to the mode management table 206 (step 1012).

(2) When no orthogonal inverse transform execution command is given from outside (NO in step 1002) (a) When the MC prediction image is represented by a set of pixel values and orthogonal transform coefficients (step 1013) YES) The orthogonal transformation coefficient of the prediction error is added to the orthogonal transformation coefficient of the MC prediction image (step 1014), and the value after the addition is stored in the orthogonal transformation coefficient memory 205 (step 101).
5). Further, the pixel value of the MC prediction image is stored in the pixel value memory 204 as it is (step 1016), and the value 1 is written in the mode management table 206 (step 101).
7).

(B) When the MC predicted image is represented only by pixel values (NO in step 1013) The orthogonal transformation coefficient of the prediction error is stored in the orthogonal transformation coefficient memory 205 as it is (step 1018). Further, the pixel value of the MC prediction image is stored in the pixel value memory 204 as it is (step 10).
19), the value 1 is written to the mode management table 206 (step 1020).

The operation of this embodiment when decoding the i-th frame will be described along the following steps.

Step 1: The image information of the (i-1) th frame stored in the image storage unit 101 is preprocessed by the previous frame preprocessing unit 103. The preprocessing is performed based on the motion vector of each block of the i-th frame. From the pre-processed image information of the (i-1) th frame, an MC prediction image can be obtained only by reading out the stored values from the pixel value memory 201 and the orthogonal transform coefficient memory 202. During the preprocessing, the quantization index and the motion vector are buffered in the codeword storage unit 102.

Step 2: The quantization index and the motion vector buffered in the codeword storage unit 102 are extracted one block at a time (in the case of an invalid block, only the motion vector is extracted), and the following processing is performed. . First, the MC prediction unit 1 based on the motion vector
At 04, MC prediction is performed to generate an MC prediction image. For the effective block, the effective block processing unit 1 receives the quantization index and the MC prediction image as inputs.
At step 05, the i-th frame image is restored. On the other hand, in the case of an invalid block, the MC predicted image is used as it is as the i-th frame image. The i-th frame image is stored in the image storage unit 101. The above operation is repeated for all the blocks of the ith frame that are buffered.

Step 3: The pixel values of the pixel value memory 204 of the image storage unit 101 are read out for one frame in a predetermined order and output as a decoded image.

Step 4: In preparation for the processing of the (i + 1) th frame, the memory and the table are exchanged so that the current frame image information of the image storage unit 101 is treated as the previous frame image information.

As can be seen from the above description, according to the present embodiment, in MC prediction, when an image extending over a plurality of blocks of the previous frame is an MC prediction image, the image of a plurality of blocks referred to is Since the information is changed to a representation of only pixel values, in MC prediction, an MC prediction image can be generated only by reading the pixel value of the frame before the relative position shifted by the motion vector. Also, in the case where an MC prediction image is generated by referring to only one block of the previous frame in MC prediction, the pixel value of the previous frame of the block at the relative position shifted by the motion vector and the orthogonal transform coefficient are simply read. Can generate an MC prediction image.

The prediction error sent from the encoding side is:
There are cases where the process is added to the MC predicted image in the orthogonal transform coefficient space and cases where the process is added to the MC predicted image in the pixel value space, and which process is performed is switched by an external control signal. Therefore, it is possible to perform control so that the former process is performed when it is desired to reduce the number of applications of the orthogonal inverse transform, and that the latter process is performed when the number of applications of the orthogonal inverse transform does not need to be reduced. It is possible.

As described above, according to the present invention, the number of orthogonal inverse transforms can be adjusted even when MC prediction is used.

In the above embodiment, the decoding of one video has been described as an example. However, the present invention can also be used as a decoding method of a multiple video decoding apparatus for decoding a plurality of videos. . The present invention can be similarly applied to a video encoding method that does not output a quantization index of a prediction error for an invalid block.

As described above, the present invention has been specifically described based on the embodiments. However, it is needless to say that the present invention is not limited to the above-described embodiments, and various changes can be made without departing from the gist of the present invention.

[0043]

As described above, according to the present invention, even when MC prediction is used on the encoding side, the number of orthogonal inverse transforms can be controlled by an external control signal. . Thereby, when MC prediction is used, the number of blocks to which orthogonal inverse transform is applied is reduced, and even when the processing speed of the orthogonal inverse transform unit is low, the orthogonal inverse transform unit is not congested. Also, when decoding multiple videos ,
It is possible to control the execution of the inverse orthogonal transformation based on the display state of the image to Re monitoring Teso on a display,
Various applications are possible depending on the method of controlling the orthogonal inverse transform.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of a video decoding method according to the present invention.

FIG. 2 is a diagram for explaining an internal configuration of an image storage unit shown in FIG.

FIG. 3 is a view showing an example of a mode management table of an image storage unit shown in FIG. 2;

FIG. 4 is an exemplary view for explaining the flow of processing of a previous frame preprocessing unit according to the embodiment;

FIG. 5 is an exemplary view for explaining the flow of processing of a previous frame preprocessing unit according to the embodiment.

FIG. 6 is a diagram illustrating an example of an internal table of a previous frame preprocessing unit according to the embodiment.

FIG. 7 is a diagram illustrating an example of a motion vector of each block of the i-th frame according to the embodiment.

FIG. 8 is a diagram illustrating an example in which an MC prediction image according to the present embodiment extends over a plurality of blocks.

FIG. 9 is a flowchart of a process for one block of an MC prediction unit according to the embodiment.

FIG. 10 is a flowchart of processing for one block by an effective block processing unit according to the embodiment;

FIG. 11 is a flowchart of processing for one block by an effective block processing unit according to the embodiment;

FIG. 12 is a diagram illustrating a configuration of a frame according to the present embodiment.

FIG. 13 is a diagram showing a configuration of a block according to the present embodiment.

FIG. 14 is a block diagram for explaining a video encoding method which is a premise of the present invention.

FIG. 15 is a block diagram showing a configuration of a video decoding method according to the present invention.

FIG. 16 is a diagram for explaining a case where an MC predicted image extends over a plurality of blocks.

FIG. 17 is a diagram for explaining a case where an MC prediction image includes only one block of pixels.

[Explanation of symbols]

 Reference Signs List 101 image storage unit 102 codeword storage unit 103 previous frame preprocessing unit 104 MC prediction unit 105 effective block processing unit 106 decoding control unit 201 previous frame pixel value memory 202 previous frame orthogonal transform coefficient memory 203 previous frame mode management Table 204 Pixel value memory of current frame 205 Orthogonal transform coefficient memory of current frame 206 Mode management table of current frame

Claims (1)

(57) [Claims] When a video is encoded by a video encoding method for orthogonally transforming a prediction error of motion-compensated inter-frame prediction for each block, image information of a previous frame is converted into a pixel value serving as a base, Stored in combination with the orthogonal transformation coefficient of the difference between the pixel value and the true pixel value,
It is determined based on the motion vector received from the encoding side whether or not the motion-compensated inter-frame predicted image extends over a plurality of blocks of the previous frame. Pre-processing is performed so that the frame image information is represented only by pixel values, and then a motion-compensated inter-frame predicted image is generated. Is used as it is as the motion-compensated inter-frame predicted image, and the orthogonal transform coefficient of the prediction error received from the encoding side is added to the motion-compensated inter-frame predicted image by an addition operation on a pixel value space involving orthogonal inverse transform, Adding the orthogonal transform coefficient of the error to the motion-compensated inter-frame predicted image by an adding operation on an orthogonal transform coefficient space without orthogonal inverse transform. Video decoding method characterized by switching the control signal to reduce the inverse orthogonal transformation number to Luke.