JP2004040512A - Image encoding method and image decoding method - Google Patents

Abstract

An object of the present invention is to improve the coding efficiency of a direct mode of a coding system in which two arbitrary reference frames are selected from a plurality of reference frames and inter-frame prediction is performed by interpolation prediction.
A direct mode scaling coefficient table in which a plurality of direct mode scaling coefficients are stored, and a direct mode coefficient is selected and used according to a first relative index value of the direct mode, so that an encoding target frame is referred to. Since a direct mode motion vector can be generated in consideration of a frame display time difference, encoding efficiency in the direct mode can be improved.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method for encoding and decoding an image signal, and a recording medium on which a program for executing the method by software is recorded.
[0002]
[Prior art]
In recent years, with the development of multimedia applications, it has become common to handle information of all kinds of media such as images, sounds, and texts in a unified manner. At this time, by digitizing all media, media can be handled in a unified manner. However, since a digitized image has a huge amount of data, an image information compression technique is indispensable for storage and transmission. On the other hand, in order to interoperate compressed image data, standardization of compression technology is also important. Standards for image compression technology include H.264 of ITU-T (International Telecommunication Union Telecommunication Standardization Sector). 261, H .; H.263, MPEG (Moving Picture Experts Group) -1, MPEG-2, MPEG-4, etc. of ISO (International Organization for Standardization). In addition, the ITU is currently using H.264 as the latest image coding standard. H.264 is under standardization, and a draft draft in the standardization process is described in H.264. 26L.
[0003]
MPEG-1,2,4, H. As a technique common to moving picture coding methods such as H.263, there is inter-frame prediction with motion compensation. In the motion compensation of these video coding methods, a frame of an input image is divided into rectangles of a predetermined size (hereinafter, referred to as blocks), and a prediction pixel is generated from a motion vector indicating a motion between frames for each block. I do.
[0004]
Hereinafter, in order to describe the inter-frame prediction with motion compensation, (1) the concept of a B picture will be described first. Since a B picture is a picture that can include an interpolation prediction block in a picture, (2) interpolation The prediction will be described. Next, since a reference frame used for interpolation prediction or the like is uniquely identified, (3) a frame number and a relative index used for identification and a method of assigning these to a reference frame will be described. Next, (4) a short-term frame buffer and a long-term frame buffer will be described, and (5) a direct mode in which inter-frame prediction is performed by pixel interpolation will be described. Finally, (6) a conventional image encoding device and (7) a conventional image decoding device will be described.
[0005]
(1) The B picture will be described with reference to FIG. FIG. 16 is a conceptual diagram of a B picture. Frm indicates a B picture to be encoded, and Ref1, Ref2, and Ref3 indicate encoded frames that can be used as reference frames for inter-frame prediction. Block Blk1 is a block inter-frame predicted from reference blocks RefBlk1 and RefBlk2, and block Blk2 is a block inter-frame predicted from reference blocks RefBlk21 and RefBlk22.
[0006]
{Circle around (2)} FIG. 17 is an explanatory diagram of interpolation prediction. RefBlk1 and RefBlk2 indicate two reference blocks used for interpolation prediction, and PredBlk indicates a prediction block obtained by the interpolation processing. Here, the block size is described as 4 × 4 pixels. X1 (i) is a pixel value of RefBlk1, X2 (i) is a pixel value of RefBlk2, and P (i) is a pixel value of PredBlk. The pixel value P (i) can be obtained by the following linear prediction equation.
[0007]
P (i) = A.X1 (i) + B.X2 (i) + C
A, B, and C are linear prediction coefficients. As the linear prediction coefficient, only an average value (when A = 1/2, B = 1/2, and C = 0 in the above equation) may be used as in MPEG-1 and MPEG-2. In some cases, a linear prediction coefficient is set in advance, the value is stored in an image coded signal, and transmitted from the image coding device to the image decoding device.
[0008]
A block predicted between frames by pixel interpolation from a plurality of reference frames is referred to as an “interpolated prediction block”. A B picture is a picture that can include an interpolation prediction block in a picture. In a B picture, a different reference frame can be selected for each interpolation prediction block, such as a block Blk1 and a block Blk2 shown in FIG.
[0009]
A picture that does not include an interpolated prediction block and can include a block for performing inter-frame prediction from one reference frame in a picture is called a P-picture, and includes only intra-frame prediction blocks that do not perform inter-frame prediction. The picture to be processed is called an I picture.
[0010]
H. In 26L, a maximum of two reference frames are used for a block of a B picture. Therefore, to distinguish the two reference frames, each reference frame is referred to as a first reference frame and a second reference frame. The motion vectors for the first reference frame and the second reference frame are called a first motion vector and a second motion vector, respectively. Taking Blk1 in FIG. 16 as an example, the first reference frame is Ref1, the second reference frame is Ref3, the first motion vector is MV1, and the second motion vector is MV2. In addition, prediction from only the first reference frame is referred to as first reference frame prediction, and prediction from only the second reference frame is referred to as second reference frame prediction. Note that it is not necessary to distinguish between reference frames and motion vectors in blocks predicted between frames from one reference frame. However, in this document, a reference frame / motion vector of a block inter-frame predicted from one reference frame is referred to as a first reference frame and a first motion vector for convenience of description.
[0011]
(3) FIG. 18 is an explanatory diagram of a frame number and a relative index. The frame number / relative index is a number for uniquely identifying a reference frame stored in the multi-frame buffer MFrmBuf. H. In 26L, a value that increases by 1 for each frame stored in the memory as a reference image is assigned as a frame number. On the other hand, the relative index is in the block as shown in FIG. Then, the relative index is used to indicate a reference frame used for inter-frame prediction of this block.
[0012]
FIG. 19 is a conceptual diagram of an image encoded signal format by a conventional image encoding device. Picture is a coded signal for one frame, Header is a coded header signal included at the beginning of the frame, Block1 is a coded signal of a block in the direct mode, Block2 is a coded signal of a block by interpolation prediction other than the direct mode, RIdx1, RIdx2 indicates a relative index, and MV1 and MV2 indicate motion vectors. The interpolation prediction block Block2 has two relative indexes RIdx1 and RIdx2 in the coded signal in this order to indicate two reference frames used for interpolation. Have in this order. Whether to use the relative index RIdx1 or RIdx2 can be determined by PredType. The relative index RIdx1 indicating the first reference frame is referred to as a first relative index, and the relative index RIdx2 indicating the second reference frame is referred to as a second relative index. The first reference frame and the second reference frame are determined by a data position in the coded stream.
[0013]
Hereinafter, a method of assigning the first relative index and the second relative index will be described with reference to FIG.
[0014]
As the value of the first relative index, first, a value starting from 0 is assigned to a reference frame having a display time earlier than the encoding target frame in the order from the closest to the encoding target frame. If a value starting from 0 is assigned to all reference frames having a display time earlier than the encoding target, then the reference frames having a display time later than the encoding target frame are assigned in order from the closest to the encoding target frame. Subsequent values are assigned.
[0015]
When the first relative index Ridx1 is 0 and the second relative index Ridx2 is 1 in FIG. 21A, as shown in FIG. 20, the first reference frame is the B picture of the frame number 14, and the second reference frame is This is a B picture of frame number 13. As the value of the second relative index, first, a value starting from 0 is assigned to a reference frame having a display time later than the encoding target frame, in order from the closest to the encoding target frame. If a value starting from 0 is assigned to all reference frames having a display time later than the encoding target, the reference frame having a display time earlier than the encoding target frame is assigned a value starting from the order closest to the encoding target frame. Subsequent values are assigned.
[0016]
The relative index in the block is represented by a variable-length codeword, and the smaller the value, the shorter the code length code is assigned. Normally, the frame closest to the encoding target frame is selected as a reference frame for inter-frame prediction. Therefore, if relative index values are assigned in the order closer to the encoding target frame as described above, encoding efficiency will increase.
[0017]
(4) FIG. 20 is a conceptual diagram of the short-term frame buffer and the long-term frame buffer. In MPEG-1 and MPEG-2, the multi-frame buffer MFrmBuf is a FIFO (First-In First-Out) format buffer in which if there is no free space in the buffer, the reference frame stored first in the buffer is discarded and a new reference frame is stored. As default. Here, consider a scene in which an object crosses in front of a stationary background. In this case, the background is once hidden by the foreground object and appears again on the screen. If the background frame coded before hiding in the object can be used as a reference frame when the background hidden in the object first appears, the coding efficiency is improved. However, the amount of memory required to hold the image data of the reference frame is enormous, and considering the practical amount of memory, the maximum number of frames that can be stored in the multi-frame buffer MFrmBuf is about several. Therefore, in the case of the above scene, when the background appears on the screen again, the background frame is already discarded from the multi-frame buffer MFrmBuf and cannot be used for prediction, and efficient inter-frame prediction cannot be performed. Often.
[0018]
Then, H. In the 26L, a buffer called a long-term frame buffer LTMem that can hold a specific frame for a long time is defined as a multi-frame buffer MFrmBuf, in addition to a FIFO-type buffer. As shown in FIG. In 26L, the multi-frame buffer MFrmBuf is virtually divided into a short-term frame buffer STMem and a long-term frame buffer LTMem. The conventional FIFO type buffer is called a short-term frame buffer STMem to distinguish it from a long-term frame buffer. The reference frame once stored in the long-term frame buffer LTMem is not discarded from the long-term frame buffer LTMem without a control instruction for the buffer. In the case of the scene described above, by storing the background frame in the long-term frame buffer LTMem, the frame can be used for prediction when the hidden background appears again. With this mechanism, the coding efficiency can be greatly increased in the case of the scene described above. In order to move the reference frame between the long-term frame buffer LTMem and the short-term frame buffer STemem, or to discard the reference frame from the long-term frame buffer LTMem, a control instruction is issued to the buffer. When performing control such as moving or discarding the reference frame on the long-term frame buffer LTMem, the image coding apparatus stores a buffer control signal RPSL indicating the control content in the coded signal as shown in FIG. . The image decoding device can perform the same control as that of the image encoding device on the long-term frame buffer LTMem of the multi-frame buffer MFrmBuf by using the buffer control signal RPSL in the encoded signal.
[0019]
In the example of FIG. 20, as shown in FIG. 19, the reference frames Ref1, Ref2, and Ref3 are stored in the short-term frame buffer STMem, and the reference frames LTRef1 and LTRef2 are stored in the long-term frame buffer LTMem. The relative index value is also assigned to the reference frame of the long-term frame buffer LTMem. Therefore, similarly to the case of the reference frame of the short-term frame buffer STMem, the frame of the long-term frame buffer LTMem selected as the reference frame by the motion compensation of the block can be indicated by the value of the relative index in the image coded signal.
[0020]
(5) FIG. 21 is an explanatory diagram of the direct mode of the conventional image encoding device. The direct mode is a mode in which a reference frame / motion vector for a current block is determined from the motion vector / reference frame used in encoding the reference frame by a method described below, and inter-frame prediction is performed by pixel interpolation. is there. Frm indicates a B picture to be encoded, and Ref1, Ref2, Ref3, and Ref4 indicate encoded frames in the multi-frame buffer MFrmBuf. RIdx1 indicates a first relative index, RIdx2 indicates a second relative index, MV01 indicates a first motion vector, and MV02 indicates a second motion vector.
[0021]
Block Blk0 is a block to be encoded in the direct mode, block Blk00 is a block located at the same position as encoding target block Blk0 in reference frame Ref3, RefBlk01 is a reference block included in reference frame Ref1, and RefBlk02 is included in reference frame Ref3. Indicates a reference block to be used.
[0022]
The motion vector MV0 refers to the frame Ref1 with the first motion vector when the block Blk00 is encoded. The first motion vector MV01 and the second motion vector MV02 used for prediction of the current block Blk0 are calculated by the following equations.
[0023]
MV01 = N1 × MV0 / D, MV02 = N2 × MV0 / D
In the above equation, N1, N2, and D are values used when calculating a direct mode motion vector, and in this detailed description, a set of the values of N1, N2, and D is referred to as a direct mode scaling coefficient. In the case of FIG. 21, the scaling coefficients for the direct mode may be N1 = 2, N2 = −1, and D = 3. The motion vector MV0 is called a scaling motion vector. Assuming that the motion of the object including the encoding target block in the screen is constant, the first motion vector MV01 and the second motion vector MV02 are represented by the display time difference between the frame Frm and the first reference frame Ref1, and The motion vector MV0 can be obtained by internally dividing the motion vector MV0 based on Frm and the display time difference between the second reference frame Ref2.
[0024]
In the direct mode, the direct mode scaling coefficient transmitted for each frame is used in common for all blocks including the frame.
[0025]
If FIG. 21 corresponds to FIG. 18, Frm in FIG. 21 corresponds to the B picture (dotted picture) in the center of FIG. 18, Ref3 in FIG. 21 corresponds to the picture of frame number 13 in FIG. Ref1 corresponds to the picture of frame number 12 in FIG. 18, and Ref2 of FIG. 21 corresponds to the B picture of frame number 11 in FIG. Non-reference frames indicated by dotted lines other than Frm in FIG. 21 are not stored in the multi-frame buffer MfrmBuf because they are not referenced from other pictures. Therefore, unlike the picture shown in FIG. 18, a relative index for referring to the frame is not assigned.
[0026]
{Circle around (6)} FIG. 15 is a block diagram showing a configuration of a conventional image encoding device. Hereinafter, the image encoding device will be described.
It is assumed that the image coding apparatus receives the image signal Img divided into blocks and performs processing for each block. The subtractor Sub subtracts the predicted image signal Pred from the image signal Img and outputs the result as a residual signal Res. The image encoding unit ImgEnc receives the residual signal Res, performs image encoding processing such as DCT transform / quantization, and outputs a residual encoded signal ERes including quantized DCT coefficients and the like. The image decoding means ImgDec receives the residual coded signal ERes, performs image decoding processing such as inverse quantization / inverse DCT transform, and outputs a residual decoded signal DRes. The adder Add adds the residual decoded signal DRes and the predicted image signal Pred, and outputs the result as a reconstructed image signal Recon. In the reconstructed image signal Recon, a signal that may be referred to in subsequent inter-frame prediction is stored in the multi-frame buffer MFrmBuf. Since the memory capacity of the multi-frame buffer MFrmBuf is finite, data of frames not used for subsequent inter-frame prediction in the multi-frame buffer MFrmBuf can be removed from the multi-frame buffer MFrmBuf.
[0027]
The motion estimating means ME inputs the reference frame RefFrms stored in the multi-frame buffer MFrmBuf, performs motion estimation, and performs a predetermined prediction among predictions by intra prediction, first reference frame prediction, second reference frame prediction, and interpolation prediction. An optimal prediction type is selected according to the method (the prediction type that can be selected differs depending on the picture type), and the first motion vector eMV1, the second motion vector eMV2, the first relative index eRIdx1, and the second relative index eRIdx2 for the block are output.
[0028]
As a method of selecting a prediction type in the motion estimation means ME, for example, there is a method of selecting a prediction type that minimizes a prediction error of each prediction type. If the selected prediction type is intra prediction, the motion vector and the relative index are not output. If the selected reference type is the first reference frame prediction, only the first relative index and the first motion vector are output, and the second reference frame is output. In the case of frame prediction, only the second relative index and the second motion vector are output, and in the case of interpolation prediction, the first and second relative indexes and the first and second motion vectors are output.
[0029]
As indicated above, H. In 26L, a reference frame whose second relative index rRIdx2 is 0 is used as a second reference frame in the direct mode. Therefore, the second relative index rRIdx2 of the value 0 is input to the motion vector / frame number buffer MVFNBuf and the direct mode vector / relative index generating means GenMVRIdx.
[0030]
The motion vector / frame number buffer MVFNBuf stores a scaling vector rMV and a frame number indicating a frame referred to by the scaling vector rMV. Since the reference frame including the scaling vector rMV is the reference frame indicated by the second relative index rRidx2, the second relative index rRIdx2 having a value of 0 is input, and the reference frame indicating the scaling vector rMV and the reference frame of the scaling vector rMV is input. 1 Relative index rRIdx1 is output.
[0031]
The direct mode vector / relative index generation unit GenMVRIdx receives the direct mode scaling coefficient SP, the scaling vector rMV, and the first relative index rRIdx1, and performs the first mode operation in the direct mode by the direct mode processing described above. A vector sMV1, a second motion vector sMV2, a first relative index sRIdx1, and a second relative index sRIdx2 are output.
[0032]
The prediction type selection means PredSel is used for prediction of the image signal Img, the reference frame RefFrms, the relative indexes sRIdx1, sRIdx2 and the motion vectors sMV1, sMV2 indicating the position of the reference block in the “direct mode”, and “other than the direct mode”. Relative indexes eRIdx1, eRIdx2 and motion vectors eMV1, eMV2 indicating the position of the reference block to be input. Then, it is determined whether the direct mode should be used for the block prediction, and the determined prediction type is output to the variable length coding unit VLC0 as the prediction type PredType.
[0033]
The selection of the prediction type in the prediction type selection unit PredSel is performed, for example, by selecting a smaller prediction error between the prediction error in the “direct mode” and the prediction error in the “non-direct mode prediction” for the input pixel. Do.
[0034]
Therefore, the direct mode is added to the prediction type PredType in addition to the intra prediction, the first reference frame prediction, the second reference frame prediction, and the interpolation prediction other than the direct mode selected by the motion estimation unit ME.
[0035]
When the prediction type PredType indicates the direct mode, the switch SW12 switches to the “1” side, and the relative indexes sRIdx1 and sRIdx2 and the motion vectors sMV1 and sMV2 are used as the relative indexes RIdx1, RIdx2 and the motion vectors MV1 and MV2. You.
[0036]
On the other hand, when the prediction type PredType indicates a mode other than the direct mode, the switch SW12 is switched to the “0” side, and the relative indexes eRIdx1 and eRIdx2 and the motion vectors eMV1 and eMV2 are used as the relative indexes RIdx1, RIdx2 and the motion vectors MV1 and MV2. Is done.
[0037]
Further, in the direct mode, the first motion vector sMV1 when the block of the coded frame is coded is used as the scaling vector. Then, the frame referred to by the first motion vector sMV1 is used as one reference frame in the direct mode.
[0038]
Accordingly, in the encoded first relative index RIdx1 and the first motion vector MV1, the first relative index RIdx1 and the first motion vector MV1 that may be used in the direct mode in the frames after the encoded frame are: It is stored in the motion vector / frame number buffer MVFNBuf.
[0039]
After determining the prediction type PredType, the first relative index RIdx1 and the first motion vector MV1 are input to the multi-frame buffer MFrmBuf, and the reference block RefBlk1 corresponding to the input first relative index RIdx1 and the first motion vector MV1 is generated. Output to the pixel interpolation means Pol. When two reference blocks are required depending on the prediction type, a reference block RefBlk2 corresponding to the second relative index RIdx2 and the second motion vector MV2 is further output from the multi-frame buffer MFrmBuf to the pixel interpolation means Pol.
[0040]
At the time of interpolation prediction, the pixel interpolation means Pol interpolates pixel values at positions corresponding to the two reference blocks RefBlk1 and RefBlk2, and outputs an interpolation block RefPol.
[0041]
When the prediction type PredType indicates interpolation prediction, the switch SW11 is switched to "1", and the interpolation block RefPol is used as the prediction image signal Pred.
[0042]
At the time of the first reference frame prediction, the multi-frame buffer MFrmBuf outputs a reference block RefBlk corresponding to the first relative index RIdx1 and the first motion vector MV1.
[0043]
At the time of the second reference frame prediction, the multi-frame buffer MFrmBuf outputs a reference block RefBlk corresponding to the second relative index RIdx2 and the second motion vector MV2.
[0044]
Although not shown, at the time of intra prediction, a block RefBlk composed of pixels of the intra prediction result is output from the multi-frame buffer MFrmBuf.
[0045]
When the prediction type PredType indicates a prediction method other than the interpolation prediction, the switch SW11 is switched to “1”, and the reference block RefBlk is used as the prediction image signal Pred.
[0046]
The variable length coding unit VLC0 performs variable length coding on the residual coded signal ERes, the relative indexes RIdx1 and RIdx2, the motion vectors MV1 and MV2, the scaling coefficient SP for direct mode, and the prediction type PredType, and outputs the coded signal BitStrm0. .
[0047]
In the conceptual diagram of the image coded signal format of the conventional image coding apparatus of FIG. 21, Block1 is an example of a block coded in the direct mode. In this block, the relative index and the motion vector information are in the coded signal. I do not have. On the other hand, Block2 is an example of a block coded by interpolation prediction other than the direct mode. In this block, relative indexes RIdx1 and RIdx2 and motion vectors MV1 and MV2 can be included in the coded image signal.
[0048]
{Circle around (7)} FIG. 23 is a block diagram showing a configuration of a conventional image decoding device. Units performing the same operations and signals having the same operations as those in the block diagram showing the configuration of the conventional image encoding device in FIG.
[0049]
The variable-length decoding means VLD0 receives the coded image signal BitStrm0, performs variable-length decoding, and calculates the residual coded signal ERes, the motion vectors MV1, MV2, relative indexes RIdx1, RIdx2, the scaling coefficient SP for direct mode, and the prediction type PredType. Output.
[0050]
The image decoding means ImgDec receives the residual coded signal ERes, performs image decoding processing such as inverse quantization / inverse DCT transform, and outputs a residual decoded signal DRes.
[0051]
The adder Add adds the residual decoded signal DRes and the predicted image signal Pred, and outputs the result as a decoded image signal DImg outside the image decoding device.
[0052]
The multi-frame buffer MFrmBuf stores the decoded image signal DImg in the buffer for inter-frame prediction.
[0053]
The motion vector / frame number buffer MVFNBuf stores a scaling vector rMV and information rRIdx1 for identifying a frame referred to by the scaling vector. The motion vector / frame number buffer MVFNBuf receives the first relative index rRIdx2 having a value of 0, and outputs a scaling vector rMV and a first relative index rRIdx1 for identifying a frame referred to by the scaling vector.
[0054]
The direct mode vector / relative index generation unit GenMVRIdx performs the same processing as the direct mode vector / relative index generation unit GenMVRIdx in FIG.
[0055]
When the prediction type PredType indicates the direct mode, the switch SW22 switches to “1”. Then, the relative indexes sRIdx1 and sRIdx2 and the motion vectors sMV1 and sMV2 are used as the relative indexes nRIdx1 and nRIdx2 and the motion vectors nMV1 and nMV2.
[0056]
When the prediction type PredType indicates a mode other than the direct mode, the switch SW22 switches to the “0” side. Then, the relative indexes RIdx1 and RIdx2 and the motion vectors MV1 and MV2 are used as the relative indexes nRIdx1 and nRIdx2 and the motion vectors nMV1 and nMV2.
[0057]
At the time of interpolation prediction, the multi-frame buffer MFrmBuf outputs a reference block RefBlk1 corresponding to the first relative index nRIdx1 and the first motion vector nMV1, and RefBlk2 corresponding to the second relative index nRIdx2 and the second motion vector nMV2. Then, the pixel interpolation means Pol outputs the pixel values corresponding to the two reference blocks RefBlk1 and RefBlk2 as the interpolation block RefPol.
[0058]
At the time of the first reference frame prediction, the multi-frame buffer MFrmBuf outputs the reference block RefBlk corresponding to the first relative index nRIdx1 and the second motion vector nMV1.
[0059]
At the time of the second reference frame prediction, the multi-frame buffer MFrmBuf outputs the reference block RefBlk corresponding to the second relative index nRIdx2 and the second motion vector nMV2.
[0060]
Although not shown, at the time of intra prediction, a block RefBlk composed of pixels of the intra prediction result is output from the multi-frame buffer MFrmBuf.
[0061]
When the prediction type PredType indicates the interpolation prediction, the switch SW21 switches to the “0” side, and the interpolation block RefPol is used as the prediction image signal Pred.
[0062]
When the prediction type PredType indicates a prediction method other than the interpolation prediction, the switch SW21 is switched to “1”, and the reference block RefBlk is used as the predicted image signal Pred.
[0063]
Among the decoded first relative index nRIdx1 and the first motion vector nMV1, the first relative index nRIdx1 and the first motion vector nMV1 that may be used in the direct mode in frames after the decoded frame are motion vector frames. It is stored in the number buffer MVFNBuf.
[0064]
As described above, the image decoding device decodes the coded image signal BitStrm0 and outputs the decoded image signal as a decoded image signal DImg.
[0065]
[Problems to be solved by the invention]
Assuming that the motion of the object including the encoding target block in the screen is constant, the display time difference between the encoding target frame and the first reference frame, and the display time difference between the encoding target frame and the second reference frame. The scaling factor for direct mode can be determined by the ratio. In this case, the scaling coefficient is (N1, N2, D) = (2, −1, 3) in the block Blk0 in FIG. 20, and the scaling coefficient is (N1, N2, D) = (5, −) in the block Blk1. 1, 6).
[0066]
However, H. et al. In 26L, only one direct mode scaling coefficient can be transmitted per frame. Therefore, the same direct mode scaling factor must be used for all direct mode blocks in a frame. As a result, in the case of FIG. 20, for example, when (N1, N2, D) = (2, -1, 3) is used as a common scaling coefficient for the frame Frm, an appropriate direct mode motion vector is used for the block Blk0. Can be generated, but an appropriate direct mode motion vector cannot be generated in the block Blk1, and the coding efficiency is degraded.
[0067]
Therefore, an object of the present invention is to provide means for selecting a scaling coefficient for a block in a direct mode according to a reference frame to be used, and to improve coding efficiency in the direct mode.
[0068]
[Means for Solving the Problems]
The first invention is
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on an encoding target frame by motion compensation from a plurality of encoded frames stored in the multi-frame buffer, A first step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the encoded frame;
A second step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A third step of encoding the prediction error and outputting an image encoded signal including the encoded signal of the prediction error;
In the second step of the image encoding method having
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the encoding target frame is set as the first reference frame,
A scaling coefficient table storing at least one scaling coefficient associated with a value of the first relative index; selecting a scaling coefficient corresponding to the first relative index from the scaling coefficient table; Calculating a motion vector for the first reference frame and a second reference frame from the coefficients;
Generating a predicted image by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame;
In the third step,
By including and including the scaling coefficient table in the image coded signal,
Since the scaling coefficient can be selected from the direct mode scaling coefficient table for each first reference frame, and the direct mode motion vector can be generated in consideration of the display time difference between the encoding target frame and the reference frame, the coding efficiency of the direct mode can be improved. Can be improved.
[0069]
The second invention is
A first step of inputting an image encoded signal including an encoded signal of a prediction error,
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on a decoding target frame by motion compensation from a plurality of decoded frames stored in the multi-frame buffer, A second step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the decoded frame;
A third step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A fourth step of generating a decoded image of a frame from the predicted image and the decoded prediction error,
A fifth step of storing a decoded image of a frame that may be used for inter-frame prediction in a multi-frame buffer;
In the first step of the image decoding method having
Decode the scaling coefficient table in the image encoded signal,
In the second step,
Selecting, as the second reference frame, a reference frame having the smallest second relative index in the display order of the decoded frames after the decoding target frame;
In the third step,
In the second reference frame, a frame referred to by a motion vector used for motion compensation of a block at the same position as a predetermined block on the decoding target frame is set as the first reference frame, and the first relative index Is selected from the scaling coefficient table, and a motion vector for the first reference frame and a second reference frame are calculated from the motion vector and the scaling coefficient. By generating a predicted image by pixel interpolation from a block obtained from a motion vector and a block obtained from a motion vector for the second reference frame,
Since the scaling coefficient can be selected from the direct mode scaling coefficient table for each first reference frame, and the direct mode motion vector can be generated in consideration of the display time difference between the encoding target frame and the reference frame, the coding efficiency of the direct mode can be improved. Can be improved.
[0070]
The third invention is
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on an encoding target frame by motion compensation from a plurality of encoded frames stored in the multi-frame buffer, A first step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the encoded frame;
A second step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
Encoding the prediction error, the third step of outputting an image encoding signal including an encoded signal of the prediction error, in the second step in the image encoding method,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the encoding target frame is set as the first reference frame,
A scaling coefficient associated with the value of the first relative index, and a scaling coefficient storing one or more prediction method types indicating pixel interpolation, prediction from the first reference frame, or prediction from the second reference frame. It has a forecast method table,
A scaling coefficient and a prediction method type corresponding to the first relative index are selected from the scaling coefficient / prediction method table, and a motion vector for the first reference frame and the second reference frame are selected from the motion vector and the scaling coefficient. Calculated to
When the prediction method type is pixel interpolation, a prediction image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame, and prediction is performed. When the method type is prediction from a first reference frame, a block obtained from a motion vector for the first reference frame is used as a prediction image, and when the prediction method type is prediction from a second reference frame, , A block obtained from a motion vector for the second reference frame as a prediction image,
In the third step,
By including the scaling coefficient / prediction method table in the image coded signal and outputting it,
Since not only the interpolation prediction but also the direct mode using only one reference frame can be used, the coding efficiency can be improved.
[0071]
The fourth invention is
A first step of inputting an image encoded signal including an encoded signal of a prediction error,
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on a decoding target frame by motion compensation from a plurality of decoded frames stored in the multi-frame buffer, A second step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the decoded frame;
A third step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A fourth step of generating a decoded image of a frame from the predicted image and the decoded prediction error,
A fifth step of storing a decoded image of a frame that may be used for inter-frame prediction in a multi-frame buffer;
In the first step of the image decoding method having
Decode the scaling coefficient / prediction method table in the image encoded signal,
In the second step,
Selecting, as the second reference frame, a reference frame having the smallest second relative index in the display order of the decoded frames after the decoding target frame;
In the third step,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the decoding target frame is set as the first reference frame,
A scaling coefficient and a prediction method type corresponding to the first relative index are selected from the scaling coefficient / prediction method table, and a motion vector for the first reference frame and the second reference frame are determined from the motion vector and the scaling coefficient. Calculated to
When the prediction method type is pixel interpolation, a prediction image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame, and prediction is performed. When the method type is prediction from a first reference frame, a block obtained from a motion vector for the first reference frame is used as a prediction image, and when the prediction method type is prediction from a second reference frame, , By using a block obtained from a motion vector for the second reference frame as a prediction image,
Since not only the interpolation prediction but also the direct mode using only one reference frame can be used, the coding efficiency can be improved.
[0072]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, specific embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram of an image encoding device according to the first embodiment. Units performing the same operations and signals having the same operations as those in the block diagram of the conventional image encoding device in FIG. 22 are denoted by the same reference numerals, and description thereof will be omitted.
[0073]
H. In the 26L, since only one direct mode scaling coefficient can be transmitted per frame, an optimal direct mode scaling coefficient for a block cannot be selected, and there is a problem in that encoding efficiency is deteriorated. Therefore, in the present invention, a direct mode scaling coefficient table storing a plurality of direct mode scaling coefficients is provided, and the direct mode scaling coefficient is selected and used from the table according to the first relative index of the direct mode. To improve the coding efficiency of the direct mode.
[0074]
FIG. 2 is a first example of a direct mode scaling coefficient table used when selecting a direct mode coefficient. RIdx1 indicates a first relative index in the direct mode, and a set of values of N1, N2, and D indicates a scaling coefficient. The scaling coefficient of each row in the table indicates a coefficient for the value of RIdx1 at the head of the row.
[0075]
Assuming that the motion of the object including the encoding target block in the screen is constant, the display time difference between the encoding target frame and the first reference frame, and the display time between the encoding target frame and the second reference frame 2 The scaling factor for direct mode can be determined by the ratio with the difference.
[0076]
In the example of FIG. 20, when the first relative index RIdx1 is 0 as in the block Blk0, the scaling coefficient is (N1, N2, D) = (2, −1, 3), and the first coefficient as in the block Blk1. When the relative index RIdx1 is 1, the scaling coefficient is (N1, N2, D) = (5, -1, 6). Similarly, the scaling coefficient can be determined in FIG. 2 based on the result of determining the scaling coefficient for the value after RIdx1 = 2.
[0077]
The direct mode scaling coefficient table SPTableBuf in FIG. 1 stores a direct mode scaling coefficient table. The direct mode scaling coefficient selection means SPSel inputs the first relative index RIdx1 obtained from the motion vector / frame number buffer MVFNBuf. Then, a row satisfying the condition of RIdx1 = rRIdx1 is searched from the first column of the direct mode scaling coefficient table, and the scaling coefficient (N1, N2, D) of the row obtained as a result of the search is used as the direct mode scaling coefficient SP. Output.
[0078]
By storing the direct mode scaling coefficient table used in the image encoding device in an encoded signal and transmitting it to the image decoding device, the same direct mode scaling coefficient can be used in the image decoding device. Therefore, the direct mode scaling coefficient table SPs is extracted from the direct mode scaling coefficient table buffer SPTableBuf, and the residual encoded signal ERes, the prediction type PredType, the relative indexes RIdx1, RIdx2, the motion vectors MV1, The variable length coding is performed together with the MV2, and the result is output as a coded signal BitStrm4.
[0079]
Here, a method of encoding the direct mode scaling coefficient table into the image encoded signal BitStrm4 will be described. In the present embodiment, four different methods are shown as methods for encoding the direct mode scaling coefficient table.
[0080]
The encoding method of the first direct mode scaling coefficient table is a method of encoding all three parameters N1, N2, and D in the image encoded signal BitStrm4. FIG. 3 shows an example of an image coded signal format according to this method. Since only the header header part differs from the image encoding signal format of the conventional image encoding device in FIG. 21, only the header part is shown. N indicates the number of direct mode scaling coefficients included in the direct mode coefficient table.
[0081]
N1 (r), N2 (r), and D (r) indicate direct mode scaling coefficients for the first relative index RIdx1 = r. The N number of direct mode scaling coefficients are stored in the header section, and the image decoding apparatus can obtain the direct mode scaling coefficient for the first relative index RIdx1 by analyzing the header section.
[0082]
The encoding method of the second direct mode scaling coefficient table defines a relational expression that holds between D, N1, and N2, and stores only two parameters of D, N1, and N2 in the image encoded signal BitStrm4. Not a way. FIG. 4 shows an example of an image coded signal format according to this method.
[0083]
Under the assumption that the motion of the object in the screen including the encoding target block is constant, the first motion vector and the second motion vector in the direct mode are obtained by changing the scaling vector to the display time of the encoding target frame and the first reference frame. The result is internally divided by the ratio of the difference and the display time difference between the current frame and the second reference frame. Assuming that the frame rate and the display time interval of the reference frame are constant, the relational expression of D = N1−N2 is satisfied in the case of internal division, so that the relation of D = N1−N2 between the image encoding device and the image decoding device is satisfied. By defining the equation, N1, N2, and D can be determined without transmitting any one parameter of N1, N2, and D to the image decoding device. FIG. 4 shows an image coded signal format when the relational expression of D = N1−N2 is satisfied and N1 is omitted. Similarly, an image coded signal format in which N2 and D are omitted can be defined.
[0084]
The third encoding method of the direct mode scaling coefficient table is a method in which one of the values of N1, N2, and D is common to all direct mode scaling coefficients in the table.
[0085]
Here, the first motion vector and the second motion vector in the direct mode are obtained by setting the scaling vector to the display time difference between the current frame and the first reference frame and the display time difference between the current frame and the second reference frame. It is considered that the result is internally divided by the ratio of
[0086]
When the second reference frame used in the direct mode is always the same reference frame in the frame, the display time difference between the encoding target frame and the second reference frame is always constant in the frame. Therefore, N2 used in calculating the second reference vector is fixed to a fixed value C, and by changing N1 and D, it is possible to generate a direct mode scaling coefficient indicating an internal portion due to a display time difference. For example, in the case of FIG. 20, C can be set to −1. FIG. 5 shows an example of an image coded signal format according to this method.
[0087]
The header section stores only one value of N2 commonly used for the direct mode scaling coefficients 1 to N.
[0088]
Similarly, when the first reference frame used in the direct mode is always the same reference frame in the frame, only the value commonly used for all the direct mode scaling coefficients is stored in the Header section for N1, and N2 is stored in the header section. , D, N sets may be stored in the Header section.
[0089]
Similarly, when the interval between the first reference frame and the second reference frame used in the direct mode is always constant within the frame, only the value commonly used for all the direct mode scaling coefficients for the header is also used for D. And N sets of N1 and N2 may be stored in the Header section.
[0090]
The fourth encoding method of the direct mode scaling coefficient table is a method of defining a calculation formula for generating N1, N2, and D.
[0091]
For example, if the frame time interval of the reference frame is constant and the first reference frame and the second reference frame are the results of internally dividing the scaling vector, D, N1, and N2 can be calculated by the following equations.
D (i) = K · (i + 1), N2 (i) = a, N1 (i) = D (i) −N2 (i)
(I is the value indicated by the first relative index, K is the frame interval of the reference frame (for example, the interval between Ref1 and Ref2 in FIG. 20), and a is the interval between the encoding target frame and the second reference frame (for example, FIG. The interval between Frm and Ref3))
[0092]
FIG. 6 shows an example of an image coded signal format according to this method. Only K and a are encoded in the encoded signal. By using the above equation for both the image encoding device and the image decoding device, the same direct mode scaling coefficient can be generated on the encoding side and the decoding side.
The above is the description of the encoding method of the direct mode scaling coefficient table.
[0093]
Next, consider a case where the motion vector in the direct mode refers to a frame stored in the long-term frame buffer as shown in FIG. FIG. 7 is an example of a direct mode scaling coefficient table. Here, a common direct mode scaling coefficient is set for all reference frames included in the long-term frame buffer. For the frame included in the long-term frame buffer, only the direct mode scaling coefficient that is commonly used for all the reference frames included in the long-term frame buffer is stored in the image coded signal format. Since the frames included in the long-term frame buffer are assumed to be images with little motion since they are reference frames for a long time, there is no problem even if all the same coefficients are assigned. For all the frames included in the long-term frame buffer, it is only necessary to send the direct mode scaling coefficient, so that the code amount can be reduced.
[0094]
In the present embodiment, the values of N1, N2, and D are integer values, and a variable length codeword can be assigned to each integer value. However, by limiting the range of the values of N1, N2, and D, more efficient variable-length codeword allocation can be performed. FIG. 8 is an example of a codeword table for a direct mode scaling coefficient. (A) is a variable length codeword table for N1, (B) is a variable length codeword table for N2, and (C) is a variable length codeword table for D.
[0095]
Assuming that the motion of the object including the encoding target block in the screen is constant, the scaling vector can be internally divided into the first motion vector and the second vector. At this time, if D and N1 are set to positive values and N2 is set to negative values, internal division can be performed. Therefore, even if the range of D and N1 values is limited to a positive value and the range of N2 values is limited to a negative value, No problem.
[0096]
Therefore, as shown in FIG. 8, variable length codewords may be assigned to D and N1 only for positive values and to N2 only for negative values. Since it is not necessary to assign a variable-length codeword to a value that is not used, coding efficiency is increased. Although the value of N1 is a positive value, a variable length codeword may be assigned to 0 as a value of 0 or more. Similarly, assuming that the value of N2 is equal to or less than 0, a variable length codeword may be assigned to 0. In this example, one variable-length codeword is assigned to each of N1, N2, and D. However, one variable-length codeword may be assigned to a combination of N1, N2, and D values.
[0097]
As described above, the image encoding apparatus according to the present embodiment selects the direct mode scaling coefficient corresponding to the first relative index of the direct mode from the direct mode scaling coefficient table, and uses the selected frame. Since the motion vector for the direct mode can be generated in consideration of the display time difference, the encoding efficiency in the direct mode can be improved.
[0098]
(Embodiment 2)
FIG. 9 is a block diagram of an image decoding device according to the second embodiment. Units performing the same operations and signals having the same operations as those in the block diagram showing the configuration of the conventional image decoding apparatus in FIG. 23 are denoted by the same reference numerals, and description thereof will be omitted.
[0099]
The variable length decoding means VLD4 receives the coded image signal BitStrm4, performs variable length decoding, and performs a residual coded signal ERes, a prediction type PredType, relative indexes RIdx1, RIdx2, motion vectors MV1, MV2, a scaling coefficient for direct mode. Output table SPs. The direct mode scaling coefficient table SPs is stored in the direct mode scaling coefficient table buffer SPTableBuf. It is assumed that the direct mode scaling coefficient selection unit SPSel performs the same processing as the direct mode scaling coefficient selection unit SPSel of the image encoding apparatus of FIG. 1 shown in the first embodiment.
[0100]
When decoding the direct mode coefficient table SPs from the image coded signal, decoding corresponding to the encoding method of the direct mode scaling coefficient table described in the first embodiment is performed.
[0101]
For example, with respect to the encoding method of the first direct mode scaling coefficient table described in the first embodiment, when a relational expression is defined between N1, N2, and D and one parameter is omitted, N1 , N2, D, unknown parameters are calculated.
[0102]
When one of the parameters is made common to all the direct mode scaling coefficients and only one common parameter is stored in the image coded signal as in the image coded signal format of FIG. Decode the parameters and use them for all direct mode scaling factors.
[0103]
When the parameter value can be calculated by a predetermined calculation formula as in the image coded signal format of FIG. 6, the parameter value obtained by using the predetermined calculation formula is used for all direct mode scaling coefficients. .
[0104]
Next, two methods will be described for processing when the direct mode scaling coefficient corresponding to the relative index rRIdx does not exist in the direct mode scaling coefficient table.
[0105]
First, the first method will be described. FIG. 10 is a flowchart for selecting a first direct mode scaling coefficient in the present embodiment. rRIdx is a first relative index in the direct mode. In process F11, it is checked whether or not the direct mode scaling coefficient for RIdx1 = rRIdx exists in the direct mode scaling coefficient table. If the direct mode scaling coefficient is present in the table, a direct mode scaling coefficient corresponding to rRIdx is selected in step F13. If the direct mode scaling coefficient does not exist in the table, the direct mode scaling coefficient having the maximum value of RIdx1 in the direct mode scaling coefficient table is selected in process F12. Then, the process of selecting the direct mode scaling coefficient is completed. Here, Ridx1 selects the largest direct mode scaling factor in the direct mode scaling factor that is not in the table, and the closest direct mode scaling factor in the table is the one with the largest RIdx1 value. Because it becomes. As described in the related art, the value of the first relative index is initially set to 0 in the order of the reference frame having the display time earlier than the encoding target frame in order from the closest to the encoding target frame. Is assigned.
[0106]
Next, the second method will be described. FIG. 11 is a flowchart for selecting a second direct mode scaling coefficient in the present embodiment. The difference from FIG. 10 is that if the direct mode scaling factor for RIdx1 = rRIdx does not exist in the direct mode scaling factor table, the default direct mode scaling factor is used in process F22.
[0107]
As described above, according to the present embodiment, it is possible to correctly decode an image-encoded signal encoded in the block diagram of the image encoding device using the image encoding method of the present invention described in Embodiment 1. Also, by using a plurality of direct mode scaling coefficients, even when there are a plurality of reference frame candidates, it is possible to generate a direct mode motion vector in consideration of the display time difference between the current frame and the reference frame. Can be improved.
[0108]
(Embodiment 3)
FIG. 12 is a block diagram of an image encoding device according to the third embodiment. Units performing the same operations and signals having the same operations as those in the block diagram of the image encoding apparatus according to the first embodiment in FIG.
[0109]
In the conventional direct mode, only interpolation prediction can be used as a prediction method. In the present embodiment, frame prediction from one reference frame can be used in the direct mode. For example, in FIG. 20, when a scene change occurs between Ref1 and Frm, there is no correlation between Ref1, Ref2 and Frm, and the effect of inter-frame prediction is lost. In this case, the prediction efficiency is higher in the prediction using only Ref3 as the reference frame than in the interpolation prediction using the reference frames Ref1 and Ref2 having no correlation.
[0110]
Therefore, if the first reference frame prediction or the second reference frame prediction using only one reference frame is used as the direct mode, the coding efficiency can be improved.
[0111]
Direct mode scaling coefficient / prediction method table The direct mode scaling coefficient / prediction method table stored in the buffer SPPRedTableBuf will be described with reference to FIG. The difference between the tables of FIGS. 2 and 13 is that a prediction method is added to the table of FIG.
[0112]
The direct mode scaling coefficient / prediction method selecting means SPPPredSel selects a set of the direct mode scaling coefficient SP and the direct mode prediction method DPred corresponding to the first relative index rRIdx1 from the direct mode scaling coefficient / prediction method table buffer SPPRedTableBuf. And output.
[0113]
When the direct mode prediction method DPred indicates the first reference frame prediction, the direct mode vector / relative index generation unit GenMVRefIdx2 outputs only the first relative index sRIdx1 and the first motion vector sMV1.
[0114]
When the direct mode prediction method DPred indicates the second reference frame prediction, the direct mode vector / relative index generation unit GenMVRefIdx2 outputs only the second relative index sRIdx2 and the second motion vector sMV2.
[0115]
When the direct mode is selected and the direct mode prediction method DPred indicates the first reference frame prediction, the prediction type selection unit PredSel switches the switch SW11 to the “1” side, and sets the first relative index RIdx1 and the first reference vector. The reference block RefBlk1 indicated by MV1 is used for prediction. When the direct mode is selected and the direct mode prediction method DPred indicates the second reference frame prediction, the prediction type selection unit PredSel switches the switch SW11 to the “1” side, and sets the second relative index RIdx2 and the second motion vector. The reference block RefBlk2 indicated by MV2 is used for prediction.
[0116]
When the direct mode prediction method DPred indicates interpolation prediction, the switch SW11 is switched to the “0” side, and the first relative index RIdx1, the reference block RefBlk1 indicated by the first reference vector MV1, the second relative index RIdx2, and the second relative index RIdx2. The reference block RefBlk2 indicated by the reference vector MV2 is used for interpolation prediction.
[0117]
The direct mode scaling coefficient / prediction method table SPPreds is stored in the coded signal so that the image coding apparatus and the image decoding apparatus can use the common direct mode scaling coefficient / prediction method table SPPreds. At this time, by using the values of the scaling coefficients N1 and N2 for the direct mode as in the method described below, the direct mode prediction method in the table is not coded, that is, the image code shown in the first embodiment is not coded. It is possible to notify the image decoding apparatus of the direct mode prediction method DPred while keeping the encoded signal format.
[0118]
In the case where a scaling coefficient for direct mode in which N2 is 0 is selected from the table of FIG. 13, it is defined that the first reference frame prediction is used instead of the interpolation prediction. At this time, if the scaling vector is MV, the first motion vector can be calculated as (N1 × MV) / D.
[0119]
Similarly, when the direct mode scaling coefficient of which N1 is 0 is selected, it is defined as the second reference frame prediction, and the first motion vector can be calculated as (N2 × MV) / D.
[0120]
The variable length coding unit VLC5 performs variable length coding on the residual coded signal ERes, the prediction type PredType, the relative indexes RefIdx1, RefIdx2, the motion vectors MV1, MV2, the direct mode scaling coefficient / prediction method table SPPreds, and outputs the coded signal. Output as BitStrm5.
[0121]
As described above, according to the present embodiment, even when there is only one reference frame having a high correlation with the encoding target frame due to a scene change or the like and the effect of interpolation prediction is lost, only the reference frame having a high correlation is used. Since the direct mode can be used, coding efficiency can be improved.
[0122]
(Embodiment 4)
FIG. 14 is a block diagram of an image decoding device according to the fourth embodiment. Units performing the same operations and signals having the same operations as in the block diagram of the image decoding apparatus according to the second embodiment in FIG.
[0123]
The variable length decoding unit VLD5 receives the coded image signal BitStrm5, performs variable length decoding, and performs a residual coded signal ERes, a prediction type PredType, relative indexes RIdx1, RIdx2, motion vectors MV1, MV2, and a scaling coefficient for direct mode. It outputs a prediction method table SPPreds.
[0124]
The direct mode scaling coefficient / prediction method table SPPPreds is stored in the direct mode scaling coefficient / prediction method table buffer SPPRedTableBuf. The direct mode scaling coefficient / prediction method selecting means SPPPredSel selects a set of the direct mode scaling coefficient SP and the direct mode prediction method DPred corresponding to the first relative index rRIdx1 from the direct mode scaling coefficient / prediction method table buffer SPPRedTableBuf. And output.
[0125]
When the direct mode prediction method DPred indicates the first reference frame prediction, the direct mode vector / relative index generation unit GenMVRefIdx2 outputs only the first relative index sRIdx1 and the first motion vector sMV1.
[0126]
When the direct mode prediction method DPred indicates the second reference frame prediction, the direct mode vector / relative index generation unit GenMVRefIdx2 outputs only the second relative index number sRIdx2 and the second motion vector sMV2.
[0127]
When the direct mode prediction method DPred indicates the first reference frame prediction when the direct mode is selected, the switch SW23 is switched to “1”, and the reference block RefBlk indicated by the first relative index nRRIdx1 and the first reference vector nMV1 is set. Used for prediction.
[0128]
When the direct mode is selected and the direct mode prediction method DPred indicates the second reference frame prediction, the switch SW23 is switched to the “1” side to change the reference block RefBlk indicated by the second relative index nRIdx2 and the second motion vector nMV2. Used for prediction.
[0129]
When the direct mode prediction method DPred indicates the interpolation prediction, the switch SW23 is switched to “0” side, and the first relative index nRIdx1, the reference block RefBlk1 indicated by the first reference vector nMV1, the second relative index nRIdx2, and the second relative index nRIdx2. The reference block RefBlk2 indicated by the reference vector nMV2 is used for interpolation prediction.
[0130]
As described above, the image decoding apparatus according to the present embodiment decodes the direct mode scaling coefficient / prediction method table in the encoded signal, and uses the direct mode scaling coefficient table according to the direct mode relative index value from the direct mode scaling coefficient table. By using the scaling coefficient, the image encoded signal encoded by the image encoding device described in the third embodiment can be correctly decoded.
[0131]
(Embodiment 5)
Furthermore, a program for realizing the configuration of the image encoding method or the image decoding method described in each of the above embodiments is recorded on a storage medium such as a flexible disk, so that the program described in each of the above embodiments is recorded. Can be easily executed in an independent computer system.
[0132]
FIG. 15 is an explanatory diagram of a storage medium for storing a program for realizing the image encoding method and the image decoding method of Embodiments 1 to 4 by a computer system.
[0133]
FIG. 15B shows the appearance, cross-sectional structure, and flexible disk as viewed from the front of the flexible disk, and FIG. 15A shows an example of the physical format of the flexible disk which is a recording medium body. The flexible disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the disk, and each track is divided into 16 sectors Se in an angular direction. ing. Therefore, in the flexible disk storing the program, an image encoding method as the program is recorded in an area allocated on the flexible disk FD.
[0134]
FIG. 15C shows a configuration for recording and reproducing the program on the flexible disk FD. When the above program is recorded on the flexible disk FD, the image coding method or the image decoding method as the above program is written from the computer system Cs via the flexible disk drive FDD. When the image encoding method is constructed in a computer system using a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.
[0135]
In the above description, the description has been made using a flexible disk as a recording medium. However, the same description can be made using an optical disk. Further, the recording medium is not limited to this, and the present invention can be similarly implemented as long as the program can be recorded, such as an IC card or a ROM cassette.
[0136]
Further, here, an application example of the image encoding device or the image decoding device described in the above embodiment and a system using the same will be described.
[0137]
FIG. 24 is a block diagram illustrating an overall configuration of a content supply system ex100 that realizes a content distribution service. A communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
[0138]
The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera on the Internet ex101 via an Internet service provider ex102 and a telephone network ex104, and base stations ex107 to ex110. Each device such as the mobile phone ex115 is connected.
[0139]
However, the content supply system ex100 is not limited to the combination as shown in FIG. 24, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
[0140]
The camera ex113 is a device such as a digital video camera capable of shooting moving images. In addition, a mobile phone can be a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access mobile phone system, or a GSM gigabit mobile access system). Or PHS (Personal Handyphone System) or the like.
[0141]
The streaming server ex103 is connected from the camera ex113 to the base station ex109 and the telephone network ex104, and enables live distribution and the like based on encoded data transmitted by the user using the camera ex113. The encoding process of the photographed data may be performed by the camera ex113, or may be performed by a server or the like that performs the data transmission process. Also, moving image data captured by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed by the LSI ex117 of the computer ex111 and the camera ex116. The image encoding / decoding software may be incorporated in any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Further, the moving image data may be transmitted by the mobile phone with camera ex115. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.
[0142]
In the content supply system ex100, the content (for example, a video image of a live music) captured by the user with the camera ex113, the camera ex116, or the like is encoded and transmitted to the streaming server ex103 as in the above-described embodiment. On the other hand, the streaming server ex103 stream-distributes the content data to the requesting client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and further, realizes personal broadcast by receiving, decoding, and reproducing the data in real time at the client. It is a system that becomes possible.
[0143]
The encoding and decoding of each device constituting this system may be performed using the image encoding device or the image decoding device described in each of the above embodiments.
[0144]
A mobile phone will be described as an example.
FIG. 25 is a diagram illustrating a mobile phone ex115 that uses the image encoding device and the image decoding device described in the above embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera unit ex203 capable of taking a picture such as a CCD camera, a still image, a picture taken by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display for displaying data obtained by decoding a received video or the like, a main unit including a group of operation keys ex204, an audio output unit ex208 such as a speaker for outputting audio, and audio input. Input unit ex205 such as a microphone for storing encoded or decoded data, such as data of captured moving images or still images, received mail data, moving image data or still image data, etc. Storage media ex207, attached storage media ex207 to mobile phone ex115 And a slot portion ex206 to ability. The storage medium ex207 stores a flash memory element, which is a kind of an electrically erasable and programmable read only memory (EEPROM), which is a nonvolatile memory that can be electrically rewritten and erased, in a plastic case such as an SD card.
[0145]
Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 is provided with a power supply circuit unit ex310, an operation input control unit ex304, an image encoding unit, and a main control unit ex311 which controls the respective units of a main body unit including a display unit ex202 and operation keys ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via the synchronous bus ex313. Connected to each other.
[0146]
When the end of the call and the power key are turned on by a user operation, the power supply circuit unit ex310 supplies power to each unit from the battery pack to activate the digital cellular phone with camera ex115 in an operable state. .
[0147]
The mobile phone ex115 converts a sound signal collected by the sound input unit ex205 into digital sound data by the sound processing unit ex305 in the voice call mode based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. This is spread-spectrum-processed by a modulation / demodulation circuit unit ex306, subjected to digital-analog conversion processing and frequency conversion processing by a transmission / reception circuit unit ex301, and then transmitted via an antenna ex201. Also, the mobile phone ex115 amplifies the received signal received by the antenna ex201 in the voice communication mode, performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum despreading processing in the modulation / demodulation circuit unit ex306, and performs analog voice After being converted into a signal, the signal is output via the audio output unit ex208.
[0148]
Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital / analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and transmits the data to the base station ex110 via the antenna ex201.
[0149]
When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.
[0150]
The image encoding unit ex312 includes the image encoding device described in the present invention, and uses the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The image data is converted into encoded image data by compression encoding, and is transmitted to the demultiplexing unit ex308. At this time, the mobile phone ex115 simultaneously transmits the audio collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.
[0151]
The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and multiplexes the resulting multiplexed data into a modulation / demodulation circuit unit. The signal is subjected to spread spectrum processing in ex306 and subjected to digital-analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201.
[0152]
When receiving data of a moving image file linked to a homepage or the like in the data communication mode, the modulation and demodulation circuit unit ex306 performs spectrum despread processing on a received signal received from the base station ex110 via the antenna ex201, and obtains the resulting multiplexed signal. The demultiplexed data is sent to the demultiplexing unit ex308.
[0153]
To decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into coded image data and audio data by separating the multiplexed data, and transmits the multiplexed data via the synchronization bus ex313. And supplies the encoded image data to the image decoding unit ex309 and the audio data to the audio processing unit ex305.
[0154]
Next, the image decoding unit ex309 is configured to include the image decoding device described in the present invention, and decodes the encoded image data by a decoding method corresponding to the encoding method described in the above embodiment. Thereby, reproduced moving image data is generated and supplied to the display unit ex202 via the LCD control unit ex302, whereby, for example, moving image data included in a moving image file linked to a homepage is displayed. At this time, the audio processing unit ex305 simultaneously converts the audio data into an analog audio signal, and supplies the analog audio signal to the audio output unit ex208. Thereby, the audio data included in the moving image file linked to the homepage is reproduced, for example. You.
[0155]
It should be noted that the present invention is not limited to the example of the system described above, and digital broadcasting using satellites and terrestrial waves has recently become a topic. As shown in FIG. Any of the decoding devices can be incorporated. Specifically, at the broadcasting station ex409, an encoded bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. The broadcasting satellite ex410 receiving this transmits a radio wave for broadcasting, receives this radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, and transmits the radio wave to a television (receiver) ex401 or a set-top box (STB) ex407 or the like. The device decodes the encoded bit stream and reproduces it. Further, the image decoding device described in the above embodiment can also be mounted on a playback device ex403 that reads and decodes an encoded bit stream recorded on a storage medium ex402 that is a recording medium. In this case, the reproduced video signal is displayed on the monitor ex404. A configuration is also conceivable in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television. At this time, the image encoding device may be incorporated in the television instead of the set-top box. Further, it is also possible to receive a signal from the satellite ex410 or the base station ex107 or the like with the car ex412 having the antenna ex411 and reproduce the moving image on a display device such as the car navigation ex413 or the like included in the car ex412.
[0156]
It should be noted that the configuration of the car navigation ex413 can be, for example, a configuration excluding the camera unit ex203 and the camera interface unit ex303 in the configuration illustrated in FIG. 26, and the same can be considered for the computer ex111 and the television (receiver) ex401. . In addition, terminals such as the mobile phone ex114 and the like have three mounting formats, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.
[0157]
As described above, the image encoding device and the image decoding device described in the above embodiment can be used in any of the devices and systems described above, and by doing so, the effects described in the above embodiment can be obtained. Obtainable.
[0158]
【The invention's effect】
As described above in detail, the image encoding method / image decoding method of the present invention includes a direct mode scaling coefficient table storing a plurality of direct mode scaling coefficients, and is provided in accordance with the first relative index value of the direct mode. By selecting and using the direct mode coefficient, a direct mode motion vector can be generated in consideration of the display time difference between the encoding target frame and the reference frame, so that the encoding efficiency in the direct mode can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image encoding device according to a first embodiment.
FIG. 2 is a diagram of a first example of a direct mode scaling coefficient table according to the first embodiment;
FIG. 3 is a diagram of a first example of an image coded signal format according to the first embodiment;
FIG. 4 is a diagram of a second example of an image coded signal format according to the first embodiment;
FIG. 5 is a diagram illustrating a third example of an image coded signal format according to the first embodiment;
FIG. 6 is a diagram illustrating a fourth example of an image coded signal format according to the first embodiment;
FIG. 7 is a diagram of a second example of a direct mode scaling coefficient table according to the first embodiment;
FIG. 8 is a diagram of a code table for scaling coefficients for direct mode according to the first embodiment.
FIG. 9 is a block diagram of an image decoding device according to a second embodiment.
FIG. 10 is a flowchart for selecting a scaling coefficient for direct mode according to the first method of the second embodiment.
FIG. 11 is a flowchart for selecting a scaling coefficient for direct mode according to a second method of the second embodiment.
FIG. 12 is a block diagram of an image decoding apparatus according to a third embodiment.
FIG. 13 is a diagram of a direct mode scaling coefficient / prediction method table according to the third embodiment;
FIG. 14 is a block diagram of an image decoding apparatus according to a fourth embodiment.
FIG. 15 is a diagram illustrating a storage medium for storing a program for realizing the image encoding method and the image decoding method according to the first to fourth embodiments by a computer system.
FIG. 16 is a conceptual diagram of a B picture.
FIG. 17 is an explanatory diagram of interpolation prediction.
FIG. 18 is an explanatory diagram of a frame number and a relative index.
FIG. 19 is a conceptual diagram of a short-term frame buffer and a long-term frame buffer.
FIG. 20 is an explanatory diagram of a direct mode of a conventional image encoding device.
FIG. 21 is a conceptual diagram of an image encoded signal format of a conventional image encoding device.
FIG. 22 is a block diagram illustrating a configuration of a conventional image encoding device.
FIG. 23 is a block diagram showing a configuration of a conventional image decoding device.
FIG. 24 is a block diagram showing the overall configuration of a content supply system.
FIG. 25 illustrates an appearance of a mobile phone.
FIG. 26 illustrates a configuration of a mobile phone.
FIG. 27 is a diagram illustrating an application example of the image encoding device or the image decoding device described in this embodiment.
[Explanation of symbols]
ImgEnc image coding means
ImgDec image decoding means
Add adder
Sub subtractor
MFrmBuf multi-frame buffer
ME motion estimation means
VLC0, VLC4, VLC5 Variable length coding means
VLD0, VLD4, VLD5 Variable length decoding means
MVBuf motion vector buffer
Pol pixel interpolation means
GenMVRdx Direct mode vector / relative index generation means
MVFNBuf motion vector frame number buffer
MVBuf motion vector buffer
SPTable Direct mode scaling coefficient table buffer
SPSel Direct mode scaling coefficient selection means
PredSel prediction type selection means
SPPredTable Scaling coefficient / prediction method table buffer for direct mode
SPPredSel Direct mode scaling coefficient / prediction method selection means
SW11 to SW13,. SW21-SW23 switch
Cs computer system
FD flexible disk
FDD flexible drive

Claims (10)

In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on an encoding target frame by motion compensation from a plurality of encoded frames stored in the multi-frame buffer, A first step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the encoded frame;
A second step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
Encoding a prediction error which is a difference between the input encoding target frame and the predicted image, and outputting an image encoded signal including an encoded signal of the prediction error. In the second step,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the encoding target frame is set as the first reference frame,
One or more scaling coefficients associated with a value of a first relative index used when calculating a motion vector for the first reference frame and a motion vector for the second reference frame by scaling from the motion vector. A scaling coefficient table corresponding to the first relative index is selected from the scaling coefficient table; a motion vector for the first reference frame; Calculate the motion vector for the frame,
A predicted image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame, and in the third step,
An image encoding method, wherein the image encoding signal is output including the scaling coefficient table. A first step of inputting an image encoded signal including an encoded signal of a prediction error,
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on a decoding target frame by motion compensation from a plurality of decoded frames stored in the multi-frame buffer, A second step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the decoded frame;
A third step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A fourth step of generating a decoded image of a frame from the predicted image and the decoded prediction error,
Storing a decoded image of a frame that may be used for inter-frame prediction in a multi-frame buffer, and a fifth step of the image decoding method,
Decoding the scaling coefficient table in the image encoded signal, in the second step,
In the decoded frame, the display order is later than the decoding target frame, and the second relative index is selected as the second reference frame having the smallest reference index. In the third step,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the decoding target frame is set as the first reference frame,
Selecting a scaling coefficient corresponding to the first relative index from the scaling coefficient table for use in calculating a motion vector for the first reference frame and a motion vector for the second reference frame by scaling from the motion vector; Calculating a motion vector for the first reference frame and a motion vector for the second reference frame from the motion vector and the scaling coefficient, and calculating a block obtained from the motion vector for the first reference frame; An image decoding method characterized by generating a predicted image by pixel interpolation from a block obtained from a motion vector for a second reference frame. In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on an encoding target frame by motion compensation from a plurality of encoded frames stored in the multi-frame buffer, A first step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the encoded frame;
A second step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
Encoding a prediction error which is a difference between the input encoding target frame and the predicted image, and outputting an image encoded signal including an encoded signal of the prediction error. In the second step,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the encoding target frame is set as the first reference frame,
A scaling coefficient associated with a value of a first relative index used when calculating a motion vector for the first reference frame and a motion vector for the second reference frame by scaling from the motion vector; A scaling coefficient / prediction method table storing one or more prediction method types indicating interpolation or prediction from a first reference frame or prediction from a second reference frame;
A scaling coefficient and prediction method type corresponding to the first relative index are selected from the scaling coefficient / prediction method table, and a motion vector for the first reference frame and the second reference frame And a motion vector for
When the prediction method type is pixel interpolation, a predicted image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame,
When the prediction method type is prediction from a first reference frame, a block obtained from a motion vector for the first reference frame is set as a prediction image,
When the prediction method type is prediction from a second reference frame, a block obtained from a motion vector for the second reference frame is used as a prediction image, and in the third step,
An image encoding method characterized by outputting the above-mentioned scaling coefficient / prediction method table in an image encoded signal. A first step of inputting an image encoded signal including an encoded signal of a prediction error,
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on a decoding target frame by motion compensation from a plurality of decoded frames stored in the multi-frame buffer, A second step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the decoded frame;
A third step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A fourth step of generating a decoded image of a frame from the predicted image and the decoded prediction error,
Storing a decoded image of a frame that may be used for inter-frame prediction in a multi-frame buffer, and a fifth step of the image decoding method,
Decoding the scaling coefficient prediction method table in the image encoding signal, in the second step,
In the decoded frame, the display order is later than the decoding target frame, and the second relative index is selected as the second reference frame having the smallest reference index. In the third step,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the decoding target frame is set as the first reference frame,
A scaling coefficient corresponding to the first relative index and a prediction method type used when calculating a motion vector for the first reference frame and a motion vector for the second reference frame by scaling from the motion vector Selecting from a coefficient / prediction method table, calculating a motion vector for the first reference frame and a motion vector for the second reference frame from the motion vector and the scaling coefficient,
When the prediction method type is pixel interpolation, a predicted image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame,
When the prediction method type is prediction from a first reference frame, a block obtained from a motion vector for the first reference frame is set as a prediction image,
When the prediction method type is prediction from a second reference frame, a block obtained from a motion vector for the second reference frame is used as a prediction image. An image signal input, a difference device that performs a difference between the image signal and the predicted image and outputs the result as a residual signal, and an image encoding unit that performs image encoding processing on the difference signal and outputs the result as a residual encoded signal. Image decoding means for decoding the residual coded signal and outputting it as a residual decoded signal, an adder for adding the residual decoded signal and a predicted image to output a reconstructed image, A multi-frame buffer to be stored, a scaling coefficient table storing at least one scaling coefficient associated with the value of the first relative index, and a motion of a block at the same position as the encoding target block in the selected reference frame A frame referred to by a vector is referred to as a first reference frame, the selected reference frame is referred to as a second reference frame, and an index relative to the first reference frame is used. Direct mode scaling coefficient selection means for selecting a scaling coefficient corresponding to a first relative index from the scaling coefficient table, and a first reference frame and a second reference frame by a predetermined method from the scaling coefficient and the motion vector. A direct mode vector / relative index for generating a motion vector to a frame and a generation unit, and prediction is performed by performing pixel interpolation of two reference blocks referenced by a motion vector for a first reference frame and a motion vector for a second reference frame. An image encoding apparatus comprising: a pixel interpolation unit that outputs an image; and a variable length encoding unit that performs variable length encoding of a prediction error and the scaling coefficient table and outputs the encoded signal as an encoded signal. Variable length decoding means for inputting an image coded signal and performing variable length decoding and outputting a residual coded signal and a scaling coefficient table, and image decoding means for decoding the residual coded signal and outputting a decoded residual signal An adder that adds the residual decoded signal and the prediction image signal and outputs a decoded image; a multi-frame buffer that stores the decoded image; and a multi-frame buffer that stores the same position as the encoding target block in the selected reference frame. A frame referred to by a motion vector of a block is a first reference frame, the selected reference frame is a second reference frame, and scaling corresponding to a first relative index which is a relative index to the first reference frame. Direct mode scaling coefficient selection means for selecting a coefficient from the scaling coefficient table; A direct mode vector / relative index for generating a motion vector to a first reference frame and a second reference frame from the first reference frame by a predetermined method, and the first motion vector for the first reference frame. A pixel interpolating means for performing pixel interpolation of a block referred to by the second motion vector with respect to a block referred to by the second motion vector and outputting the predicted image signal as the predicted image signal. Decoding device. A storage medium storing a program for performing image encoding by a computer, the program storing, on the computer, a block on an encoding target frame from a plurality of encoded frames stored in a multi-frame buffer. In order to select a first reference frame and a second reference frame to be referred to when obtaining by motion compensation, a first relative index and a second relative index given to the encoded frame are used. A first step of selecting the first or second at least one reference frame;
A second step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
Encoding the prediction error, the third step of outputting an image encoding signal including an encoded signal of the prediction error, in the second step in the image encoding method,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the encoding target frame is set as the first reference frame,
A scaling coefficient table storing at least one scaling coefficient associated with a value of the first relative index; selecting a scaling coefficient corresponding to the first relative index from the scaling coefficient table; Calculating a motion vector for the first reference frame and a second reference frame from the coefficients;
Generating a predicted image by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame;
In the third step,
A storage medium for outputting an encoded image signal including the scaling coefficient table. A storage medium storing a program for performing image decoding by a computer, wherein the program stores
A first step of inputting an image encoded signal including an encoded signal of a prediction error,
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on a decoding target frame by motion compensation from a plurality of decoded frames stored in the multi-frame buffer, A second step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the decoded frame;
A third step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A fourth step of generating a decoded image of a frame from the predicted image and the decoded prediction error,
Storing a decoded image of a frame that may be used for inter-frame prediction in a multi-frame buffer, and a fifth step of the image decoding method,
Decode the scaling coefficient table in the image encoded signal,
In the second step,
Selecting, as the second reference frame, a reference frame having the smallest second relative index in the display order of the decoded frames after the decoding target frame;
In the third step,
In the second reference frame, a frame referred to by a motion vector used for motion compensation of a block at the same position as a predetermined block on the decoding target frame is set as the first reference frame, and the first relative index is set. Is selected from the scaling coefficient table, and a motion vector for the first reference frame and a second reference frame are calculated from the motion vector and the scaling coefficient. A storage medium for causing a predicted image to be generated by pixel interpolation from a block obtained from a motion vector and a block obtained from a motion vector for the second reference frame. A computer-readable storage medium storing a program for performing image encoding, wherein the program stores
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on an encoding target frame by motion compensation from a plurality of encoded frames stored in the multi-frame buffer, A first step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the encoded frame;
A second step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
Encoding the prediction error, the third step of outputting an image encoding signal including an encoded signal of the prediction error, in the second step in the image encoding method,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the encoding target frame is set as the first reference frame,
A scaling coefficient associated with the value of the first relative index, and a scaling coefficient storing one or more prediction method types indicating pixel interpolation, prediction from the first reference frame, or prediction from the second reference frame. It has a forecast method table,
A scaling coefficient and a prediction method type corresponding to the first relative index are selected from the scaling coefficient / prediction method table, and a motion vector for the first reference frame and the second reference frame are determined from the motion vector and the scaling coefficient. Calculated to
When the prediction method type is pixel interpolation, a prediction image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame, and prediction is performed. When the method type is prediction from a first reference frame, a block obtained from a motion vector for the first reference frame is used as a prediction image, and when the prediction method type is prediction from a second reference frame, , A block obtained from a motion vector for the second reference frame as a prediction image,
In the third step,
A storage medium for outputting an encoded image signal including the scaling coefficient / prediction method table. A storage medium storing a program for performing image decoding by a computer, wherein the program stores
A first step of inputting an image encoded signal including an encoded signal of a prediction error,
In order to select a first reference frame and a second reference frame to be referred to when obtaining a block on a decoding target frame by motion compensation from a plurality of decoded frames stored in the multi-frame buffer, A second step of selecting the first or second at least one reference frame using a first relative index and a second relative index given to the decoded frame;
A third step of generating a predicted image by pixel interpolation from a block obtained by motion compensation on the first or second at least one reference frame;
A fourth step of generating a decoded image of a frame from the predicted image and the decoded prediction error,
Storing a decoded image of a frame that may be used for inter-frame prediction in a multi-frame buffer, and a fifth step of the image decoding method,
Decode the scaling coefficient / prediction method table in the image encoded signal,
In the second step,
Selecting, as the second reference frame, a reference frame having the smallest second relative index in the display order of the decoded frames after the decoding target frame;
In the third step,
In the second reference frame, a frame referred to by a motion vector used in motion compensation of a block at the same position as a predetermined block on the decoding target frame is set as the first reference frame,
A scaling coefficient and a prediction method type corresponding to the first relative index are selected from the scaling coefficient / prediction method table, and a motion vector for the first reference frame and the second reference frame are determined from the motion vector and the scaling coefficient. Calculated to
When the prediction method type is pixel interpolation, a prediction image is generated by pixel interpolation from a block obtained from a motion vector for the first reference frame and a block obtained from a motion vector for the second reference frame, and prediction is performed. When the method type is prediction from a first reference frame, a block obtained from a motion vector for the first reference frame is used as a prediction image, and when the prediction method type is prediction from a second reference frame, A block obtained from a motion vector for the second reference frame as a predicted image.

Images