CN1838775B - Moving picture coding apparatus, moving picture decoding apparatus, moving picture coding method and moving picture decoding method - Google Patents

Abstract

A moving image coding device, which encodes a moving image composed of frame images in time sequence through displacement compensation, the device includes: a reference image generating part for generating several different a reference image; a displacement compensating part, which calculates a displacement compensation value for a predetermined area to be encoded by using the generated reference image; and a sending part, which is used to send reference image information and an indication displacement A combination of compensation value information; wherein, the reference image information is a combination of reference frame image identification information and information indicating image processing.

Description

Mobile image encoding device and method, moving image decoding device and method

This application is based on the international filing date of November 29, 2002 filed on May 28, 2004, entitled "Moving image encoding device, moving image decoding device, moving image encoding method, moving image decoding method, program and stored program A divisional application filed in Chinese patent application 02823828.1 for a computer-readable recording medium.

technical field

The present invention relates to a moving image encoding device, a moving image decoding device, a moving image encoding method, and a moving image decoding method.

Background technique

As an example of a conventional mobile image coding system, a mobile image coding system based on the "H.26L coding system" described in "ITU-T SG16VCEG-M81, H.26L Test Model Long Term Number 8 (TML-8)" is described below. An image encoding device and a moving image decoding device. FIG. 1 shows a configuration of the aforementioned moving image coding device 20 , and FIG. 2 shows a configuration of the aforementioned moving image decoding device 50 .

The mobile image encoding device shown in FIG. 1 reduces the redundant expression in the time direction by displacement compensation inter-frame prediction, and further reduces the redundancy in the spatial direction by orthogonal transformation, thereby performing a moving image (an input video signal) information compression. Figure 3 shows an explanatory diagram of motion compensated inter prediction.

Hereinafter, the operation of the moving image encoding device 20 shown in FIG. 1 will be described with reference to these drawings.

The input video signal 1 consists of the timing of frame images. Here, it is assumed that a frame image to be encoded is divided into square areas (macroblocks) of 16×16 pixels, and encoding processing by the moving image encoding device 20 and decoding processing by the moving image decoding device 50 are performed in units of these macroblocks. In addition, a frame image divided into macroblock units is defined as "frame image signal 2".

According to the "H.26L coding system", it can be used as a "prediction mode", which is an "INTRA prediction mode", which is used to use the pixel values of the coded adjacent areas in the same frame image (for example, adjacent to the frame image signal to be encoded 2) for spatial prediction, and several "INTER prediction modes" that use coded frame images for motion-compensated inter prediction that vary over time.

This configuration of the "H.26L coding system" enables effective information compression by switching the "prediction mode" of the macroblock according to the location characteristics of the input video signal 1 .

"Displacement-compensated inter-frame prediction" is a technique for finding an image signal pattern similar to that in a frame image signal 2 within a predetermined search range of a reference frame image 5; for detecting an image signal pattern between two image signal patterns The amount of spatial displacement is used as "displacement vector 3"; and is used to encode and send "displacement compensation related information", which includes "displacement vector 3", "prediction mode" and "reference frame number" and "displacement vector 3" calculated according to Predicted residual signal 9".

According to the "H.26L coding system", as shown in FIG. 3, seven kinds of "INTER prediction modes" are available. More precisely, in fact, in addition to these prediction modes, when the video is static, "skip mode" is available, such as a prediction mode that directly copies pixels at the same position as the reference frame image 5 (encoded frame image).

As shown in Figure 3, displacement vector 3 is detected in units of 16×16 pixels in “mode 1”, detected in units of 8×16 pixels in “mode 2”, and detected in units of 16×8 pixels in “mode 3”. Units are detected in units of 8x8 pixels in "mode 4", in units of 8x4 pixels in "mode 5", in units of 8x4 pixels in "mode 6", and In "mode 7" it is detected in units of 4x4 pixels.

That is to say, these seven prediction modes can subdivide the displacement detection unit in the macroblock, and can accurately grasp the displacement occurring in various macroblocks.

First, an input section 31 sends the frame image signal 2 to the displacement detection section 32 and the spatial prediction section 35 .

Then, the displacement detection section 32 detects that the number of displacement vectors 3 conforms to the predetermined prediction mode 4 of the received frame image signal 2 by referring to the reference frame image 5 from the frame memory 34 .

At the same time, the spatial prediction unit 35 performs spatial prediction using the encoded pixel values of the neighboring regions of the same frame image from the frame memory 34 . Spatial prediction component 35 can perform spatial prediction by several methods.

Second, the displacement detection part 32 sends the displacement vector 3 detected by all "INTER prediction modes" shown in FIG.

Then, the displacement compensating part 33 generates a predicted image signal (one macroblock unit) 6 through displacement compensation using the reference frame image 5 and several displacement vectors 3 from the frame memory 34 and the prediction mode from the displacement detecting part 32. 4 combinations.

Third, the displacement compensating section 33 sends information about the predicted image signal 6 generated by displacement compensation, the prediction mode 4 , the displacement vector 3 and the coding efficiency to the prediction mode judging section 36 . On the other hand, the spatial prediction section 35 sends to the prediction mode judging section 36 information about the predicted image signal 7 generated by spatial prediction, the prediction mode (if there are several types of spatial prediction) 4 and the encoding efficiency.

Fourth, the prediction mode judging unit 36 evaluates all the "INTER prediction modes" shown in FIG. 3 in units of macroblocks, so as to select the "INTER prediction mode" that determines the highest coding efficiency.

Also, if the "INTRA prediction mode" is higher in encoding efficiency than the "INTER prediction mode", the prediction mode judging section 36 similarly evaluates the "INTRA prediction mode", and selects the "INTRA prediction mode".

Then, the prediction mode judging section 36 sends the predicted image signal (one macroblock unit) 8 generated by the selected prediction mode 4 to the subtracter 37 .

In addition, when the "INTER prediction mode" is selected as the prediction mode 4, the prediction mode judging section 36 sends the "displacement compensation related information" to the variable length encoding section 40, wherein the displacement compensation related information includes the selected "INTER prediction mode The number of displacement vectors 3 set in " (16 per macroblock), etc. On the other hand, when "INTRA prediction mode" is selected as the prediction mode 4, the prediction mode judging section 36 does not transmit the displacement vector 3.

Fifth, an orthogonal transform section 38 that generates an orthogonal transform coefficient by applying the orthogonal transform to the difference (a prediction residual signal 9) between the frame image signal 2 and the predicted image signal 8 from the subtractor 37 10.

Sixth, the quantization section 39 generates a quantized orthogonal transformation coefficient 11 by quantizing the orthogonal transformation coefficient 10 from the orthogonal transformation section 38 .

Seventh, the variable length encoding section 40 entropy-encodes the quantized orthogonal transform coefficient 11 from the quantization section 39 and the prediction mode 4 (and the displacement vector 3) from the prediction mode judging section 36 to multiplex them Used in compressed stream 12.

The variable-length encoding section 40 can transmit the compressed stream 12 to the moving image decoding device 50 in units of macroblocks, or transmit the compressed stream 12 in units of frame images.

Also, an inverse quantization section 41 generates an orthogonal transformation coefficient 13 by dequantizing the quantized orthogonal transformation coefficient 11 from the quantization section 39 . Then, an inverse orthogonal transform section 42 generates a prediction residual signal 14 by inversely orthogonally transforming the orthogonal transform coefficient 13 from the inverse quantization section 14 .

Next, in an adder, the prediction residual signal 14 from the inverse orthogonal transform section 42 and the predicted image signal 8 from the prediction mode judging section 36 are added together to generate a frame image signal 15 .

The frame image signal 15 in units of macroblocks is stored in the frame memory 34 . In the frame memory 34, the reference frame image 5 of the frame image unit used for subsequent encoding processing, and the information (pixel value or displacement vector) of the encoded macroblock of the frame image currently being encoded have been stored.

Next, the operation of the moving image decoding device 10 shown in FIG. 2 is described.

First, after receiving the compressed stream 12, the variable length decoding section 71 detects a sync word indicating the header of each frame, and restores displacement vector 3, prediction mode 4 and quantized orthogonal transform coefficient 11 for each macroblock unit.

Then, the variable-length decoding section 71 sends the quantized orthogonal transform coefficient 11 to the inverse quantization section 76, and sends the prediction mode to the switcher 75. In addition, when the prediction mode 4 is "INTER prediction mode", the variable length decoding part 71 sends the displacement vector 3 and the prediction mode 4 to the displacement compensation part 72; when the prediction mode 4 is "INTER prediction mode", the prediction mode 4 to the spatial prediction component 74.

Next, when the prediction mode 4 is "INTER prediction mode", the displacement compensating part 72 uses the displacement vector 3 and the prediction mode 4 from the variable length decoding part 71, refers to the reference frame image 5 from the frame memory 73, and generates a prediction Image signal6.

On the other hand, when the prediction mode 4 is "INTRA prediction mode", the spatial prediction section 74 refers to the coded image signal of the adjacent area from the frame memory 73, and generates a predicted image signal 7.

Next, the switcher 75 selects one of the predicted image signals 6 and 7 according to the prediction mode 4 from the variable length decoding section 71 , thereby determining the predicted image signal 8 .

At the same time, the quantized orthogonal transform coefficient 11 decoded by the variable length decoding section 71 is dequantized by the inverse quantization section 76 to be restored to the orthogonal transform coefficient 10 . Then, the orthogonal transform coefficient 10 is inversely orthogonally transformed by the inverse orthogonal transform unit 77 to be restored to the prediction residual signal 9 .

Then, at the adder 78, the predicted image signal 8 from the switcher 75 and the predicted residual signal 9 from the inverse orthogonal transform section 77 are added together, whereby the frame image signal 2 is restored to be sent to the output section 80. The output section 80 outputs the signal with a predetermined time to a display device, (not shown), whereby an output video signal (a kind of moving image) 1A is reproduced.

In addition, the restored frame image signal 2 is stored in the frame memory 73 so as to be used in subsequent decoding processing.

In "TML-8", displacement compensation using the concept of "specific position" is realized. Figure 4 shows this "specific position" along with integer image positions, 1/2 image positions and 1/4 image positions. Incidentally, in "TML-8", displacement compensation of 1/4 pixel precision is realized.

In Fig. 4, it is assumed that the displacement vector 3 detected by the displacement detection part 32 indicates an integer pixel position (pixel position of (1 pixel, 1 pixel)) "D" in a reference frame image 5, which is consistent with the frame image to be encoded The integer pixel position "A" in signal 2 is correlated. In this case, the pixel value of the pixel position "D" in the reference frame image 5 becomes a "displacement compensation value" which is related to the pixel position "A" in the frame image signal 2 to be encoded.

In the next step, assume that the displacement vector 3 indicates a 1/2 pixel position (the pixel position of (1/2 pixel, 1/2 pixel)) "E" in a reference frame image 5, which is identical to that in the frame image signal 2 to be encoded. The integer pixel position "A" is associated. In this case, an interpolated value is obtained by independently running a 6-stage filter (1, -5, 20, 20, -5, 1)/32 on pixel values vertically and horizontally at integer pixel positions in the reference frame image 5 , this interpolated value serves as a "displacement compensation value", which is related to the pixel position "A" in the frame image signal 2 to be encoded.

In the next step, assume that the displacement vector 3 indicates a 1/4 pixel position (the pixel position of (1/4 pixel, 1/4 pixel)) "F" or "G" in a reference frame image 5, which is consistent with the frame to be encoded Integer pixel positions "A" in image signal 2 are correlated. In this case, a linear interpolation value of a pixel value adjacent to an integer pixel position and a pixel value adjacent to a 1/2 pixel position is used as a "displacement compensation value", which is consistent with the pixel position "A" in the frame image signal 2 to be encoded " related.

For example, when the displacement vector 3 indicates the pixel position "F" in the reference frame image 5, which is related to the pixel position "A" in the frame image signal 2 to be encoded, the pixel values of the adjacent integer pixel positions and the adjacent 1/2 pixel The pixel value of the position, the average value of the pixel values of the four points surrounding the pixel position "F" is used as the "displacement compensation value", which is related to the pixel position "A" in the frame image signal 2 to be encoded.

In addition, when the displacement vector 3 indicates the pixel position "G" in the reference frame image 5, it is related to the integer pixel position A in the frame image signal 2 to be coded, sandwiching the 1/2 pixel position of the pixel position "G" horizontally The average value of the pixel values of the 2 points is used as the "displacement compensation value", which is related to the pixel position A in the frame image signal 2 to be encoded.

Further, when the displacement vector indicates the pixel position in the reference frame image 5 (N+3/4 pixel, M+3/4 pixel: N and M are given integers), it is consistent with the pixel position in the frame image signal 2 to be encoded The integer pixel position A is related, and the "displacement compensation value" related to the integer pixel position in the frame image signal 2 to be encoded becomes (N, M), (N, M+1), (N+1) in the reference frame image 5 , M) and (N+1, M+1) the average value of the pixel values. Here, (N+3/4 pixels, M+3/4 pixels: N and M are given integers) in the reference frame image 5 is the aforementioned "specific position".

For example, when the displacement vector 3 indicates the pixel position "H" in the reference frame image 5 (for example, a "special position"), it is related to the integer pixel position "A" in the frame image signal 2 to be coded, and is related to the pixel position "A" to be coded. The "displacement compensation value" related to the pixel position "A" in the encoded frame image signal 2 is not the value calculated in the case of the aforementioned 1/4 pixel position (for example, pixel position "F"), but is calculated by (A+B+ C+D)/4 calculated value.

As described above, in the "H.26L coding system", many "INTER prediction modes" are available for refined displacement compensation. In addition, displacement compensation based on integer pixel positions, 1/2 pixel positions, 1/4 pixel positions and specific positions are all available. With the foregoing configuration, when a prediction configuration is refined, a mechanism for preventing the breakage of the predicted image signal 8 is introduced, which works even if the frame image signal 2 whose prediction is not realized is input.

1/4 image precision calculations are performed by linear interpolation of pixel values at adjacent pixel locations. In this way, in the frequency domain space, a low-pass type operation is provided in order to generate a smooth predicted image signal 6 .

Additionally, when position-specific based displacement compensation is used, a "displacement compensation value" is calculated based on the average of 4 pixel values at adjacent integer pixel positions, thereby generating a further smoothed predicted image signal. If Gaussian noise is superimposed on the predicted image signal, this smoothing can achieve the effect of reducing prediction errors when the noise component becomes large.

In this way, in the "H.26L coding system" defined by "TML-8", if noise is superimposed on the reference frame image 5, or, if many high-pass components are included in the reference frame image 5, and an error in prediction is not possible Tolerant, which improves coding efficiency by using 1/4-pixel precision calculations and position-specific displacement compensation.

However, in the conventional "H.26L coding system", the following problems are conceivable to occur.

First, when the pixel position in the frame image signal 2 to be encoded has a displacement indicating a pixel position (N+3/4 pixel, M+3/4 pixel: N and M are given integers) equal to the "specific position" vector, the computed "displacement compensation value" is always strongly smoothed, and, in particular, there is the problem that refined displacement compensation is hindered at high rates (the first problem).

That is, in the conventional "H.26L coding system", a specific position is defined by the absolute value of the displacement vector 3 . In this way, as shown in FIG. 5, for example, when the blocks A, B, C, D, and E move in parallel to the lower right side (3/4 pixel, 3/4 pixel), based on the displacement vector MV=(MVx, MVy )=(3/4, 3/4) for smooth displacement compensation. Optionally, displacement compensation is performed by supplying a displacement vector different from a real displacement based on displacement vector MV=(MVx, MVy)=(1/2, 3/4) or (4/3, 1). Here, MVx indicates the X element of the displacement vector, and MVy indicates the Y element of the displacement vector.

In particular, as shown in FIG. 5, in the conventional "H.26L coding system", when the block to be coded is E, and the displacement vector MV of the block E is (MVxE, MVyE), one by "MVxE% 4=3" and "MVyE%4=3" compressed regions are always a "singular location", and the smoothed pixel value is selected as the "displacement compensation value" of block E. Here, "%" is a quotient remainder calculation symbol, and the unit expressing the displacement vector MV is 1/4 pixel.

Thus, in the "H.26L coding system", since the displacement vector (3/4, 3/4) indicates a smooth pixel value occurring in a real (1/2, 1/2) pixel position, there exists a "singular position" equal to The problem that the expression of the pixel value of the pixel position (N+3/4 pixel, M+3/4 pixel: N and M are given integers) is hindered.

Second, in generating 1/4 pixel precision predicted image signals, the effects of prediction refinement and prediction smoothing are expected at high and low rates, respectively. However, for predictive smoothing at low rates, displacement compensation with 1/4 pixel precision is unnecessary, while displacement compensation with 1/2 pixel precision is sufficient. Therefore, detection of a displacement vector with 1/4 pixel precision for smooth prediction, which occupies half of the parameter space of the displacement vector, is redundant, which is also a problem.

Therefore, the present invention is based on the aforementioned problems, and its aim is to express a predicted image signal with light overhead and to provide displacement compensation at different levels of pixel precision.

Contents of the invention

A first feature of the present invention is summarized as a moving picture encoding apparatus for encoding a moving picture composed of frame images time-series by displacement compensation. The moving image encoding device includes: a displacement vector detecting part for detecting a displacement vector of a predetermined region to be coded in a frame image; a predicting part for predicting the displacement vector of a predetermined region to be coded in a frame image The displacement vector of the coded predetermined area; a judging unit for judging whether the displacement vector detected by the displacement vector detection unit is a predetermined displacement vector set according to the displacement vector predicted by the prediction unit; and a switching unit based on the displacement vector Whether the displacement vector detected by the detection component is a predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined area to be encoded.

A second feature of the present invention is summarized as a moving image decoding device for decoding a moving image composed of frame image timing by displacement compensation. The moving image decoding device includes: a displacement vector decoding part for decoding a displacement vector of a predetermined region to be decoded in a frame image; a predicting part for decoding a displacement vector of a predetermined region decoded in a frame image Predicting the displacement vector of the predetermined area to be decoded; a judging unit for judging whether the displacement vector detected by the displacement vector detection unit is a predetermined displacement vector set according to the displacement vector predicted by the prediction unit; and a switching unit based on Whether the displacement vector decoded by the displacement vector decoding part is a predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined area to be decoded.

A third feature of the present invention is summarized as a moving picture encoding method for encoding a moving picture composed of frame images in time series by displacement compensation. The moving image encoding method includes: a step A for detecting the displacement vector of a predetermined area to be encoded in the frame image; a step B for predicting the area to be encoded by using the displacement vector of the encoded predetermined area in the frame image The displacement vector of the predetermined area; a step C, judging whether the displacement vector detected in the step A is a predetermined displacement vector set according to the displacement vector predicted in the step B; and a step D, based on the detected displacement vector in the step A Whether the displacement vector is a predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined area to be encoded.

A fourth feature of the present invention is summarized as a moving image decoding method for decoding a moving image composed of frame image timings through displacement compensation. The moving image decoding method includes: a step A for decoding the displacement vector of a predetermined region to be decoded in the frame image; a step B for predicting the displacement vector of the predetermined region to be decoded in the frame image The displacement vector of the decoded predetermined area; a step C, judging whether the decoded displacement vector in step A is a predetermined displacement vector set according to the predicted displacement vector in step B; and a step D, based on the decoded displacement vector in step A Whether the passed displacement vector is a predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined area to be decoded.

A fifth feature of the present invention is summarized as a program that causes a computer to function as a moving image encoding device to encode moving images composed of frame image time series by displacement compensation. The moving image encoding device includes: a displacement vector detecting part for detecting a displacement vector of a predetermined region to be coded in a frame image; a predicting part for predicting a displacement vector of a predetermined region to be coded in a frame image The displacement vector of the predetermined area; a judging part, which is used to judge whether the displacement vector detected by the displacement vector detection part is a predetermined displacement vector set according to the displacement vector predicted by the prediction part; and a switching part, which is detected according to the displacement vector detection part Whether the displacement vector is the predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined region to be encoded.

A sixth feature of the present invention is summarized as a program that causes a computer to function as a moving image decoding device to decode a moving image composed of frame image timing through displacement compensation. The moving image decoding device includes: a displacement vector decoding part for decoding a displacement vector of a predetermined region to be decoded in a frame image; a predicting part for predicting the displacement vector of a predetermined region to be decoded in the frame image The displacement vector of the predetermined area decoded; a judging unit for judging whether the displacement vector decoded by the displacement vector decoding unit is a predetermined displacement vector set according to the displacement vector predicted by the prediction unit; and a switching unit for decoding the unit according to the displacement vector Whether the decoded displacement vector is the predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined area to be decoded.

A seventh feature according to the present invention is summarized as a computer-readable recording medium storing a program for causing a computer to function as a moving image encoding device to encode moving images composed of frame image timing by displacement compensation. The moving image encoding device includes: a displacement vector detecting part for detecting a displacement vector of a predetermined region to be coded in a frame image; a predicting part for predicting a displacement vector of a predetermined region to be coded in a frame image The displacement vector of the predetermined area; a judging part, which is used to judge whether the displacement vector detected by the displacement vector detection part is a predetermined displacement vector set according to the displacement vector predicted by the prediction part; and a switching part, which is detected according to the displacement vector detection part Whether the displacement vector is the predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined region to be encoded.

In the seventh feature of the present invention, the predetermined displacement vector is preferably set to a value different from the displacement vector predicted by the prediction means.

Furthermore, in the seventh feature of the present invention, when the difference information between the displacement vector predicted by the predicting means and the displacement vector detected by the displacement vector detecting means is a predetermined value, the judging means preferably judges the displacement detected by the displacement vector detecting means vector is the predetermined displacement vector.

An eighth feature of the present invention is summarized as a computer-readable recording medium storing a program for causing a computer to function as a moving image decoding device to decode moving images composed of frame image timing by displacement compensation. The moving image decoding device includes: a displacement vector decoding part for decoding a displacement vector of a predetermined region to be decoded in a frame image; a predicting part for predicting the displacement vector of a predetermined region to be decoded in the frame image The displacement vector of the predetermined area decoded; a judging unit for judging whether the displacement vector decoded by the displacement vector decoding unit is a predetermined displacement vector set according to the displacement vector predicted by the prediction unit; and a switching unit for decoding the unit according to the displacement vector Whether the decoded displacement vector is the predetermined displacement vector is used to switch the calculation method of the displacement compensation value of the predetermined region to be decoded.

In the eighth feature of the present invention, the predetermined displacement vector is preferably set to a value different from the displacement vector predicted by the prediction means.

Furthermore, in the eighth feature of the present invention, when the difference information between the displacement vector predicted by the predicting section and the displacement vector decoded by the displacement vector decoding section is a predetermined value, the judging section preferably judges the displacement vector decoded by the displacement vector decoding section. vector is the predetermined displacement vector.

A ninth feature of the present invention is summarized as a moving image coding apparatus that encodes a moving image sequentially composed of frame images by displacement compensation. The mobile image encoding device includes: a reference image generating part for generating several different reference images by performing several different image processes on a reference frame image; a displacement compensating part for using the generated reference image to generate A displacement compensation value is calculated for a predetermined area; and a sending unit is used for sending a combination of reference image information used for calculating the displacement compensation value and information indicating the displacement compensation value. The reference image information is a combination of reference frame image identification information and information indicating image processing.

A tenth feature of the present invention is summarized as a moving image coding apparatus that encodes a moving image sequentially composed of frame images by displacement compensation. The mobile image encoding device includes: a reference image generation unit for generating a reference image subjected to predetermined image processing from a reference frame image according to a predetermined area encoding condition to be encoded; A reference image for image processing to calculate a displacement compensation value for a predetermined area to be encoded.

An eleventh feature of the present invention is summarized as a moving image decoding device that decodes a moving image sequentially composed of frame images by displacement compensation. The moving image decoding device includes: a reference image generating part, which is used to generate several different reference images by performing several different image processing on the reference frame image; a decoding part, which is used to calculate the displacement compensation value in the moving image coding device the reference image information used for decoding; and a displacement compensating means for calculating a displacement compensation value of a predetermined area to be decoded by using a reference image specified by the generated reference image information. Reference image information is a combination of reference frame image identification information and image processing information.

A twelfth feature of the present invention is summarized as a moving image decoding device that decodes a moving image sequentially composed of frame images by displacement compensation. The moving image decoding device includes: a reference image generating part for generating a reference image subjected to predetermined image processing from a reference frame image according to encoding conditions of a predetermined region to be decoded; The reference image for image processing is used to calculate the displacement compensation value of the predetermined area to be decoded.

The thirteenth feature of the present invention is summarized as a moving picture encoding method that encodes a moving picture composed of frame images time-sequentially by displacement compensation. The moving image encoding method includes: a step A of generating several different reference images by performing several different image processes on the reference frame image; a step B of calculating displacement compensation of a predetermined area to be encoded by using the generated reference images value; and a step C of sending a combination of reference image information used to calculate the displacement compensation value and information indicating the displacement compensation value. The reference image information is a combination of identification information of a reference frame image and information indicating image processing.

The fourteenth feature of the present invention is summarized as a moving picture encoding method that encodes a moving picture composed of frame images in time series by displacement compensation. The moving image encoding method includes: a step A of generating a reference image that has undergone predetermined image processing from a reference frame image according to the encoding conditions of a predetermined region to be encoded; and a step B of using the generated reference image that has undergone predetermined image processing The reference image is used to calculate the displacement compensation value of the predetermined area to be encoded.

A fifteenth feature of the present invention is summarized as a moving picture decoding method that decodes a moving picture sequentially composed of frame pictures by displacement compensation. The moving image decoding method includes: a step A for generating several different reference images by performing several different image processing on the reference frame image; a step B for using Decoding the reference image information; and a step C of calculating the displacement compensation value of the predetermined area to be decoded by using the reference image specified by the generated reference image information. Reference image information is a combination of reference frame image identification information and image processing information.

The sixteenth feature of the present invention is summarized as a moving picture decoding method which decodes a moving picture composed of frame picture time series by displacement compensation. The moving image decoding method includes: a step A of generating a reference image subjected to predetermined image processing from a reference frame image according to encoding conditions of a predetermined region to be decoded; and a step B of generating a reference image subjected to predetermined image processing by using the generated The reference image is used to calculate the displacement compensation value of the predetermined area to be decoded.

A seventeenth feature of the present invention is summarized as a program that causes a computer to function as a moving image encoding device to encode moving images sequentially composed of frame images by displacement compensation. The mobile image encoding device includes: a reference image generating part for generating several different reference images by performing several different image processes on a reference frame image; a displacement compensating part for using the generated reference image to generate calculating a displacement compensation value in a predetermined area; and a sending component, configured to send a combination of reference image information used for calculating the displacement compensation value and information indicating the displacement compensation value. The reference image information is a combination of reference frame image identification information and information indicating image processing.

An eighteenth feature of the present invention is summarized as a program that causes a computer to function as a moving image encoding device to encode moving images sequentially composed of frame images by displacement compensation. The mobile image encoding device includes: a reference image generating section for generating a reference image subjected to predetermined image processing from a reference frame image according to encoding conditions of a predetermined area to be encoded; and a displacement compensating section for A reference image for predetermined image processing, and a displacement compensation value is calculated for a predetermined region to be coded.

A nineteenth feature of the present invention is summarized as a program that causes a computer to function as a moving image decoding device that decodes moving images composed of frame image timing by displacement compensation. The moving image decoding device includes: a reference image generating part, which is used to generate several different reference images by performing several different image processing on the reference frame image; a decoding part, which is used to calculate the displacement compensation value in the moving image coding device the reference image information used for decoding; and a displacement compensating means for calculating a displacement compensation value of a predetermined area to be decoded by using a reference image specified by the generated reference image information. Reference image information is a combination of reference frame image identification information and image processing information.

A twentieth feature of the present invention is summarized as a program that causes a computer to function as a moving image decoding device that decodes moving images composed of frame image timing through displacement compensation. The moving image decoding device includes: a reference image generating part for generating a reference image subjected to predetermined image processing from a reference frame image according to encoding conditions of a predetermined region to be decoded; The reference image for image processing is used to calculate the displacement compensation value of the predetermined area to be decoded.

A twenty-first feature of the present invention is summarized as a computer-readable recording medium storing a program for causing a computer to function as a moving image encoding device to encode moving images composed of frame image time series by displacement compensation. The mobile image encoding device includes: a reference image generating part for generating several different reference images by performing several different image processes on a reference frame image; a displacement compensating part for using the generated reference image to generate calculating a displacement compensation value in a predetermined area; and a sending component, configured to send a combination of reference image information used for calculating the displacement compensation value and information indicating the displacement compensation value. The reference image information is a combination of reference frame image identification information and information indicating image processing.

In the twenty-first feature of the present invention, preferably, the displacement compensating means switches the reference image used to calculate the displacement compensation value in units of detected displacement vectors, and the transmitting means transmits the reference image information and the indicated displacement in units of detected displacement vectors A combination of compensation value information.

In the twenty-first feature of the present invention, preferably, the reference image information is a combination of identification information indicating the detection displacement vector unit and image processing information, and the sending unit is used to send the reference image information, reference frame image A combination of identification information and displacement compensation value information indicating each predetermined area to be encoded.

In addition, in the twenty-first feature of the present invention, preferably, the image processing is a process of changing the spatial resolution, and, when a reference image of low spatial resolution is used, the displacement compensating means is used to reduce the calculation of the displacement vector The displacement vector precision to use.

In addition, in the twenty-first feature of the present invention, preferably, the reference image information dynamically changes the combination of the reference frame image identification information and the instruction image processing information according to the encoding conditions of the predetermined area to be encoded.

Further, in the twenty-first feature of the present invention, preferably, according to the encoding conditions of the predetermined area to be encoded, the reference image information dynamically changes the combination of the identification information indicating the detected displacement vector unit and the image processing information.

A twenty-second feature of the present invention is summarized as a computer-readable recording medium storing a program for causing a computer to function as a moving image encoding device to encode moving images composed of frame image timing by displacement compensation. The mobile image encoding device includes: a reference image generation unit for generating a reference image subjected to predetermined image processing from a reference frame image according to a predetermined area encoding condition to be encoded; A reference image for image processing to calculate a displacement compensation value for a predetermined area to be encoded.

In the twenty-second feature of the present invention, preferably, the reference image generating means is configured to generate a reference image subjected to predetermined image processing according to a type of the detection displacement vector unit.

In addition, in the twenty-second feature of the present invention, preferably, according to the quantization step, the reference image generating means is configured to generate a reference image subjected to predetermined image processing.

A twenty-third feature of the present invention is summarized as a computer-readable recording medium storing a program for causing a computer to function as a moving image decoding device to decode moving images composed of frame image timing by displacement compensation. The moving image decoding device includes: a reference image generating part, which is used to generate several different reference images by performing several different image processing on the reference frame image; a decoding part, which is used to calculate the displacement compensation value in the moving image coding device the reference image information used for decoding; and a displacement compensating means for calculating a displacement compensation value of a predetermined area to be decoded by using a reference image specified by the generated reference image information. Reference image information is a combination of reference frame image identification information and image processing information.

In the twenty-third feature of the present invention, preferably, the decoding means decodes the reference image information and the information indicating the displacement compensation value in units of detection displacement vectors, and the displacement compensation means switches the calculation of displacement compensation in units of detection displacement vectors The reference image to use for the value.

In the twenty-third feature of the present invention, preferably, the reference image information is a combination of identification information indicating the unit of the detected displacement vector and information indicating image processing, and the decoding means is configured to pair The reference image information, the reference frame image identification information and the information indicating the displacement compensation value are decoded. In addition, preferably, the displacement compensation component is used to calculate the displacement compensation value of the predetermined area to be decoded by using the generated reference image information and the reference image specified by the reference frame image identification information.

Also, in the twenty-third feature of the present invention, preferably, the image processing is a process of changing the spatial resolution, and, when a reference image of low spatial resolution is used, the displacement compensating means is used to reduce the calculation displacement compensation The displacement vector precision to use for the value.

In addition, in the twenty-third feature of the present invention, preferably, the reference image information dynamically changes a combination of identification information of a reference frame image and information indicating image processing according to encoding conditions of a predetermined area to be decoded.

Further, in the twenty-third feature of the present invention, preferably, according to the coding conditions of the predetermined area to be decoded, the reference image information dynamically changes the combination of the identification information indicating the detected displacement vector unit and the image processing information.

A twenty-fourth feature of the present invention is summarized as a computer-readable recording medium storing a program for causing a computer to function as a moving image decoding device to decode moving images composed of frame image timing by displacement compensation. The moving image decoding device includes: a reference image generating part for generating a reference image subjected to predetermined image processing from a reference frame image according to encoding conditions of a predetermined region to be decoded; The reference image for image processing is used to calculate the displacement compensation value of the predetermined area to be decoded.

In addition, in the twenty-fourth feature of the present invention, preferably, the reference image generating means is configured to generate a reference image subjected to predetermined image processing according to a type of the detection displacement vector unit.

Further, in the twenty-fourth feature of the present invention, preferably, according to the quantization step, the reference image generation means is for generating a reference image subjected to predetermined image processing.

A twenty-fifth feature of the present invention is a moving image encoding device that encodes a moving image sequentially composed of frame images by displacement compensation. The mobile image encoding device includes: a reference image generating part for generating several different reference images by performing several different image processes on a reference frame image; a 3-dimensional displacement vector generating part for detecting The displacement vector is associated with information indicating the image processing performed for the reference image, and is used to generate a 3-dimensional displacement vector; a displacement compensation component calculates a displacement compensation value for a predetermined area to be encoded by using the generated reference image; And a sending part, used for sending the combination of the 3-dimensional displacement vector and the information indicating the displacement compensation value.

A twenty-sixth feature of the present invention is a moving image decoding device for decoding a moving image composed of frame images in time series by displacement compensation. The mobile image decoding device includes: a reference image generating unit for generating several different reference images by performing several different image processing on a reference frame image; a decoding unit for converting the 3D The displacement vector is decoded; and a displacement compensating part calculates a displacement compensation value for the predetermined area to be decoded by using the reference image specified by the generated 3-dimensional displacement vector.

A twenty-seventh feature of the present invention is a moving image encoding method for encoding a moving image sequentially composed of frame images by displacement compensation. The moving image coding method comprises: a step A of generating several different reference images by performing several different image processes on the reference frame image; a step B of performing The image processing information is associated to generate a 3-dimensional displacement vector; a step C is used to calculate the displacement compensation value of the predetermined area to be encoded by using the generated reference image; and a step D is used to send the 3-dimensional displacement Combination of a vector and information indicating the displacement compensation value.

A twenty-eighth feature of the present invention is a moving image decoding method for decoding a moving image composed of frame images time-series by displacement compensation. The moving image decoding method includes: a step A for generating several different reference images by performing several different image processes on a reference frame image; a step B for converting the 3-dimensional displacement vector of the predetermined area to be decoded performing decoding; and a step C of calculating a displacement compensation value for the predetermined area to be decoded by using the reference image specified by the generated 3-dimensional displacement vector.

A twenty-ninth feature of the present invention is a program that causes a computer to function as a moving image encoding device that encodes a moving image composed of frame image time series by displacement compensation. The mobile image encoding device includes: a reference image generating part for generating several different reference images by performing several different image processes on a reference frame image; a 3-dimensional displacement vector generating part for detecting The displacement vector is associated with information indicating the image processing performed for the reference image, and is used to generate a 3-dimensional displacement vector; a displacement compensation component calculates a displacement compensation value for a predetermined area to be encoded by using the generated reference image; And a sending part, used for sending the combination of the 3-dimensional displacement vector and the information indicating the displacement compensation value.

A thirtieth feature of the present invention is a program that causes a computer to function as a moving image decoding device that decodes a moving image composed of frame image timing by displacement compensation. The mobile image decoding device includes: a reference image generating unit for generating several different reference images by performing several different image processing on a reference frame image; a decoding unit for converting the 3D The displacement vector is decoded; and a displacement compensating part calculates a displacement compensation value for the predetermined area to be decoded by using the reference image specified by the generated 3-dimensional displacement vector.

A thirty-first feature of the present invention is a computer-readable recording medium storing a program for causing a computer to function as a moving image encoding device to encode moving images composed of frame image time series by displacement compensation. The mobile image encoding device includes: a reference image generating part for generating several different reference images by performing several different image processes on a reference frame image; a 3-dimensional displacement vector generating part for detecting The displacement vector is associated with information indicating the image processing performed for the reference image, and is used to generate a 3-dimensional displacement vector; a displacement compensation component calculates a displacement compensation value for a predetermined area to be encoded by using the generated reference image; And a sending part, used for sending the combination of the 3-dimensional displacement vector and the information indicating the displacement compensation value.

In the thirty-first feature of the present invention, preferably, the reference image generating section performs filtering processing for generating several different reference images by using filters having several different passbands, and the 3-dimensional displacement vector identification filter device.

In addition, in the thirty-first feature of the present invention, preferably, the 3-dimensional displacement vector predicting means is used to predict the 3-dimensional displacement vector by using the correlation of the coded predetermined region in the frame image and the predetermined region to be coded , and the sending unit is configured to send the combination of the difference information between the 3-dimensional displacement vector generated by the 3-dimensional displacement vector generation unit and the 3-dimensional displacement vector predicted by the 3-dimensional displacement vector prediction unit and the information indicating the displacement compensation value.

Also, in the thirty-first feature of the present invention, preferably, the 3-dimensional displacement vector predicting means predicts the 3-dimensional displacement vector by switching the context in the encoding algorithm.

Further, in the thirty-first feature of the present invention, preferably, the image processing is processing of changing the spatial resolution, and the 3-dimensional displacement vector generating means is used to reduce the 3-dimensional displacement vector of the low spatial resolution reference image. precision.

A thirty-second feature of the present invention is a computer-readable recording medium storing a program for causing a computer to function as a moving image decoding device to decode a moving image composed of frame image timing by displacement compensation. The mobile image decoding device includes: a reference image generating unit for generating several different reference images by performing several different image processing on a reference frame image; a decoding unit for converting the 3D The displacement vector is decoded; and a displacement compensating part calculates a displacement compensation value for the predetermined area to be decoded by using the reference image specified by the generated 3-dimensional displacement vector.

In the thirty-second feature of the present invention, preferably, the reference image generating section performs filtering processing for generating several different reference images by using filters having several different passbands, and the 3-dimensional displacement vector identification filter device.

In addition, in the thirty-second feature of the present invention, preferably, the 3-dimensional displacement vector predicting means is used to predict the 3-dimensional displacement vector by using the correlation of the decoded predetermined region and the predetermined region to be decoded in the frame image , and the displacement compensating part is used to calculate the displacement compensation value of the predetermined region to be decoded by using the difference information between the 3-dimensional displacement vector decoded by the decoding part and the 3-dimensional displacement vector predicted by the 3-dimensional displacement vector prediction part.

Further, in the thirty-second feature of the present invention, preferably, the 3-dimensional displacement vector predicting means predicts the 3-dimensional displacement vector by switching the context in the encoding algorithm.

Description of drawings

FIG. 1 shows a schematic diagram of a mobile image encoding device according to the conventional art;

FIG. 2 shows a schematic diagram of a mobile image decoding device according to conventional techniques;

Fig. 3 shows the division mode of macroblocks in the INTER prediction mode according to the conventional technology;

Figure 4 shows the concept of a specific location according to conventional techniques;

FIG. 5 shows a method for calculating a predicted displacement vector in a moving image encoding device according to an embodiment of the present invention;

FIG. 6 shows a functional block diagram of a displacement compensation component of a mobile image encoding device according to an embodiment of the present invention;

Fig. 7 shows the concept of judging "specific position" in the mobile image encoding device according to an embodiment of the present invention;

FIG. 8 shows an operation flowchart of the displacement compensation part of the moving image encoding device according to an embodiment of the present invention;

FIG. 9 shows a flowchart of decoding processing in a mobile image decoding device according to an embodiment of the present invention;

FIG. 10 shows the concept of judging a "special position" in a moving image encoding device according to an improved embodiment of the present invention;

Fig. 11 shows the concept of judging "specific position" in the mobile image encoding device according to the improved embodiment of the present invention;

FIG. 12 shows a schematic diagram of a mobile image encoding device according to an embodiment of the present invention;

FIG. 13 shows a schematic diagram of a mobile image decoding device according to an embodiment of the present invention;

Fig. 14 shows the coding syntax in the macroblock unit in the H.26L coding system used in the embodiment of the present invention;

Fig. 15 has shown the example of the reference frame code table used in the embodiment of this invention;

Figure 16 shows a part of the reference frame code table used in this embodiment of the invention;

Fig. 17 shows the coding syntax in the macroblock unit in the H.26L coding system used in the improved embodiment of the present invention;

Fig. 18 has shown the macroblock mode code table used in this improved embodiment of the invention;

FIG. 19 shows a schematic diagram of a mobile image encoding device according to an embodiment of the present invention;

FIG. 20 shows a schematic diagram of a mobile image decoding device according to an embodiment of the present invention;

FIG. 21 is a layered reference image for explaining an embodiment of the invention;

FIG. 22 is used to explain a method of generating a layered reference image according to an embodiment of the invention;

FIG. 23 is a diagram showing a method of calculating a predicted displacement vector in a moving image encoding device according to an embodiment of the present invention;

FIG. 24 shows a flow chart of a displacement compensation operation in a moving image encoding device according to an embodiment of the present invention;

Fig. 25 has explained the method for generating hierarchical reference image according to the improved embodiment of the present invention;

FIG. 26 shows a computer-readable recording medium storing a program for causing a computer to function as a moving image encoding device or a moving image decoding device of the embodiment of the invention.

Detailed ways

(Example 1)

In the first embodiment of the present invention, a moving image encoding device 20 and a moving image decoding device 50 are described, in which an improvement is introduced in respect of "singular position" displacement compensation (the first problem), where the problem is that the conventional "TML-8 "Define the "H.26L coding system" problem.

According to this embodiment, other operations are similar to those of the moving image coding device and the moving image decoding device described in "TML-8" except for the displacement compensation in "Specific Position". As such, specific details will be omitted and the description will focus on the differences.

In particular, the embodiment of the present invention differs from the configurations of the moving image encoding device 20 and the moving image decoding device 50 of the conventional embodiment in the different configurations of the displacement compensating parts 33 and 72 of the present invention and the conventional art.

In the embodiment of the present invention, since the displacement compensating part 33 of the moving image encoding device 20 and the displacement compensating part 72 of the moving image encoding device 50 have the same configuration. Thus, the displacement compensating section 33 of the moving image encoding device 20 will be described hereinafter.

Incidentally, the moving image encoding device 20 of this embodiment is for encoding a moving image (an input video signal 1) composed of frame image timings based on displacement compensation. The moving picture decoding device 50 of this embodiment is for decoding a moving picture (an output video signal 1A) composed of frame picture timings based on displacement compensation.

In addition, in the moving image coding apparatus 20 of this embodiment, the displacement detecting section 32 constitutes a displacement vector detecting section for detecting a displacement vector 3 of a predetermined area (for example, a macroblock) to be coded in a frame image. In the moving image decoding device 50 of this embodiment, the variable length decoding section 71 constitutes a displacement vector decoding section for decoding the displacement vector 3 of a predetermined area to be decoded (for example, a macroblock) in a frame image.

As shown in FIG. 6, the displacement compensating section 33 of the moving image coding apparatus 20 of this embodiment includes a displacement vector input section 33a, a reference frame image input section 33b, a predicted displacement vector calculation section 33c, a judging section 33d, and a Predicted image signal generating section 33e.

According to this embodiment, the prediction displacement vector calculation section 33c constitutes a prediction section by using the displacement vectors (for example, MV _A =(MVx _A , MVy _A ), MV _B =(MVx _B , MVy _B ), _MVC =(MVx _C , MVy _C )), used to predict the prediction displacement vector PMV _E =(PMVx _E , PMVy _E ) of a predetermined area to be coded (one macroblock E).

In addition, the judging part 33d constitutes a judging part for judging whether the displacement vector _MVE = ( _MVxE , _MVyE ) detected by the displacement vector detection part (displacement detection part 32) is based on the prediction part (predicted displacement vector calculation part 33c ) Predicted displacement vector MPV _E = (PMVx _E , PMVy _E ) A predicted displacement vector (displacement vector indicating a "specific position") is set.

Further, the predicted image signal generation part 33e constitutes a switching part, which detects whether the displacement vector _MVE = ( _MVxE , _MVyE ) detected by the displacement vector detection part (displacement detection part 32) is a predetermined displacement vector (indicating "specific position ” for switching the method of calculating the “displacement compensation value” of the predetermined area to be coded (the method of generating the predicted image signal 6 ).

The displacement vector input part 33a is connected to the judging part 33d for receiving the displacement vector _MVE detected by the displacement detecting part 32 so as to send the displacement vector _MVE to the judging part 33d.

The reference frame image input part 33b is connected to the predicted displacement vector calculation part 33c and the judgment part 33d, and is used to extract the displacement vectors of the adjacent regions (macroblocks A, B and C) of the predetermined region to be encoded, which are stored in the frame memory 34 , so that these vectors are sent to the predicted displacement vector calculation section 33c. Likewise, the reference frame image input part 33 is used to extract the reference frame image 5 stored in the frame memory 34, so as to send the reference frame image 5 to the judging part 33d.

The predicted displacement vector calculating section 33c is connected to the reference frame image input section 33b and the judging section 33d, for example, by using displacement vectors (encoded in the frame image) of adjacent areas (macroblocks A, B, C) of the predetermined area to be encoded. The displacement vectors of the predetermined area) _MVA , MV _B and _MVC , which are stored in the frame memory 34, are used to calculate the predicted displacement vector PMV _E = (PMVx _E , PMVy _E ), which is the predetermined area (macroblock) to be encoded ) displacement vector _MVE = (MVx _E , MVy _E ) prediction value.

Here, PMVx _E indicates a horizontal element (one X element) of the predicted displacement vector, and PMVy _E indicates a vertical element (one Y element) of the predicted displacement vector.

According to "TML-8", in order to encode the displacement vector efficiently, the displacement vector of the predetermined area to be encoded is predicted, and the encoding process included in the reference frame image 5 is used by a prediction system called "median prediction". The displacement vectors of the neighboring regions are encoded.

In Fig. 5, due to the displacement vectors MV of adjacent areas (macroblocks) A, B and C: MV _A =(MVx _A , MVy _A ), MV _B =(MVx _B , MVy _B ) and MV _C =(MVx _C , MVy _C ) has been encoded, the average value of the displacement vector horizontal elements MVx _A , MVx _B and MVx _C can be obtained, which is set as the horizontal element PMVx _E of the prediction displacement vector of the predetermined area (macroblock) E to be encoded, And the average value of vertical elements MVy _A , MVy _B and MVy _C of the displacement vectors is also obtained, which is set as the vertical element PMVy _E of the prediction displacement vector of the predetermined area (macroblock) E to be coded.

For example, in a case of a prediction mode (one coding mode), in which the adjacent regions (macroblocks) A, B, and C used for calculating the predicted displacement vector PMV are outside the frame image or have no displacement vector, the predicted displacement vector PMV is set to 0 vector.

Further, when there are no 3 or more displacement vectors among the adjacent areas (macroblocks) A, B, and C used for calculating the predicted displacement vector PMV of the predetermined area (macroblock) E to be encoded, by using the assumed adjacent area ( The displacement vectors of the macroblocks) A, B, and C are set to 0 vectors or other assumptions, the value of the predicted displacement vector PMV _E of the predetermined area (macroblock) E to be coded can always be obtained.

In addition, the predicted displacement vector calculation section 33c calculates difference information MVD=(MVDx, MVDy) between the displacement vector MV _E from the displacement vector input section 33a and the predicted displacement vector PMV _E. Here, the X element MVDx of the difference information is obtained by calculating "MVx _E -PMVx _E ", and the Y element MVDy of the difference information is obtained by calculating "MVy _E -PMVy _E ".

In the "H.26L coding system", in order to improve the transmission efficiency, the displacement vector MV is coded and transmitted in the form of the aforementioned difference information MVD.

The judgment section 33d is connected to a displacement vector input section 33a, a reference frame image input section 33b, a predicted displacement vector calculation section 33c, and a predicted image signal generation section 33e. The judging part 33d is used for judging the method of generating the predicted image signal 6 (a method of calculating a displacement compensation value) based on the displacement vector _MVE from the displacement vector input part 33a and the predicted displacement vector PMV _E from the predicted displacement vector calculation part 33c, The method is thus put into the predicted image signal generating section 33e.

Specifically, as shown _in FIG. 7, the judging part 33d judges whether the displacement vector _{MVE is} A "singular position" is indicated, and the judgment result is put into the predicted image signal generating section 33e.

Hereinafter, the unit of expressions of the predicted displacement vector PMV _E and the displacement vector MV _E is 1/4 pixel.

First, in the case of " _PMVyE %4=0 or 1" (for example, _PMVyE is in the first phase), " _MVxE %4=3" and " _MVyE %4=3", the judging part 33d It is judged that the displacement vector _MVE (MVx _E , MVy _E ) of the predetermined area (macroblock E) to be coded indicates a "special position".

Second, in the case of "PMVy _E % 4 = 2 or 3" (for example, PMVy _E is in the second phase), "MVx _E % 4 = 1" and "MVy _E % 4 = 1", the judging part 33d judges that the displacement vector _MVE (MVx _E , MVy _E ) of the predetermined area to be coded (macroblock E) indicates the "special position".

As a result, the area indicated by the predicted displacement vector PMV _E is adjusted so as not to be superimposed on this "specific position". That is, the predicted displacement vector _PMVE predicted by the predicted displacement vector calculation section 33c (prediction section) is set to be different from the displacement vector (predetermined displacement vector) indicating the "specific position".

In the case of PMVy _E in the first phase, in practice, the displacement compensation of the conventional "H.26L coding system" is applied (for example, the displacement vector indicating the pixel position a (see FIG. 7 ) indicates the "singular position").

In addition, in case PMVy _E is in the second phase, the displacement vectors indicating different pixel positions b (see Fig. 7) are assumed to indicate "specific positions".

However, the obtained "displacement compensation value" is smoothed as the average value of the pixel values of 4 adjacent integer pixel positions around the displacement vector, and other displacement compensation processing is similar to that of the conventional "H.26L coding system" .

The predicted image signal generating section 33e is connected to the judging section 33d for generating the predicted image signal 6 by switching the "predicted image signal 6 generation method" of a predetermined region (macroblock) to be coded.

In particular, in other cases, when it is judged that the displacement vector MVE of a predetermined area (macroblock _E ) to be encoded detected by the displacement detecting section 32 indicates a "special position", the predicted image signal generating section 33e smoothes the prediction of the predetermined area The image signal 6 is taken as the average value of the pixel values of 4 adjacent integer pixel positions around the pixel position indicated by the displacement vector _MVE , and the predicted image signal 6 of the predetermined area is generated through the traditional "H.26L coding system".

In addition, in this embodiment, the displacement compensating component 33 and the displacement detecting component 32 are separately arranged. However, the displacement compensating part 33 and the displacement detecting part 32 may be integrally arranged.

FIG. 8 shows the operation of the aforementioned displacement compensating part 33 .

In step 401, the predicted displacement vector calculation unit 33c calculates the predicted displacement vector based on the coded displacement vectors MVA, MV _B and MV _C of adjacent regions (macroblocks _A , B and C in FIG. 5) in the same frame image PMV _E , which is a predictive value of the displacement vector of the predetermined area to be encoded (macroblock E in FIG. 5 ) in the frame image.

_{In step 402, the judging part 33d judges whether the displacement vector MV E indicates a} " unique location".

In step 403, when the aforementioned displacement vector _MVE indicates a "specific position", the predicted image signal generation unit 33e generates a predicted image signal 6 of a predetermined area in a smooth form.

In step 404, when the aforementioned displacement vector _MVE does not indicate a "specific position", the predictive image signal generating unit 33e generates a predictive image signal 6 of a predetermined area through the conventional "H.26L coding system".

Next, the decoding processing steps in the moving image decoding device 50 are described below with reference to FIG. 9 .

In step 501, the variable-length decoding section 71 detects a "sync word" indicating the header of a picture (each frame of image constituting the input video signal 1).

In step 502, the variable length decoding section 71 decodes the "picture header" of the aforementioned picture. The "picture header" includes "picture type information", and the "picture type information" is used to judge that the picture is a picture (hereinafter referred to as "I picture") in which "all the macroblocks composing the picture are coded through the INTRA prediction mode." " is also a "picture using INTER prediction mode (hereinafter referred to as "P picture")". Also, the picture header includes a quantization parameter value and the like among orthogonal transform coefficients.

Next, data decoding processing for each macroblock layer composed of predetermined syntax is performed.

In step 503, the variable length decoding section 71 decodes "RUN" in the macroblock layer. This "RUN" indicates the number of repeated macroblocks for which the macroblock layer data is 0, and the same number of macroblocks (jump MBs) as the number of "RUN" to which the skip mode is applied are generated.

In step 505, if the macroblock is a skipped MB, actually, a 16×16 pixel area at the same position on the predetermined reference frame image 5 stored in the frame memory 73 is used for the prediction image signal 6. This processing is performed by the variable length decoding section 72 sending to the displacement compensating section 72 a displacement vector whose value is 0 and an identification number of a predetermined reference frame image.

In step 506, if the macroblock is not a skipped MB, it is determined whether the "RUN" of the MB indicates the last MB of the picture.

In step 507, if the "RUN" of the MB is the last MB, the variable length decoding of the picture is terminated, and the variable length decoding of the next picture starts.

In step 508, if the "RUN" is neither a jump MB nor the last MB, for example, if the "RUN" is a normal MB, the variable length decoding section 71 decodes "MB_Type" (a macroblock type) . With this "MB_Type", prediction mode 4 of a predetermined area (macroblock) to be decoded is established. In step 509, it is judged whether the established prediction mode 4 is "INTER prediction mode".

In step 510, if the prediction mode 4 is "INTRA prediction mode", the variable length decoding section 71 decodes "intra_pred_mode". In step 511 , the spatial prediction section 74 performs spatial prediction from the pixel values of the adjacent area based on "intra_pred_mode", thereby generating the predicted image signal 7 .

If the prediction mode 4 is "INTER prediction mode", the prediction mode 4 is one of modes 1 to 7 shown in FIG. 3 . Thus, at this time, the number of "Ref_frames (number of reference frame images)" and "MVDs (difference information of displacement vectors)" to be decoded are established. Based on such information, the variable length decoding section 71 decodes a combination of "Ref_frame" and "MVD".

However, in step 512, since the judgment of whether "Ref_frame" has been multiplexed is integrated in the aforementioned "picture type information", it is necessary to judge whether "Ref_frame" exists according to the value of "picture type information".

In step 513, if "Ref_frame" exists, the variable length decoding section 71 decodes the "Ref_frame", and then, in step 514, the variable length decoding section 71 decodes "MVD". If "Ref_frame" does not exist, only "MVD" is decoded in step 514 .

In step 514, based on the prediction mode 4 established by the obtained "Ref_frame", "MVD" and "MB_Type", displacement vectors MV corresponding to all 4x4 blocks in the MB are recovered.

In step 515, the displacement compensating section 72 generates a predicted image signal 6 for each 4×4 block based on the “Ref_frame” and the displacement vector MV. The treatment of "specific locations" is reflected here.

In step 516 , the variable length decoding section 71 restores the quantized orthogonal transform coefficient 11 . In step 517 , the inverse quantization section 76 restores the orthogonal transform coefficient 10 . In step 518 , the inverse orthogonal transform section 77 restores the prediction residual signal 9 .

In step 519, at the adder 78, the predicted image signal 8 from the switcher 75 and the predicted residual signal 9 from the inverse orthogonal transform section 77 are added to obtain the frame image signal 2 of the MB. Then, decoding processing of the next MB is performed.

(Operation/Effect of Moving Image Coding and Decoding Device According to Embodiment 1)

According to the moving image coding apparatus of this embodiment, the predicted image signal generation section 33e switches the calculation method of the displacement compensation value of the predetermined area to be encoded according to the judgment result of the judgment section 33d, for example, the predicted displacement vector predicted by the predicted displacement vector calculation section 33c. For this reason, it is possible to represent pixel values of the same pixel position (N+3/4 pixels, M+3/4 pixels: N and M are given integers) as "specific positions".

In addition, it is possible to solve the problem that strong smoothing is always applied to "displacement compensation values" in regions with displacement vectors indicating the same pixel positions as "specific positions".

The method of generating predicted image signals or displacement compensation values described in this embodiment is just an example. A given method necessary for realizing the switching displacement compensation value calculation method performed according to the present embodiment can also be used.

(improved embodiment 1A)

A modified embodiment 1A of the moving image encoding device 20 and moving image decoding device 50 of the foregoing embodiment 1 is described below. Hereinafter, only differences from the foregoing Embodiment 1 are described.

Regarding the moving image encoding device 20 and moving image decoding device 50 according to this modified embodiment, improvements are introduced in the displacement compensating section 33 of the moving image encoding device 20 and the displacement compensating section 72 of the moving image decoding device 50 of the previous embodiments. The displacement compensating section 33 of the moving image encoding device 20 and the displacement compensating section 72 of the moving image decoding device 50 are the same. Thus, hereinafter, the displacement compensating section 33 of the moving image encoding device 20 will be described.

According to this improved embodiment, the judging part 33d _of the displacement compensating part ₃₃ judges _whether the _displacement vector is Indicates a "specific location". Then, the judgment section 33d reports the judgment result to the predicted image signal generation section 33e. Hereinafter, the unit of the displacement vector expression _MVE = (MVx _E , MVy _E ) is 1/4 pixel.

First, in the case of " _PMVxE %4=0 or 1" (for example, _PMVxE is in the first phase), " _MVxE %4=3" and " _MVyE %4=3", the judging part 33d It is _{judged that the displacement vector MVE (MVx E ,} MVy _E ) indicates a "specific position".

Second, in the case of "PMVx _E % 4 = 2 or 3" (for example, PMVx _E is in the second phase), "MVx _E % 4 = 1" and "MVy _E % 4 = 1", the judging part 33d judges that the displacement vector _MVE (MVx _E , MVy _E ) indicates the "specific position".

As a result, the area indicated by the predicted displacement vector PMV _E is adjusted so as not to be superimposed on this "specific position". That is, the displacement vector indicating the "specific position" is set to be different from the predicted displacement vector _PMVE .

As described above, the improved embodiment 1A has the effect of reducing the possibility of superposition of displacement vectors that become "singular positions" in real number displacements by using the predictive coding structure of displacement vectors. For example, in FIG. 5, when the second phase of the predicted displacement vector PMV is 2, even blocks A, B, C, D and E move in parallel along the lower right with (3/4 pixel, 3/4 pixel), For example, even in the case of displacement vector MV=(MVx, MVy)=(3/4, 3/4), displacement vector MV=(MVx, MVy)=(3/4, 3/4) does not necessarily indicate "Special location".

(improved embodiment 1B)

A modified embodiment 1B of the moving image encoding device 20 and the moving image decoding device 50 of the foregoing embodiment 1 is described below. Hereinafter, only differences from Embodiment 1 are described.

In the moving image encoding device 20 and the moving image decoding device 50 according to this modified embodiment, changes are made in the displacement compensating section 33 of the moving image encoding device 20 and the displacement compensating section 72 of the moving image decoding device 50 of the foregoing embodiments. . The displacement compensating section 33 of the moving image encoding device 20 and the displacement compensating section 72 of the moving image decoding device 50 are the same. Thus, hereinafter, the displacement compensating section 33 of the moving image encoding device 20 will be described.

According to this improved embodiment, the judging part 33d of the displacement compensating part 33 judges the displacement vector of the predetermined area to be coded (macroblock E) detected by the displacement detecting part 32 according to the aforementioned difference information MVD _E = (MVDx _E , MVDy _E ) Whether MV _E indicates a "specific location". Then, the judgment section 33d reports the judgment result to the predicted image signal generation section 33e.

That is to say, the judging part 33d forms a judging part, when the difference information of the displacement vector PMV _E detected by the prediction part (predicted displacement vector calculation part 33c) and the displacement vector MVE detected by the displacement vector detection part (displacement detection part 32) When MVD _E is equal to a predetermined value, it is used to judge that the displacement vector _MVE detected by the displacement vector detection unit (displacement detection unit 32) is a predetermined displacement vector (a displacement vector indicating a “specific position”).

Hereinafter, the unit of the displacement vector expression _MVE = (MVx _E , MVy _E ) is 1/4 pixel.

For example, in the case of "MVxE%4=3" and "MVyE%4=3", the judging part 33d judges that the displacement vector _MVE = ( _MVxE , _MVyE ) detected by the displacement detecting part 32 indicates a "specific position ".

In this case, for example, if the smoothing operation is performed using the quotient of the displacement vector difference information _MVDE , it is necessary to change the calculation method of the pixel value in the "specific position".

That is to say, according to the conventional technology, the foregoing embodiment 1 and the improved embodiment 1, displacement compensation is performed based on the average value of pixel values at integer pixel positions surrounding the "specific position". However, if the "singular position" judgment is made using the quotient of the displacement vector difference information _MVDE , the "singular position" itself appears as an integer pixel position, and the integer pixel position for obtaining the average value may not be established. Thus, the displacement compensation operation is performed as follows.

In FIG. 11, since "MVDxE%4=3" and "MVDyE%4=3" (in this case, "PMVxE%4=1" and "PMVyE%4=1"), the predicted image generating section 33e It is judged that the displacement vector _MVE = (MVx _E , MVy _E ) detected by the displacement detection part 32 indicates a "specific position". Here, actually, the displacement vector _MVE = (MVx _E , MVy _E ) indicates an integer pixel position "D".

As shown in FIG. 11, in this case, the predictive image generation section 33e makes the displacement compensation value of the predetermined area (one macroblock or subblock) use the pixel values (MVx _E +2, MVy _E +2) in the reference frame image 5 ), the pixel value (MVx _E +2, MVy _E -2), the pixel value (MVx _E -2, MVy _E +2) and the average value of the pixel value (MVx _E -2, MVy _E -2).

Alternatively, in addition to pixel values originating from integer pixel positions in the reference frame image 5, the predicted image generating section 33e may also obtain displacement compensation values of predetermined regions (macroblocks or subblocks) to be coded. In particular, the displacement compensation value can also be the pixel value ((MVx _E /4)*4, (MVy _E /4)*4), (((MVx _E +4)/4)*4, (MVy _E /4 )*4), ((MVx _E /4)*4, ((MVy _E +4)/4)*4) and (((MVx _E +4)/4)*4, ((MVy _E +4) /4)*4) average value.

According to this modified embodiment 1B, in the moving image encoding device 20, it is possible to transmit the real number displacement vector (MVx _E , MVy _E ) and the displacement vector _MVE =(MVx _E , MVy _E )= Choose between (1, 1) for sending.

(Example 2)

The moving image encoding device 20 and the moving image decoding device 50 according to Embodiment 2 of the present invention are described below. In this embodiment, displacement compensation using "specific positions" is not performed.

The description of Embodiment 1 has been referred to in order to cope with the transmission problem of the displacement vector MV (or the difference information MVD of the displacement vector) indicating a "specific position", for example, the transmission of the real displacement vector MV is blocked. However, there is still a possibility that the real displacement vector MV cannot be transmitted.

Thus, in the present embodiment, the moving image encoding device 20 and the moving image decoding device 50 will be described below. The moving image encoding and decoding apparatus provides a strongly smooth predicted image signal by separately preparing a "displacement compensation value" like in the "specific position" of embodiment 1, and a usual "displacement compensation value" except for the "specific position" and by signaling the identification information of these two predicted image signals together with the reference frame image number, it is possible to perform displacement compensation from the reference image whose smoothing level varies from one predetermined area to be coded to another . In this way, displacement compensation can be performed without using "singular positions".

FIG. 12 is a schematic diagram of a moving picture encoding device 20 of this embodiment, and FIG. 13 is a schematic diagram of a moving picture decoding device 50. As shown in FIG.

In the embodiment, as in the case of Embodiment 1, the moving picture coding device 20 and the moving picture decoding device 50 are described below, wherein an improvement is introduced in the displacement compensation of "singular position", where the problem is that the conventional "TML-8" definition A question in "H.26L Coding System" (the first question).

In this Embodiment 2, other operations are similar to those of the moving image encoding device 20 and the moving image decoding device 50 described in "TML-8", except that displacement compensation is performed from a reference image instead of using a "special position". As such, a detailed description will be omitted, and the following description will only focus on differences.

In addition to introducing improvements in the displacement compensating parts 33 and 72 and the variable length coding part 40, and adding new reference image generating parts 45 and 80, the basic structure of the moving picture encoding device 20 and the moving picture decoding device 50 according to this embodiment Operations are virtually the same as those of the moving image encoding device 20 and the moving image decoding device 50 according to the conventional art.

According to this embodiment, the reference image generation components 45 and 80 are used to generate several different reference images (typically a first reference image or a second reference image with strong smoothing applied) by performing several different image processings on the reference frame image 5 17 . Here, as with the aforementioned image processing, processing of changing the smoothing level, processing of changing the spatial resolution, and the like are conceivable. In this embodiment, as image processing, the following describes the case of using changing smoothing level processing.

In addition, the displacement compensating section 33 is used to calculate a displacement compensation value (a predicted image signal 6 ) of a predetermined area (macroblock) to be encoded by using the reference image 17 instead of the reference frame image 5 .

The variable length coding section 40 constitutes a sending section for sending "information of a reference image 17 (a reference frame code: Ref_frame)" and "information indicating a displacement compensation value ( Prediction Residual Signal Data Coding Syntax: Text Coding Syntax)" combination.

In addition, the variable-length decoding section 71 constitutes a decoding section for decoding "reference image information (Ref_frame)" and "information indicating a displacement compensation value (prediction residual signal data encoding grammar)".

The variable length decoding section 71 sends the displacement vector 3 , the prediction mode 4 and the "reference frame code (one Ref_frame)" 4A to the displacement compensating section 72 .

Further, the displacement compensating section 72 is used to calculate a displacement compensation value of a predetermined region (one macroblock) to be encoded by using the reference image 17 designated by "reference image information (Ref_frame)" instead of the reference frame image 5.

According to this embodiment, "the information of the reference image 17 (reference frame code: Ref_frame)" is an "identification information of the reference frame image (reference frame image number)" and "information indicating the smoothing level (the first reference image or the second reference image)" combination.

First, according to this embodiment, the reference image generation section 45 generates a general reference image (hereinafter referred to as a first reference image) that does not have a "specific position". Except that an initial "displacement compensation value" is used, even for the "displacement compensation value" of the same position as the "specific position" of "TML-8", the "displacement compensation value" of the first reference image is equal to that of "TML-8" "Displacement compensation value" for each pixel position of integer pixel position, 1/2 pixel position and 1/4 pixel position.

In the case where the "displacement compensation value" is at position (1/4 pixel, 1/4 pixel) instead of the "specific position" of "TML-8", the same pixel position as the "specific position" of "TML-8" The "displacement compensation value" of the first reference image is generated as a 4-point average of the pixel values of adjacent integer pixel positions and adjacent 1/2 pixel positions.

Second, by using the first reference image, a reference image (hereinafter referred to as the second reference image) is generated, which provides a reference for the "displacement compensation value" of the "specific position" similar to "TML-8" that is strongly smoothed image. Here, the second reference image can be generated by performing image processing using various smoothing filters on each pixel value of the first reference image. For example, by separately running a 3-stage filter (1, 4, 1)/6, which has vertical and horizontal smoothing effects on pixel values at each pixel location of the first reference image with 1/4 pixel precision, it is possible to generate /4 pixel precision of the second reference image.

According to the "H.26L encoding system", several encoded frame images temporally different from each other are prepared as reference frame image number 5, and these can be used as reference images for displacement compensation. In addition, the identification information bits of these coded frame images different from each other over time are distinguished by reference frame image numbers.

According to this embodiment, the identification information of the first reference image or the identification information of the second reference image, for example, the information for generating the reference image and the image number of the reference frame are combined and sent.

In this way, displacement compensation can be performed using reference images whose smoothness levels of predetermined regions to be coded (for example, one macroblock) are different from each other without using "special positions".

In this case, for the reference frame image 5 which is the encoded frame image temporally different from each other, the reference image generating sections 45 and 80 generate first and second reference images of different smoothness levels. Therefore, these reference frame images can be used as “reference images” for displacement compensation in the displacement compensating sections 33 and 72 .

FIG. 14 shows a "coding syntax of each macroblock unit according to the H.26L coding system" used in this embodiment. According to this embodiment, there is no change in the encoding syntax of each macroblock unit of the H.26L encoding system. However, the definition of "Ref_frame" is changed to a combination of a "reference frame image number" and "reference image generation method identification information", for example, a "reference frame code".

As shown in FIG. 14, even when a prediction mode (for example, mode 7) requiring detection of several displacement vectors in one macroblock is applied, encoding can be performed without including several "MB_TYPEs", "Ref_frames", etc. information.

That is, by using the encoding syntax, it is possible to repeatedly transmit the difference information MVD of the displacement vector and the prediction residual signal data encoding syntax (a text encoding syntax) in response to the transmission action of "MB_TYPEs", "Ref_frames" and the like. Here, the coding syntax of the prediction residual signal data is obtained by variable-length coding the quantized orthogonal transform coefficients 11 .

FIG. 15 shows an example of a reference frame code (a Ref_frame) based on a combination of reference frame picture number and reference picture generation method identification information.

Here, as shown in FIG. 15 , the same reference frame codes ("0" to "4") as the reference frame picture numbers labeled with conventional H.26L are used for the first reference picture, and the newly added reference frame codes ( "5" to "9") are used for the second reference image.

According to this embodiment, the second reference image is smoothed more strongly than the first reference image, which is a usual reference image, and the second reference image is an image that does not have the spatial resolution that the original image (reference frame image number 5) has. reference image.

In this way, when using a reference image of a stronger smoothing level, encoding distortion is much suppressed, and displacement compensation efficiency is improved, the second reference image is used. In this way, the second reference image is less likely to be selected compared to the first reference image.

Therefore, in many cases, the first reference image, which is a general reference image, is selected as a reference image for displacement compensation. In this case, the reference frame code table for defining the reference frame code to be transmitted is similar to the reference frame code table for defining the reference frame code (Ref_frame) of the "H.26L coding system" shown in FIG. 16 . In this way, compared with the case of the conventional "H.26L encoding system", there is no increase in the number of bits caused by the change of the reference frame code.

Further, in the case of performing displacement compensation using the second reference image with a stronger smoothing level, the reference frame code to be transmitted requires a relatively long encoding length. However, it is less likely to use a second reference image, and the increased number of bits for the reference frame code has less impact than the increased displacement compensation efficiency using a second reference image with a strong smoothing level. In this way, highly efficient encoding can be obtained.

(Operation/Effect of Moving Image Coding and Decoding Device According to Embodiment 2)

According to the moving image encoding device 20 of the present invention, by using reference images of two different smoothness levels, for example, a normal reference image (first reference image) and a strong smoothing level reference image (second reference image) formed by the reference image generation section 45 image), displacement compensation can be performed using reference images of different smoothness levels in predetermined areas to be encoded (for example, a macroblock).

In addition, according to the mobile image encoding device 20 of the present invention, by generating a reference frame code combining "reference image generation method identification information" and "reference frame image number", the smoothing level of the reference image is signaled. In addition, it is possible to solve the problem that strong smoothing is always applied to "displacement compensation value" having a displacement vector area indicating the same pixel position as "specific position" in "H.26L encoding system".

In this embodiment, the filter used to generate the second reference image which is a strong smoothness level reference image is just an example. Prediction based on different feature reference images can be achieved by using filters for processing other than smoothing.

In addition, in FIG. 11 , for simplicity of description, the maximum number of reference frame images used for reference image generation is "5". However, the present invention is not limited to this value, and the maximum image number of the reference frame can be optionally set.

According to "TML-8", the maximum number of reference frame images used for reference image generation in the moving image encoding device 20 and the moving image decoding device 50 is known and given.

Further, in practical applications, the maximum number of reference frame images may be determined by a method or based on information of the compressed stream 12 from the moving image encoding device 20 to the moving image decoding device 50 .

In any case, in the moving image encoding device 20 and the moving image decoding device 50, the maximum number of reference frame images used for reference image generation is uniquely determined. In this way, the reference frame code table can be uniquely determined according to the maximum number of reference frame images used for reference image generation.

Further, in the reference frame code table defining the reference frame code (Ref_frames) in Fig. 16, it is the "reference frame code (a Ref_frame)" of the combination of "reference frame image number" and "second reference image", which is in The "reference frame code (one Ref_frame)" which is the combination of the "reference frame image number" and the "first reference image" is assigned afterward. However, those smaller reference frame picture numbers in the second reference picture of the strong smoothing level can be arranged at higher positions in the reference frame code table based on the assumption that the reference pictures are frequently used immediately after the displacement compensation time off.

In addition, the reference frame code table can be uniquely determined according to the above-mentioned coding conditions of the predetermined region (macroblock) to be coded. Alternatively, the reference frame code can be dynamically changed according to the aforementioned encoding conditions. Here, for the aforementioned encoding conditions, prediction modes (for example, to detect displacement vector unit types), quantization steps (QP values), and the like are conceivable. Here, for the kind of unit of the detection displacement vector, for example, the size of a sub-block of the detection displacement vector, etc. are conceivable.

As a specific example, the following describes the case of dynamically changing the reference frame code according to the quantization step. Here, when low bit rate encoding is applied, the second reference picture of a strong smoothing level may be frequently used. Therefore, if the quantization step is equal to or lower than the predetermined threshold value, the "reference frame code" including the "second reference image" is arranged at a lower position in the reference frame code table. If the quantization step exceeds a predetermined threshold value, the "reference frame code" including the "second reference image" is arranged at a higher position in the reference frame code table.

As mentioned above, the "reference frame code table" that defines the "reference frame code" shown in Fig. 15 is just an example, where the reference frame code is a set of "reference frame image number" and "reference image generation method identification information" combination. According to this embodiment, it is possible to use a given reference frame code table executed according to this embodiment in order to realize the need of switching between "reference frame image number" and "reference image generation method identification information".

In addition, according to this embodiment, the smoothing level of the reference image can be automatically switched to be uniquely determined according to the encoding conditions of the predetermined area (macroblock) to be encoded instead of the smoothing level used in the displacement compensation using the reference frame code (Ref_frame). Reference to explicit signaling of image smoothing levels.

That is, the reference image generation section 45 can generate the reference image 17 of a predetermined smoothness level according to the encoding conditions (unit of detection displacement vector, quantization step, etc.) of a predetermined region (macroblock) to be encoded.

For example, complex displacements may occur in areas encoded in "macroblock mode (MB_Type: one prediction mode)", where the predetermined area (macroblock) to be encoded is very finely divided, therefore, the reference image used for displacement compensation High pixel value precision may not be required.

Further, in a macroblock with a large number of quantization steps (one QP value), the reference image used for displacement compensation may not require high pixel value precision.

In this way, for those macroblocks where the number of units (sub-blocks) of the detected displacement vector or the quantization step exceeds a predetermined threshold value, the second reference image with a strong smoothing level can always be used.

In this case, since the reference image generated according to the encoding condition is uniquely determined, there is no need to use the smoothing level identification information of the image number of the reference frame. Thus, compared with the "H.26L coding system", the number of bits caused by changing the reference frame code or the macroblock mode code does not increase.

(improved embodiment 2A)

A modified embodiment 2A of the foregoing embodiment 2 is described below. Hereinafter, only the differences between this modified example and Example 2 are described.

According to the foregoing Embodiment 2, reference images (first and second reference images) 17 of two different kinds of generation methods (smoothing levels) are formed. As a result, the formed "reference image generation method identification information (indicating smoothness level information)" and "reference frame image number (identification information of reference frame image)" are combined to generate "reference frame image code (reference frame image information)". In this way, displacement compensation is performed by using a reference image having a different level of smoothness for each predetermined region (macroblock) to be coded.

However, switching of the smoothing level is possible only for the macroblock unit assigned the picture number unit of the reference frame.

Thus, in this modified embodiment, the moving image encoding device 20 and the moving image decoding device 50 capable of transmitting a combination of "reference frame code" and "sub-block unit (one detection displacement vector unit)" are described below. The "reference frame code" is generated by combining the reference frame image number and reference image generation method identification information. A sub-block is a portion of a macro-block on which motion compensation is performed.

The basic operations of the moving image encoding and decoding devices 20 and 50 according to this modified embodiment are substantially the same as those of the moving image encoding and decoding devices 20 and 50 of Embodiment 2 described above.

According to this modified embodiment, the displacement compensating section 33 switches the reference image (first reference image or second reference image) 17 used for the calculated displacement compensation value in units of detected displacement vectors (macroblock units).

The variable length coding unit 40 sends a combination of "reference image information (Ref_frame)" and "indicative displacement compensation value information (prediction residual signal data encoding syntax)" in a detection displacement vector unit (sub-block unit).

In addition, the variable-length decoding section 71 decodes "reference image information (reference frame code)" and "information indicating displacement compensation (prediction residual signal data coding syntax)" in detection displacement vector units (sub-block units).

Further, the displacement compensating section 72 switches the reference image 17 used for calculating the displacement compensation value in units of detected displacement vectors (sub-block units).

Here, as an example of "reference frame code (Ref_frame)" based on a combination of "reference frame image number" and "reference image generation method identification information", it is assumed that it is similar to the reference frame code in Embodiment 2.

FIG. 17 shows macroblock unit-based encoding syntax according to this embodiment.

According to this improved embodiment, in a macroblock, it is necessary to send the "reference frame code (Ref_frame)" generated by combining the "reference frame image number" and the "reference image generation method identification information" several times in subblock units.

The number of "reference frame codes (Ref_frames)" can be notified by the "macroblock type (one MB_type)" because the type and the number of subblocks are sent by the "macroblock type (MB_type)".

For example, if the macroblock type is "INTER prediction mode (mode 7)", the number of "reference frame codes (Ref_frames)" to be sent is "16".

According to this modified embodiment, in the "sub-block" unit which is the unit of detection displacement vectors in a macroblock, it is possible to perform displacement compensation by using reference images 17 of different smoothing levels.

In addition, it is possible to perform displacement compensation using a different reference frame image number 5 for each sub-block. In addition, in response to the shape and displacement of the frame image signal 2, a higher degree of freedom of displacement compensation can be performed.

Further, according to this improved embodiment, the smoothing level of the reference image can be automatically switched to be uniquely determined according to the encoding conditions of the predetermined area (macroblock or sub-block) of the detected displacement vector, instead of using the reference frame code (Ref_frame) for displacement compensation Explicit signaling of the smoothing level of the reference image.

(improved embodiment 2B)

Modified Embodiment 2B of Embodiment 2 is described below. The operations of the moving image encoding device 20 and the moving image decoding device 50 according to the embodiment of the present invention are the same as those of the moving image encoding device described in "TML-8", except that displacement compensation is performed using different reference images without using "singular positions". 20 and the operation of the moving image decoding device 50 are similar. Therefore, a detailed description thereof is omitted, and the description is focused on different points.

According to the foregoing Embodiment 2, reference images (first and second reference images) of two different kinds of generation methods (smoothing levels) are formed. As a result, the formed "reference image generation method identification information" and "reference frame image number" are combined to generate a "reference frame code". In this way, it is possible to perform displacement compensation in which the level of smoothness is different for each predetermined region to be coded (for example, one macroblock: a unit of assigning a reference frame image number).

According to this embodiment, reference images (first and second reference images) of two different smoothing levels are formed, and the formed "reference image generation method identification information" and "macroblock mode" are combined to generate "macroblock mode code". In this way, displacement compensation can be performed in which the smoothing level is different for each predetermined region (macroblock) to be coded.

The basic operations of the moving image encoding device 20 and the moving image decoding device 50 according to this modified embodiment are practically the same as those of the moving image encoding device 20 and the moving image decoding device 50 according to the foregoing Embodiment 2.

According to this improved embodiment, "reference image information (macroblock mode code)" is a "indicating detection displacement vector unit unit identification information (macroblock mode: MB_Type)" and "indicating smoothness level information (first reference image or second reference image)".

The variable length encoding part 40 sends a "reference image information (macroblock mode code)", "reference frame image identification information (reference frame image number: Ref_frame)" and A combination of "information indicating a displacement compensation value (prediction residual signal data encoding syntax)".

In addition, the variable-length decoding section 71 performs "reference image information (macroblock mode code: MB_Types)", "reference frame image identification information (Ref_frame)" and "indicated displacement" in units of predetermined regions (macroblocks) to be encoded. The prediction residual signal data encoding syntax information of the compensation value" is decoded.

Further, the displacement compensation component 72 uses the reference image 17 specified by the "reference image information (MB_Type)" and "reference frame image identification information (Ref_frame)" to replace the reference frame image 5, thereby calculating the displacement compensation of the predetermined area (macroblock) to be encoded value.

Fig. 14 shows macroblock unit-based coding syntax according to the H.26L coding system. According to this improved embodiment, there is no change from the conventional "encoding syntax in units of macroblocks of the H.26L encoding system". However, the definition of "macroblock mode (MB_Type)" is changed to be expressed by a combination of "macroblock mode" and "reference image generation method identification information".

FIG. 18 shows an example of "macroblock mode code (MB_Type)" based on the combination of "macroblock mode" and "reference image generation method identification information".

As shown in FIG. 18, regarding the same macroblock mode as the conventional H.26L macroblock mode, execution of displacement compensation is instructed from the first reference picture. With regard to the newly added macroblock mode, the execution of displacement compensation is commanded from the second reference image of the strong smoothing level.

According to this modified embodiment, since the macroblock mode is assigned to one macroblock unit, it is possible to perform displacement compensation from reference images of different smoothness levels in units of macroblocks.

Further, according to this improved embodiment, the smoothing level of the reference image can be automatically switched to be uniquely determined according to the encoding conditions of the area to be encoded (macroblock), instead of smoothing the reference image for displacement compensation by using the macroblock mode code (MB_Type) level of explicit signaling.

(improved embodiment 2C)

A modified embodiment 2C of the foregoing embodiment 2 is described below. According to this improved embodiment, the second reference image of the foregoing embodiment 2 is strongly smoothed, and a displacement vector with the same precision as that of the first reference image is not always required in displacement compensation. Thus, a configuration in which the displacement vector is changed when the second reference image is used is described below. Hereinafter, differences of this modified embodiment from the foregoing Embodiment 2 are described.

According to this modified embodiment, when using a reference image (second frame image) of a strong smoothing level, the displacement compensating section 33 reduces the precision of the displacement vector for calculating the displacement compensation value (predicted image signal 6). In particular, in this case the displacement compensating means 33 are used to reduce the accuracy of the horizontal and vertical components of the displacement vector.

According to this according embodiment, like in the case of the foregoing Embodiment 2, after generating the first reference image, the reference image generation section 45 generates a strongly smoothed reference image as the second reference image in which the spatial resolution is reduced to 1/2 pixel precision or integer pixel precision.

As an example of a strongly smoothed reference image generation method in a second reference image with a spatial resolution of 1/2 pixel precision, the following describes the downlink by running a (1, 2, 1)/4 filter on the first reference image sampling method.

Also, as an example of a reference image generation method that undergoes strong smoothing with a spatial resolution of integer pixel precision, the following describes the downlink by running a (1, 2, 1)/4 filter on the smoothed 1/2 pixel precision image sampling method.

According to this improved embodiment, as an example of a "reference frame code" based on a combination of "reference frame image number" and "reference image generation method identification information", it is assumed that it is similar to the reference frame code in the aforementioned embodiment 2.

For example, when a strongly smoothed reference image whose spatial resolution is reduced to 1/2 pixel precision is generated as the second reference image, assuming that the displacement vector of the first reference image is (MVx, MVy), the displacement of the second reference image Perform the following permutations in the vector.

Pixel position of the first reference image:

MVx, MVy (1/4 pixel unit)

Pixel position of the second reference image:

MVx//2, MVy//2

Here, "//" indicates an integer division with a rounding operation in the direction of zero.

In addition, if the strongly smoothed reference image whose spatial resolution is reduced to integer pixel precision is generated as the second reference image, assuming that the displacement vector of the first reference image is (MVx, MVy), the following is performed in the displacement vector of the second reference image replacement.

Pixel position of the first reference image:

MVx, MVy (1/4 pixel unit)

Pixel position of the second reference image:

MVx//4, MVy//4

Here, "//" indicates an integer division with a rounding operation in the direction of zero.

Thus, the second reference image has several values for the same displacement vector MV=(MVx, MVy).

For example, (3,3), (2,3), (3,2) and (2,2) represent the same displacement vector if the second reference image is reduced to 1/2 pixel precision.

Therefore, in the second reference image, when encoding is performed at the variable-length encoding section 40, a generated displacement vector (such as (2, 2)) with a small encoding amount can be transmitted as a representative value.

Alternatively, in the second reference image, the difference information MVD between one predicted displacement vector PMV and one displacement vector can be calculated by using the displacement vector after performing permutation according to the spatial resolution, and the difference information of the displacement vector to be transmitted value can be lowered. In other words, the amount of encoding can be reduced.

In this case, if the difference information MVD between the predicted displacement vector and this displacement vector is calculated from the second reference image of large resolution to the first reference image, the permutation is performed inversely to increase the second reference image displacement The spatial resolution of the vector.

Through the aforementioned changes, if noise is superimposed on the reference frame image 5, coding efficiency can be obtained by referring to the second reference image with low spatial resolution.

In addition, this improved embodiment is characterized in that the spatial resolution of the displacement vector is reduced in the second reference image of low spatial resolution, and redundancy in encoding time of the displacement vector is avoided.

Further, according to this improved embodiment, the spatial resolution of the reference image can be automatically switched to be uniquely determined according to the encoding conditions of the area to be encoded (macroblock), instead of using the reference frame code (Ref_frame) for displacement compensation. Explicit signaling of the spatial resolution of the image.

For example, compound displacement may occur in a region coded with "macroblock mode (MB_Type: a prediction mode)" in which a predetermined region (macroblock) to be coded is finely divided, so the reference for displacement compensation Images may not require high pixel value precision.

Furthermore, in macroblocks with a large number of quantization steps (QP values), the reference image for displacement compensation may not require high pixel value precision.

In this way, for macroblocks, where the number of detected displacement vector units (sub-blocks) or quantization steps exceeds a predetermined threshold, a strongly smoothed second reference image (eg low spatial resolution) can always be used.

In this case, since the generated reference image is uniquely determined according to the encoding conditions, information for identifying the smoothing level by using the reference frame image number is unnecessary. Therefore, compared with the "H.26L encoding system", there is no increase in the amount of bits caused by changes in the reference frame code or the macroblock mode code.

Further, the smoothing level and the spatial resolution can be automatically switched to be uniquely determined according to the aforementioned encoding conditions (macroblock mode, quantization step, etc.).

In this case, for a macroblock in which the number of sub-block divisions or the number of quantization steps exceeds a predetermined threshold, the spatial resolution is reduced to 1/2 pixel precision or integer pixel precision, by always using a reference Image, which uses a filter to apply a strong level of smoothing, and by reducing the horizontal and vertical components of the displacement vector, the amount of encoding of the resulting displacement vector is reduced.

(Example 3)

The moving image encoding device 20 and moving image decoding device 50 according to Embodiment 3 will be described below. The description of Embodiment 2 has mentioned the moving image encoding device 20 and the moving image decoding device 50 . The mobile image encoding and decoding device is notified by signaling of "reference image information (reference frame code or macroblock mode code)", which is "reference image generation method identification information (first reference image or second reference image)" and " The combination of the reference frame image number" enables displacement compensation to be performed for each predetermined region (macroblock) to be coded using a normal reference image (first reference image) and a strong smoothing reference image (second reference image).

In this embodiment, the following describes the moving image encoding device 20 and the moving image decoding device 50, which constitute a pyramid of layers for each spatial resolution so that reference images having three different pixel precisions are generated.

FIG. 19 shows a schematic diagram of a moving image encoding device 20 of this embodiment, and FIG. 20 shows a schematic diagram of a moving image decoding device 50 of this embodiment.

In addition to introducing some improvements in the displacement detection part 32, the displacement compensation parts 33 and 72, and the variable length coding part 40, and in addition to adding the hierarchical reference image generation parts 46 and 81, the moving image coding according to this embodiment The basic operations of the device 20 and the moving image decoding device 50 are completely the same as those of the moving image encoding device 20 and the moving image decoding device 50 according to the conventional art.

According to this embodiment, the hierarchical reference image generation parts 46 and 81 constitute a reference image generation part for generating several different reference images (hierarchical reference images 18) by performing several different image processes on the reference frame image 15. . Here, as the aforementioned image processing, processing of changing the smoothing level, processing of changing the spatial resolution, and the like are conceivable. The description of this embodiment will refer to the processing for changing the spatial resolution used as the aforementioned image processing.

Furthermore, each of the hierarchical reference image generating sections 46 and 81 generates one reference image (one hierarchical reference image 18 ) having several spatial resolutions by performing filtering processing through several different passband filters. Here, "indicating image processing information, for example, indicating spatial resolution information (layer)" is an identification of the filter.

Further, the displacement detecting part 32 constitutes a 3-dimensional displacement vector generating part, by combining the "displacement vector (MVx, MVy)" detected using the reference image (layered reference image 18) with the "indicating reference image (layered reference image 18) ) spatial resolution information (layer)” to generate a “3-dimensional displacement vector (layer, MVx, MVy)”.

The displacement detection component 32 can be used to reduce the precision of the 3-dimensional displacement vector of the reference image with low spatial resolution (such as layer 3, etc.).

In addition, the displacement compensation section 33 (72) calculates a displacement compensation value for a predetermined area (macroblock) to be encoded (decoded) by using a reference image (layered reference image 18) instead of the reference frame image 5.

Further, the displacement compensating part 33 constitutes a 3D displacement vector predicting part, by using the correlation between the coded predetermined area (encoded macroblock) and the to-be-encoded predetermined area (to-be-encoded macroblock) in the frame image (such as context switching in encoding algorithms), used to predict 3D displacement vectors.

The variable length coding section 40 constitutes a sending section for sending a combination of "3-dimensional displacement vector" and "information indicating displacement compensation value".

Incidentally, variable length coding section 40 may transmit a combination of difference information (layer D, MVDx, MVDy) and information indicating a displacement compensation value, wherein the difference information is a 3-dimensional displacement vector generated by displacement detection section 32 (layer , MVx, MVy) and the difference information between the 3-dimensional displacement vector (P layer, PMVx, PMVy) predicted by the displacement compensating part 33.

In addition, the variable length decoding section 71 constitutes a decoding section for decoding a 3-dimensional displacement vector of a predetermined area to be decoded.

First, a concept used in this embodiment will be described with reference to FIGS. 21 and 22 .

Each layered reference image generating section 46 and 81 generates 3 layers for the reference frame image 5 for displacement compensation via each displacement compensating section 33 and 72 .

First, as shown in Figures 21 and 22, each layered reference image generation unit 46 and 81 upsamples the reference frame image 5 through an 8-stage filter to generate a 1/4 pixel precision layer 1, which is a layered Refer to image 18. An example of an 8-stage filter used here is as follows:

For 1/4 pixel positions: (-3, 12, -37, 229, 71, -21, 6, -1)/256

For 2/4 pixel positions: (-3, 12, -39, 158, 158, -39, 12, -3)/256

For 3/4 pixel positions: (-1, 6, -21, 71, 229, -37, 12, -3)/256

Here, for pixel values at integer pixel positions, pixel values at the same positions of the reference frame image 5 are copied. The pixel values at 1/4 pixel positions, 2/4 pixel positions and 3/4 pixel positions between the integer pixel positions are obtained by multiplying and adding the filter coefficients of the pixel values at the aforementioned integer pixel positions. This filtering process is performed separately in the horizontal and vertical directions.

The filtering process is described in the upsampling operation of 1/8 pixel precision of the conventional "TML-8", and therefore, a detailed description is omitted here.

Second, each layered reference image generation unit 46 and 81 downsamples the generated layer 1 of 1/4 pixel precision through a 3-stage filter (low-pass band type filter), thereby generating a layer of 1/2 pixel precision 2, which is a layered reference image 18 . An example of a 3-stage filter used here is "(1, 2, 1)/4".

Third, each layered reference image generation unit 46 and 81 down-samples the generated layer 2 of 1/2 pixel precision through a 3-stage filter (low-pass band type filter), thereby generating a layer 3 of integer pixel precision , which is a hierarchical reference image 18 . The 3-stage filter used here is the same as used before.

Note that in the case of the conventional technology, layer 1 has 1/4 pixel precision, but the pixel value at the 1/4 pixel position is not obtained by linear interpolation, but calculated by performing the aforementioned filtering process, thereby maintaining the original image (refer to The spatial resolution of the frame image 5).

As described above, each layered reference image generation section 46 and 81 generates reference images (layers 1 to 3) with several spatial resolutions by filtering processing with filters having several different passbands.

According to this embodiment, the displacement compensation section 33 performs displacement compensation by using the hierarchical reference image 18 generated in the aforementioned manner.

In this event, displacement vector 3 is not a 2-item set (2-dimensional displacement vector) of (MVx, MVy), but a 3-item set (3-item displacement vector) of (layer, MVx, MVy).

The displacement detecting section 32 detects a 3-dimensional displacement vector (layer, MVx, MVy) instead of a 2-dimensional displacement vector (MVx, MVy).

Layer 2 has half the spatial resolution of layer 1, and layer 3 has half the spatial resolution of layer 2. Therefore, the following permutations are performed.

Resolution of layer 1: MVx, MVy (1/4 pixel unit),

Resolution of layer 2: MVx//2, MVy//2,

Resolution of layer 3: MVx//4, MVy//4

Among them, "//" represents an integer division accompanied by a rounding operation in the zero direction.

Thus, layers 2 and 3 have several values (MVx, MVy) for the same displacement vector.

For example, in layers 2 and 3, (2, 3, 3), (2, 2, 3), (2, 2, 3) and (2, 2, 2) represent the same displacement vector.

Therefore, in the upper layer (layer 2 or layer 3), when encoding is performed at the variable-length encoding section 40, a displacement vector with a small encoding amount such as (2, 2, 2) can be transmitted as a representative value.

Alternatively, in layers 2 and 3, the difference information MVD between a predicted displacement vector PMV and a displacement vector can be calculated by using displacement vectors permuted according to the resolution of each layer, and the displacement vector to be transmitted The difference information MVD value can be reduced. In other words, the amount of encoding can be reduced.

In this case, when the difference information MVD between this displacement vector and the predicted displacement vector PMV of the large spatial resolution displacement vector of layer 1 is calculated from such a layer, the permutation is performed inversely to increase The spatial resolution of the displacement vector.

By using the aforementioned concept, as shown in FIG. 23 , by exactly the same method as that of the aforementioned Embodiment 1, the predicted displacement vector calculating section 33c of the displacement compensating section 22 predicts a 3-dimensional displacement vector extended to several spatial resolutions.

In FIG. 23, “PMV _XE ” represents the horizontal component of the prediction displacement vector _PMV E of the prediction region (macroblock) E to be coded, and “PMV _YE ” represents the prediction displacement vector PMV of the prediction region (macroblock) E to be coded. Vertical component of _E.

"P layer _E " indicates the prediction spatial resolution of the prediction region (macroblock) E to be coded.

Each of “MVDX _E ” and “MVDY _E ” represents difference information of a displacement vector of ¼ pixel precision of a prediction region (macroblock) E to be encoded. "Layer _D " represents predicted spatial resolution difference information of a predicted region (macroblock) E to be coded.

In this way, encoding processing is performed on the 3-item group of (layer D, MVD _X , PMV _Y ).

Incidentally, the displacement compensating section 33 may calculate the prediction 3 The difference information of the 3-dimensional displacement vector and the 3-dimensional displacement vector detected by the displacement vector detection section 32, and performs displacement compensation by using the calculated 3-dimensional displacement vector difference information.

Here, it is generally predicted that if layer D is concentrated towards zero, the transition seen from layer 1 or layer 3 is not uniform.

Thus, according to this embodiment, an "adaptive coding algorithm" can be implemented using conventional techniques, and a further extension adding 3 states to the P-layer context model can be performed to use the correlation with neighboring area macroblocks.

As described above, for this layer, it is possible to perform encoding using correlation with neighboring area macroblocks.

In the foregoing description, prediction difference encoding using the P layer and the layer D is performed. However, the implementation of the predictive difference coding is the same as that of the context model.

Therefore, not layer D but this layer itself can be coded using a coding algorithm by directly using the correlation with neighboring macroblocks or using an intermediate value such as P layer (using context switching) as a context model.

Incidentally, since the concept and details of the aforementioned "adaptive algorithm" are described as "based on the context of an adaptive binary coding algorithm", a detailed description is omitted here.

Similar to the foregoing embodiments 1 and 2 and the improved embodiment, the present invention refers to a new displacement compensation, in which "2-dimensional displacement vector" is expanded to include "information (layer) indicating image processing (spatial resolution)" 3-dimensional displacement vector" instead of the transmitted "2-dimensional displacement vector" and "information indicating image processing (information indicating spatial resolution)".

By referring to FIG. 24, the displacement compensation operation in the moving image coding apparatus 20 of the embodiment of the present invention is described below.

In step 1201 , the hierarchical reference image generating section 46 generates a hierarchical reference image 18 by using the reference frame image 5 extracted from the frame memory 34 .

In step 1202 , the displacement detection section 32 detects a 3-dimensional displacement vector of a predetermined region (macroblock) to be encoded by referring to the hierarchical reference image 18 from the hierarchical reference image generation section 46 .

In step 1203 , the displacement compensating section 33 generates the predicted image signal 6 based on the 3-dimensional displacement vector from the displacement detecting section 32 and the hierarchical reference image 18 from the hierarchical reference image generating section 46 .

According to this embodiment, if noise is superimposed on the reference frame image 5, coding efficiency is obtained by referring to the adaptive low spatial resolution layer image.

In addition, low spatial resolution layer images (such as layer 2 or 3) are characterized by reduced spatial resolution of the 3D displacement vectors, and redundancy in the encoding of the 3D displacement vectors is avoided.

Further, according to this embodiment, a 3-dimensional displacement vector is applied. The vector distribution in the 3-dimensional displacement vector parameter space is spatially continuous and can improve coding efficiency.

The generation method of the predicted image signal and the displacement compensation value according to this embodiment is just an example. It is possible to use the optional generation method required to realize switching of the calculation method of the displacement compensation value in this embodiment.

(Operation/Effect of Moving Image Coding and Decoding Device According to Embodiment 3)

According to the moving image coding apparatus of the present invention, since the displacement detecting section 32 generates a 3-dimensional displacement vector based on the layered reference image 17, it is possible to perform displacement compensation of different pixel accuracy for each predetermined region to be coded.

(improved embodiment 3A)

A modified embodiment 3A of the foregoing embodiment 3 is described below. Differences between this modified example and Embodiment 3 are described below with reference to FIG. 25 .

First, this modified embodiment differs from the foregoing Embodiment 3 in that displacement compensation is performed with 1/4 pixel precision instead of 1/2 pixel precision.

That is to say, according to this improved embodiment, each layered reference image generating part 46 and 81 passes the 6-stage filter (1, -5, 20, 20, -5, 1 )/32 is applied to the reference frame image 5 to perform upsampling, resulting in layer 1 with 1/2 pixel precision.

Second, this modified embodiment differs from the foregoing Embodiment 3 in that each hierarchical reference image generating section 46 and 81 applies the smoothing filter "(1, 2, 1)/4" horizontally and vertically, respectively on layer 1, resulting in layer 2.

Third, the difference between this improved embodiment and the aforementioned embodiment 3 is that the number of layers is 2 (layer 1 and layer 2) and both have the same spatial resolution.

In this way, the following situation is realized:

Pixel position of layer 1: MVx, MVy (1/2 pixel unit),

Pixel position of layer 2: MVx, MVy (1/2 pixel unit)

By adding the foregoing changes, and performing "intermediate value prediction" based on the same method as the conventional technique, 3-dimensional displacement vectors (layer D, MVDx, MVDy) are encoded. The "adaptive coding algorithm" differs from the foregoing embodiment 3 in that: the context of the P layer _E has gone through two states.

With the foregoing changes, when performing displacement compensation with 1/2 pixel precision, if noise is superimposed on the reference frame image 5, it is possible to perform displacement compensation switching to adaptive low resolution.

In particular, in low-rate encoding, assuming that 1/4-pixel precision is unnecessary, there is a tendency to use a smoothed image with 1/4-pixel precision displacement compensation. Therefore, a system that explicitly switches the image to low spatial resolution is described herein as an embodiment of this improvement.

Incidentally, according to this modified embodiment, the filter "(1, 2, 1)/4" used to generate the layer 2 image is a simple low-pass type filter. However, an edge-clamp type smoothing filter may be used.

For example, among the smoothing filters of the edge clamping type, there is an "intermediate filter" for obtaining an intermediate value of an area of 3×3 pixels, and it is also possible to use US Patent No. 6,041,145 titled "Apparatus and method for smoothing image signals, coding image The "Dynamic Weighting Filter" described in "Apparatus and Method for Decoding Images and Apparatus and Method for Decoding Images".

The aforementioned "Dynamic Weighting Filter" is performed by calculating the absolute value of the difference between the smoothed central pixel value and its approximate pixel value, by applying filter coefficients inversely proportional to the absolute value of the difference to the surrounding pixel values (nearby 8) to perform adaptive smooth.

Incidentally, a program for causing the computer 100 to function as the moving image encoding device 20 or the moving image decoding device 50 of the present invention may be stored in a computer-readable recording medium.

As the computer-readable recording medium, as shown in FIG. 26, for example, a floppy disk 101, a compact disk 102, an IC chip 103, a magnetic tape cassette 104, and the like can be cited. According to such a computer-readable recording medium storing a program, the aforementioned program can be easily stored, transferred, sold, and the like.

industrial application

As described above, according to the present invention, it is possible to express a predicted image signal with light overhead, and to provide displacement compensation of different pixel precision.

Claims (50)

1. moving picture coding apparatus, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this equipment comprises:

A reference picture generates parts, is used for handling by a reference frame image being carried out some different images, generates some different reference pictures;

A displacement compensator is by using the reference picture that generates, for presumptive area to be encoded is calculated a bit shift compensation value; And

A transmit block is used to send a combination of calculating the information of employed reference image information of described bit shift compensation value and the described bit shift compensation value of indication;

Wherein, described reference image information is the identification information of a reference frame image and the combination of indicating the information of described image processing.

2. according to the moving picture coding apparatus of claim 1, wherein,

Displacement compensator is used for switching the employed described reference picture of displacement calculating offset to survey motion vector unit, and

Described transmit block is used for to survey the combination that motion vector unit sends the information of described reference image information and the described bit shift compensation value of indication.

3. moving picture coding apparatus, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this equipment comprises:

A reference picture generates parts, is used for handling by a reference frame image being carried out some different images, generates some different reference pictures;

A displacement compensator is by using the reference picture that generates, for presumptive area to be encoded is calculated a bit shift compensation value; And

A transmit block is used to send a combination of calculating the information of employed reference image information of described bit shift compensation value and the described bit shift compensation value of indication;

Wherein, described reference image information is a combination of indicating the identification information of the unit that surveys motion vector and indicating the information of described image processing, and,

Described transmit block is used for sending the identification information of described reference image information, a described reference frame image and indicating the combination of the information of described bit shift compensation value with presumptive area unit to be encoded.

4. according to the moving picture coding apparatus of claim 1, wherein, described reference image information dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication according to the encoding condition of presumptive area to be encoded.

5. according to the moving picture coding apparatus of claim 2, wherein, described reference image information dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication according to the encoding condition of presumptive area to be encoded.

6. according to the moving picture coding apparatus of claim 3, wherein, described reference image information is surveyed the identification information of described motion vector unit and is indicated the action attitude that is combined into of the information of described image processing to change indication according to the encoding condition of presumptive area to be encoded.

7. according to any 1 moving picture coding apparatus among the claim 1-6, wherein,

Described reference picture generates the filtration treatment of the filter of parts by use being had some different passbands and carries out as described image processing, is used to generate described some different reference pictures, and

Indicate the information of described image processing, identify described filter.

8. according to any 1 moving picture coding apparatus among the claim 1-6,

Described reference picture generates parts and carries out as described image processing by the processing that will change spatial resolution, be used to generate described some different reference pictures, when using the described reference picture of low spatial resolution, described displacement compensator is used to reduce the employed motion vector precision of displacement calculating offset

Indicate the information of described image processing, identify described spatial resolution.

9. moving picture decoding apparatus, it will be decoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this equipment comprises:

A reference picture generates parts, is used for generating some different reference pictures by a reference frame image is carried out some different image processing;

Decoding parts are used for decoding to calculating the employed reference image information of bit shift compensation value at moving picture coding apparatus; And

A displacement compensator by using the described reference picture of the described reference image information appointment that generates, is used to calculate the bit shift compensation value of presumptive area to be decoded,

Wherein, described reference image information is the identification information of a described reference frame image and the combination of indicating the information of described image processing.

10. according to the moving picture decoding apparatus of claim 9, wherein,

The decoding parts are used for coming the information of described reference image information and the described bit shift compensation value of indication is decoded to survey motion vector unit, and

Described displacement compensator is used for switching the employed described reference picture of displacement calculating offset to survey motion vector unit.

11. a moving picture decoding apparatus, it will be decoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this equipment comprises:

A reference picture generates parts, is used for generating some different reference pictures by a reference frame image is carried out some different image processing;

Decoding parts are used for decoding to calculating the employed reference image information of bit shift compensation value at moving picture coding apparatus; And

A displacement compensator by using the described reference picture of the described reference image information appointment that generates, is used to calculate the bit shift compensation value of presumptive area to be decoded,

Wherein, described reference image information is that the identification information of motion vector unit and the combination of the information of the described image processing of indication are surveyed in an indication.

12. according to the moving picture decoding apparatus of claim 9, wherein, described reference image information according to the decode condition of presumptive area to be decoded, dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication.

13. according to the moving picture decoding apparatus of claim 10, wherein, described reference image information according to the decode condition of presumptive area to be decoded, dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication.

14. moving picture decoding apparatus according to claim 11, wherein, described reference image information is according to the decode condition of presumptive area to be decoded, to the attitude change of taking action that is combined into of identification information of indicating the unit that surveys described motion vector and the information of indicating described image processing.

15. according to any 1 moving picture decoding apparatus among the claim 9-14, wherein,

Described reference picture generates the filtration treatment of the filter of parts by use being had some different passbands and carries out as described image processing, is used to generate described some different reference pictures, and

Indicate the information of described image processing, identify described filter.

16. according to any 1 moving picture decoding apparatus among the claim 9-14,

Described reference picture generates parts and carries out as described image processing by the processing that will change spatial resolution, is used to generate described some different reference pictures,

When using the described reference picture of low spatial resolution, described displacement compensator is used to reduce the employed motion vector precision of displacement calculating offset,

Indicate the information of described image processing, identify described spatial resolution.

17. a mobile method for encoding images, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this method comprises:

A steps A by a reference frame image is carried out some different image processing, generates some different reference pictures;

A step B by using the reference picture that generates, calculates the bit shift compensation value of presumptive area to be encoded; And

A step C is used to send the combination of the information of calculating employed reference image information of described bit shift compensation value and the described bit shift compensation value of indication,

Wherein, described reference image information is the identification information of reference frame image and the combination of indicating the information of described image processing.

18. according to the mobile method for encoding images of claim 17, wherein,

In step B, the employed described reference picture of displacement calculating offset is switched to survey motion vector unit, and

In step C, the combination of the information of described reference image information and the described bit shift compensation value of indication is sent out to survey motion vector unit.

19. a mobile method for encoding images, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this method comprises:

A steps A by a reference frame image is carried out some different image processing, generates some different reference pictures;

A step B by using the reference picture that generates, calculates the bit shift compensation value of presumptive area to be encoded; And

A step C is used to send the combination of the information of calculating employed reference image information of described bit shift compensation value and the described bit shift compensation value of indication,

Wherein, described reference image information is the identification information and the combination of indicating described image processing data of the unit of an indication detection motion vector, and,

In step C, the identification information of described reference image information, a described reference frame image and indicate the combination of described bit shift compensation value information to be sent out with presumptive area unit to be encoded.

20. according to the mobile method for encoding images of claim 17, wherein, described reference image information dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication according to the encoding condition of presumptive area to be encoded.

21. according to the mobile method for encoding images of claim 18, wherein, described reference image information dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication according to the encoding condition of presumptive area to be encoded.

22. mobile method for encoding images according to claim 19, wherein, described reference image information is surveyed the identification information of described motion vector unit and is indicated the action attitude that is combined into of described image processing data to change indication according to the encoding condition of presumptive area to be encoded.

23. according to any 1 the mobile method for encoding images among the claim 17-22, wherein,

In steps A, by execution the filtration treatment that use has the filter of some different passbands is carried out as described image processing, generate described some different reference pictures, and

Indicate the information of described image processing, identify described filter.

24. according to any 1 the mobile method for encoding images among the claim 17-22, wherein,

In steps A, carry out as described image processing by the processing that will change spatial resolution, generate described some different reference pictures,

In step B, when using the described reference picture of low spatial resolution, reduce the employed motion vector precision of displacement calculating offset,

Indicate the information of described image processing, identify described spatial resolution.

25. a moving picture decoding method, it will be decoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this method comprises:

A steps A by a reference frame image is carried out some different image processing, is used to generate some different reference pictures;

A step B is used for decoding to calculating the employed reference image information of bit shift compensation value at moving picture coding apparatus; With

A step C by using the reference picture of the described reference image information appointment that generates, calculates the bit shift compensation value of presumptive area to be decoded,

Wherein, described reference image information is the identification information of a described reference frame image and the combination of indicating the information of described image processing.

26. according to the moving picture decoding method of claim 25, wherein,

In step B, the information of reference image information and the described bit shift compensation value of indication is decoded to survey motion vector unit, and

In step C, the employed described reference picture of displacement calculating offset is switched to survey motion vector unit.

27. a moving picture decoding method, it will be decoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this method comprises:

A steps A by a reference frame image is carried out some different image processing, is used to generate some different reference pictures;

A step B is used for decoding to calculating the employed reference image information of bit shift compensation value at moving picture coding apparatus; With

A step C by using the reference picture of the described reference image information appointment that generates, calculates the bit shift compensation value of presumptive area to be decoded,

Wherein, described reference image information is that the identification information of motion vector unit and the combination of the information of the described image processing of indication are surveyed in an indication.

28. according to the moving picture decoding method of claim 25, described reference image information according to the decode condition of presumptive area to be decoded, dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication.

29. according to the moving picture decoding method of claim 26, described reference image information according to the decode condition of presumptive area to be decoded, dynamically changes the identification information of a described reference frame image and the combination of the information of the described image processing of indication.

30. moving picture decoding method according to claim 27, wherein, described reference image information according to the decode condition of presumptive area to be decoded, is surveyed the identification information of described motion vector unit and is indicated the action attitude that is combined into of described image processing data to change indication.

31. according to any 1 the moving picture decoding method among the claim 25-30, wherein,

In steps A, the filtration treatment of the filter by use being had some different passbands is carried out as described image processing, generates described some different reference pictures, and

Indicate the information of described image processing, identify described filter.

32. according to any 1 the moving picture decoding method among the claim 25-30,

In steps A, carry out as described image processing by the processing that will change spatial resolution, be used to generate described some different reference pictures,

In step C, when using the described reference picture of low spatial resolution, reduced the employed motion vector precision of displacement calculating offset.

33. a moving picture coding apparatus, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this equipment comprises:

A reference picture generates parts, handles by a reference frame image being carried out some different images, is used to generate some different reference pictures;

One 3 dimension motion vector generates parts, is associated with the information of indication to the image processing of reference frame image execution by the motion vector that will use reference picture to detect, and is used to generate one 3 dimension motion vector;

A displacement compensator is by using the reference picture that generates, for presumptive area to be encoded is calculated a bit shift compensation value; And

A transmit block, the combination that is used to send one 3 dimension motion vector and indicates the information of displacement offset.

34. according to the moving picture coding apparatus of claim 33, wherein,

Described reference picture generates parts by carrying out the filtration treatment of using the filter with some different passbands, is used to generate described some different reference pictures, and

Described 3 dimension motion vectors identify described filter.

35. a moving picture coding apparatus, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, it is characterized in that this equipment comprises:

A reference picture generates parts, handles by a reference frame image being carried out some different images, is used to generate some different reference pictures;

One 3 dimension motion vector generates parts, is associated with the information of indication to the image processing of reference frame image execution by the motion vector that will use reference picture to detect, and is used to generate one 3 dimension motion vector;

A displacement compensator is by using the reference picture that generates, for presumptive area to be encoded is calculated a bit shift compensation value;

One 3 dimension motion vector prediction parts by using the presumptive area of encode in the two field picture and the correlation between the presumptive area to be encoded, are used to predict that 3 tie up motion vectors; And

A transmit block is used to send the combination of difference information between the described 3 dimension motion vectors that one 3 dimension motion vector generates 3 dimension motion vectors that parts generate and described 3 dimension motion vectors prediction parts predictions and indication displacement offset information.

36. according to the moving picture coding apparatus of claim 35, wherein, described 3 dimension motion vector prediction parts are used to predict described 3 dimension motion vectors by switching the encryption algorithm context.

37. according to the moving picture coding apparatus of claim 33, wherein,

Image processing is the processing that changes spatial resolution, and

Described 3 dimension motion vectors generate parts and are used to reduce the precision that described 3 of low spatial resolution reference picture is tieed up motion vectors.

38. a moving picture decoding apparatus is used for by bit shift compensation, will be decoded by the mobile image that the two field picture sequential is formed, and it is characterized in that this equipment comprises:

A reference picture generates parts, handles by a reference frame image being carried out some different images, is used to generate some different reference pictures;

Decoding parts are used for 3 dimension motion vectors of presumptive area to be decoded are decoded; And

A displacement compensator is by using the described reference picture by described 3 dimension motion vector appointments that generates, for presumptive area to be decoded is calculated a bit shift compensation value.

39. according to the moving picture decoding apparatus of claim 38, wherein,

Described reference picture generates parts by carrying out the filtration treatment of using the filter with some different passbands, is used to generate described some different reference pictures, and

Described 3 dimension motion vectors identify described filter.

40. a moving picture decoding apparatus is used for by bit shift compensation, will be decoded by the mobile image that the two field picture sequential is formed, and it is characterized in that this equipment comprises:

A reference picture generates parts, handles by a reference frame image being carried out some different images, is used to generate some different reference pictures;

Decoding parts are used for 3 dimension motion vectors of presumptive area to be decoded are decoded;

One 3 dimension motion vector prediction parts by using the presumptive area of decode in the two field picture and the correlation between the presumptive area to be decoded, are used to predict that 3 tie up motion vectors; And

A displacement compensator is tieed up the described 3 poor information of tieing up between the motion vectors that motion vector prediction parts are predicted by the 3 dimension motion vectors and described 3 that use the parts of decoding to decode, and calculates the bit shift compensation value of presumptive area to be decoded.

41. according to the moving picture decoding apparatus of claim 40, wherein, described 3 dimension motion vector prediction parts are predicted described 3 dimension motion vectors by switching the context of encryption algorithm.

42. a mobile method for encoding images, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, and this method comprises:

A steps A by a reference frame image is carried out some different image processing, generates some different reference pictures;

A step B is associated by the information that will use reference picture motion vector that detects and the image processing that is designated as the reference frame image execution, is used to generate one 3 dimension motion vector;

A step C by using the described reference picture that generates, calculates the bit shift compensation value of presumptive area to be encoded; And

A step D, the combination that is used to send one 3 dimension motion vector and indicates the information of displacement offset.

43. according to the mobile method for encoding images of claim 42, wherein,

In steps A, by carrying out the filtration treatment of using filter, generate described some different reference pictures with some different passbands, and

Described 3 dimension motion vectors identify described filter.

44. a mobile method for encoding images, it will be encoded by the mobile image that the two field picture sequential constitutes by bit shift compensation, and this method comprises:

A steps A by a reference frame image is carried out some different image processing, generates some different reference pictures;

A step B is associated by the information that will use reference picture motion vector that detects and the image processing that is designated as the reference frame image execution, is used to generate one 3 dimension motion vector;

A step C by using the described reference picture that generates, calculates the bit shift compensation value of presumptive area to be encoded;

A step e by using the presumptive area of encode in the two field picture and the correlation between the presumptive area to be encoded, is used to predict that 3 tie up motion vectors; And

Between the described 3 dimension motion vectors of predicting in 3 dimension motion vectors that generate among a step D, step B and the step e the combination of poor information and the information of indicating the displacement offset be sent out.

45., wherein, in step e, predict described 3 dimension motion vectors by switching the encryption algorithm context according to the mobile method for encoding images of claim 44.

46. according to the mobile method for encoding images of claim 42, wherein,

Image processing is the processing that changes spatial resolution, and

In step B,, reduce the precision of described 3 dimension motion vectors for the reference picture of low spatial resolution.

47. a moving picture decoding method is used for by bit shift compensation, will be decoded by the mobile image that the two field picture sequential is formed, this method comprises:

A steps A is used for handling by a reference frame image being carried out some different images, generates some different reference pictures;

A step B is used for 3 dimension motion vectors of presumptive area to be decoded are decoded; With

A step C is by using the described reference picture by described 3 dimension motion vector appointments that generates, for presumptive area to be decoded is calculated a bit shift compensation value.

48. according to the moving picture decoding method of claim 47, wherein,

In steps A, have the filtration treatment of some different pass-band filters by carrying out to use, generate some different reference pictures, and

Described 3 dimension motion vectors identify described filter.

49. a moving picture decoding method is used for by bit shift compensation, will be decoded by the mobile image that the two field picture sequential is formed, this method comprises:

A steps A is used for handling by a reference frame image being carried out some different images, generates some different reference pictures;

A step B is used for 3 dimension motion vectors of presumptive area to be decoded are decoded;

A step D by using the presumptive area of decode in the two field picture and the correlation between the presumptive area to be decoded, is used to predict that 3 tie up motion vectors; And

A step C, by use between the described 3 dimension motion vectors of predicting among the 3 dimension motion vectors of decoding among the step B and the step D poor information, calculate the bit shift compensation value of presumptive area to be decoded.

50., wherein, in step D, predict described 3 dimension motion vectors by switching the encryption algorithm context according to the moving picture decoding method of claim 49.

Images