HK1224114B - Image encoding device and method, image decoding device and method - Google Patents

Description

Image encoding device and method, and image decoding device and method

This application is a divisional application of the invention patent application with application number 201280005061.6 (PCT/JP2012/000061), application date January 6, 2012 (submission date July 11, 2013), and invention name "Motion image encoding device, motion image decoding device, motion image encoding method and motion image decoding method".

Technical Field

The present invention relates to a technology of an image coding device and an image coding method for efficiently coding an image, and an image decoding device and an image decoding method for efficiently decoding a coded image.

Background Art

For example, in international standard video coding methods such as MPEG (Moving Picture Experts Group) and "ITU-TH.26x", the input video frame is divided into rectangular blocks (coding target blocks), and a prediction image is generated by performing prediction processing on the coding target blocks using the encoded image signal. Information compression is performed by performing orthogonal transformation and quantization processing on the prediction error signal, which is the difference between the coding target block and the predicted image, in units of blocks.

For example, in MPEG-4 AVC/H.264 (ISO/IEC 14496-10│ITU-T H.264), which is an international standard, intra-frame prediction processing from encoded nearby pixels or motion compensation prediction processing between adjacent frames is performed (for example, see Non-Patent Document 1).

In MPEG-4 AVC/H.264, in the intra prediction mode of luma, one prediction mode can be selected from a plurality of prediction modes in units of blocks.

FIG. 10 is an explanatory diagram showing intra prediction modes when the luma block size is 4×4 pixels.

In FIG10 , white circles represent pixels in the block to be coded, and black circles represent pixels used for prediction, which are pixels in already coded adjacent blocks.

In FIG10 , nine intra-frame prediction modes, Mode 0 to Mode 8, are prepared, and Mode 2 is a mode for performing average value prediction, which predicts pixels within the encoding target block using the average value of adjacent pixels of the upper and left blocks.

Modes other than Mode 2 perform directional prediction. Mode 0 is vertical prediction, generating a predicted image by vertically repeating adjacent pixels in the upper block. For example, when the image is a vertical stripe pattern, select Mode 0.

Mode 1 is horizontal prediction, which generates a predicted image by repeating adjacent pixels of the left block in the horizontal direction. For example, when the image is a horizontal stripe pattern, select Mode 1.

Modes 3 to 8 generate a predicted image by generating interpolated pixels in a predetermined direction (direction indicated by an arrow) using adjacent pixels of the upper or left block.

The block size of luminance to which intra prediction is applied can be selected from 4×4 pixels, 8×8 pixels, and 16×16 pixels. In the case of 8×8 pixels, 9 intra prediction modes are specified as in the case of 4×4 pixels.

In the case of 16×16 pixels, four intra prediction modes (average prediction, vertical prediction, horizontal prediction, and planar prediction) are defined.

Planar prediction is a mode in which pixels generated by interpolating adjacent pixels of an upper block and adjacent pixels of a left block in a diagonal direction are used as prediction values.

The intra-frame prediction mode that performs directional prediction generates prediction values in a direction predetermined by the mode, such as 45 degrees. Therefore, when the direction of the boundary (edge) of the object in the block is consistent with the direction indicated by the prediction mode, the prediction efficiency becomes high and the coding amount can be reduced.

However, sometimes there is only a slight deviation between the direction of the edge and the direction indicated by the prediction mode, or even if the direction is the same but the edge in the encoding object block is slightly deformed (shaking, bending, etc.), a large local prediction error will occur, and the prediction efficiency will drop sharply.

In order to prevent this decline in prediction efficiency, in the directional prediction of 8×8 pixels, prediction processing is performed by smoothing the encoded adjacent pixels to generate a smoothed prediction image, thereby reducing the prediction error caused by slight deviations in the prediction direction and slight deformations at the edges.

Non-Patent Document 1: MPEG-4 AVC (ISO/IEC 14496-10)/ITU-TH.264 Standard

Since conventional image encoding devices are configured as described above, generating a smoothed prediction image can reduce the resulting prediction error even if there is a slight deviation in the prediction direction or slight distortion at the edge. However, in Non-Patent Document 1, smoothing is not performed on blocks other than 8×8 pixel blocks, and only general smoothing is performed on 8×8 pixel blocks.

In practice, the same problem exists for blocks of sizes other than 8×8 pixels: even if the patterns of the predicted image and the target image are similar, small mismatches at the edges can cause large localized prediction errors, significantly reducing prediction efficiency. Furthermore, even within blocks of the same size, different factors, such as the quantization parameter used to quantize the prediction error signal, the position of pixels within the block, and the prediction mode, lead to different processing for reducing localized prediction errors. Therefore, simply providing general smoothing processing cannot sufficiently reduce prediction errors.

Furthermore, there is the following problem: when performing average value prediction, since all predictions within the block are taken as the average value of the pixels adjacent to the block, the prediction signal of the pixels located at the block boundary is likely to become a discontinuous signal relative to the surrounding encoded pixels. On the other hand, because image signals are generally signals with high spatial directional correlation, prediction errors are likely to occur at the boundary part of the block due to the above-mentioned discontinuity.

Summary of the Invention

The present invention is proposed to solve the above-mentioned problems, and its object is to provide an image encoding device, an image decoding device, an image encoding method, and an image decoding method that can reduce locally occurring prediction errors and improve image quality.

Regarding the image encoding device according to the present invention, when the intra-frame prediction unit generates a predicted image by performing intra-frame prediction processing using the encoded image signal within the frame, a filter is selected from one or more pre-prepared filters according to the states of various parameters involved in the encoding of the filtering object block, and filtering processing is performed on the predicted image using the filter, and the predicted image after filtering processing is output to the differential image generation unit.

According to the present invention, when the intra-frame prediction unit generates a predicted image by performing intra-frame prediction processing using an encoded image signal within a frame, a filter is selected from one or more pre-prepared filters according to the states of various parameters involved in the encoding of the filtering object block, and filtering processing is performed on the predicted image using the filter, and the filtered predicted image is output to the differential image generation unit. Therefore, it has the effect of reducing locally occurring prediction errors and improving image quality.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a diagram showing the structure of a moving picture encoding apparatus according to Embodiment 1 of the present invention.

FIG2 is a diagram showing the configuration of the moving picture decoding apparatus according to the first embodiment of the present invention.

FIG3 is a flowchart showing the processing contents of the moving picture encoding apparatus according to Embodiment 1 of the present invention.

FIG4 is a flowchart showing the processing contents of the moving picture decoding apparatus according to the first embodiment of the present invention.

FIG5 is an explanatory diagram showing how a coding block of the maximum size is divided hierarchically into a plurality of coding blocks.

FIG6(a) shows the distribution of partitions after division, and FIG6(b) is an explanatory diagram showing, using a quadtree diagram, how coding modes m( ^Bn ) are assigned to partitions after hierarchical division.

FIG7 is an explanatory diagram showing an example of intra-frame prediction parameters (intra-frame prediction modes) that can be selected in each partition _Pin within ^the coding block ^Bn .

FIG8 is an explanatory diagram showing an example of pixels used when generating predicted values of ^pixels within a partition _Pin when _lin = _min = ^{4 .}

FIG. 9 is an explanatory diagram showing an example of reference pixel arrangement when N=5.

FIG. 10 is an explanatory diagram showing the intra prediction mode of Non-Patent Document 1 when the luminance block size is 4×4 pixels.

FIG. 11 is an explanatory diagram showing an example of the distance between an encoded image and a filtering target pixel within a frame used when generating a predicted image.

FIG. 12 is an explanatory diagram showing a specific arrangement of reference pixels of a filter.

FIG. 13 is an explanatory diagram showing an example of a table for determining which filter to use for each combination of an intra prediction mode index and a partition size.

FIG. 14 is an explanatory diagram showing an example of simplification of filtering processing when average value prediction is performed.

FIG15 is an explanatory diagram showing an example of a bit stream when a filter selection table index is added to a sequence level header.

FIG16 is an explanatory diagram showing an example of a bit stream when a filter selection table index is added to a picture-level header.

FIG17 is an explanatory diagram showing an example of a bit stream when a filter selection table index is added to a slice header.

FIG18 is an explanatory diagram showing an example of a bit stream when a filter selection table index is added to a reference block header.

FIG. 19 is an explanatory diagram showing an example different from FIG. 13 of a table for determining which filter to use for each combination of an intra prediction mode index and a partition size.

20 is an explanatory diagram showing an example of a table for determining whether or not to perform smoothing processing on reference pixels when generating an inter-prediction image for each combination of an intra prediction mode index and a partition size.

Explanation of symbols

1: Encoding control unit (encoding control unit)

2: Block division unit (block division unit)

3: Switch (intra-frame prediction unit, motion compensation prediction unit)

4: Intra-frame prediction unit (intra-frame prediction unit)

5: Motion Compensated Prediction Unit (Motion Compensated Prediction Unit)

6: Subtraction unit (difference image generation unit)

7: Transformation and quantization unit (image compression unit)

8: Inverse quantization and inverse transformation unit

9: Addition Department

10: Intra-frame prediction memory

11: Loop filter section

12: Motion compensation prediction frame memory

13: Variable length coding unit (variable length coding means)

51: Variable length decoding unit (variable length decoding means)

52: Switch (intra-frame prediction unit, motion compensation prediction unit)

53: Intra-frame prediction unit (intra-frame prediction unit)

54: Motion Compensation Prediction Unit (Motion Compensation Prediction Unit)

55: Inverse quantization and inverse transformation unit (difference image generation unit)

56: Adding unit (decoded image generating unit)

57: Intra-frame prediction memory

58: Loop filter section

59: Motion Compensated Prediction Frame Memory

100: filter selection table index

DETAILED DESCRIPTION

Hereinafter, in order to explain the present invention in more detail, embodiments will be described with reference to the accompanying drawings.

Implementation Method 1

In this first embodiment, a motion image encoding device and a motion image decoding device are described. The motion image encoding device inputs each frame image of a video, generates a predicted image by performing intra-frame prediction processing from encoded nearby pixels or performing motion compensation prediction processing between adjacent frames, performs compression processing based on orthogonal transformation and quantization on a prediction error signal which is a differential image between the predicted image and the frame image, and then performs variable-length coding to generate a bit stream. The motion image decoding device decodes the bit stream output from the motion image encoding device.

The moving picture coding apparatus of the first embodiment is characterized by dividing the video signal into regions of various sizes to perform intra-frame and inter-frame adaptive coding in response to local changes in the spatial and temporal directions of the video signal.

Video signals generally exhibit local variations in complexity across space and time. Spatially, within a given video frame, there are patterns with uniform signal characteristics across relatively wide image regions, such as the sky and walls, as well as patterns with complex textures within smaller image regions, such as figures and finely textured paintings.

Even when viewed temporally, although the changes in the time-direction patterns of the sky and the wall are locally small, the contours of moving people and objects undergo rigid and non-rigid motion in time, so the changes in time are large.

The encoding process generates a prediction error signal with low signal power and entropy through temporal and spatial prediction, thereby reducing the overall encoding amount. If the parameters used for prediction can be evenly applied to the largest possible image signal area, the encoding amount of the parameter can be reduced.

On the other hand, if the same prediction parameters are applied to an image signal pattern that varies greatly temporally and spatially, the prediction error increases, and thus the coding amount of the prediction error signal cannot be reduced.

Therefore, for image signal patterns that vary greatly temporally and spatially, it is desirable to reduce the area to be predicted, thereby reducing the power and entropy of the prediction error signal even if the amount of parameter data used for prediction is increased.

In order to perform coding adapted to the general properties of such video signals, the moving picture coding apparatus according to the first embodiment hierarchically divides the video signal into regions based on a predetermined maximum block size, and performs prediction processing and prediction error coding processing on each divided region.

The video signal processed by the motion image encoding device of this embodiment 1 is an arbitrary video signal whose video frame has horizontal and vertical two-dimensional columns of digital samples (pixels). In addition to color video signals of arbitrary color spaces such as YUV signals having a brightness signal and two color difference signals, RGB signals output from a digital camera element, there are also monochrome image signals, infrared image signals, etc.

The grayscale of each pixel may be 8 bits, 10 bits, 12 bits, or the like.

However, in the following description, unless otherwise specified, it is assumed that the input video signal is a YUV signal. In addition, it is assumed that the two color difference components U and V are subsampled with respect to the luminance component Y and are in a 4:2:0 format.

Furthermore, the unit of processed data corresponding to each video frame is called a "picture." In this first embodiment, a "picture" is described as a signal of a video frame that undergoes progressive scanning. However, if the video signal is an interlaced signal, a "picture" may also be a field image signal that constitutes a video frame.

FIG1 is a diagram showing the structure of a moving picture encoding apparatus according to Embodiment 1 of the present invention.

In Figure 1, the encoding control unit 1 implements the following processing: determines the maximum size of the coding block that serves as the processing unit when implementing intra-frame prediction processing (intra-frame prediction processing) or motion compensation prediction processing (inter-frame prediction processing), and determines the upper limit number of layers when the maximum-sized coding block is divided into layers.

Furthermore, the encoding control unit 1 performs processing for selecting an encoding mode suitable for each hierarchically divided encoding block from one or more available encoding modes (one or more intra-frame encoding modes and one or more inter-frame encoding modes).

Furthermore, the encoding control unit 1 performs processing to determine, for each coding block, the quantization parameter and transform block size used when compressing the differential image, and the intra-frame prediction parameters or inter-frame prediction parameters used when performing prediction processing. The quantization parameter and transform block size are included in the prediction error coding parameters and are output to the transform/quantization unit 7, the inverse quantization/inverse transform unit 8, and the variable length coding unit 13.

Furthermore, the encoding control unit 1 constitutes an encoding control unit.

When a video signal representing an input image is input, the block division unit 2 performs processing such that the input image represented by the video signal is divided into coding blocks of the maximum size determined by the coding control unit 1, and the coding blocks are divided hierarchically until the upper limit number of layers determined by the coding control unit 1 is reached. Furthermore, the block division unit 2 constitutes a block division unit.

The switching switch 3 implements the following processing: if the coding mode selected by the coding control unit 1 is the intra-frame coding mode, the coding block divided by the block division unit 2 is output to the intra-frame prediction unit 4; if the coding mode selected by the coding control unit 1 is the inter-frame coding mode, the coding block divided by the block division unit 2 is output to the motion compensation prediction unit 5.

The intra-frame prediction unit 4 performs the following processing: when receiving the coding block divided by the block division unit 2 from the switching switch 3, it uses the encoded image signal in the frame and generates a predicted image by performing intra-frame prediction processing on the coding block based on the intra-frame prediction parameters output from the encoding control unit 1.

However, after generating the above-mentioned predicted image, the intra-frame prediction unit 4 selects a filter from one or more pre-prepared filters based on the states of various parameters known at the time when a predicted image identical to the above-mentioned predicted image is generated in the motion image decoding device, uses the filter to perform filtering processing on the above-mentioned predicted image, and outputs the filtered predicted image to the subtraction unit 6 and the addition unit 9.

Specifically, the filter is uniquely determined based on the state of at least one of the following four parameters as the above-mentioned parameters.

Parameters (1)

The block size of the predicted image above

Parameters (2)

The quantization parameter determined by the encoding control unit 1

Parameters (3)

The distance between the coded image signal in the frame used to generate the predicted image and the pixel to be filtered

Parameters (4)

The intra-frame prediction parameters determined by the encoding control unit 1

Furthermore, the switch 3 and the intra prediction unit 4 constitute an intra prediction unit.

When the coding control unit 1 has selected the inter-frame coding mode as the coding mode appropriate for the coding block divided by the block division unit 2, the motion-compensated prediction unit 5 generates a predicted image by performing motion-compensated prediction processing on the coding block based on the inter-frame prediction parameters output from the coding control unit 1, using one or more reference images stored in the motion-compensated prediction frame memory 12. The switch 3 and the motion-compensated prediction unit 5 constitute a motion-compensated prediction unit.

The subtraction unit 6 generates a difference image (= coded block - predicted image) by subtracting the predicted image generated by the intra prediction unit 4 or the motion compensation prediction unit 5 from the coded block divided by the block division unit 2. The subtraction unit 6 constitutes a difference image generation unit.

The transform/quantization unit 7 performs a transform process (e.g., a DCT (discrete cosine transform) or an orthogonal transform such as a KL transform pre-designed for a specific learning sequence) on the difference image generated by the subtraction unit 6, using the transform block size included in the prediction error coding parameters output from the encoding control unit 1 as a unit. Furthermore, the transform coefficients of the difference image are quantized using the quantization parameters included in the prediction error coding parameters, thereby outputting the quantized transform coefficients as compressed data for the difference image. Furthermore, the transform/quantization unit 7 constitutes image compression means.

The inverse quantization and inverse transformation unit 8 performs the following processing: by using the quantization parameters included in the prediction error coding parameters output from the encoding control unit 1, the compressed data output from the transformation and quantization unit 7 is inverse quantized, and the inverse transformation processing (for example, inverse DCT (inverse discrete cosine transform), inverse KL transform, etc.) of the inverse quantized compressed data is performed in units of the transformation block size included in the prediction error coding parameters, thereby outputting the compressed data after the inverse transformation processing as a local decoding prediction error signal.

The adding unit 9 performs the following processing: by adding the local decoded prediction error signal output from the inverse quantization/inverse transform unit 8 and the prediction signal representing the predicted image generated by the intra-frame prediction unit 4 or the motion compensation prediction unit 5, a local decoded image signal representing the local decoded image is generated.

The intra prediction memory 10 is a storage medium such as RAM that stores the local decoded image represented by the local decoded image signal generated by the adder 9 as an image to be used in the next intra prediction process by the intra prediction unit 4 .

The loop filter unit 11 performs processing to compensate for coding distortion included in the local decoded image signal generated by the adder 9 and outputs the local decoded image represented by the local decoded image signal after coding distortion compensation to the motion compensation prediction frame memory 12 as a reference image.

The motion-compensated prediction frame memory 12 is a recording medium such as RAM that stores the local decoded image filtered by the loop filter unit 11 as a reference image to be used by the motion-compensated prediction unit 5 in the next motion-compensated prediction process.

The variable length coding unit 13 performs variable length coding on the compressed data output from the transform/quantization unit 7, the coding mode and prediction error coding parameters output from the coding control unit 1, and the intra-frame prediction parameters output from the intra-frame prediction unit 4 or the inter-frame prediction parameters output from the motion compensation prediction unit 5, thereby generating a bit stream of coded data that multiplexes the compressed data, coding mode, prediction error coding parameters, and intra-frame prediction parameters/inter-frame prediction parameters. Furthermore, the variable length coding unit 13 constitutes a variable length coding unit.

FIG2 is a diagram showing the configuration of the moving picture decoding apparatus according to the first embodiment of the present invention.

In FIG2 , the variable-length decoding unit 51 performs variable-length decoding of compressed data, coding modes, prediction error coding parameters, and intra-frame prediction parameters/inter-frame prediction parameters for each hierarchically divided coding block from the coded data multiplexed in the bitstream, outputs the compressed data and prediction error coding parameters to the inverse quantization/inverse transform unit 55, and outputs the coding mode and intra-frame prediction parameters/inter-frame prediction parameters to the switch 52. Furthermore, the variable-length decoding unit 51 constitutes a variable-length decoding unit.

The switching switch 52 performs the following processing: when the coding mode involved in the coding block output from the variable length decoding unit 51 is the intra-frame coding mode, the intra-frame prediction parameters output from the variable length decoding unit 51 are output to the intra-frame prediction unit 53; when its coding mode is the inter-frame coding mode, the inter-frame prediction parameters output from the variable length decoding unit 51 are output to the motion compensation prediction unit 54.

The intra prediction unit 53 performs a process of generating a predicted image by performing an intra prediction process on the coding block based on the intra prediction parameters output from the switch 52 using the decoded image signal in the frame.

Among them, after generating the above-mentioned prediction image, the intra-frame prediction internal 53 selects a filter from one or more pre-prepared filters according to the states of various parameters known at the time of generating the above-mentioned prediction image, uses the filter to perform filtering processing on the above-mentioned prediction image, and outputs the filtered prediction image to the addition unit 56.

Specifically, as the various parameters described above, the filter is uniquely determined based on the state of at least one of the following four parameters. The parameters used are predetermined to be the same as those used in the previously described moving picture encoding device. That is, the parameters used in the moving picture encoding device and the moving picture decoding device are unified so that when the intra-frame prediction unit 4 on the moving picture encoding device performs filtering using parameters (1) and (4), the intra-frame prediction unit 53 on the moving picture decoding device similarly performs filtering using parameters (1) and (4).

Parameters (1)

The block size of the above predicted image

Parameters (2)

Quantization parameter of variable length decoding performed by the variable length decoding unit 51

Parameters (3)

The distance between the decoded image signal in the frame used to generate the predicted image and the pixel to be filtered

Parameters (4)

Intra-frame prediction parameters that are variable-length decoded by the variable-length decoding unit 51

The switch 52 and the intra prediction unit 53 constitute an intra prediction unit.

The motion-compensated prediction unit 54 generates a predicted image by performing motion-compensated prediction on the coding block based on the intra-frame prediction parameters output from the switch 52, using one or more reference images stored in the motion-compensated prediction frame memory 59. The switch 52 and the motion-compensated prediction unit 54 constitute a motion-compensated prediction unit.

The inverse quantization/inverse transformation unit 55 performs the following processing: using the quantization parameter included in the prediction error coding parameter output from the variable length decoding unit 51, it inversely quantizes the compressed data associated with the coded block output from the variable length decoding unit 51. By performing inverse transformation processing (e.g., inverse DCT (inverse discrete cosine transform) or inverse KL transform) on the inversely quantized compressed data in units of the transform block size included in the prediction error coding parameter, the inversely transformed compressed data is output as a decoded prediction error signal (a signal representing the difference image before compression). Furthermore, the inverse quantization/inverse transformation unit 55 constitutes a difference image generation unit.

The addition unit 56 performs processing to generate a decoded image signal representing a decoded image by adding the decoded prediction error signal output from the inverse quantization/inverse transform unit 55 and a prediction signal representing a predicted image generated by the intra prediction unit 53 or the motion compensation prediction unit 54. Furthermore, the addition unit 56 constitutes a decoded image generation unit.

The intra prediction memory 57 is a recording medium such as RAM that stores the decoded image represented by the decoded image signal generated by the adder 56 as an image used by the intra prediction unit 53 in the next intra prediction process.

The loop filter unit 58 performs processing to compensate for coding distortion included in the decoded image signal generated by the adder 56 , and outputs a decoded image represented by the decoded image signal after coding distortion compensation as a reference image to the motion compensation prediction frame memory 59 .

The motion compensation prediction frame memory 59 is a recording medium such as RAM that stores the decoded image subjected to the loop filtering process by the loop filter unit 58 as a reference image to be used in the next motion compensation prediction process by the motion compensation prediction unit 54 .

In FIG. 1 , it is assumed that the components of the moving picture coding apparatus, namely, the coding control unit 1, the block division unit 2, the switch 3, the intra-frame prediction unit 4, the motion-compensated prediction unit 5, the subtraction unit 6, the transform/quantization unit 7, the inverse quantization/inverse transform unit 8, the addition unit 9, the loop filter unit 11, and the variable-length coding unit 13, are each configured using dedicated hardware (e.g., a semiconductor integrated circuit or a single-chip microcomputer having a CPU mounted thereon). However, if the moving picture coding apparatus is configured using a computer, a program recording the processing details of the coding control unit 1, the block division unit 2, the switch 3, the intra-frame prediction unit 4, the motion-compensated prediction unit 5, the subtraction unit 6, the transform/quantization unit 7, the inverse quantization/inverse transform unit 8, the addition unit 9, the loop filter unit 11, and the variable-length coding unit 13 may be stored in a memory of the computer, and the CPU of the computer may execute the program stored in the memory.

FIG3 is a flowchart showing the processing contents of the moving picture encoding apparatus according to Embodiment 1 of the present invention.

In FIG. 2 , it is assumed that the variable-length decoding unit 51, the switch 52, the intra-frame prediction unit 53, the motion-compensated prediction unit 54, the inverse quantization/inverse transform unit 55, the addition unit 56, and the loop filter unit 58, which are components of the moving picture decoding apparatus, are each configured using dedicated hardware (e.g., a semiconductor integrated circuit or a single-chip microcomputer equipped with a CPU). However, if the moving picture decoding apparatus is configured using a computer, a program recording the processing contents of the variable-length decoding unit 51, the switch 52, the intra-frame prediction unit 53, the motion-compensated prediction unit 54, the inverse quantization/inverse transform unit 55, the addition unit 56, and the loop filter unit 58 may be stored in a memory of the computer, and the CPU of the computer may execute the program stored in the memory.

FIG4 is a flowchart showing the processing contents of the moving picture decoding apparatus according to the first embodiment of the present invention.

The following describes the operation.

First, the processing contents of the moving picture encoding apparatus in FIG. 1 will be described.

First, the encoding control unit 1 determines the maximum size of the coding block that serves as the processing unit when performing intra-frame prediction processing (intra-frame prediction processing) or motion compensation prediction processing (inter-frame prediction processing), and determines the upper limit number of layers when hierarchically dividing the coding block of the maximum size (step ST1 of Figure 3).

As a method for determining the maximum size of a coding block, for example, a method of determining a size corresponding to the resolution of an input image for all pictures may be considered.

Another possible method is to quantify the difference in complexity of local motion in the input image as a parameter, and determine the maximum size to be a small value in images with intense motion and a large value in images with less motion.

For example, the following method can be considered to set the upper limit of the number of layers: the more intense the motion of the input image, the deeper the number of layers to detect finer motions; and if the motion of the input image is small, the number of layers is reduced.

Furthermore, the encoding control unit 1 selects an encoding mode suitable for each hierarchically divided encoding block from one or more available encoding modes (M intra-frame encoding modes and N inter-frame encoding modes) (step ST2).

The method for selecting the coding mode based on the coding control unit 1 is a well-known technology, so detailed description is omitted. For example, there are methods such as: using any available coding mode, performing coding processing on the coding block, verifying the coding efficiency, and selecting the coding mode with the best coding efficiency from multiple available coding modes.

Furthermore, the encoding control unit 1 determines, for each encoding block, a quantization parameter and a transform block size used when compressing a differential image, and determines intra-frame prediction parameters or inter-frame prediction parameters used when performing a prediction process.

The encoding control unit 1 outputs prediction error coding parameters including quantization parameters and transform block size to the transform/quantization unit 7, inverse quantization/inverse transform unit 8, and variable length encoding unit 13. Furthermore, the prediction error coding parameters are output to the intra prediction unit 4 as needed.

When a video signal representing an input image is input, the block division unit 2 divides the input image represented by the video signal into coding blocks of the maximum size determined by the coding control unit 1, and divides the coding blocks into layers until the upper limit number of layers determined by the coding control unit 1 is reached.

Here, FIG5 is an explanatory diagram showing how a coding block of the maximum size is hierarchically divided into a plurality of coding blocks.

In the example of FIG5 , the largest-sized coding block is coding block B ⁰ of layer 0, which has a size of (L ⁰ , M ⁰ ) according to the luminance component.

In the example of FIG. 5 , the maximum-sized coding block B ⁰ is used as a starting point, and the coding block B ⁿ is obtained by hierarchically dividing the coding block B 0 to a predetermined depth determined separately using a quad-tree structure.

At depth n, coding block ^Bn is an image region of size ( ^Ln , ^Mn ).

However, ^Ln and ^Mn may be the same or different, but the example in FIG5 shows a case where ^Ln = ^Mn .

Hereinafter, the size of coding block ^Bn is defined as ( ^Ln , ^Mn ) in the luma component of coding block ^Bn .

Since the block dividing unit 2 performs quad-tree division, (Ln ⁺¹ , Mn ⁺¹ ) = ( ^Ln /2, ^Mn /2) always holds.

In a color video signal (4:4:4 format) where all color components have the same number of samples, such as an RGB signal, the size of all color components is ( ^Ln , ^Mn ). However, when processing the 4:2:0 format, the size of the coding block of the corresponding color difference component is ( ^Ln /2, ^Mn /2).

Hereinafter, the coding mode that can be selected in the coding block ^Bn of the nth layer is denoted as m( ^Bn ).

In the case of a color video signal having multiple color components, the coding mode m( ^Bn ) can be configured to use an independent mode for each color component. However, unless otherwise specified, the following description will be based on the coding mode for the luminance component of a coding block in a YUV signal 4:2:0 format.

Among the coding modes m( ^Bn ), there are one or more intra-frame coding modes (collectively referred to as "INTRA") and one or more inter-frame coding modes (collectively referred to as "INTER"). As described above, the coding control unit 1 selects the coding mode with the highest coding efficiency for the coding block ^Bn from all coding modes available for the picture or a subset thereof.

As shown in FIG5 , the coding block ^Bn is further divided into one or more prediction processing units (partitions).

Hereinafter, the partitions belonging to the coding block ^Bn are denoted as _Pin ⁽ i: partition number of the nth layer).

The coding mode m(B ⁿ ) contains information on how the partition P _i ⁿ belonging to the coding block B ⁿ is divided.

All partitions P _i ⁿ are predicted according to the coding mode m(B ⁿ ), but for each partition P _i ⁿ , independent prediction parameters can be selected.

The encoding control unit 1 generates a block division state as shown in FIG. 6 , for example, for the maximum-sized encoding block, and determines the encoding block B ⁿ .

The shaded areas in FIG6(a) represent the distribution of partitions after division, and FIG6(b) shows, using a quadtree diagram, how coding modes m( ^Bn ) are assigned to partitions after hierarchical division.

In FIG6(b), the nodes surrounded by squares represent nodes (coding blocks ^Bn ) to which coding mode m( ^Bn ) is assigned.

When the encoding control unit 1 selects the optimal encoding mode m( ^Bn ) for each partition _P in the ^encoding block ^Bn , if the encoding mode m( ^Bn ) is an intra-frame encoding mode (step ST3), the switching switch 3 outputs the ^partition _P in the encoding block ^Bn divided by the block dividing unit 2 to the intra-frame prediction unit 4.

On the other hand, if the coding mode m(B ⁿ ) is the inter-frame coding mode (step ST3 ), the partitions _P ⁱⁿ of the coding block B ⁿ divided by the block dividing unit 2 are output to the motion compensation prediction unit 5 .

Upon receiving ^the partition _Pin of the coding block ^Bn from the switching switch 3, the intra-frame coding prediction unit 4 uses the encoded image signal within the frame and, based on the intra-frame prediction parameters output from the coding control unit 1, performs intra-frame prediction processing ^on the partition _Pin of the coding block ^Bn to generate an intra-frame predicted image _Pin ( ^step ST4).

However, after generating the intra-frame prediction image ^{P in} _, the intra-frame prediction unit 4 selects a filter from one or more pre-prepared filters based on the states of various parameters known at the time of generating a prediction image identical to ^the intra-frame prediction image _P in the motion picture decoding device, and performs filtering processing on the intra-frame prediction image _P using the ^selected filter.

When the intra-frame prediction unit 4 performs filtering processing on the intra-frame ^{prediction image P in} _, it outputs the filtered intra-frame prediction image _P ⁱⁿ to the subtraction unit 6 and the addition unit 9. Since the same intra-frame prediction image ^{P in} _can be generated even in the moving picture decoding apparatus of FIG. 2 , the intra-frame prediction parameters are output to the variable-length coding unit 13.

The outline of the processing contents of the intra prediction unit 4 is as described above, and the details of the processing contents will be described later.

Upon receiving ^the partition _Pin of the coding block ^Bn from the switching switch 3, the motion-compensated prediction unit 5 uses one or more reference images stored in the motion-compensated prediction frame memory 12 and, based on the inter-frame prediction parameters output from the encoding control unit 1, performs motion-compensated prediction processing on the partition _Pin of the coding block ^Bn , thereby generating ^{an inter} -frame prediction image _Pin (step ST5).

Furthermore, the technology for generating a predicted image by performing motion compensation prediction processing is a well-known technology, and therefore detailed description thereof will be omitted.

When the intra prediction unit 4 or the motion compensation prediction unit 5 generates a predicted image (intra ^prediction image _Pin , inter ^prediction image _Pin ), the subtraction unit 6 generates a ^difference image by subtracting the predicted image (intra prediction image _Pin , inter prediction image _Pin ⁾ generated by the intra prediction unit 4 or the motion compensation prediction unit 5 from the partition Pin of the coding block _Bn divided by the block division unit ² , ^and outputs a prediction error signal _ein representing the ^difference image to the transformation and quantization unit 7 (step ST6).

When the transform/quantization unit 7 receives the prediction error signal e _i ⁿ representing the differential image from the subtraction unit 6, it performs transform processing (for example, DCT (discrete cosine transform), orthogonal transform processing such as KL transform that is pre-designed for a specific learning series) on the differential image in units of the transform block size included in the prediction error coding parameter output from the encoding control unit 1, and quantizes the transform coefficients of the differential image using the quantization parameter included in the prediction error coding parameter, thereby outputting the quantized transform coefficients as compressed data of the differential image to the inverse quantization/inverse transform unit 8 and the variable length coding unit 13 (step ST7).

When receiving the compressed data of the differential image from the transform/quantization unit 7, the inverse quantization/inverse transformation unit 8 uses the quantization parameter included in the prediction error coding parameter output from the encoding control unit 1 to inverse quantize the compressed data of the differential image, and performs inverse transformation processing (for example, inverse transformation processing such as inverse DCT (inverse discrete cosine transform) and inverse KL transform) on the inverse quantized compressed data in units of the transform block size included in the prediction error coding parameter, thereby outputting the compressed data after the inverse transformation processing as a local decoding prediction error signal ^{e in} _hat (due to electronic application, the "＾" attached to the alphabetical character is represented as hat) to the addition unit 9 (step ST8).

Upon receiving the local decoding prediction error signal ein _- ^hat from the inverse quantization/inverse transform unit 8, the adder 9 adds the local decoding prediction error signal ^ein- _hat to a prediction signal representing a prediction image (intra-frame prediction image _Pin , inter-frame prediction image _Pin ) generated by the intra-frame prediction unit ⁴ or the motion compensation prediction unit ⁵ , thereby generating a local decoded image, which is the local decoding partition image _Pin ^- hat or a local decoding coding block image as a collection thereof (step ST9).

When generating a local decoded image, the adder 9 stores a local decoded image signal representing the local decoded image in the intra prediction memory 10 , and outputs the local decoded image signal to the loop filter 11 .

The processing of steps ST3 to ST9 is repeatedly performed until the processing of all the hierarchically divided coding blocks ^Bn is completed. When the processing of all the coding blocks ^Bn is completed, the process shifts to step ST12 (steps ST10 and ST11).

The variable length coding unit 13 performs entropy coding on the compressed data output from the transform/quantization unit 7, the coding mode (including information indicating the segmentation status of the coding block) and the prediction error coding parameters output from the coding control unit 1, the intra-frame prediction parameters output from the intra-frame prediction unit 4, or the inter-frame prediction parameters output from the motion compensation prediction unit 5.

The variable length coding unit 13 multiplexes the compressed data as the encoding result of the entropy encoding, the encoding mode, the prediction error encoding parameter, and the encoded data of the intra prediction parameter/inter prediction parameter to generate a bit stream (step ST12).

When receiving the local decoded image signal from the adder 9, the loop filter unit 11 compensates for the coding distortion contained in the local decoded image signal, and stores the local decoded image represented by the local decoded image signal after coding distortion compensation as a reference image in the motion compensation prediction frame memory 12 (step ST13).

The filtering processing performed by the loop filter unit 11 can be performed in units of the maximum coding block or individual coding blocks of the local decoded image signal output from the adder 9, or can be performed in units of a plurality of maximum coding blocks, or can be performed by aggregating one picture after outputting the local decoded image signal of one picture.

Next, the processing contents of the intra prediction unit 4 will be described in detail.

FIG7 is an explanatory diagram showing an example of intra-frame prediction parameters (intra-frame prediction modes) that can be selected in each partition _Pin within ^the coding block ^Bn .

In the example of FIG. 7 , the intra prediction modes and the prediction direction vectors indicated by the intra prediction modes are shown, and the relative angles between the prediction direction vectors decrease as the number of selectable intra prediction modes increases.

The intra prediction unit 4 performs intra prediction processing on the partition _Pin based on the intra prediction parameters for the partition ^Pin and the selection parameters of the filter used to generate ^{the intra} prediction _{image Pin} .

The following describes the intra-frame processing of generating an intra-frame prediction signal of the luminance signal based on the intra-frame prediction parameters (intra-frame prediction mode) of the luminance signal for the ^partition _Pin .

Here, it is assumed ^that the size of ^{the partition} _Pin is _lin × _min pixels.

FIG8 is an explanatory diagram showing an example of pixels used when generating pixel prediction values within ^a partition _Pin when _lin = _min = ^{4 .}

In FIG8 , (2× _lin +1) pixels of the coded upper partition and ^{(2×min )} pixels of the left partition adjacent to ^partition _Pin are used as pixels for prediction. However, the pixels used for prediction may be more or less than those shown in _FIG8 .

In addition, although one adjacent row or column of pixels is used for prediction in FIG8 , two rows or two columns of pixels, or more, may be used for prediction.

When the index value of the intra- ^frame prediction mode for partition _Pin is 2 (average prediction), the average value of the adjacent pixels of the upper partition and the adjacent pixels of the left partition is used as the prediction value of all pixels ⁱⁿ partition _Pin to generate an intermediate prediction image.

When the intra prediction mode index value is other than 2 (average prediction), the predicted value of the pixel in the ^partition _Pin is generated according to the prediction direction vector _vp = (dx, dy) represented by the partition value.

Here, it is assumed that the relative coordinates (with the upper left pixel of the partition as the origin) within the partition ^Pin of the _pixel generating the predicted value (prediction target pixel) are (x, y).

The position of the reference pixel used in the prediction becomes the intersection of A shown below and the adjacent pixel.

Here, k is a negative scalar value.

When the reference pixel is at an integer pixel position, the integer pixel is set as the predicted value of the prediction target pixel.

On the other hand, when the reference pixel is not located at an integer pixel position, an interpolated pixel generated from integer pixels adjacent to the reference pixel is used as a predicted value.

In the example of Figure 8, since the reference pixel is not at an integer pixel position, the predicted value is calculated by interpolation based on two pixels adjacent to the reference pixel. However, the predicted value is not limited to two adjacent pixels; an interpolated pixel can be generated based on two or more adjacent pixels and used as the predicted value.

Then, by performing filtering processing (described later) on the intermediate prediction image formed by the prediction values within ^the partition _Pin generated according to the above steps, a final intra-frame prediction image _Pin is obtained, ^and the intra-frame prediction image _Pin ^is output to the subtraction unit 6 and the addition unit 9.

In addition, the intra-frame prediction parameters used to generate the intra-frame prediction image _P ⁱⁿ are output to the variable-length coding unit 13 for multiplexing in the bit stream.

The following describes specific filtering processing.

A filter to be used is selected from at least one filter prepared in advance by a method described later, and filtering processing is performed on each pixel of the intermediate prediction image according to the following equation (1).

In the formula (1), a _n (n=0, 1, ..., N) is a filter coefficient including coefficients related to reference pixels (a ₀ , a ₁ , ..., a _N-1 ) and an offset coefficient a _N .

P _n (n=0, 1, ..., N-1) represents reference pixels of the filter including the filtering target p _0. N is an arbitrary number of reference pixels.

s(p _n ) represents the luminance value of each reference pixel, and s hat (p ₀ ) represents the luminance value after filtering at the filtering target pixel p ₀ .

However, the filter coefficients may be configured without the offset coefficient a _N. Furthermore, the luminance value s(p _n ) of each reference pixel within the partition P _i ⁿ may be the luminance value of each pixel in the intermediate prediction image, or may be the luminance value after filtering is performed only at the pixel position where filtering is completed.

For the luminance value s( _pn ) of each reference pixel outside ^the partition _P in a coded area, the coded luminance value (decoded luminance value) is used; if the area is not yet coded, a substitute signal value is selected in a prescribed order from the luminance values s( _pn ) of each reference pixel within the partition _P ^defined above and the coded luminance values of the coded area (for example, the signal value at the nearest position among the candidate pixels is selected).

FIG. 9 is an explanatory diagram showing an example of reference pixel arrangement when N=5.

When performing the above ^- mentioned filtering process, the larger ^the size ( _line × _minute ) of ^the partition _Pin , the more likely it is that nonlinear edges will exist in the input image, and since deviations from the prediction direction of the intermediate prediction image are likely to occur, it is preferable to smooth the intermediate prediction image.

Furthermore, the larger the quantization value of the prediction error, the greater the quantization distortion generated in the decoded image. Since the prediction accuracy of the intermediate prediction image generated based on the encoded pixels adjacent to ^the partition _Pin is reduced, it is preferred to prepare a smoothed prediction image that roughly represents the ^partition _Pin .

Furthermore, even for pixels within the same ^partition _Pin , since the farther the pixel is from the encoded pixel adjacent to the partition _Pin used in generating the ^intermediate prediction image, the more likely it is that an offset such as an edge will occur between the intermediate prediction image and the input image, it is better to smooth the prediction image to suppress the drastic increase in prediction error when an offset occurs.

Furthermore, intra-frame prediction for generating an intermediate predicted image is configured to perform prediction using one of two different methods: average prediction, where all predicted values within a prediction block are the same, or prediction using a prediction direction vector vp _. Furthermore, even when using prediction using the prediction direction vector _vp , pixels not located at integer pixel positions are generated by interpolating between pixels whose predicted values are directly used as reference pixels at integer pixel positions and at least two reference pixels. The placement of pixels within the prediction block whose predicted values are used differs depending on the direction of the prediction direction vector _vp . Therefore, the properties of the predicted image differ depending on the intra-frame prediction mode, and the optimal filtering process also differs. Therefore, it is preferable to change the filter strength, the number of reference pixels used by the filter, the reference pixel placement, and other factors according to the intra-frame prediction mode index value.

Therefore, the filter selection process is configured to select a filter in consideration of the following four parameters (1) to (4).

(1) Size of ^partition _Pin ^{(l in} _{× min} ⁱⁿ )

(2) Quantization parameters included in the prediction error coding parameters

(3) The distance between the group of coded pixels used in generating the intermediate prediction image (“pixels used in prediction” shown in FIG8 ) and the pixel to be filtered

(4) Index value of the intra-frame prediction mode when generating the intermediate prediction image

Specifically, the system is configured such that as ^{the size (l in × m in} ₎ ^of _the ^partition _Pin increases, the quantization value determined by the quantization parameter increases, and the distance between the target pixel to be filtered and the coded ^pixel groups to the left and above the partition _Pin increases, a filter with a stronger smoothing strength or a filter with a larger number of reference pixels is used. Figure 11 is an example of the distance between the target pixel to be filtered and the coded pixel groups to the left and above the ^partition _Pin. Furthermore, the system is configured to switch the filter strength, the number of reference pixels used in the filter, the reference pixel arrangement, and the like using the index value of the intra-frame prediction mode.

That is, for each combination of the above-mentioned parameter combinations, by matching an appropriate filter from a pre-prepared filter group, adaptive selection of a filter corresponding to the above-mentioned parameters is achieved. In addition, for example, when combining parameters (3) and (4), the definition of the "distance from the encoded pixel group" of parameter (3) can also be appropriately changed according to the "intra-frame prediction mode" of parameter (4). In other words, instead of fixing the definition of the distance from the encoded pixel group as shown in Figure 11, it can be set to a distance that depends on the prediction direction, such as the distance from the "reference pixel" shown in Figure 8. By processing in this way, adaptive filtering processing can be achieved that also takes into account the relationship between multiple parameters such as parameters (3) and (4).

Furthermore, among these parameter combinations, a combination in which no filtering is performed may be prepared corresponding to “no filtering.” Furthermore, as a definition of filter strength, the weakest filter may be defined as “no filtering.”

Furthermore, since the four parameters (1) to (4) are already known parameters even in the video decoding apparatus, no additional information to be encoded is generated in order to perform the above-mentioned filtering process.

Moreover, in the above description, the filter switching is performed by preparing the necessary number of filters in advance for adaptive selection, but the filter switching can also be achieved by defining the filter as a function of the above filter selection parameter in a manner of calculating the filter based on the value of the above filter selection parameter.

Furthermore, in the above description, a case is shown in which a filter is selected by considering four parameters (1) to (4), but a filter may be selected by considering at least one parameter among the four parameters (1) to (4).

The following describes a configuration example of filtering processing in which appropriate filters are matched to each combination of parameter combinations from a pre-prepared filter group to perform adaptive selection of filters, using the case of using parameters (1) and (4) as an example.

The filter used in the above-mentioned filtering process example is defined as follows.

Filter with filter index 1 (reference pixel number N=3)

a ₀ = 3/4, a ₁ = 1/8, a ₂ = 1/8

Filter with filter index 2 (reference pixel number N=3)

a ₀ =1/2, a ₁ =1/4, a ₂ =1/4

Filter with filter index 3 (reference pixel number N=3)

a ₀ =1/4, a ₁ =3/8, a ₂ =3/8

Filter index 4 (reference pixel number N = 5)

a ₀ =1/4, a ₁ =3/16, a ₂ =3/16, a ₃ =3/16, a ₄ =3/16

However, assuming that the filtering process is based on equation (1) with no offset coefficient a _N (a _N = 0), and assuming that three types of filters are used this time, the reference pixel arrangement of the filters is as shown in FIG. 12 .

Next, FIG13 is an explanatory diagram showing an example of a table indicating filters used in each intra prediction mode for each size of ^a partition _Pin . Assuming that the available _Pin sizes are 4×4 pixels, 8×8 pixels, 16×16 pixels, 32×32 pixels, and 64×64 pixels, the correspondence between ^the intra prediction mode index value and the intra prediction direction is as shown in FIG7 .

In addition, filter index 0 indicates that no filtering is performed. Generally, when directional prediction or average value prediction is used, because the following trend exists, as shown in the table in FIG13 , by considering the characteristics of the image in intra-frame prediction, the filter to be used is mapped in the table for each combination of parameters (1) and (4), thereby enabling appropriate filter switching to be achieved by referring to the table.

Horizontal and vertical edges are often found in artificial objects such as buildings, and they often have clear, linear edges. Therefore, horizontal and vertical prediction can often achieve high-precision prediction. Therefore, it is best not to perform smoothing during horizontal and vertical prediction.

Generally, because image signals have high spatial continuity, when using average value prediction that impairs the continuity of coded pixels adjacent to ^the partition _Pin , it is preferable to improve continuity by smoothing the pixels around the ^block boundaries to the left and above the partition _Pin .

·The larger the area of a region with oblique directionality, the more likely it is that edges will be distorted and have nonlinear shapes. Therefore, when using oblique direction prediction, it is better to apply a larger partition size, a stronger smoothing filter, and a larger number of reference pixels.

Generally, if the partition size is too large, the spatial variation of signal values within the partition becomes diverse, resulting in only very coarse predictions for directional and average predictions, and increasing the number of areas where high-precision prediction is difficult. In such areas, smoothing simply blurs the image and cannot be expected to improve prediction efficiency. Therefore, it is ideal to omit filtering at such a partition size because this avoids unnecessary computational overhead (for example, in Figure 13, filtering is not performed for partition sizes larger than 32×32 pixels).

Furthermore, when the reference pixel in the filtering process is a pixel within ^the partition _Pin , the luminance value of the intermediate prediction image is used as the luminance value of the reference pixel, which can sometimes simplify the filtering process. For example, when the intra prediction mode is average value prediction, the filtering process for ^the partition _Pin can be simplified to the following filtering process for each region shown in FIG. 14 .

Region A (the upper left pixel of ^partition _Pin )

Filter with filter index 1 (no change)

a ₀ = 3/4, a ₁ = 1/8, a ₂ = 1/8 (reference pixel number N = 3)

Filter with filter index 2 (no change)

a ₀ = 1/2, a ₁ = 1/4, a ₂ = 1/4 (reference pixel number N = 3)

Filter with filter index 3 (no change)

a ₀ = 1/4, a ₁ = 3/8, a ₂ = 3/8 (refer to the number of pixels N = 3)

Filter with filter index 4

a ₀ = 5/8, a ₁ = 3/16, a ₂ = 3/16 (referring to the number of pixels N = 3)

Region B (the pixels at the top of the ^partition _Pin outside of region A)

Filter with filter index 1

a ₀ = 7/8, a ₂ = 1/8 (reference pixel number N = 2)

Filter with filter index 2

a ₀ = 3/4, a ₂ = 1/4 (reference pixel number N = 2)

Filter with filter index 3 (no change)

a ₀ = 5/8, a ₂ = 3/8 (reference pixel number N = 2)

Filter with filter index 4

a ₀ = 13/16, a ₂ = 3/16 (refer to the number of pixels N = 2)

Region C (the pixels at the left end of the ^partition _Pin outside region A)

Filter with filter index 1

a ₀ = 7/8, a ₁ = 1/8 (reference pixel number N = 2)

Filter with filter index 2

a ₀ = 3/4, a ₁ = 1/4 (reference pixel number N = 2)

Filter with filter index 3

a ₀ = 5/8, a ₁ = 3/8 (reference pixel number N = 2)

Filter with filter index 4

a ₀ = 13/16, a ₁ = 3/16 (refer to the number of pixels N = 2)

Region D (pixels in the ^partition _Pin outside regions A, B, and C)

All filter indexed filters:

No filtering

Even if the filtering process is simplified as described above, the filtering process result is the same as before the simplification.

By eliminating redundant portions of actual processing in this manner, it is possible to increase the speed of filtering processing.

While the table in Figure 13 was used in the above example, other tables may be used. For example, when reducing processing load is more important than improving coding performance, the table in Figure 19 ^may be used instead of the table in Figure 13. With this table, filtering is performed only for average value predictions of partitions _Pin sizes of 4×4 pixels, 8×8 pixels, and 16×16 pixels. Therefore, fewer prediction modes are filtered than when the table in Figure 13 is used, thus suppressing the increase in computational complexity associated with filtering. In this case, by also utilizing the simplified filtering process when the intra-frame prediction mode is average value prediction, filtering with a significantly lower processing load can be achieved.

Furthermore, when ease of implementation is also important, as with the above-mentioned filtering process, it is assumed that filtering is performed only in the case of average value prediction, and the filter to be used further is not switched according to the size ^of the partition _Pin , and the same filter (for example, the filter with filter index 2) can be always used. In this case, although the improvement effect of the coding performance based on the filter is reduced by the amount corresponding to not performing ^the processing corresponding to the size of the partition _Pin , the circuit scale (the number of lines of code in the case of software) during implementation can be suppressed. This filtering process is a filter that considers only parameter (4) among the four parameters (1) to (4).

As an implementation method for the filtering process, even if it is not implemented in a form in which a filter index corresponding to the filter is selected using a reference table, a form in which the filtering process is directly performed on each size of the ^partition _Pin , or a form in which the filtering process is directly performed on each pixel position of each size of ^the partition _Pin can be used. In this way, even if it is not implemented in a reference table, as long as the predicted image obtained as a result of the filtering process is equivalent, the implementation method is not relevant.

In addition, in the example described above, a method of using only one table for switching filters was described, but it is also possible to prepare multiple such tables and encode the filter selection table index 100 as header information in one of the formats shown in Figures 15 to 18, so that the filter selection table can be switched using a specified unit.

For example, as shown in FIG15 , by adding a filter selection table index 100 to the sequence level header, it is possible to perform filtering processing that is appropriate to the characteristics of the sequence, compared to the case where only a single table is used.

Moreover, similar to the smoothing process performed on the reference image during the intra-frame prediction of the 8×8 pixel block in MPEG-4 and AVC/H.264 described above, even in a case where the reference pixels when generating the intermediate prediction image of the partition _Pin in the intra-frame prediction unit ⁴ are used as pixels on which the coded pixels adjacent to the ^partition _Pin are smoothed, the same filtering process as in the above example can be performed on the intermediate prediction image.

On the other hand, because the effects of smoothing and filtering the reference pixels when generating the intermediate prediction images overlap, sometimes using both processes simultaneously only yields a slight performance improvement compared to performing either process alone. Therefore, when prioritizing computational complexity, a configuration can be employed where filtering is not performed on the intermediate prediction images for the partition _Pin where smoothing is performed on the reference ^pixels when generating the intermediate prediction images. For example, consider a case where filtering is performed only for average value prediction, as shown in the table in FIG19 , while smoothing is performed only for specific directional predictions, as shown in FIG20 . In FIG20 , '1' indicates smoothing is performed, and '0' indicates not smoothing.

The intra-frame prediction parameters used to generate the intra-frame prediction image _P ⁱⁿ are output to the variable-length coding unit 13 in order to be multiplexed in the bit stream.

The color difference signal within ^the partition _Pin is also subjected to intra-frame prediction processing based on the intra-frame prediction parameters (intra-frame prediction mode) in the same order as the luminance signal, and the intra-frame prediction parameters used in generating the intra-frame prediction image are output to the variable length coding unit 13.

However, the intra-frame prediction of the chrominance signal may be configured to perform the filtering process described above in the same manner as the luminance signal, or may not perform the filtering process described above.

Next, the processing contents of the moving picture decoding apparatus in FIG2 will be described.

When the variable length decoding unit 51 receives the bit stream output from the image encoding apparatus of FIG. 1 , it performs variable length decoding on the bit stream and decodes frame size information in units of a sequence of pictures having one or more frames or in units of pictures (step ST21 of FIG. 4 ).

The variable length decoding unit 51 determines the maximum size of the coding block that serves as the processing unit when performing intra-frame prediction processing (intra-frame prediction processing) or motion compensation prediction processing (inter-frame prediction processing) in the same steps as the coding control unit 1 in Figure 1, and determines the upper limit number of layers when hierarchically dividing the coding block of the maximum size (step ST22).

For example, in an image encoding apparatus, when the maximum size of a coding block is determined according to the resolution of an input image, the maximum size of the coding block is determined according to previously decoded frame size information.

Furthermore, when information indicating the maximum size of a coding block and the upper limit number of layers is multiplexed in a bitstream, the information decoded from the bitstream is referenced.

The coding mode m(B ⁰ ) of the maximum-sized coding block B ⁰ multiplexed in the bitstream includes information indicating the division state of the maximum-sized coding block B ^0. Therefore, the variable-length decoding unit 51 decodes the coding mode m(B ⁰ ) of the maximum-sized coding block B ⁰ multiplexed in the bitstream and determines each hierarchically divided coding block B ⁿ (step ST23).

When determining each coding block ^Bn , the variable length decoding unit 51 decodes the coding mode m( ^Bn ) of the coding block ^Bn , and determines the partition ^Pin belonging to the coding block ^Bn based on the partition _Pin information belonging to the coding mode _m ( ^{Bn )} .

When the partition _P ⁱⁿ belonging to the coding block B ⁿ is determined, the variable length decoding unit 51 decodes the compressed data, the coding mode, the prediction error coding parameters, and the intra prediction parameters/inter prediction parameters for each partition _P ⁱⁿ (step ST24).

That is, when the coding mode m(B ⁿ ) assigned to the coding block B ⁿ is the intra coding mode, intra prediction parameters are decoded for each partition P _i ⁿ belonging to the coding block.

When the coding mode m(B ⁿ ) assigned to the coding block B ⁿ is the inter coding mode, inter prediction parameters are decoded for each partition P _i ⁿ belonging to the coding block.

The partition serving as the prediction processing unit is further divided into one or more partitions serving as the transform processing unit based on the transform block size information included in the prediction error coding parameter, and compressed data (transformed and quantized transform coefficients) is decoded for each partition serving as the transform processing unit.

When the coding mode m( ^Bn ) of the partition _P in ^the coding block ^Bn from the variable length decoding unit 51 is the intra coding mode (step ST25), the switch 52 outputs the intra prediction parameters output from the variable length decoding unit 51 to the intra prediction unit 53.

On the other hand, when the coding mode m(B ⁿ ) of ^the partition _Pin is the inter coding mode (step ST25 ), the inter prediction parameters output from the variable length decoding unit 51 are output to the motion compensation prediction unit 54 .

Upon receiving the intra-frame prediction parameters from the switch 52, the intra-frame prediction unit 53, like the intra-frame prediction unit 4 in FIG1 , uses the decoded image signal within the frame and, based on the intra-frame prediction parameters, performs intra-frame prediction processing ^on the partition _P in the coding block ^Bn to generate an intra- ^frame predicted image _P (step ST26).

However, after generating the intra-frame prediction image P in the same manner as the intra-frame prediction unit 4 in _FIG. ¹ , the intra-frame prediction unit 53 selects a filter from one or more pre-prepared filters based on the states of various parameters known at the time of generating the intra- ^frame prediction image _P in the same manner as the intra- ^frame prediction unit 4 in FIG. 1 , and performs filtering processing on the intra-frame prediction image _P in using the filter, and uses the filtered intra-frame prediction image _P ⁱⁿ as the final intra-frame prediction image.

That is, the filtering process is performed by selecting a filter using the same parameters as those used for filter selection in the intra prediction unit 4 and by using the same method as the filter selection method in the intra prediction unit 4 .

For example, the intra-frame prediction unit 4 is configured to correspond to the filter index 0 when no filtering processing is performed, and the four pre-prepared filters are respectively corresponded to the filter indexes 1 to 4. When filtering processing is performed with reference to the table in Figure 13, the intra-frame prediction unit 53 also defines the same filters and filter indexes as the intra-frame prediction unit 4, and performs filtering processing by selecting a filter based on the size of the partition _Pin and the index of the ^intra -frame prediction mode as the intra-frame prediction parameter with reference to the table in Figure 13.

In addition, as in the example described above, it is also possible to configure such that: when a table defining filters used in units of parameter combinations is prepared and filters are switched by referring to the table, the filter selection table index 100 is decoded as header information in the form of one of Figures 15 to 18, and the table represented by the decoded filter selection table index 100 is selected from the same table group as that of the pre-prepared motion picture encoding device, and the filter is selected by referring to the table.

Upon receiving the inter-frame prediction parameters from the switching switch 52, the motion-compensated prediction unit 54 uses one or more reference frames stored in the motion-compensated prediction frame memory 59 to perform motion-compensated prediction processing on the partition _P in the coding block _B according to the inter- ^frame prediction parameters, thereby generating an inter- ^frame prediction image _P (step ST27).

The inverse quantization and inverse transformation unit 55 uses the quantization parameters included in the prediction error coding parameters output from the variable length decoding unit 51 to inverse quantize the compressed data related to the coding block output from the variable length decoding unit 51, and uses the transformation block size included in the prediction error coding parameters as a unit to perform inverse transformation processing (for example, inverse transformation processing such as inverse DCT (inverse discrete cosine transform) and inverse KL transform) on the inverse quantized compressed data, and outputs the compressed data after the inverse transformation processing as a decoded prediction error signal (a signal representing the differential image before compression) to the addition unit 56 (step ST28).

When the addition unit 56 receives the decoded prediction error signal from the inverse quantization/inverse transformation unit 55, it generates a decoded image by adding the decoded prediction error signal and a prediction signal representing a predicted image generated by the intra-frame prediction unit 53 or the motion compensation prediction unit 54, stores the decoded image signal representing the decoded image in the intra-frame prediction memory 57, and outputs the decoded image signal to the loop filter unit 58 (step ST29).

The processes of steps ST23 to ST29 are repeatedly performed until the processes on all the hierarchically divided coding blocks ^Bn are completed (step ST30).

When receiving the decoded image signal from the adder 56, the loop filter unit 58 compensates for the coding distortion contained in the decoded image signal, and stores the decoded image represented by the decoded image signal after coding distortion compensation as a reference image in the motion compensation prediction frame memory 59 (step ST31).

The filtering process performed by the loop filter unit 58 can be performed in units of the largest coding block or each coding block of the decoded image signal output from the adder 56, or can be performed by aggregating the decoded image signals of one picture after outputting the decoded image signals of macroblocks equivalent to one picture.

As can be seen from the above, according to the present embodiment 1, since the intra-frame prediction unit 4 constructed as a motion image encoding device uses the encoded image signal within the frame, when generating the intra-frame predicted image by performing intra-frame prediction processing, a filter is selected from one or more pre-prepared filters according to the states of various parameters involved in the encoding of the filtering object block, and the predicted image is filtered using the filter, thereby achieving the effect of reducing the prediction error that occurs locally and improving the image quality.

Furthermore, according to the first embodiment, since the intra-frame prediction unit 4 is configured to select a filter by taking into account at least one of the parameters (1) to (4), it is possible to suppress a local prediction error caused by slight nonlinear distortion of the edge of the encoding target image or angular deviation when performing directional prediction, and a prediction error at the boundary of a block caused by loss of continuity with adjacent encoded signals when performing average value prediction, thereby achieving an effect of improving prediction efficiency. The parameters (1) are ^the size ( _lin × _min ) of ^the partition _Pin , (2) are the quantization parameters included in ^the prediction error coding parameters, (3) are the distances between the encoded pixel group used in generating the intermediate prediction image and the filtering target pixel, and (4) are the index values of the intra-frame prediction mode used in generating the intermediate prediction image.

According to the first embodiment, since the intra-frame prediction unit 53 configured as a motion picture decoding device uses the decoded image signal within the frame, when generating an intra-frame predicted image by performing intra-frame prediction processing, a filter is selected from one or more pre-prepared filters based on the states of various parameters involved in the decoding of the filtering target block, and filtering processing is performed on the predicted image using the filter. This has the effect of reducing the prediction error that occurs locally, and generating an intra-frame predicted image on the motion picture decoding device side that is the same as the intra-frame predicted image generated on the motion picture encoding device side.

Furthermore, according to the first embodiment, since the intra prediction unit 53 is configured to select a filter by taking into account at least one of the parameters (1) to (4), it is possible to suppress a local prediction error caused by slight nonlinear distortion of the edge of the image to be coded or by angular deviation when performing directional prediction, and a prediction error at the boundary of a block caused by loss of continuity with adjacent coded signals when performing average value prediction, thereby achieving an effect of generating an intra prediction image on the side of the moving picture decoding apparatus that is the same as the intra prediction image generated on the side of the moving picture coding apparatus. The parameters (1) are ^the size ( _lin × _min ) of the ^partition _Pin , (2) are the quantization parameters included in ^the prediction error coding parameters, (3) are the distances between the coded pixel group used in generating the intermediate prediction image and the pixel to be filtered, and (4) are the index values of the intra prediction mode used in generating the intermediate prediction image.

Implementation Method 2

In the above-mentioned embodiment 1, the following method is represented, that is, the intra-frame prediction unit 4 uses the encoded image signal in the frame, and when generating the intra-frame predicted image by performing intra-frame prediction processing, a filter is selected from one or more pre-prepared filters according to the states of various parameters involved in the encoding of the filtering object block, and the predicted image is filtered using the filter. However, when a Wiener filter is designed in which the sum of square errors between the encoding object block and the predicted image is minimized, and the degree of reduction in prediction error becomes higher when the above-mentioned Wiener filter is used compared with the use of the filter selected from one or more pre-prepared filters, the predicted image can also be filtered using the above-mentioned Wiener filter instead of the selected filter.

The following describes the processing details in detail.

In the first embodiment described above, the intra-frame prediction units 4 and 53 select a filter from one or more pre-prepared filters based on the states of various parameters related to the encoding of the target block to be filtered. However, when the filter is selected based on the four parameters (1) to (4), an appropriate filter can be selected from the selection candidates. However, if an optimal filter exists outside the selection candidates, "optimal filtering" cannot be performed.

In this embodiment 2, it is characterized in that: an optimal filter is designed on the side of the motion image encoding device in units of pictures to implement filtering processing, and the filter coefficients of the filter are encoded, and the filtering processing using the filter is implemented on the side of the motion image decoding device by decoding the filter coefficients.

The intra prediction unit 4 of the moving picture encoding device generates an intra prediction image _P ⁱⁿ by performing an intra prediction process on the partition P ⁿ of the coding block B ⁿ , similarly to the first embodiment.

In addition, the intra-frame prediction unit 4 uses the same method as the above-mentioned embodiment 1 to select a filter from one or more pre-prepared filters according to the states of various parameters related to the encoding of the filtering target block, and uses the filter to filter the intra-frame prediction image P _i ⁿ .

After the intra-frame prediction unit 4 determines the intra-frame prediction parameters in all the coding blocks ^Bn in the picture, it designs a Wiener filter for each area using the same filter in the picture (area with the same filter index) so that the sum of the square errors between the input image and the intra-frame prediction image in the area (the mean square error in the object area) is minimized.

The Wiener filter can calculate the filter coefficient w using the autocorrelation matrix R _s's' of the intermediate predicted image signal s' and the cross-correlation matrix R _ss' between the input image signal s and the intermediate predicted image signal s' according to the following equation (4). The sizes of the matrices R _s's' and R _ss' correspond to the number of filter taps required.

When the intra-frame prediction unit 4 designs the Wiener filter, it assumes that the sum of square errors in the filter design object area when the Wiener filter is used to perform filtering processing is D1, the coding amount when encoding information related to the Wiener filter (for example, filter coefficients) is R1, and the sum of square errors in the filter design object area when the filter is selected in the same way as in the above-mentioned embodiment 1 to perform filtering processing is D2, and confirms whether the following formula (5) holds.

D1+λ·R1＜D2 (5)

Here, λ is a constant.

When equation (5) holds true, the intra prediction unit 4 performs filtering using the Wiener filter instead of the filter selected in the same manner as in the first embodiment.

On the other hand, when the equation (5) does not hold, filtering processing is performed using a filter selected in the same manner as in the first embodiment.

Here, the evaluation is performed using the sum of squared errors D1 and D2, but the present invention is not limited thereto. Instead of the sum of squared errors D1 and D2, another measure representing prediction distortion, such as the sum of absolute values of errors, may be used for evaluation.

When the intra prediction unit 4 performs filtering using a Wiener filter, filter update information indicating the filter coefficients of the Wiener filter and the filter index to be replaced with the Wiener filter is required.

Specifically, when the number of filters that can be selected by filtering processing using filter selection parameters is set to L, and each filter is assigned an index of 0 to L-1, for each index, when using a designed Wiener filter, it is necessary to encode a "1" value as filter update information, and when using a pre-prepared filter, it is necessary to encode a "0" value as filter update information.

The variable-length coding unit 13 performs variable-length coding on the filter update information output from the intra prediction unit 4 and multiplexes the coded data of the filter update information into a bit stream.

Here, although the case is shown in which a Wiener filter is designed so that the mean square error between the input image and the predicted image in each area using the same filter in the picture is minimized, it can also be constructed so that a Wiener filter is designed so that the mean square error between the input image and the predicted image in each area using the same filter is minimized for other specific area units that are not picture units. It can also be possible to design a filter only for a specific picture, or to design the above-mentioned filter only in accordance with specific conditions (for example, when a scene change detection function is added and a picture with a scene change is detected).

The variable-length decoding unit 51 of the moving picture decoding device performs variable-length decoding on the filter update information from the encoded data multiplexed into the bit stream.

The intra prediction unit 53 generates ^an intra prediction image _Pin by performing an intra prediction process ^on the partition _Pin of the coding block ^Bn , similarly to the first embodiment.

Upon receiving the filter updating information from the variable-length decoding unit 51 , the intra prediction unit 53 refers to the filter updating information and checks whether or not the filter of the corresponding index has been updated.

When the result of confirmation is that the filter of a certain area is replaced by a Wiener filter, the intra-frame prediction unit 53 reads the filter coefficient of the Wiener filter contained in the filter update information, determines the Wiener filter, and uses the Wiener filter to perform filtering processing on the intra-frame prediction image P _i ⁿ .

On the other hand, in the region not replaced by the Wiener filter, a filter is selected by the same method as in the first embodiment, and filtering processing is performed on the intra- ^frame prediction image _Pin using this filter.

As can be seen from the above, according to the present embodiment 2, since the Wiener filter is designed to minimize the sum of square errors between the block of the encoding object and the predicted image, when the degree of reduction in the prediction error becomes higher than that of using a filter selected from one or more pre-prepared filters, the Wiener filter is used instead of the selected filter to perform filtering processing on the predicted image, so that the effect of further reducing the locally occurring prediction error can be achieved compared with the above-mentioned embodiment 1.

Furthermore, within the scope of the present invention, the various embodiments can be freely combined, arbitrary components of the various embodiments can be modified, or arbitrary components can be omitted in the various embodiments.

Industrial applicability

The present invention is suitable for an image encoding device that needs to encode an image efficiently, and is also suitable for an image decoding device that needs to decode an encoded image efficiently.

Claims (4)

1. An image encoding device, characterized in that it comprises: The intra-prediction unit, when the coding mode corresponding to the coded block obtained by segmenting the input image is an intra-coding mode, performs intra-prediction processing on the block that becomes the unit of prediction processing of the coded block, and generates a prediction image. In the intra-frame prediction unit, when performing average prediction, filtering is applied to the top and left-hand predicted pixels within the block that is the unit of prediction processing, using an intermediate prediction value defined as the average of multiple neighboring pixels of the block that becomes the unit of prediction processing, and the neighboring pixels of the predicted pixel. When the predicted target pixel is a pixel located at the top of the block other than the top-left pixel within the block that becomes the unit of prediction processing for the coded block, the filter coefficients involved in the intermediate prediction value are set to 3/4, and the filter coefficients involved in the pixel adjacent to the top of the predicted target pixel are set to 1/4. 2. An image decoding device, characterized in that it comprises: The intra-prediction unit, when the coding mode involved in the coding block is intra-coding mode, performs intra-prediction processing on the block that becomes the unit of prediction processing of the coding block, and generates a prediction image. In the intra-prediction unit, when the intra-prediction parameter represents average prediction, when filtering is performed on the top and left-end pixels within the block that is the unit of prediction processing, using an intermediate prediction value defined as the average of multiple neighboring pixels of the block that becomes the unit of prediction processing, and the neighboring pixels of the prediction target pixel, When the predicted target pixel is a pixel located at the top of the block other than the top-left pixel within the block that becomes the unit of prediction processing for the coded block, the filter coefficients involved in the intermediate prediction value are set to 3/4, and the filter coefficients involved in the pixel adjacent to the top of the predicted target pixel are set to 1/4. 3. An image encoding method, characterized in that it comprises: The intra-frame prediction processing step, when the coding mode corresponding to the coded block obtained by segmenting the input image is an intra-frame coding mode, performs intra-frame prediction processing on the block that becomes the unit of prediction processing of the coded block to generate a prediction image. In the intra-frame prediction processing step, when performing average prediction, filtering is applied to the top and left-hand predicted pixels within the block that is the unit of the prediction processing, using an intermediate prediction value defined as the average of multiple neighboring pixels of the block that becomes the unit of the prediction processing, and the neighboring pixels of the predicted target pixel. When the predicted target pixel is a pixel located at the top of the block other than the top-left pixel within the block that becomes the unit of prediction processing for the coded block, the filter coefficients involved in the intermediate prediction value are set to 3/4, and the filter coefficients involved in the pixel adjacent to the top of the predicted target pixel are set to 1/4. 4. An image decoding method, characterized in that it comprises: The intra-frame prediction processing step, when the coding mode involved in the coding block is an intra-frame coding mode, performs intra-frame prediction processing on the block that becomes the unit of prediction processing for the coding block, generating a prediction image. In the intra-frame prediction processing step, when the intra-frame prediction parameter represents average prediction, when filtering is performed on the top and left-hand predicted pixels within the block that is the unit of the prediction processing, using an intermediate prediction value defined as the average of multiple neighboring pixels of the block that becomes the unit of the prediction processing, and the neighboring pixels of the predicted target pixel, the filtering process is performed on these pixels. When the predicted target pixel is a pixel located at the top of the block other than the top-left pixel within the block that becomes the unit of prediction processing for the coded block, the filter coefficients involved in the intermediate prediction value are set to 3/4, and the filter coefficients involved in the pixel adjacent to the top of the predicted target pixel are set to 1/4.