HK1234555B - Image encoding device and method, image decoding device and method and storage medium - Google Patents

Description

Image encoding device and method, image decoding device and method, and recording medium

This application is a divisional application of the invention patent application with application number 201280021843.9 (PCT/JP2012/003555), application date May 30, 2012 (submission date November 5, 2013), and invention name "Motion image encoding device, motion image decoding device, motion image encoding method and motion image decoding method".

Technical Field

The present invention relates to an image encoding device and an image encoding method for efficiently encoding an image, an image decoding device and an image decoding method for decoding an efficiently encoded image, and an image prediction device.

Background Art

For example, in international standard image coding methods such as MPEG (Moving Picture Experts Group) and "ITU-TH.26x", the input image frame is divided into rectangular blocks (coding blocks), and the coding blocks are subjected to prediction processing using the encoded image signals to generate predicted images. The prediction error signal, which is the difference between the coding block and the predicted image, is orthogonally transformed and quantized in block units to perform information compression.

For example, in MPEG-4 AVC/H.264 (ISO/IEC 14496-10 | ITU-T H.264), which is an international standard, intra-frame prediction processing or motion compensation prediction processing between adjacent frames is performed based on coded nearby pixels (for example, see Non-Patent Document 1).

In MPEG-4 AVC/H.264, in the intra prediction mode of luma, one prediction mode can be selected from a plurality of prediction modes in units of blocks.

FIG. 14 is an explanatory diagram showing the intra prediction mode in a case where the block size of luminance is 4×4 pixels.

In Figure 14, white circles within a block represent pixels to be coded, and black circles represent already coded pixels used for prediction. When the luma block size is 4×4 pixels, nine intra-frame prediction modes, Mode 0 to Mode 8, are defined.

In FIG14 , mode 2 is a mode for performing average value prediction, in which pixels within a block are predicted using the average value of adjacent pixels above and to the left of the block.

Modes other than Mode 2 perform directional prediction. Mode 0 is vertical prediction, generating a predicted image by vertically repeating adjacent pixels above a block. For example, when using a vertical stripe pattern, select Mode 0.

Mode 1 is horizontal prediction, which generates a predicted image by repeating the adjacent pixels to the left of the block in the horizontal direction. For example, mode 1 is selected when a horizontal stripe pattern is used.

In Modes 3 to 8, already coded pixels above or to the left of a block are used to generate interpolated pixels in a predetermined direction (direction indicated by an arrow) to generate a predicted image.

Here, the luma block size for intra-frame prediction can be selected from 4×4 pixels, 8×8 pixels, and 16×16 pixels. In the case of 8×8 pixels, nine intra-frame prediction modes are defined, similar to 4×4 pixels. However, the pixels used for prediction are not the coded pixels themselves, but rather pixels obtained by applying filtering to these pixels.

On the other hand, in the case of 16×16 pixels, in addition to the intra prediction modes related to average value prediction, vertical prediction, and horizontal prediction, four intra prediction modes called Plane prediction are defined.

The intra prediction mode related to Plane prediction is a mode in which pixels generated by interpolating already coded adjacent pixels above and to the left of a block in an oblique direction are used as prediction values.

In addition, in the directional prediction mode, the prediction value is generated by repeating the adjacent pixels of the block in a predetermined direction (prediction direction) or interpolated pixels generated based on the adjacent pixels. Therefore, when the direction of the boundary (edge) of the target within the prediction object block is consistent with the prediction direction as shown in Figure 15, and the signal value within the block is constant along the prediction direction, the prediction efficiency becomes high and the symbol amount can be reduced.

Non-Patent Document 1: MPEG-4 AVC (ISO/IEC 14496-10)/ITU-T H.264 Standard

Summary of the Invention

Conventional moving picture coding devices are configured as described above. If the direction of the boundary (edge) of an object within a prediction block coincides with the prediction direction and the signal value within the prediction block is constant along the prediction direction, high-precision prediction can be achieved by using directional prediction. However, even if the direction of the boundary (edge) of an object within a prediction block coincides with the prediction direction, prediction errors can increase if the signal value varies along the prediction direction, as shown in FIG16 .

The present invention is completed to solve the above-mentioned problems, and its purpose is to obtain an image encoding device, an image decoding device, an image encoding method, an image decoding method and an image prediction device, which can achieve high-precision prediction to improve image quality even when the signal value changes along the prediction direction.

In the image encoding device of the present invention, when the intra-frame prediction parameter indicates horizontal prediction processing, the intra-frame prediction unit adds a value proportional to the change in the horizontal brightness value of the pixel adjacent to the top of the block to the brightness value of the pixel adjacent to the left of each block serving as the unit of prediction processing for the coding block, and uses the added value as the prediction value of the predicted image. When the intra-frame prediction parameter indicates vertical prediction processing, the intra-frame prediction unit adds a value proportional to the change in the vertical brightness value of the pixel adjacent to the top of the block to the brightness value of the pixel adjacent to the left of the block, and uses the added value as the prediction value of the predicted image.

According to the present invention, the intra-frame prediction unit is constructed to add a value proportional to the amount of change in the horizontal brightness value of the pixel adjacent to the upper side of the block to the brightness value of the pixel adjacent to the left side of the block when the intra-frame prediction parameter indicates horizontal prediction processing, and to use the added value as the prediction value of the predicted image when the intra-frame prediction parameter indicates vertical prediction processing, and to add a value proportional to the amount of change in the vertical brightness value of the pixel adjacent to the left side of the block to the brightness value of the pixel adjacent to the upper side of the block, and use the added value as the prediction value of the predicted image. Therefore, it has the effect of improving image quality by achieving high-precision prediction even when the signal value changes along the prediction direction.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a diagram showing the structure of a moving picture encoding apparatus according to Embodiment 1 of the present invention.

FIG2 is a flowchart showing the processing contents (moving picture encoding method) of the moving picture encoding apparatus according to Embodiment 1 of the present invention.

FIG3 is a diagram showing the configuration of the moving picture decoding apparatus according to Embodiment 1 of the present invention.

FIG4 is a flowchart showing the processing contents (moving picture decoding method) of the moving picture decoding device according to Embodiment 1 of the present invention.

FIG5 is an explanatory diagram showing an example in which a maximum coding block is hierarchically divided into a plurality of coding blocks.

FIG6(a) is an explanatory diagram showing the distribution of coding blocks and prediction blocks after division, and FIG6(b) is an explanatory diagram showing the state of allocation of coding mode m( ^Bn ) by hierarchical division.

FIG7 is an explanatory diagram showing an example of intra-frame prediction parameters (intra-frame prediction modes) selectable by each prediction ^block _P in the coding block ^Bn .

^FIG8 is an explanatory diagram showing an example of pixels used when generating predicted values of _pixels in a prediction block ^Pin when _lin = _min = ⁴ .

Figure 9 is an explanatory diagram showing relative coordinates with the upper left pixel in the prediction ^block _Pin as the origin.

FIG. 10 is an explanatory diagram showing an example of adjacent pixels of a left prediction block that are referred to in order to calculate the amount of change in luminance value added to a prediction value in conventional vertical prediction.

FIG. 11 is an explanatory diagram showing an example of a scaling value of a luminance value variation added to a prediction value in a conventional vertical prediction.

FIG. 12 is an explanatory diagram showing an example of adjacent pixels of an upper prediction block that are referred to in order to calculate the amount of change in luminance value added to a prediction value in conventional horizontal prediction.

FIG. 13 is an explanatory diagram showing an example of a scaling value of a luminance value variation added to a prediction value in a conventional method of horizontal prediction.

FIG. 14 is an explanatory diagram showing the intra prediction mode in a case where the block size of luminance is 4×4 pixels.

FIG. 15 is an explanatory diagram showing an example of a predicted image predicted with high accuracy by horizontal prediction.

FIG. 16 is an explanatory diagram showing an example in which a large prediction error occurs when prediction is performed using horizontal prediction.

FIG17 is an explanatory diagram showing an example of intra-frame prediction parameters (intra-frame prediction modes) selectable by each prediction ^block _P in the coding block ^Bn .

(Explanation of Symbols)

1: Block division unit (block division unit); 2: Coding control unit (coding control unit); 3: Switch; 4: Intra-frame prediction unit (intra-frame prediction unit); 5: Motion compensation prediction unit (motion compensation prediction unit); 6: Subtraction unit (quantization unit); 7: Transformation/quantization unit (quantization unit); 8: Inverse quantization/inverse transformation unit; 9: Addition unit; 10: Intra-frame prediction memory (intra-frame prediction unit); 11: Loop filter unit; 12: Motion compensation prediction frame memory (motion compensation prediction unit); Detection unit); 13: variable length coding unit (variable length coding unit); 31: variable length decoding unit (variable length decoding unit); 32: inverse quantization/inverse transform unit (inverse quantization unit); 33: switching switch; 34: intra-frame prediction unit (intra-frame prediction unit); 35: motion compensation unit (motion compensation prediction unit); 36: addition unit; 37: memory for intra-frame prediction (intra-frame prediction unit); 38: loop filter unit; 39: motion compensation prediction frame memory (motion compensation prediction unit).

DETAILED DESCRIPTION

Hereinafter, in order to explain the present invention in more detail, embodiments of the present invention will be described with reference to the accompanying drawings.

Implementation method 1.

FIG1 is a diagram showing the structure of a moving picture encoding apparatus according to Embodiment 1 of the present invention.

In Figure 1, the block division unit 1 performs the following processing: if an image signal representing an input image is input, the input image is divided into coding blocks of the maximum size determined by the coding control unit 2, that is, the maximum coding block, and until the upper limit of the number of levels determined by the coding control unit 2 is reached, the maximum coding block is hierarchically divided into each coding block.

Specifically, the block division unit 1 divides the input image into coding blocks according to the division determined by the coding control unit 2 and outputs the coding blocks. Each coding block is divided into one or more prediction blocks serving as prediction processing units.

In addition, the block division section 1 constitutes a block division unit.

The encoding control unit 2 determines the maximum size of a coding block that serves as a processing unit when performing prediction processing and determines the upper limit number of hierarchical divisions of the maximum-sized coding block, thereby determining the size of each coding block.

Furthermore, the encoding control unit 2 performs processing for selecting a coding mode with the highest coding efficiency for the coding blocks output from the block division unit 1 from one or more selectable coding modes (one or more intra-frame coding modes and one or more inter-frame coding modes).

In addition, the coding control unit 2 implements the following processing: when the coding mode with the highest coding efficiency is the intra-frame coding mode, for each prediction block as the prediction processing unit, the frame intra prediction parameters used when the intra-frame prediction processing is implemented for the coding block in the intra-frame coding mode are determined; when the coding mode with the highest coding efficiency is the inter-frame coding mode, for each prediction block as the prediction processing unit, the frame inter prediction parameters used when the inter-frame prediction processing is implemented for the coding block in the inter-frame coding mode are determined.

Furthermore, the encoding control unit 2 performs processing for determining prediction difference encoding parameters to be supplied to the transform/quantization unit 7 and the inverse quantization/inverse transform unit 8 .

In addition, the encoding control unit 2 constitutes an encoding control unit.

The switching switch 3 implements the following processing: if the coding mode determined by the coding control unit 2 is the intra-frame coding mode, the coding block output from the block division unit 1 is output to the intra-frame prediction unit 4; if the coding mode determined by the coding control unit 2 is the inter-frame coding mode, the coding block output from the block division unit 1 is output to the motion compensation prediction unit 5.

The intra-frame prediction unit 4 performs the following processing: for the coding block output from the switching switch 3, according to each prediction block as the prediction processing unit, while referring to the local decoded image stored in the memory 10 for intra-frame prediction, an intra-frame prediction process (intra-frame prediction process) using the intra-frame prediction parameters determined by the encoding control unit 2 is performed to generate an intra-frame prediction image.

In addition, when the intra-frame prediction processing when generating the predicted image is a horizontal prediction processing, the intra-frame prediction unit 4 adds a value proportional to the change in the horizontal brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the prediction block, and determines the added value as the predicted value of the predicted image. When the intra-frame prediction processing when generating the predicted image is a vertical prediction processing, the intra-frame prediction unit 4 adds a value proportional to the change in the vertical brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the prediction block, and determines the added value as the predicted value of the predicted image.

The intra prediction unit 4 and the intra prediction memory 10 constitute an intra prediction unit.

The motion compensation prediction unit 5 performs the following processing: according to the prediction block unit as the prediction processing unit, it compares the coding block output from the switching switch 3 and the local decoded image of more than one frame stored in the motion compensation prediction frame memory 12 to search for the motion vector, and uses the motion vector and the inter-frame prediction parameter determined by the encoding control unit 2 to perform inter-frame prediction processing (motion compensation prediction processing) on the coding block according to the prediction block unit to generate an inter-frame prediction image.

The subtraction unit 6 performs the following processing: subtracts the intra-frame prediction image generated by the intra-frame prediction unit 4 or the inter-frame prediction image generated by the motion compensation prediction unit 5 from the coding block output by the block division unit 1, and outputs the predicted differential signal (differential image) as the subtraction result to the transformation/quantization unit 7.

The transform/quantization unit 7 performs the following processing: referring to the predicted differential coding parameters determined by the coding control unit 2, an orthogonal transform processing (for example, DCT (discrete cosine transform), orthogonal transform processing such as KL transform with a basis designed in advance for a specific learning sequence) is performed on the predicted differential signal output from the subtraction unit 6 to calculate the transform coefficient, and referring to the predicted differential coding parameters, the transform coefficient is quantized, and the compressed data as the quantized transform coefficient is output to the inverse quantization/inverse transform unit 8 and the variable length coding unit 13.

Furthermore, the subtraction unit 6 and the transform/quantization unit 7 constitute a quantization unit.

The inverse quantization/inverse transform unit 8 performs the following processing: referring to the predicted differential coding parameters determined by the coding control unit 2, the compressed data output from the transform/quantization unit 7 is inversely quantized, and referring to the predicted differential coding parameters, an inverse orthogonal transform is performed on the transform coefficients of the compressed data after inverse quantization, and a local decoded predicted differential signal equivalent to the predicted differential signal output from the subtraction unit 6 is calculated.

The addition unit 9 performs the following processing: adding the local decoded prediction differential signal calculated by the inverse quantization/inverse transform unit 8 and the intra-frame prediction image generated by the intra-frame prediction unit 4 or the inter-frame prediction image generated by the motion compensation prediction unit 5 to calculate a local decoded image equivalent to the coding block output from the block division unit 1.

The intra-frame prediction memory 10 is a recording medium that stores the local decoded image calculated by the adding unit 9 .

The loop filter unit 11 performs a predetermined filtering process on the local decoded image calculated by the adder unit 9 and outputs the filtered local decoded image.

The motion-compensated prediction frame memory 12 is a recording medium that stores the local decoded image after the filtering process.

The variable length coding unit 13 performs the following processing: variable length coding is performed on the compressed data output from the transform/quantization unit 7, the output signal of the coding control unit 2 (block segmentation information within the maximum coding block, coding mode, prediction differential coding parameters, intra-frame prediction parameters or inter-frame prediction parameters), and the motion vector output from the motion compensation prediction unit 5 (when the coding mode is the inter-frame coding mode) to generate a bit stream.

In addition, the variable length coding unit 13 constitutes a variable length coding unit.

In the example of FIG. 1 , it is assumed that the block division unit 1, encoding control unit 2, switch 3, intra-frame prediction unit 4, motion-compensated prediction unit 5, subtraction unit 6, transform/quantization unit 7, inverse quantization/inverse transform unit 8, addition unit 9, intra-frame prediction memory 10, loop filter unit 11, motion-compensated prediction frame memory 12, and variable-length coding unit 13, which are components of the moving picture coding apparatus, are each configured by dedicated hardware (e.g., a semiconductor integrated circuit or a single-chip microcomputer having a CPU mounted thereon). However, if the moving picture coding apparatus is configured by a computer, a program describing the processing details of the block division unit 1, encoding control unit 2, switch 3, intra-frame prediction unit 4, motion-compensated prediction unit 5, subtraction unit 6, transform/quantization unit 7, inverse quantization/inverse transform unit 8, addition unit 9, loop filter unit 11, and variable-length coding unit 13 may be stored in a memory of the computer, and the program stored in the memory may be executed by the CPU of the computer.

FIG2 is a flowchart showing the processing contents (moving picture encoding method) of the moving picture encoding apparatus according to Embodiment 1 of the present invention.

FIG3 is a diagram showing the configuration of the moving picture decoding apparatus according to Embodiment 1 of the present invention.

In Figure 3, the variable length decoding unit 31 implements the following processing: if the bit stream generated by the motion image encoding device of Figure 1 is input, the compressed data, block division information, coding mode, intra-frame prediction parameters (when the coding mode is intra-frame coding mode), inter-frame prediction parameters (when the coding mode is inter-frame coding mode), predicted differential coding parameters and motion vectors (when the coding mode is inter-frame coding mode) are variable-length decoded from the bit stream.

In addition, the variable length decoding unit 31 constitutes a variable length decoding means.

The inverse quantization/inverse transform unit 32 performs the following processing: referring to the prediction differential coding parameters variable-length decoded by the variable-length decoding unit 31, inverse quantizing the compressed data variable-length decoded by the variable-length decoding unit 31, and performing inverse orthogonal transform processing on the transform coefficients of the compressed data after inverse quantization with reference to the prediction differential coding parameters, and calculating a decoded prediction differential signal that is the same as the locally decoded prediction differential signal output from the inverse quantization/inverse transform unit 8 of Figure 1.

In addition, the inverse quantization/inverse transformation unit 32 constitutes an inverse quantization unit.

The switching switch 33 implements the following processing: if the coding mode variable-length decoded by the variable length decoding unit 31 is the intra-frame coding mode, the intra-frame prediction parameters variable-length decoded by the variable length decoding unit 31 are output to the intra-frame prediction unit 34; if the coding mode variable-length decoded by the variable length decoding unit 31 is the inter-frame coding mode, the inter-frame prediction parameters and motion vector variable-length decoded by the variable length decoding unit 31 are output to the motion compensation unit 35.

The intra-frame prediction unit 34 performs the following processing: for the decoding block (a block equivalent to the "coding block" of the motion image encoding device in Figure 1) determined based on the block segmentation information and the encoding mode variable-length decoded by the variable-length decoding unit 31, for each prediction block as a prediction processing unit, while referring to the decoded image stored in the intra-frame prediction memory 37, an intra-frame prediction process (intra-frame prediction process) using the intra-frame prediction parameters output from the switching switch 33 is performed to generate an intra-frame prediction image.

In addition, when the intra-frame prediction processing when generating the predicted image is a horizontal prediction processing, the intra-frame prediction unit 34 adds a value proportional to the change in the horizontal brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the left of the prediction block, and determines the added value as the predicted value of the predicted image. When the intra-frame prediction processing when generating the predicted image is a vertical prediction processing, the intra-frame prediction unit 34 adds a value proportional to the change in the vertical brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the left of the prediction block, and determines the added value as the predicted value of the predicted image.

The intra prediction unit 34 and the intra prediction memory 37 constitute an intra prediction unit.

The motion compensation unit 35 performs the following processing: for the decoded blocks determined based on the block segmentation information and the encoding mode variable-length decoded by the variable-length decoding unit 31, according to each prediction block as a prediction processing unit, while referring to the decoded image stored in the motion compensation prediction frame memory 39, an inter-frame prediction process (motion compensation prediction process) using the motion vector and inter-frame prediction parameters output from the switching switch 33 is performed to generate an inter-frame prediction image.

The addition unit 36 performs the following processing: adding the decoded prediction differential signal calculated by the inverse quantization/inverse transformation unit 32 and the intra-frame prediction image generated by the intra-frame prediction unit 34 or the inter-frame prediction image generated by the motion compensation unit 35 to calculate a decoded image that is the same as the local decoded image output from the addition unit 9 in Figure 1.

The intra-frame prediction memory 37 is a recording medium that stores the decoded image calculated by the adder 36 .

The loop filter unit 38 performs a predetermined filtering process on the decoded image calculated by the adder unit 36 and outputs the decoded image after the filtering process.

The motion compensation prediction frame memory 39 is a recording medium that stores decoded images after filtering.

In the example of FIG. 3 , it is assumed that the variable length decoding unit 31, the inverse quantization/inverse transform unit 32, the switch 33, the intra-frame prediction unit 34, the motion compensation unit 35, the addition unit 36, the intra-frame prediction memory 37, the loop filter unit 38, and the motion-compensated prediction frame memory 39, which are components of the moving picture decoding device, are each configured by dedicated hardware (e.g., a semiconductor integrated circuit or a single-chip microcomputer having a CPU mounted thereon). However, if the moving picture decoding device is configured by a computer, a program describing the processing contents of the variable length decoding unit 31, the inverse quantization/inverse transform unit 32, the switch 33, the intra-frame prediction unit 34, the motion compensation unit 35, the addition unit 36, and the loop filter unit 38 may be stored in a memory of the computer, and the program stored in the memory may be executed by the CPU of the computer.

FIG4 is a flowchart showing the processing contents (moving picture decoding method) of the moving picture decoding device according to Embodiment 1 of the present invention.

Next, the operation will be explained.

In this embodiment 1, a motion image encoding device and a motion image decoding device are described, wherein the motion image encoding device takes each frame image of the image as an input image, implements intra-frame prediction based on encoded nearby pixels or motion compensation prediction between adjacent frames, implements compression processing based on orthogonal transform/quantization on the obtained prediction differential signal, and then performs variable-length encoding to generate a bit stream, and the motion image decoding device decodes the bit stream output from the motion image encoding device.

The moving picture encoding apparatus of FIG. 1 is characterized in that it divides the video signal into blocks of various sizes and performs intra-frame/inter-frame adaptive encoding in response to local changes in the spatial and temporal directions of the video signal.

In general, image signals have the characteristic of locally varying signal complexity in space and time. When viewed spatially, within a given image frame, there may be patterns with uniform signal characteristics across relatively wide image regions, such as the sky or a wall, but also patterns with complex texture patterns within smaller image regions, such as figures or paintings with subtle textures.

Even when observing over time, although the local changes in the time direction of the pattern of the sky and the wall are small, the changes over time of moving people and objects are large because their outlines undergo rigid/non-rigid motion over time.

In the encoding process, a prediction differential signal with low signal power or entropy is generated through temporal/spatial prediction to reduce the overall symbol amount. However, as long as the parameters used in the prediction can be evenly applied to the largest possible image signal area, the symbol amount of the parameter can be reduced.

On the other hand, if the same prediction parameters are applied to a large image area for an image signal pattern that varies greatly temporally and spatially, prediction errors increase, and thus the code amount of the prediction difference signal increases.

Therefore, in areas with large temporal/spatial variations, it is preferable to reduce the block size for prediction processing using the same prediction parameters, increase the amount of data of the parameters used in the prediction, and reduce the power/entropy of the prediction differential signal.

In this embodiment 1, in order to perform encoding that is adapted to the general properties of such image signals, the following structure is adopted: the prediction processing is initially started from a specified maximum block size, the image signal area is hierarchically divided, and for each divided area, the prediction processing and the encoding processing of its prediction difference are adapted.

Regarding the motion image encoding device in Figure 1, the image signal format of the processing object is set to be, in addition to color image signals of arbitrary color spaces such as YUV signals composed of a luminance signal and two color difference signals, RGB signals output from a digital camera element, etc., it is also set to arbitrary image signals such as monochrome image signals and infrared image signals whose image frames are composed of horizontal and vertical two-dimensional digital sampling (pixel) columns.

The grayscale of each pixel can be 8 bits, 10 bits, 12 bits, etc.

For convenience, in the following description, unless otherwise specified, it is assumed that the video signal of the input image is a YUV signal, and a 4:2:0 format signal in which the two color difference components U and V are subsampled relative to the luminance component Y is processed.

In addition, a processing data unit corresponding to each frame of a video signal is referred to as a "picture".

In this first embodiment, the "picture" is described as an image frame signal scanned sequentially (progressive scanning), but when the image signal is an interlaced scanning signal, the "picture" can also be a field image signal that is a unit constituting an image frame.

First, the processing contents of the moving picture encoding apparatus in FIG. 1 will be described.

First, the encoding control unit 2 determines the size of the maximum coding block used for encoding the picture to be encoded (current picture) and the upper limit of the number of hierarchical divisions of the maximum coding block (step ST1 in FIG. 2 ).

As a method for determining the size of the maximum coding block, for example, the same size can be determined for all pictures based on the resolution of the image signal of the input image, or the difference in the complexity of the local motion of the image signal of the input image can be quantified as a parameter, and a small size can be determined for pictures with intense motion, and a large size can be determined for pictures with less motion.

As a method for determining the upper limit of the number of segmentation levels, there is a method of determining the same number of levels for all pictures based on the resolution of the image signal of the input image, or a method of setting the number of levels to be increased when the motion of the image signal of the input image is intense so as to be able to detect more subtle motion, and setting the number of levels to be suppressed when the motion is small.

Furthermore, the encoding control unit 2 selects an encoding mode corresponding to each hierarchically divided encoding block from one or more available encoding modes (step ST2 ).

That is, the encoding control unit 2 hierarchically divides each image region of the maximum encoding block size into encoding blocks having the encoding block size until the upper limit of the number of division levels determined in advance is reached, and determines the encoding mode for each encoding block.

Among the coding modes, there are one or more intra-frame coding modes (collectively referred to as "INTRA") and one or more inter-frame coding modes (collectively referred to as "INTER"). The coding control unit 2 selects the coding mode corresponding to each coding block from all coding modes available in the picture or a subset thereof.

Each coding block hierarchically divided by a block division unit 1 described later is further divided into one or more prediction blocks as units for performing prediction processing, and the division status of the prediction blocks is also included as information in the coding mode.

The method for selecting the coding mode of the coding control unit 2 is a well-known technology, so detailed description is omitted. For example, there are the following methods: use any available coding mode to implement coding processing for the coding block to verify the coding efficiency, and select the coding mode with the best coding efficiency from multiple available coding modes.

Furthermore, the encoding control unit 2 determines, for each encoding block, a quantization parameter and a transform block size used when compressing a differential image, and determines prediction parameters (intra-frame prediction parameters or inter-frame prediction parameters) used when performing a prediction process.

When the coding block is further divided into prediction block units for performing prediction processing, prediction parameters (intra-frame prediction parameters or inter-frame prediction parameters) can be selected for each prediction block.

Furthermore, in a coding block whose coding mode is the intra-frame coding mode, since the encoded pixels adjacent to the prediction block are used when performing intra-frame prediction processing as described later, encoding needs to be performed in units of prediction blocks, so the selectable transform block size is limited to be less than the size of the prediction block.

The encoding control unit 2 outputs the prediction difference encoding parameters including the quantization parameter and the transform block size to the transform/quantization unit 7 , the inverse quantization/inverse transform unit 8 , and the variable length encoding unit 13 .

Furthermore, the encoding control unit 2 outputs the intra prediction parameters to the intra prediction unit 4 as needed.

Furthermore, the encoding control unit 2 outputs the inter-frame prediction parameters to the motion compensation prediction unit 5 as needed.

If the image signal of the input image is input, the block division unit 1 divides the image signal of the input image into the maximum coding block size determined by the coding control unit 2, and further hierarchically divides the divided maximum coding block into coding blocks determined by the coding control unit 2, and outputs the coding blocks.

Here, FIG5 is an explanatory diagram showing an example in which a maximum coding block is hierarchically divided into a plurality of coding blocks.

In FIG5 , the largest coding block is a coding block having a size of (L ⁰ , M ⁰ ) for the luminance component described as “layer 0”.

The largest coding block is used as a starting point, and coding blocks are obtained by hierarchically dividing the block up to a predetermined depth determined separately using a quadtree structure.

At depth n, a coding block is an image region of size ( ^Ln , ^Mn ).

^Ln and ^Mn may be the same or different, but FIG. 5 shows the case where ^Ln = ^Mn .

Hereinafter, the coding block size determined by the coding control unit 2 is defined as the size (L ⁿ , M ⁿ ) of the luminance component of the coding block.

Since quadtree partitioning is performed, (Ln ⁺¹ , Mn ⁺¹ ) = ( ^Ln /2, ^Mn /2) always holds.

Furthermore, in a color video signal (4:4:4 format) such as an RGB signal where all color components have the same number of samples, the size of all color components is ( ^Ln , ^Mn ). However, when processing the 4:2:0 format, the encoding block size of the corresponding color difference component is ( ^Ln /2, ^Mn /2).

Hereinafter, it is assumed that ^Bn represents the coding block of the nth layer, and m( ^Bn ) represents the coding mode selectable in the coding block ^Bn .

In the case of a color video signal composed of multiple color components, the coding mode m(B ⁿ ) can be configured to use a separate mode for each color component or a common mode for all color components. Unless otherwise specified, the following description refers to the coding mode corresponding to the luma component of a YUV signal and a coding block in the 4:2:0 format.

As shown in FIG6 , the coding block ^Bn is divided by the block division unit 1 into one or more prediction blocks representing prediction processing units.

Hereinafter, the prediction block belonging to the coding block ^Bn will be referred to as _P ⁱⁿ (i is the prediction block number in the nth layer). FIG5 shows an example ^of _P00 ^and _P10 .

Coding mode m(B ⁿ ) includes information on how to perform prediction block division of coding block B ⁿ .

Regarding all prediction ^blocks _Pin , prediction processing is performed according to the coding mode m( ^Bn ), but individual prediction parameters (intra-frame prediction parameters or inter-frame prediction parameters) can be selected for each prediction ^block _Pin .

The encoding control unit 2 generates a block division state as shown in FIG. 6 , for example, for the largest encoding block, and determines the encoding blocks.

The rectangles surrounded by dotted lines in FIG6(a) represent coding blocks, and the blocks filled with diagonal lines within each coding block represent the division status of each prediction block.

Fig. 6(b) shows the allocation of coding mode m( ^Bn ) by hierarchical partitioning using a quadtree graph in the example of Fig. 6(a). Nodes surrounded by squares in Fig. 6(b) are nodes (coding blocks) to which coding mode m( ^Bn ) is allocated.

The information on the quadtree pattern is output from the encoding control unit 2 to the variable-length encoding unit 13 together with the encoding mode m(B ⁿ ) and multiplexed into the bit stream.

When the coding mode m(B ⁿ ) determined by the coding control unit 2 is the intra coding mode (m(B ⁿ )∈INTRA), the switch 3 outputs the coding block B ⁿ output from the block division unit 1 to the intra prediction unit 4 .

On the other hand, when the coding mode m(B ⁿ ) determined by the coding control unit 2 is the inter-coding mode (m(B ⁿ )∈INTER), the coding block B ⁿ output from the block division unit 1 is output to the motion compensation prediction unit 5 .

In the intra-frame prediction unit 4, if the coding mode m( ^Bn ) determined by the coding control unit 2 is the intra-frame coding mode (in the case where m( ^Bn )∈INTRA) and the coding block ^Bn is received from the switch 3 (step ST3), the intra-frame prediction unit 4 performs intra-frame prediction processing on each prediction block P in the coding block ^Bn using the intra-frame prediction parameters determined by the coding control unit 2 while referring to the local decoded image stored in the intra- ^frame prediction memory ₁₀ , thereby generating an intra-frame prediction image P _INTRAi ⁿ (step ST4).

In addition, the motion picture decoding device needs to generate an intra-frame prediction image that is completely identical to the intra-frame prediction image P _INTRAi ⁿ , so the intra-frame prediction parameters used in generating the intra-frame prediction image P _INTRAi ⁿ are output from the encoding control unit 2 to the variable length encoding unit 13 and multiplexed into the bit stream.

The details of the processing performed by the intra-frame prediction unit 4 will be described later.

In the motion-compensated prediction unit 5, if the coding mode m( ^Bn ) determined by the coding control unit 2 is the inter-frame coding mode (in the case where m( ^Bn )∈INTER), and the coding block ^Bn is received from the switching switch 3 (step ST3), each prediction block _P in the coding block ^Bn is compared ^with the filtered local decoded image stored in the motion-compensated prediction frame memory 12 to search for a motion vector. Then, using the motion vector and the inter-frame prediction parameters determined by the coding control unit 2, inter-frame prediction processing is performed on each prediction ^block _P in the coding block ^Bn to generate an inter-frame prediction image _PINTERi ⁿ (step ST5).

Furthermore, the moving picture decoding apparatus needs to generate an inter-frame prediction image identical to the inter-frame prediction image _PINTERi ⁿ , so the inter-frame prediction parameters used in generating the inter-frame prediction image _PINTERi ⁿ are output from the encoding control unit 2 to the variable-length encoding unit 13 and multiplexed into the bitstream.

In addition, the motion vector searched by the motion compensation prediction unit 5 is also output to the variable length coding unit 13 and multiplexed into the bit stream.

Upon receiving the coding block ^Bn from the block partitioning unit 1, the subtraction unit 6 subtracts either the intra- ^frame prediction image P _INTRAi ⁿ generated by the intra-frame prediction unit 4 or the inter-frame prediction image P _INTERi ⁿ generated by the motion-compensated prediction unit 5 from the prediction block P in the coding _block ^Bn , and outputs a prediction difference signal _e in which the subtraction result is ^obtained to the transform/quantization unit 7 (step ST6).

Upon receiving the prediction differential signal _e ⁱⁿ from the subtraction unit 6, the transform/quantization unit 7 performs an orthogonal transform process (e.g., DCT (discrete cosine transform), KL transform with a basis design performed on a predetermined learning sequence, or ^other orthogonal transform process) on the prediction differential signal _e in reference to the prediction differential coding parameters determined by the coding control unit 2, and calculates the transform coefficients.

Furthermore, the transform/quantization unit 7 quantizes the transform coefficients with reference to the prediction difference coding parameters, and outputs the compressed data of the quantized transform coefficients to the inverse quantization/inverse transform unit 8 and the variable length coding unit 13 (step ST7).

Upon receiving the compressed data from the transform/quantization unit 7 , the inverse quantization/inverse transform unit 8 inversely quantizes the compressed data with reference to the prediction difference encoding parameter determined by the encoding control unit 2 .

In addition, the inverse quantization/inverse transform unit 8 refers to the predicted differential coding parameters, performs inverse orthogonal transform processing (for example, inverse DCT, inverse KL transform, etc.) on the transform coefficients of the compressed data after inverse quantization, calculates the local decoded predicted differential signal equivalent to the predicted differential signal e _i ⁿ output from the subtraction unit 6, and outputs it to the addition unit 9 (step ST8).

If the adding unit 9 receives the local decoded prediction differential signal from the inverse quantization/inverse transform unit 8, it adds the local decoded prediction differential signal to one of the intra-frame prediction image P _INTRAi ⁿ generated by the intra-frame prediction unit 4 and the inter-frame prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 5, thereby calculating the local decoded image (step ST9).

Furthermore, the adding unit 9 outputs the local decoded image to the loop filtering unit 11 and stores the local decoded image in the intra-frame prediction memory 10 .

This locally decoded image becomes an encoded image signal used in subsequent intra-frame prediction processing.

Upon receiving the local decoded image from the adder 9 , the loop filter unit 11 performs a predetermined filtering process on the local decoded image and stores the filtered local decoded image in the motion-compensated prediction frame memory 12 (step ST10 ).

The filtering process performed by the loop filter unit 11 may be performed in units of the largest coding block or each coding block of the input local decoded image, or may be performed collectively for one picture after one picture of local decoded images is input.

In addition, as examples of prescribed filtering processing, there can be cited processing that filters block boundaries in a manner that prevents discontinuity (block noise) at the coding block boundaries from becoming noticeable, filtering processing that compensates for distortion of a local decoded image in a manner that minimizes the error between the image signal of Figure 1 as the input image and the local decoded image, and the like.

However, when performing filtering processing to compensate for the distortion of the local decoded image in a manner that minimizes the error between the image signal of Figure 1 as the input image and the local decoded image, it is necessary to refer to the image signal in the loop filter unit 11, so it is necessary to change the motion image encoding device of Figure 1 so that the image signal is input to the loop filter unit 11.

The processes of steps ST3 to ST9 are repeatedly performed until the processes of all the hierarchically divided coding blocks ^Bn are completed. When the processes of all the coding blocks ^Bn are completed, the process proceeds to step ST13 (steps ST11 and ST12).

The variable-length coding unit 13 performs variable-length coding on the compressed data output from the transform/quantization unit 7, the block division information within the largest coding block output from the coding control unit 2 (quadtree information in FIG. 6B ), the coding mode m (B ⁿ ) and the prediction difference coding parameters, the intra-frame prediction parameters (when the coding mode is the intra-frame coding mode) or inter-frame prediction parameters (when the coding mode is the inter-frame coding mode) output from the coding control unit 2, and the motion vector (when the coding mode is the inter-frame coding mode) output from the motion compensation prediction unit 5, and generates a bitstream representing these coding results (step ST13).

Next, the processing contents of the intra prediction unit 4 will be described in detail.

FIG7 is an explanatory diagram showing an example of intra-frame prediction parameters (intra-frame prediction modes) selectable by each prediction ^block _P in the coding block ^Bn .

FIG7 shows an intra-frame prediction mode and a prediction direction vector represented by the intra-frame prediction mode. In the example of FIG7 , the design is such that the relative angles between the prediction direction vectors decrease as the number of selectable intra-frame prediction modes increases.

As described above, the intra ^- frame prediction unit 4 refers to the intra-frame prediction parameters of the prediction ^block _Pin , performs intra-frame prediction processing on the prediction ^block _Pin , and generates an intra-frame prediction image _Pintran . However, the intra-frame processing for generating an intra- ^frame prediction signal of the prediction block _Pin in the luminance signal is described here.

The size of ^the prediction ^block _Pin is set to _lin × _min pixels ^.

FIG8 is an explanatory diagram showing an example of pixels used when generating predicted values of _pixels in a ^prediction block ^Pin in the case of _lin = _min = 4 ^.

In FIG8 , the coded pixels ( ² × _lin +1) above and the coded pixels ⁽ 2× _min ) to the left of the _prediction ^block Pin are used as pixels for prediction. However, the number of pixels used for prediction may be more or less than the number of pixels shown in FIG8 .

In addition, in Figure 8, one row or one column of pixels near the prediction ^{block Pin} _is used for prediction, but two rows or two columns, or more than two rows of pixels may be used for prediction.

When the index value of the intra-frame prediction mode for the prediction ^block _Pin is 0 (vertical prediction), the prediction value of the pixel in the prediction ^block _Pin is calculated according to the following formula (1) to generate a predicted image.

S'(x, y)=S(x,-1)+(S(-1, y)-S(-1,-1))/t (t)

Among them, the coordinates (x, y) are relative coordinates with the upper left pixel in the prediction block P _i ⁿ as the origin (refer to Figure 9), S'(x, y) is the predicted value at the coordinates (x, y), and S(x, y) is the brightness value (decoded brightness value) of the encoded pixel at the coordinates (x, y).

In this way, a value proportional to S(-1, y)-S(-1, -1) representing the amount of change ⁱⁿ the vertical brightness value of the coded pixel adjacent to the left of the prediction ^block _Pin (the pixel surrounded by the thick frame in Figure 10) is added to the brightness value S(x, -1) of the coded pixel above the _prediction block Pin, which is the prediction value of the vertical prediction in the previous (MPEG-4AVC/H.264) method (a value obtained by scaling S(-1, y)-S(-1, -1) representing the amount of change in the vertical brightness value to 1/t) is added, and the added value is determined as the prediction value of the predicted image, thereby realizing vertical prediction that follows the change in the brightness value in the prediction direction.

However, if the predicted value does not fall within the range of possible values for the luminance value, the value is rounded off so as to fall within the range.

In addition, the above-mentioned 1/t may be set as a fixed value, or may be set as a variable that changes according to the coordinates (x, y).

For example, if t=2x ⁺¹ , as shown in FIG11 , the scaling value decreases in order from the left column, such as 1/2, 1/4, 1/8, and 1/16. Therefore, the greater the distance from the left adjacent coded pixel of the prediction ^block _Pin , the smaller the change in the added vertical brightness value.

Therefore, the farther the prediction target pixel is from the already encoded pixel adjacent to the left of the prediction ^{block Pin} _and the lower the correlation becomes, the smaller the influence of the already encoded pixel adjacent to the left of the prediction ^block _Pin can be. Therefore, a high-precision prediction corresponding to the correlation between the already encoded pixels adjacent to the left of the ^prediction block _Pin can be performed.

Furthermore, the block size of the prediction ^block _P in which the prediction process of equation (1) is performed may be limited. Generally, with large block sizes, various signal variations are easily included within the block, making it less likely that directional prediction can be used to perform high-precision prediction. Therefore, for example, equation (1) is not applied to prediction ^blocks _P in block sizes of 16×16 pixels or larger, and the prediction value of the conventional vertical prediction (the luminance value S(x, -1) of the coded pixel adjacent to the top of the prediction block ^P ₎ is used. Equation (1) is applied only to blocks smaller than 16×16 pixels. This improves prediction performance compared to conventional vertical prediction and suppresses an increase in the amount of computation.

In addition, when the index value of the intra- ^frame prediction mode for the prediction block _Pin is 1 (horizontal prediction), the prediction value of the pixel in the prediction ^block _Pin is calculated according to the following formula (2) to generate a predicted image.

S'(x, y)=S(-1, y)+(S(x,-1)-S(-1,-1))/u (2)

Among them, the coordinates (x, y) are relative coordinates with the upper left pixel in the prediction block P _i ⁿ as the origin (refer to Figure 9), S'(x, y) is the predicted value at the coordinates (x, y), and S(x, y) is the brightness value (decoded brightness value) of the encoded pixel at the coordinates (x, y).

In this way, a value proportional to S(x, -1)-S(-1, -1) representing the amount of change in the horizontal brightness value of the coded pixel adjacent to the top of the prediction ^block _Pin (the pixel surrounded by the thick frame in Figure 12) is added to the brightness value S(-1, y) of the coded pixel adjacent to the left of the _prediction block Pin, which is the prediction value of the horizontal prediction in the ^previous (MPEG-4AVC/H.264) (a value obtained by scaling S(x, -1)-S(-1, -1) representing the amount of change in the horizontal brightness value to 1/u) and the added value is determined as the prediction value of the predicted image, thereby realizing horizontal prediction that follows the change in the brightness value in the prediction direction.

However, if the predicted value does not fall within the range of possible values for the luminance value, the value is rounded off so as to fall within the range.

In addition, the above-mentioned 1/u may be set as a fixed value, or may be set as a variable that changes with the coordinates (x, y).

For example, if u=2y ⁺¹ , as shown in FIG13, the scaling value decreases from the top row in the order of 1/2, 1/4, 1/8, and 1/16, so the greater the distance from the adjacent encoded pixel above the prediction block _P ⁱⁿ , the smaller the change in the added horizontal brightness value.

Therefore, the farther the distance from the already-encoded pixels adjacent to the prediction ^{block Pin} _is , and the lower the correlation is, the smaller the influence of the already-encoded pixels adjacent to the prediction ^{block Pin} _is . Therefore, a high-precision prediction corresponding to the correlation between the already-encoded pixels adjacent to the prediction ^{block Pin} _can be performed.

Furthermore, the block size of the prediction ^block _P in which the prediction processing of equation (2) is performed may be limited. Generally, with large block sizes, various signal variations are more likely to be included within the block, making it less likely that directional prediction can be performed with high accuracy. Therefore, for example, equation (2) is not applied to prediction ^blocks _P in block sizes of 16×16 pixels or larger, and the prediction value of conventional horizontal prediction (the luminance value S(-1, y) of the already coded pixel adjacent to the left of the prediction block _P) ^is used. Equation (2) is applied only to blocks smaller than 16×16 pixels. This improves prediction performance compared to conventional horizontal prediction and suppresses an increase in the amount of computation.

In addition, when the index value of the intra-frame prediction mode for the prediction ^block _Pin is 2 (average value prediction), the average value of the coded pixels adjacent to the top of the prediction ^{block Pin} _and the coded pixels adjacent to the left of the prediction _block ^Pin is used as the prediction value of the pixel in the prediction ^{block Pin} _to generate a predicted image.

When the index value of the intra-frame prediction mode is other than 0 (vertical prediction), 1 (horizontal prediction), or 2 (average prediction), the predicted value of the pixel in the prediction block _Pin is generated based on the prediction direction vector _υp = ( ^dx , dy) represented by the index value.

As shown in FIG9 , if the upper left pixel of the prediction _block ^Pin is set as the origin and the relative coordinates within the ^prediction block _Pin are set to (x, y), the position of the reference pixel used in the prediction becomes the intersection of the following L and the adjacent pixel.

Here, k is a negative scalar value.

When the reference pixel is at an integer pixel position, the integer pixel is used as the predicted value of the prediction target pixel. When the reference pixel is not at an integer pixel position, an interpolated pixel generated from integer pixels adjacent to the reference pixel is used as the predicted value.

In the example of FIG8 , since the reference pixel is not located at an integer pixel position, the prediction value is generated by interpolating two pixels adjacent to the reference pixel. Furthermore, the prediction value may be generated by interpolating pixels based on two or more adjacent pixels, rather than just two adjacent pixels.

By increasing the number of pixels used in the interpolation process, the interpolation accuracy of the interpolated pixels is improved. On the other hand, since the complexity of the operations required for the interpolation process increases, in the case of a motion picture encoding device that requires high encoding performance even though the computational load is large, it is preferable to generate interpolated pixels based on more pixels.

Through the same steps, predicted pixels corresponding to all pixels of the luminance signal in the prediction ^block _Pin are generated, and an intra-frame predicted image _Pintran ^is output.

In addition, the intra-frame prediction parameters used for generating the intra ^{-frame prediction image PINTRAin} _are output to the variable-length coding unit 13 in order to be multiplexed into the bit stream.

In addition, similar to the intra-frame prediction of 8×8 pixel blocks in MPEG-4 AVC/H.264 described above, the pixels used for intra-frame prediction may not be the pixels in the encoded adjacent blocks themselves, but the results of filtering these pixels may be used.

For the color difference signal of the prediction ^block _Pin , the same steps as the luminance signal are used to implement intra-frame prediction processing based on intra-frame prediction parameters (intra-frame prediction mode), and the intra-frame prediction parameters used in generating the intra-frame prediction image are output to the variable length coding unit 13.

The intra-frame prediction parameters (intra-frame prediction modes) selectable in the color difference signal do not need to be the same as those in the luminance signal. In addition, the vertical prediction and horizontal prediction may also be the conventional (MPEG-4 AVC/H.264) prediction methods.

For example, in the case of a YUV signal in 4:2:0 format, the color difference signals (U and V signals) are signals obtained by reducing the resolution of the luminance signal (Y signal) to 1/2 in both the horizontal and vertical directions. Compared with the luminance signal, the image signal has low complexity and is easier to predict, so the number of selectable intra-frame prediction parameters (intra-frame prediction modes) is less than that of the luminance signal, and the vertical and horizontal predictions are also set to the previous simple prediction methods, thereby almost without reducing the prediction efficiency and achieving a reduction in the amount of symbols required to encode the intra-frame prediction parameters (intra-frame prediction modes) and low computational complexity in the prediction processing.

In addition, the scaling value 1/t used for vertical prediction and the scaling value 1/u used for horizontal prediction can also be agreed upon in advance between the motion image encoding device and the motion image decoding device, but the intra-frame prediction unit 4 of the motion image encoding device can output t and u to the variable length encoding unit 13 according to sequence units or picture units, and the variable length encoding unit 13 can perform variable length encoding on t and u and include them in the bit stream, and the motion image decoding device can perform variable length decoding on t and u from the bit stream for use.

In this way, t and u can be adaptively controlled in sequence units or picture units, thereby achieving prediction processing that further corresponds to the characteristics of the video signal of the input image.

In addition, the block size for the vertical prediction using formula (1) and the horizontal prediction using formula (2) can also be agreed upon in advance between the motion image encoding device and the motion image decoding device, but the intra-frame prediction unit 4 of the motion image encoding device can also output the ON/OFF flag of each block size representing the block size for the vertical prediction using formula (1) and the horizontal prediction using formula (2) to the variable length encoding unit 13 according to the sequence unit or the picture unit. The variable length encoding unit 13 performs variable length encoding on the above-mentioned ON/OFF flag and includes it in the bit stream, and the motion image decoding device performs variable length decoding on the above-mentioned ON/OFF flag from the bit stream for use.

In this way, the block size of the vertical prediction using equation (1) and the horizontal prediction using equation (2) can be adaptively controlled in sequence units or picture units, thereby enabling prediction processing that further corresponds to the characteristics of the image signal of the input image.

Next, the processing contents of the moving picture decoding device in FIG. 3 will be described in detail.

If the variable length decoding unit 31 inputs the bit stream generated by the motion image encoding device in Figure 1, it implements variable length decoding processing on the bit stream (step ST21 in Figure 4) and decodes the frame size information according to the sequence unit composed of more than one frame of pictures or the picture unit.

At this time, among the parameter t of the scaling value used in the vertical prediction, the parameter u of the scaling value used in the horizontal prediction, and the ON/OFF flag of each block size indicating the block size of the vertical prediction using formula (1) and the horizontal prediction using formula (2), as long as any one of them is variable-length encoded and multiplexed into the bit stream, it is decoded according to the unit (sequence unit or picture unit) encoded by the motion image encoding device of Figure 1.

The variable length decoding unit 31 determines the maximum coding block size and the upper limit of the number of division hierarchies determined by the encoding control unit 2 of the moving picture encoding device of FIG. 1 through the same steps as the moving picture encoding device (step ST22 ).

For example, when the maximum coding block size and the upper limit of the number of division levels are determined based on the resolution of the video signal, the maximum coding block size is determined based on the decoded frame size information using the same procedure as that of the moving picture encoding device.

When the maximum coding block size and the upper limit of the number of division layers are multiplexed into the bitstream on the moving picture coding device side, the values obtained by decoding the bitstream are used.

Hereinafter, in the moving picture decoding apparatus, the maximum coding block size is referred to as the maximum decoding block size, and the maximum coding block is referred to as the maximum decoding block.

The variable length decoding unit 31 decodes the division state of the maximum decoding block as shown in FIG6 in units of the determined maximum decoding block. Based on the decoded division state, the decoding blocks (blocks corresponding to the "coding blocks" in the moving image encoding device of FIG1) are hierarchically determined (step ST23).

Next, the variable length decoding unit 31 decodes the coding mode assigned to the decoding block. Based on the information included in the decoded coding mode, the decoding block is further divided into one or more prediction processing units, namely prediction blocks, and prediction parameters assigned to the prediction block units are decoded (step ST24).

When the coding mode assigned to the decoding block is the intra coding mode, the variable length decoding unit 31 decodes the intra prediction parameters for each of one or more prediction blocks included in the decoding block and serving as a prediction processing unit.

Furthermore, the variable length decoding unit 31 divides the decoding block into one or more transform blocks serving as transform processing units based on the transform block size information included in the predicted differential coding parameters, and decodes the compressed data (transformed/quantized transform coefficients) for each transform block (step ST24).

In the switching switch 33, if the coding mode m( ^Bn ) variable-length-decoded by the variable-length decoding unit 31 is an intra-frame coding mode (in the case of m( ^Bn )∈INTRA), the intra-frame prediction parameters of the prediction block units variable-length-decoded by the variable-length decoding unit 31 are output to the intra-frame prediction unit 34.

On the other hand, if the coding mode m(B ⁿ ) variable-length-decoded by the variable-length decoding unit 31 is an inter-frame coding mode (in the case of m(B ⁿ )∈INTER), the inter-frame prediction parameters and motion vectors of the prediction block units variable-length-decoded by the variable-length decoding unit 31 are output to the motion compensation unit 35.

When the coding mode m( ^Bn ) variable-length-decoded by the variable-length decoding unit 31 is the intra coding mode (m( ^Bn )∈INTRA) (step ST25), the intra prediction unit 34 receives the intra prediction parameters for the prediction block output from the switch 33, and performs intra prediction processing on each prediction block P in the decoded block ^Bn using the intra prediction parameters through the same steps as those of the intra prediction unit ⁴ in FIG. 1 while referring to the decoded image stored in the intra prediction memory ₃₇ , thereby generating an intra prediction image P _INTRAi ⁿ (step ST26).

That is, when the index value of the intra prediction mode for the prediction ^block _Pin is 0 (vertical prediction), the intra prediction unit 34 calculates the prediction value of the pixels in the prediction ^block _Pin according to the above equation (1) to generate the intra ^prediction image _Pintran .

Furthermore, when the index value of the intra prediction mode for the prediction ^block _Pin is 1 (horizontal prediction), the prediction values of the pixels in the prediction ^block _Pin are calculated according to the above equation (2) to generate the ^intra prediction image _Pintrain .

However, when the block size of the vertical prediction using formula (1) and the horizontal prediction using formula (2) is limited, intra-frame prediction processing is performed in the prediction block Pin of a size other than the block size of the vertical prediction using ^formula (1) and the horizontal prediction using _formula (2) using the previous (MPEG-4AVC/H.264) vertical prediction and horizontal prediction.

When the coding mode m( ^Bn ) variable-length-decoded by the variable-length decoding unit 31 is an inter-frame coding mode (m( ^Bn )∈INTER) (step ST25), the motion compensation unit 35 receives the motion vector and inter-frame prediction parameters of the prediction block unit output from the switch 33, and performs inter-frame prediction processing on each prediction block P in the decoded block using the motion vector and inter-frame prediction parameters while referring to the filtered decoded image stored in the motion- ^{compensated prediction frame memory 39} _, thereby generating an inter-frame prediction image _PINTERi ⁿ (step ST27).

If the inverse quantization/inverse transform unit 32 receives compressed data and predicted differential coding parameters from the variable length decoding unit 31, it performs inverse quantization on the compressed data with reference to the predicted differential coding parameters through the same steps as the inverse quantization/inverse transform unit 8 in Figure 1, and performs inverse orthogonal transform processing on the transform coefficients of the inverse quantized compressed data with reference to the predicted differential coding parameters, and calculates a decoded predicted differential signal that is the same as the locally decoded predicted differential signal output from the inverse quantization/inverse transform unit 8 in Figure 1 (step ST28).

The addition unit 36 adds the decoded prediction differential signal calculated by the inverse quantization/inverse transform unit 32 to either the intra-frame prediction image P _INTRAi ⁿ generated by the intra-frame prediction unit 34 or the inter-frame prediction image P _INTERi ⁿ generated by the motion compensation unit 35 to calculate a decoded image, outputs it to the loop filter unit 38, and stores the decoded image in the intra-frame prediction memory 37 (step ST29).

This decoded image becomes a decoded image signal used in subsequent intra-frame prediction processing.

In the loop filter unit 38, if the processing of steps ST23 to ST29 for all decoding blocks ^Bn is completed (step ST30), the prescribed filtering processing is performed on the decoded image output from the addition unit 36, and the decoded image after filtering is stored in the motion compensation prediction frame memory 39 (step ST31).

The filtering process of the loop filter unit 38 may be performed in units of the largest decoded block or each decoded block of the input decoded image, or may be performed collectively for one picture after one picture of decoded images is input.

Examples of the predetermined filtering process include filtering of block boundaries to prevent discontinuity (block noise) at the coding block boundaries from becoming noticeable and filtering to compensate for distortion of a decoded image.

This decoded image becomes a reference image for motion compensation prediction and also becomes a reconstructed image.

As can be seen from the above, according to this embodiment 1, the intra-frame prediction unit 4 of the motion image encoding device is constructed to add a value proportional to the amount of change in the horizontal brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the left of the prediction block when the intra-frame prediction processing when generating the prediction image is a horizontal prediction processing, and to determine the added value as the prediction value of the prediction image; when the intra-frame prediction processing when generating the prediction image is a vertical prediction processing, add a value proportional to the amount of change in the vertical brightness value of the pixel adjacent to the left of the prediction block to the brightness value of the pixel adjacent to the upper side of the prediction block, and determine the added value as the prediction value of the prediction image, thereby achieving the effect of improving image quality by achieving high-precision prediction even when the signal value changes along the prediction direction.

In addition, according to the first embodiment, the intra-frame prediction unit 34 of the motion image decoding device is configured to, when the intra-frame prediction processing when generating the predicted image is a horizontal prediction processing, add a value proportional to the amount of change in the horizontal brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the left of the prediction block, and determine the added value as the predicted value of the predicted image; when the intra-frame prediction processing when generating the predicted image is a vertical prediction processing, add a value proportional to the amount of change in the vertical brightness value of the pixel adjacent to the left of the prediction block to the brightness value of the pixel adjacent to the upper side of the prediction block, and determine the added value as the predicted value of the predicted image. This has the effect of achieving high-precision prediction even when the signal value changes along the prediction direction, thereby improving image quality.

According to this embodiment 1, the scaling value 1/u used when the frame internal prediction units 4 and 34 perform horizontal prediction processing is configured such that the scaling value related to the row farther from the pixel adjacent to the top of the prediction block is set to a smaller value. Therefore, the farther the distance between the pixels adjacent to the top of the prediction block and the lower the correlation, the smaller the influence of the pixels adjacent to the top of the prediction block can be. As a result, the effect of high-precision prediction can be achieved.

In addition, regarding the scaling value 1/t used when the frame internal prediction units 4 and 34 implement vertical prediction processing, the scaling value related to the column that is farther away from the pixel adjacent to the left of the prediction block is set to a smaller value. Therefore, the farther the distance between the pixels adjacent to the left of the prediction block is, the lower the correlation becomes. The smaller the influence of the pixels adjacent to the left of the prediction block can be. As a result, the effect of high-precision prediction can be achieved.

In addition, in this embodiment 1, an example is shown in which the scaling value of the Nth row (the Nth row from the top of the prediction block) in the prediction block when the intra-frame prediction unit 4, 34 performs horizontal prediction processing is 1/ ^2N+1 (=1/2, 1/4, 1/8, 1/16,...), and the scaling value of the Mth column (the Mth column from the left end of the prediction block) in the prediction block when the intra-frame prediction unit 4, 34 performs vertical prediction processing is 1/ ^2M+1 (=1/2, 1/4, 1/8, 1/16,...). However, this is only an example, and any value may be used as long as the scaling value of the row farther from the top of the prediction block when the intra-frame prediction unit 4, 34 performs horizontal prediction processing is smaller, and the scaling value of the column farther from the left end of the prediction block when the intra-frame prediction unit 4, 34 performs vertical prediction processing is smaller.

Implementation method 2.

In the above-mentioned embodiment 1, when the intra-frame prediction processing when generating a predicted image is a vertical prediction processing, the intra-frame prediction units 4 and 34 add, to the luminance values of the pixels adjacent to the upper side of the prediction block, a value obtained by multiplying the vertical luminance value change of the pixels adjacent to the left of the prediction block by a scaling value set for each column in the prediction block, and determine the added value as the predicted value of the predicted image. However, in order to achieve low-calculation processing, it is also possible to add, to the luminance values of the pixels adjacent to the upper side of the prediction block for a predetermined number of columns starting from the left end of the prediction block, a value proportional to the vertical luminance value change of the pixels adjacent to the left of the prediction block, and determine the added value as the predicted value of the predicted image. However, for the remaining columns in the prediction block, the luminance values of the pixels adjacent to the upper side of the prediction block are determined as the predicted value of the predicted image.

In addition, for the same reason, when the intra-frame prediction processing when generating the predicted image is a horizontal prediction processing, the brightness values of the pixels adjacent to the left of the prediction block for a specified number of rows starting from the upper end of the prediction block can be added with the value obtained by multiplying the horizontal brightness value change of the pixels adjacent to the top of the prediction block by the scaling value set for each row in the prediction block, and the added value is determined as the predicted value of the predicted image, but for the remaining rows in the prediction block, the brightness values of the pixels adjacent to the left of the prediction block are determined as the predicted value of the predicted image.

The following describes in detail the processing contents of the intra prediction units 4 and 34.

When the index value of the intra prediction mode for the ^prediction block _Pin is 0 (vertical prediction), the intra prediction units 4 and 34 calculate the predicted values of the pixels in the prediction ^block _Pin according to the following equation (4) to generate a predicted image.

Here, B is an integer greater than 0. When x<B in the above formula of formula (4) is applied, if the calculated predicted value exceeds the range of values that the brightness value can take, the value is rounded so that the predicted value converges to the range.

The smaller the value of B, the lower the computational effort can be. When B = 0, this is consistent with the previous (MPEG-4AVC/H.264) vertical prediction that uses only the luminance value S(x, -1) of the coded (decoded) pixel adjacent to the prediction block _P ⁱⁿ .

The value of B can also be changed according to the block size of the prediction block _P ⁱⁿ . Generally, as the predicted block size increases, various signal variations are more likely to be included within the block, making it difficult to predict in a single direction. Therefore, the situation where directional prediction can be used to achieve high-precision prediction decreases.

Therefore, B ≥ 1 is set only when the block size of the prediction _block ^Pin is smaller than a predetermined size, and B = 0 is set when the block size of the prediction _block ^Pin is larger than the predetermined size.

For example, if the specified size is 16×16 pixels, then B = 0 in prediction ^blocks _Pin with block sizes larger than 16×16 pixels. This allows for the same reduction in computational processing as with conventional vertical prediction. Specifically, in block sizes with B = 0, the conditional determination process of whether x < B or x ≥ B is unnecessary. Therefore, conventional vertical prediction is performed without performing this conditional determination process, eliminating any increase in computational processing associated with conventional vertical prediction.

On the other hand, in a prediction ^block _Pin having a block size smaller than 16×16 pixels, such as 4×4 pixels and 8×8 pixels, B≥1 is satisfied, and thus prediction performance can be improved compared to conventional vertical prediction.

For example, in a prediction ^block _Pin with a block size of 4×4 pixels, when B=1, the above formula (4) is applied to the leftmost column in the prediction ^block _Pin , and a value proportional to the change in the vertical brightness value of the pixel adjacent to the left of the prediction block is added.

On the other hand, for the 2nd to 4th columns from the left end in the prediction ^block _Pin , the following equation (4) is applied without adding a value proportional to the vertical luminance value change of the left adjacent pixel of the prediction block.

By setting the value of B to a small value in this manner, an increase in the amount of calculation can be significantly suppressed.

In addition, as an actual device, as shown in the above formula (4), the calculation formula of the prediction value can be separately constructed for the pixels at the position of x<B and the pixels at the position of x≥B. It can also be constructed so that for all pixels in the prediction block P _i ⁿ , after copying the luminance value S(x, -1) of the adjacent coded (decoded) pixel above the prediction block P _i ⁿ as the prediction value of the previous vertical prediction, the value obtained by scaling S(-1, y)-S(-1, -1) to 1/t is added only to the pixels at the position of x<B, etc. As long as the prediction value equivalent to the above formula can be calculated, it can be constructed arbitrarily.

In addition, when the index value of the intra-frame prediction mode for the prediction ^block _Pin is 1 (horizontal prediction), the intra-frame prediction unit 4, 34 calculates the prediction value of the pixel in the prediction block _Pin according to the following formula ( ⁵ ) to generate a predicted image.

Here, C is an integer greater than 0. When x<C in the above formula of formula (5) is applied, if the calculated predicted value exceeds the range of values that the brightness value can take, the value is rounded so that the predicted value converges within the range.

The smaller the value of C, the lower the computational effort can be. When C = 0, it is consistent with the previous (MPEG-4AVC/H.264) horizontal prediction that uses only the luminance value S(-1, y) of the encoded (decoded) pixel adjacent to the left of the prediction ^block _Pin .

The value of C can also be changed according to the block size of the prediction ^{block Pin} _. Generally, as the predicted block size increases, various signal variations are more likely to be included within the block, making it difficult to predict in a single direction. Therefore, the situation where high-precision prediction can be performed using directional prediction decreases.

Therefore, C ≥ 1 is set only when the block size of the prediction _block ^Pin is smaller than a predetermined size, and C = 0 is set when the block size of the prediction _block ^Pin is greater than or equal to the predetermined size.

For example, if the specified size is 16×16 pixels, then C = 0 in prediction blocks _Pin with block sizes larger than 16×16 pixels. This allows the reduction of computational processing, similar to conventional horizontal prediction. ^Specifically , in block sizes with C = 0, the conditional determination process of whether y < C or y ≥ C is unnecessary. Therefore, conventional horizontal prediction is performed without performing this conditional determination process, eliminating any increase in computational processing associated with conventional horizontal prediction.

On the other hand, in prediction blocks _Pin with a block size smaller than 16×16 pixels, such as 4×4 pixels and 8× ⁸ pixels, C≥1, so the prediction performance can be improved compared to conventional horizontal prediction, and the increase in the amount of calculation can be greatly suppressed.

For example, in a prediction ^block _Pin with a block size of 4×4 pixels, when C=1, the above formula (5) is applied to the top row in the prediction ^block _Pin , and a value proportional to the change in the horizontal brightness value of the pixels adjacent to the top of the prediction block is added.

On the other hand, for the 2nd to 4th rows from the top in the prediction ^block _Pin , the following equation (5) is applied without adding a value proportional to the amount of change in the horizontal luminance value of the upper adjacent pixels of the prediction block.

By setting the value of C to a small value in this manner, an increase in the amount of calculation can be significantly suppressed.

In addition, as an actual device, as shown in the above formula (5), the calculation formula of the prediction value can be constructed separately for pixels at the position y<C and pixels at the position y≥C. It can also be constructed so that for all pixels in the prediction block P _i ⁿ , after copying the luminance value S(-1, y) of the encoded (decoded) pixel adjacent to the left of the prediction block P _i ⁿ as the previous horizontal prediction prediction value, the value obtained by scaling S(x, -1)-S(-1, -1) to 1/u is added only to the pixels at the position y<C, etc. As long as the prediction value equivalent to the above formula can be calculated, it can be constructed arbitrarily.

In addition, regarding 1/t, B as the scaling value used in vertical prediction (intra-block information representing the column within the prediction block, in which the luminance value of the pixel adjacent to the upper side of the prediction block is added with a value proportional to the vertical luminance value change of the pixel adjacent to the left side of the coding block), and 1/u, C as the scaling value used in horizontal prediction (intra-block information representing the row within the prediction block, in which the luminance value of the pixel adjacent to the left side of the prediction block is added with a value proportional to the horizontal luminance value change of the pixel adjacent to the upper side of the prediction block), it is also possible to pre-agreed between the motion image encoding device and the motion image decoding device, but the frame intra prediction unit 4 of the motion image encoding device may output t, u, B, C to the variable length coding unit 13 in sequence units or picture units, and the variable length coding unit 13 may perform variable length encoding on t, u, B, C and include them in the bit stream, and the motion image decoding device may perform variable length decoding on t, u, B, C from the bit stream for use.

In this way, t, u, B, and C can be adaptively controlled in sequence units or picture units, thereby enabling prediction processing that further corresponds to the characteristics of the video signal of the input image.

In addition, the block size for the vertical prediction using equation (4) and the horizontal prediction using equation (5) may be agreed upon in advance between the motion image encoding device and the motion image decoding device, but the intra-frame prediction unit 4 of the motion image encoding device may output an ON/OFF flag for each block size representing the block size for the vertical prediction using equation (4) and the horizontal prediction using equation (5) to the variable length encoding unit 13 according to a sequence unit or a picture unit, and the variable length encoding unit 13 may perform variable length encoding on the ON/OFF flag and include it in the bit stream, and the motion image decoding device may perform variable length decoding on the ON/OFF flag from the bit stream for use.

In this way, the block size of the vertical prediction using equation (4) and the horizontal prediction using equation (5) can be adaptively controlled in sequence units or picture units, thereby enabling prediction processing that further corresponds to the characteristics of the image signal of the input image.

As can be seen from the above, according to this embodiment 2, the intra-frame prediction units 4 and 34 are configured to, when the intra-frame prediction processing when generating the predicted image is the horizontal prediction processing, add a value proportional to the horizontal brightness value change of the pixel adjacent to the top of the prediction block to the brightness values of the pixels adjacent to the left of the prediction block for a specified number of rows starting from the upper end of the prediction block, and determine the added value as the predicted value of the predicted image, but determine the brightness value of the pixel adjacent to the left of the prediction block as the predicted value of the predicted image for the remaining rows in the prediction block, thereby suppressing the increase in the amount of calculation and improving the prediction efficiency of the horizontal prediction.

In addition, the intra-frame prediction units 4 and 34 are configured to, when the intra-frame prediction processing when generating a predicted image is a vertical prediction processing, add a value proportional to the vertical brightness value change of the pixel adjacent to the left of the prediction block to the brightness values of the pixels adjacent to the upper side of the prediction block for a predetermined number of columns starting from the left end of the prediction block, and determine the added value as the predicted value of the predicted image, but determine the brightness value of the pixel adjacent to the upper side of the prediction block as the predicted value of the predicted image for the remaining columns within the prediction block, thereby suppressing the increase in the amount of calculation and improving the prediction efficiency of the vertical prediction.

Implementation method 3.

The structural diagram of the moving picture encoding device in this embodiment 3 is the same as FIG. 1 shown in the above-mentioned embodiment 1, and the structural diagram of the moving picture decoding device in this embodiment 3 is the same as FIG. 3 shown in the above-mentioned embodiment 1.

Next, the operation will be explained.

In this embodiment 3, the following motion image encoding device and motion image decoding device are described, wherein the motion image encoding device takes each frame image of the image as an input image, implements intra-frame prediction from encoded nearby pixels or motion compensation prediction between adjacent frames, implements compression processing based on orthogonal transform/quantization on the obtained prediction differential signal, and then performs variable-length encoding to generate a bit stream, and the motion image decoding device decodes the bit stream output from the motion image encoding device.

The moving picture encoding apparatus of FIG. 1 is characterized in that it divides the video signal into blocks of various sizes and performs intra-frame/inter-frame adaptive encoding in accordance with local variations of the video signal in the spatial and temporal directions.

In general, image signals have the characteristic of locally varying signal complexity in space and time. When viewed spatially, within a given image frame, there may be patterns with uniform signal characteristics across relatively wide image regions, such as the sky or a wall, but also patterns with complex texture patterns within smaller image regions, such as figures or paintings with subtle textures.

Even when observing over time, although the local changes in the time direction of the pattern of the sky and the wall are small, the changes over time of moving people and objects are large because their outlines undergo rigid/non-rigid motion over time.

In the encoding process, a prediction differential signal with low signal power or entropy is generated through temporal/spatial prediction to reduce the overall symbol amount. However, as long as the parameters used in the prediction can be evenly applied to the largest possible image signal area, the symbol amount of the parameter can be reduced.

On the other hand, if the same prediction parameters are applied to a large image area for an image signal pattern that varies greatly temporally and spatially, prediction errors increase, and thus the code amount of the prediction difference signal increases.

Therefore, in areas with large temporal/spatial variations, it is preferable to reduce the block size for prediction processing using the same prediction parameters, increase the amount of data of the parameters used in the prediction, and reduce the power/entropy of the prediction differential signal.

In this embodiment 3, in order to perform encoding that is adapted to the general properties of such image signals, the following structure is adopted: the prediction processing is initially started from a specified maximum block size, the image signal area is hierarchically divided, and for each divided area, the prediction processing and the encoding processing of its prediction difference are adapted.

Regarding the motion image encoding device in Figure 1, the image signal format of the processing object is set to be, in addition to color image signals of arbitrary color spaces such as YUV signals composed of a luminance signal and two color difference signals, RGB signals output from a digital camera element, etc., it is also set to arbitrary image signals such as monochrome image signals and infrared image signals whose image frames are composed of horizontal and vertical two-dimensional digital sampling (pixel) columns.

The grayscale of each pixel can be 8 bits, 10 bits, 12 bits, etc.

For convenience, in the following description, unless otherwise specified, it is assumed that the video signal of the input image is a YUV signal, and a 4:2:0 format signal in which the two color difference components U and V are subsampled relative to the luminance component Y is processed.

In addition, a processing data unit corresponding to each frame of a video signal is referred to as a "picture".

In this third embodiment, the "picture" is described as an image frame signal scanned sequentially (progressive scanning), but when the image signal is an interlaced scanning signal, the "picture" can also be a field image signal that is a unit constituting an image frame.

First, the processing contents of the moving picture encoding apparatus in FIG. 1 will be described.

First, the encoding control unit 2 determines the size of the maximum coding block used for encoding the picture to be encoded (current picture) and the upper limit of the number of hierarchical divisions of the maximum coding block (step ST1 in FIG. 2 ).

As a method for determining the size of the maximum coding block, for example, the same size can be determined for all pictures based on the resolution of the image signal of the input image, or the difference in the complexity of the local motion of the image signal of the input image can be quantified as a parameter, and a small size can be determined for pictures with intense motion, and a large size can be determined for pictures with less motion.

As a method for determining the upper limit of the number of segmentation levels, there is a method of determining the same number of levels for all pictures based on the resolution of the image signal of the input image, or a method of setting the number of levels to be increased when the motion of the image signal of the input image is intense so as to be able to detect more subtle motion, and setting the number of levels to be suppressed when the motion is small.

Furthermore, the encoding control unit 2 selects an encoding mode corresponding to each hierarchically divided encoding block from one or more available encoding modes (step ST2 ).

That is, the encoding control unit 2 hierarchically divides each image region of the maximum encoding block size into encoding blocks having the encoding block size until the upper limit of the number of division levels determined in advance is reached, and determines the encoding mode for each encoding block.

Among the coding modes, there are one or more intra-frame coding modes (collectively referred to as "INTRA") and one or more inter-frame coding modes (collectively referred to as "INTER"). The coding control unit 2 selects the coding mode corresponding to each coding block from all coding modes available in the picture or a subset thereof.

Each coding block hierarchically divided by a block division unit 1 described later is further divided into one or more prediction blocks as units for performing prediction processing, and the division status of the prediction blocks is also included as information in the coding mode.

The method for selecting the coding mode of the coding control unit 2 is a well-known technology, so detailed description is omitted. For example, there are the following methods: use any available coding mode to implement coding processing for the coding block to verify the coding efficiency, and select the coding mode with the best coding efficiency from multiple available coding modes.

Furthermore, the encoding control unit 2 determines, for each encoding block, a quantization parameter and a transform block size used when compressing a differential image, and determines prediction parameters (intra-frame prediction parameters or inter-frame prediction parameters) used when performing a prediction process.

When the coding block is further divided into prediction block units for performing prediction processing, prediction parameters (intra-frame prediction parameters or inter-frame prediction parameters) can be selected for each prediction block.

Furthermore, in a coding block whose coding mode is the intra-frame coding mode, since the encoded pixels adjacent to the prediction block are used when performing intra-frame prediction processing as described later, encoding needs to be performed in units of prediction blocks, so the selectable transform block size is limited to be less than the size of the prediction block.

The encoding control unit 2 outputs the prediction difference encoding parameters including the quantization parameter and the transform block size to the transform/quantization unit 7 , the inverse quantization/inverse transform unit 8 , and the variable length encoding unit 13 .

Furthermore, the encoding control unit 2 outputs the intra prediction parameters to the intra prediction unit 4 as needed.

Furthermore, the encoding control unit 2 outputs the inter-frame prediction parameters to the motion compensation prediction unit 5 as needed.

If the image signal of the input image is input, the block division unit 1 divides the image signal of the input image into the maximum coding block size determined by the coding control unit 2, and further hierarchically divides the divided maximum coding block into coding blocks determined by the coding control unit 2, and outputs the coding blocks.

Here, FIG5 is an explanatory diagram showing an example in which a maximum coding block is hierarchically divided into a plurality of coding blocks.

In FIG5 , the largest coding block is a coding block having a size of (L ⁰ , M ⁰ ) for the luminance component described as “layer 0”.

The largest coding block is used as a starting point, and coding blocks are obtained by hierarchically dividing the block up to a predetermined depth determined separately using a quadtree structure.

At depth n, a coding block is an image region of size ( ^Ln , ^Mn ).

^Ln and ^Mn may be the same or different, but FIG. 5 shows the case where ^Ln = ^Mn .

Hereinafter, the coding block size determined by the coding control unit 2 is defined as the size (L ⁿ , M ⁿ ) of the luminance component of the coding block.

Since quadtree partitioning is performed, (Ln ⁺¹ , Mn ⁺¹ ) = ( ^Ln /2, ^Mn /2) always holds.

Furthermore, in a color video signal (4:4:4 format) such as an RGB signal where all color components have the same number of samples, the size of all color components is ( ^Ln , ^Mn ). However, when processing the 4:2:0 format, the encoding block size of the corresponding color difference component is ( ^Ln /2, ^Mn /2).

Hereinafter, it is assumed that ^Bn represents the coding block of the nth layer, and m( ^Bn ) represents the coding mode selectable in the coding block ^Bn .

In the case of a color video signal composed of multiple color components, the coding mode m(B ⁿ ) can be configured to use a separate mode for each color component or a common mode for all color components. Unless otherwise specified, the following description refers to the coding mode corresponding to the luma component of a YUV signal and a coding block in the 4:2:0 format.

As shown in FIG6 , the coding block ^Bn is divided by the block division unit 1 into one or more prediction blocks representing prediction processing units.

Hereinafter, the prediction block belonging to the coding block ^Bn will be referred to as _P ⁱⁿ (i is the prediction block number in the nth layer). FIG5 shows an example ^of _P00 ^and _P10 .

Coding mode m(B ⁿ ) includes information on how to perform prediction block division of coding block B ⁿ .

Regarding all prediction ^blocks _Pin , prediction processing is performed according to the coding mode m( ^Bn ), but individual prediction parameters (intra-frame prediction parameters or inter-frame prediction parameters) can be selected for each prediction ^block _Pin .

The encoding control unit 2 generates a block division state as shown in FIG. 6 for the largest encoding block, for example, and determines the encoding blocks.

The rectangles surrounded by dotted lines in FIG6(a) represent each coding block, and the blocks filled with diagonal lines within each coding block represent the division status of each prediction block.

Fig. 6(b) shows the allocation of coding mode m( ^Bn ) by hierarchical partitioning using a quadtree graph in the example of Fig. 6(a). Nodes surrounded by squares in Fig. 6(b) are nodes (coding blocks) to which coding mode m( ^Bn ) is allocated.

The information on the quadtree pattern is output from the encoding control unit 2 to the variable-length encoding unit 13 together with the encoding mode m(B ⁿ ) and multiplexed into the bit stream.

When the coding mode m(B ⁿ ) determined by the coding control unit 2 is the intra coding mode (m(B ⁿ )∈INTRA), the switch 3 outputs the coding block B ⁿ output from the block division unit 1 to the intra prediction unit 4 .

On the other hand, when the coding mode m(B ⁿ ) determined by the coding control unit 2 is the inter-coding mode (m(B ⁿ )∈INTER), the coding block B ⁿ output from the block division unit 1 is output to the motion compensation prediction unit 5 .

In the intra-frame prediction unit 4, if the coding mode m( ^Bn ) determined by the coding control unit 2 is the intra-frame coding mode (in the case where m( ^Bn )∈INTRA) and the coding block ^Bn is received from the switch 3 (step ST3), the intra-frame prediction unit 4 performs intra-frame prediction processing on each prediction block P in the coding block ^Bn using the intra-frame prediction parameters determined by the coding control unit 2 while referring to the local decoded image stored in the intra- ^frame prediction memory ₁₀ , thereby generating an intra-frame prediction image P _INTRAi ⁿ (step ST4).

In addition, the motion picture decoding device needs to generate an intra-frame prediction image that is completely identical to the intra-frame prediction image P _INTRAi ⁿ , so the intra-frame prediction parameters used in generating the intra-frame prediction image P _INTRAi ⁿ are output from the encoding control unit 2 to the variable length encoding unit 13 and multiplexed into the bit stream.

The details of the processing performed by the intra-frame prediction unit 4 will be described later.

In the motion-compensated prediction unit 5, if the coding mode m( ^Bn ) determined by the coding control unit 2 is the inter-frame coding mode (in the case where m( ^Bn )∈INTER), and the coding block ^Bn is received from the switching switch 3 (step ST3), each prediction block _P in the coding block ^Bn is compared ^with the filtered local decoded image stored in the motion-compensated prediction frame memory 12 to search for a motion vector. Then, using the motion vector and the inter-frame prediction parameters determined by the coding control unit 2, inter-frame prediction processing is performed on each prediction ^block _P in the coding block ^Bn to generate an inter-frame prediction image _PINTERi ⁿ (step ST5).

Furthermore, the moving picture decoding apparatus needs to generate an inter-frame prediction image identical to the inter-frame prediction image _PINTERi ⁿ , so the inter-frame prediction parameters used in generating the inter-frame prediction image _PINTERi ⁿ are output from the encoding control unit 2 to the variable-length encoding unit 13 and multiplexed into the bitstream.

In addition, the motion vector searched by the motion compensation prediction unit 5 is also output to the variable length coding unit 13 and multiplexed into the bit stream.

Upon receiving the coding block ^Bn from the block partitioning unit 1, the subtraction unit 6 subtracts either the intra- ^frame prediction image P _INTRAi ⁿ generated by the intra-frame prediction unit 4 or the inter-frame prediction image P _INTERi ⁿ generated by the motion-compensated prediction unit 5 from the prediction block P in the coding _block ^Bn , and outputs a prediction difference signal _e in which the subtraction result is ^obtained to the transform/quantization unit 7 (step ST6).

Upon receiving the prediction differential signal _e ⁱⁿ from the subtraction unit 6, the transform/quantization unit 7 performs an orthogonal transform process (e.g., DCT (discrete cosine transform), KL transform with a basis design performed on a predetermined learning sequence, or ^other orthogonal transform process) on the prediction differential signal _e in reference to the prediction differential coding parameters determined by the coding control unit 2, and calculates the transform coefficients.

Furthermore, the transform/quantization unit 7 quantizes the transform coefficients with reference to the prediction difference coding parameters, and outputs the compressed data of the quantized transform coefficients to the inverse quantization/inverse transform unit 8 and the variable length coding unit 13 (step ST7).

Upon receiving the compressed data from the transform/quantization unit 7 , the inverse quantization/inverse transform unit 8 inversely quantizes the compressed data with reference to the prediction difference encoding parameter determined by the encoding control unit 2 .

In addition, the inverse quantization/inverse transform unit 8 refers to the predicted differential coding parameters, performs inverse orthogonal transform processing (for example, inverse DCT, inverse KL transform, etc.) on the transform coefficients of the compressed data after inverse quantization, calculates the local decoded predicted differential signal equivalent to the predicted differential signal e _i ⁿ output from the subtraction unit 6, and outputs it to the addition unit 9 (step ST8).

If the adding unit 9 receives the local decoded prediction differential signal from the inverse quantization/inverse transform unit 8, it adds the local decoded prediction differential signal to one of the intra-frame prediction image P _INTRAi ⁿ generated by the intra-frame prediction unit 4 and the inter-frame prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 5, thereby calculating the local decoded image (step ST9).

Furthermore, the adding unit 9 outputs the local decoded image to the loop filtering unit 11 and stores the local decoded image in the intra-frame prediction memory 10 .

This locally decoded image becomes an encoded image signal used in subsequent intra-frame prediction processing.

Upon receiving the local decoded image from the adder 9 , the loop filter unit 11 performs a predetermined filtering process on the local decoded image and stores the filtered local decoded image in the motion-compensated prediction frame memory 12 (step ST10 ).

The filtering process performed by the loop filter unit 11 may be performed in units of the largest coding block or each coding block of the input local decoded image, or may be performed collectively for one picture after one picture of local decoded images is input.

In addition, as examples of prescribed filtering processing, there can be cited processing that filters block boundaries in a manner that prevents discontinuity (block noise) at the coding block boundaries from becoming noticeable, filtering processing that compensates for distortion of a local decoded image in a manner that minimizes the error between the image signal of Figure 1 as the input image and the local decoded image, and the like.

However, when performing filtering processing to compensate for the distortion of the local decoded image in a manner that minimizes the error between the image signal of Figure 1 as the input image and the local decoded image, it is necessary to refer to the image signal in the loop filter unit 11, so it is necessary to change the motion image encoding device of Figure 1 so that the image signal is input to the loop filter unit 11.

The processes of steps ST3 to ST9 are repeatedly performed until the processes of all the hierarchically divided coding blocks ^Bn are completed. When the processes of all the coding blocks ^Bn are completed, the process proceeds to step ST13 (steps ST11 and ST12).

The variable-length coding unit 13 performs variable-length coding on the compressed data output from the transform/quantization unit 7, the block division information within the largest coding block output from the coding control unit 2 (quadtree information in FIG. 6B ), the coding mode m (B ⁿ ) and the prediction difference coding parameters, the intra-frame prediction parameters (when the coding mode is the intra-frame coding mode) or inter-frame prediction parameters (when the coding mode is the inter-frame coding mode) output from the coding control unit 2, and the motion vector (when the coding mode is the inter-frame coding mode) output from the motion compensation prediction unit 5, and generates a bitstream representing these coding results (step ST13).

Next, the processing contents of the intra prediction unit 4 will be described in detail.

FIG17 is an explanatory diagram showing an example of intra-frame prediction parameters (intra-frame prediction modes) selectable by each prediction ^block _Pin within a coding block ^Bn . _NI represents the number of intra-frame prediction modes.

FIG17 shows an intra-frame prediction mode and a prediction direction vector represented by the intra-frame prediction mode. In the example of FIG17 , the design is such that the relative angles between the prediction direction vectors decrease as the number of selectable intra-frame prediction modes increases.

As described above, the intra ^- frame prediction unit 4 refers to the intra-frame prediction parameters of the prediction ^block _Pin , performs intra-frame prediction processing on the prediction ^block _Pin , and generates an intra-frame prediction image _Pintran . However, the intra-frame processing for generating an intra- ^frame prediction signal of the prediction block _Pin in the luminance signal is described here.

The size of ^the prediction ^block _Pin is set to _lin × _min pixels ^.

FIG8 is an explanatory diagram showing an example of pixels used when generating predicted values of _pixels in a ^prediction block ^Pin in the case of _lin = _min = 4 ^.

In FIG8 , the coded pixels ( ² × _lin +1) above and the coded pixels ⁽ 2× _min ) to the left of the _prediction ^block Pin are used as pixels for prediction. However, the number of pixels used for prediction may be more or less than the number of pixels shown in FIG8 .

In addition, in Figure 8, one row or one column of pixels near the prediction ^{block Pin} _is used for prediction, but two rows or two columns, or more than two rows of pixels may be used for prediction.

When the index value of the intra-frame prediction mode for the prediction ^block _Pin is 0 (planar prediction), the value obtained by interpolating the distance between the prediction object pixel in the prediction ^block _Pin and the above-adjacent coded pixels and ^the _left -adjacent coded pixels of the prediction _{block Pin} ^is used as the prediction value to generate the predicted image.

When the index value of the intra-frame prediction mode for the prediction ^block _Pin is 1 (vertical prediction), the prediction value of the pixel in the prediction ^block _Pin is calculated according to the following formula (1) to generate a predicted image.

S'(x, y)=S(x,-1)+(S(-1, y)-S(-1,-1))/t (1)

Among them, the coordinates (x, y) are relative coordinates with the upper left pixel in the prediction block P _i ⁿ as the origin (refer to Figure 9), S'(x, y) is the predicted value at the coordinates (x, y), and S(x, y) is the brightness value (decoded brightness value) of the encoded pixel at the coordinates (x, y).

In this way, a value proportional to S(-1, y)-S(-1, -1) representing the amount of change ⁱⁿ the vertical brightness value of the coded pixel adjacent to the left of the prediction ^block _Pin (the pixel surrounded by the thick frame in Figure 10) is added to the brightness value S(x, -1) of the coded pixel above the _prediction block Pin, which is the prediction value of the vertical prediction in the previous (MPEG-4AVC/H.264) method (a value obtained by scaling S(-1, y)-S(-1, -1) representing the amount of change in the vertical brightness value to 1/t) is added, and the added value is determined as the prediction value of the predicted image, thereby realizing vertical prediction that follows the change in the brightness value in the prediction direction.

However, if the predicted value does not fall within the range of acceptable luminance values, the value may be rounded to fall within that range. While this slightly increases the amount of computation required due to rounding, it can prevent predicted values from falling outside the acceptable luminance range, thereby reducing prediction errors.

In addition, the above-mentioned 1/t may be set as a fixed value, or may be set as a variable that changes according to the coordinates (x, y).

For example, if t=2x ⁺¹ , as shown in FIG11 , the scaling value decreases in order from the left column, such as 1/2, 1/4, 1/8, and 1/16. Therefore, the greater the distance from the left adjacent coded pixel of the prediction ^block _Pin , the smaller the change in the added vertical brightness value.

Therefore, the farther the prediction target pixel is from the already encoded pixel adjacent to the left of the prediction ^{block Pin} _and the lower the correlation becomes, the smaller the influence of the already encoded pixel adjacent to the left of the prediction ^block _Pin can be. Therefore, a high-precision prediction corresponding to the correlation between the already encoded pixels adjacent to the left of the ^prediction block _Pin can be performed.

In addition, when t=2x ⁺¹ , the equation (1) can be expressed by an equation based on bit shift as follows.

S'(x, y)=S(x,-1)+(S(-1, y)-S(-1,-1))>>(x+1) (1a)

In formula (1a), “>>a” represents an operation of a right arithmetic shift by a bits.

By using a shift operation instead of the division in equation (1), high-speed operation can be achieved when the method is installed on a computer.

However, S(-1, y)-S(-1, -1) can also take negative values, so sometimes, depending on the installation environment (compiler), ">>" is not processed as an arithmetic shift but as a logical shift, and the calculation result is different from formula (1).

Therefore, as an approximate expression of the expression (1) in the case of t=2x ⁺¹ that does not depend on the installation environment, the following expression (1b) can be cited.

S'(x, y)=S(x,-1)+S(-1, y)>>(x+1)-S(-1,-1)>>(x+1) (1b)

In formula (1b), the brightness values S(-1, y) and S(-1, -1) are first shifted to the right by (x+1) bits before the subtraction operation is performed. Therefore, if the brightness value is defined by a positive value, the arithmetic shift and the logical shift will obtain the same calculation result.

Furthermore, the block size of the prediction ^block _P in which the prediction process of equation (1) is performed can also be limited to a specific size. Generally, with large block sizes, various signal variations are easily included within the block, making it less likely that directional prediction can be used to achieve high-precision prediction. Therefore, for example, equation (1) ^{is not applied to prediction blocks P in a block size of 16×16 pixels or larger. Instead} _, equation (1) is applied only to blocks smaller than 16×16 pixels as the predicted value for conventional vertical prediction (the luminance value S(x,-1) of the already encoded pixel adjacent to the top of the ^{prediction block} _P ). This improves prediction performance and suppresses an increase in computational complexity compared to conventional vertical prediction.

In addition, when the index value of the intra-frame prediction mode for the prediction ^block _Pin is 2 (horizontal prediction), the prediction value of the pixel in the prediction ^block _Pin is calculated according to the following formula (2) to generate a predicted image.

S'(x, y)=S(-1, y)+(S(x,-1)-S(-1,-1))/u (2)

Among them, the coordinates (x, y) are relative coordinates with the upper left pixel in the prediction block P _i ⁿ as the origin (refer to Figure 9), S'(x, y) is the predicted value at the coordinates (x, y), and S(x, y) is the brightness value (decoded brightness value) of the encoded pixel at the coordinates (x, y).

In this way, a value proportional to S(x, -1)-S(-1, -1) representing the amount of change in the horizontal brightness value of the coded pixel adjacent to the top of the prediction ^block _Pin (the pixel surrounded by the thick frame in Figure 12) is added to the brightness value S(-1, y) of the coded pixel adjacent to the left of the _prediction block Pin, which is the prediction value of the horizontal prediction in the ^previous (MPEG-4AVC/H.264) method (a value obtained by scaling S(x, -1)-S(-1, -1) representing the amount of change in the horizontal brightness value to 1/u) is added, and the added value is determined as the prediction value of the predicted image, thereby realizing horizontal prediction that follows the change in the brightness value in the prediction direction.

However, if the predicted value does not fall within the range of acceptable luminance values, the value may be rounded to fall within that range. While this rounding process slightly increases the computational complexity, it can prevent predicted values from falling outside the acceptable luminance range, thereby reducing prediction errors.

In addition, the above-mentioned 1/u may be set as a fixed value, or may be set as a variable that changes according to the coordinates (x, y).

For example, if u=2y ⁺¹ , as shown in FIG13, the scaling value decreases from the top row in the order of 1/2, 1/4, 1/8, and 1/16, so the greater the distance from the adjacent encoded pixel above the prediction block _P ⁱⁿ , the smaller the change in the added horizontal brightness value.

Therefore, the farther the distance from the already-encoded pixels adjacent to the prediction ^{block Pin} _is , and the lower the correlation is, the smaller the influence of the already-encoded pixels adjacent to the prediction ^{block Pin} _is . Therefore, a high-precision prediction corresponding to the correlation between the already-encoded pixels adjacent to the prediction ^{block Pin} _can be performed.

In addition, when u=2y ⁺¹ , the equation (2) can be expressed by an equation based on bit shifting as follows.

S'(x, y)=S(-1, y)+(S(x,-1)-S(-1,-1))>>(y+1) (2a)

In formula (2a), “>>a” represents an operation of arithmetic shift to the right by a bits.

By using a shift operation instead of the division in equation (2), high-speed operation can be achieved when the method is installed on a computer.

However, S(x, -1)-S(-1, -1) can also take negative values, so sometimes, depending on the installation environment (compiler), ">>" is not processed as an arithmetic shift but as a logical shift, and the calculation result is different from formula (2).

Therefore, as an approximate expression of the expression (2) in the case of u=2y ⁺¹ that does not depend on the installation environment, the following expression (2b) can be given.

S'(x, y)=S(-1, y)+S(x,-1)>>(y+1)-S(-1,-1)>>(y+1) (2b)

In formula (2b), the brightness values S(x, -1) and S(-1, -1) are first shifted to the right by (y+1) bits before the subtraction operation is performed. Therefore, if the brightness value is defined by a positive value, the arithmetic shift and the logical shift will obtain the same calculation result.

Furthermore, the block size of the prediction ^block _P in which the prediction process of equation (2) is performed can also be limited to a specific size. Generally, with large block sizes, various signal variations are easily included within the block, making it less likely that directional prediction can be used to achieve high-precision prediction. Therefore, for example, in prediction blocks ^P in a block size of 16×16 _pixels or larger, equation (2) is not applied, and instead the prediction value of conventional horizontal prediction (the luminance value S(-1, y) of the already coded pixel adjacent to the left of the ^prediction block _P ) is used. Equation (2) is applied only to blocks smaller than 16×16 pixels. This improves prediction performance and suppresses an increase in computational complexity compared to conventional horizontal prediction.

In addition, when the index value of the intra-frame prediction mode for the prediction ^block _Pin is 3 (average value (DC) prediction), the average value of the coded pixels adjacent to the top of the prediction ^block _Pin and the average value of the coded pixels adjacent to the left of the prediction ^block _Pin is used as the prediction value of the pixel in the prediction ^block _Pin to generate a predicted image.

When the index value of the intra-frame prediction mode is other than 0 (planar prediction), 1 (vertical prediction), 2 (horizontal prediction), and 3 (average value (DC) prediction), the predicted value of the pixel in the prediction ^block _Pin is generated according to the prediction direction vector _υp = (dx, dy) represented by the index value.

As shown in FIG9 , if the upper left pixel of the prediction _block ^Pin is set as the origin and the relative coordinates within the ^prediction block _Pin are set to (x, y), the position of the reference pixel used in the prediction becomes the intersection of the following L and the adjacent pixel.

Here, k is a negative scalar value.

When the reference pixel is at an integer pixel position, the integer pixel is used as the predicted value of the prediction target pixel. When the reference pixel is not at an integer pixel position, an interpolated pixel generated from integer pixels adjacent to the reference pixel is used as the predicted value.

In the example of FIG8 , since the reference pixel is not located at an integer pixel position, the prediction value is obtained by interpolating two pixels adjacent to the reference pixel. Furthermore, the prediction value can be generated by interpolating pixels not only from two adjacent pixels but also from two or more adjacent pixels.

By increasing the number of pixels used in the interpolation process, the interpolation accuracy of the interpolated pixels is improved. On the other hand, since the complexity of the calculations required for the interpolation process increases, in the case of a motion picture encoding device that requires high encoding performance even if the calculation load is large, it is preferable to generate interpolated pixels based on a larger number of pixels.

Through the same steps, predicted pixels corresponding to all pixels of the luminance signal in the prediction ^block _Pin are generated, and an intra-frame predicted image _Pintran ^is output.

In addition, the intra-frame prediction parameters used for generating ^{the intra-frame prediction image PINTRAin} _are output to the variable-length coding unit 13 in order to be multiplexed into the bit stream.

In addition, similar to the intra-frame prediction of 8×8 pixel blocks in MPEG-4AVC/H.264 described above, the pixels used for intra-frame prediction may not be the pixels in the encoded adjacent blocks themselves, but the results of filtering these pixels may be used.

For the color difference signal of the prediction ^block _Pin , the same steps as the luminance signal are used to implement intra-frame prediction processing based on intra-frame prediction parameters (intra-frame prediction mode), and the intra-frame prediction parameters used in generating the intra-frame prediction image are output to the variable length coding unit 13.

However, the intra-frame prediction parameters (intra-frame prediction modes) selectable in the chrominance signal do not need to be the same as those for the luminance signal. In addition, the vertical prediction and horizontal prediction may also be the conventional (MPEG-4 AVC/H.264) prediction methods.

For example, in the case of a YUV signal in 4:2:0 format, the color difference signals (U and V signals) are signals obtained by reducing the resolution of the luminance signal (Y signal) to 1/2 in both the horizontal and vertical directions. Compared with the luminance signal, the image signal has low complexity and is easier to predict, so the number of selectable intra-frame prediction parameters (intra-frame prediction modes) is less than that of the luminance signal. The vertical and horizontal predictions are also set to the previous simple prediction methods, thereby almost without reducing the prediction efficiency and achieving a reduction in the amount of symbols required to encode the intra-frame prediction parameters (intra-frame prediction modes) and low computational complexity in the prediction processing.

In addition, the scaling value 1/t used for vertical prediction and the scaling value 1/u used for horizontal prediction can also be agreed upon in advance between the motion image encoding device and the motion image decoding device, but the intra-frame prediction unit 4 of the motion image encoding device can output t and u to the variable length encoding unit 13 according to sequence units or picture units, and the variable length encoding unit 13 can perform variable length encoding on t and u and include them in the bit stream, and the motion image decoding device can perform variable length decoding on t and u from the bit stream for use.

By adaptively controlling t and u in sequence units or picture units in this manner, it is possible to implement prediction processing that further corresponds to the characteristics of the video signal of the input image.

In addition, the block size for the vertical prediction using formula (1) and the horizontal prediction using formula (2) can also be agreed upon in advance between the motion image encoding device and the motion image decoding device, but the intra-frame prediction unit 4 of the motion image encoding device can output the ON/OFF flag of each block size representing the block size for the vertical prediction using formula (1) and the horizontal prediction using formula (2) to the variable length encoding unit 13 according to the sequence unit or the picture unit, and the variable length encoding unit 13 can perform variable length encoding on the above-mentioned ON/OFF flag and include it in the bit stream, and the motion image decoding device can perform variable length decoding on the above-mentioned ON/OFF flag from the bit stream for use.

In this way, the block size of the vertical prediction using equation (1) and the horizontal prediction using equation (2) can be adaptively controlled in sequence units or picture units, thereby enabling prediction processing that further corresponds to the characteristics of the image signal of the input image.

Next, the processing contents of the moving picture decoding device in FIG. 3 will be described in detail.

If the variable length decoding unit 31 inputs the bit stream generated by the motion image encoding device in Figure 1, it implements variable length decoding processing on the bit stream (step ST21 in Figure 4) and decodes the frame size information according to the sequence unit composed of more than one frame of pictures or the picture unit.

At this time, among the parameter t of the scaling value used in the vertical prediction, the parameter u of the scaling value used in the horizontal prediction, and the ON/OFF flag of each block size indicating the block size of the vertical prediction using formula (1) and the horizontal prediction using formula (2), as long as any one of them is variable-length encoded and multiplexed into the bit stream, it is decoded according to the unit (sequence unit or picture unit) encoded by the motion image encoding device of Figure 1.

The variable length decoding unit 31 determines the maximum coding block size and the upper limit of the number of division hierarchies determined by the encoding control unit 2 of the moving picture encoding device of FIG. 1 through the same steps as the moving picture encoding device (step ST22 ).

For example, when the maximum coding block size and the upper limit of the number of division levels are determined based on the resolution of the video signal, the maximum coding block size is determined based on the decoded frame size information using the same procedure as that of the moving picture encoding device.

When the maximum coding block size and the upper limit of the number of division layers are multiplexed into the bitstream on the moving picture coding device side, the values obtained by decoding the bitstream are used.

Hereinafter, in the moving picture decoding apparatus, the maximum coding block size is referred to as the maximum decoding block size, and the maximum coding block is referred to as the maximum decoding block.

The variable length decoding unit 31 decodes the division state of the maximum decoding block as shown in FIG6 in units of the determined maximum decoding block. Based on the decoded division state, the decoding blocks (blocks corresponding to the "coding blocks" in the moving image encoding device of FIG1) are hierarchically determined (step ST23).

Next, the variable length decoding unit 31 decodes the coding mode assigned to the decoding block. Based on the information included in the decoded coding mode, the decoding block is further divided into one or more prediction processing units, namely prediction blocks, and prediction parameters assigned to the prediction block units are decoded (step ST24).

That is, when the coding mode assigned to the decoding block is the intra coding mode, the variable length decoding unit 31 decodes the intra prediction parameters for each of one or more prediction blocks included in the decoding block and serving as prediction processing units.

On the other hand, when the coding mode assigned to the decoding block is the inter-frame coding mode, the inter-frame prediction parameters and motion vectors are decoded for each of one or more prediction blocks included in the decoding block and serving as prediction processing units (step ST24).

Furthermore, the variable length decoding unit 31 divides the decoding block into one or more transform blocks serving as transform processing units based on the transform block size information included in the predicted differential coding parameters, and decodes the compressed data (transformed/quantized transform coefficients) for each transform block (step ST24).

In the switching switch 33, if the coding mode m( ^Bn ) variable-length-decoded by the variable-length decoding unit 31 is an intra-frame coding mode (in the case of m( ^Bn )∈INTRA), the intra-frame prediction parameters of the prediction block units variable-length-decoded by the variable-length decoding unit 31 are output to the intra-frame prediction unit 34.

On the other hand, if the coding mode m(B ⁿ ) variable-length-decoded by the variable-length decoding unit 31 is an inter-frame coding mode (in the case of m(B ⁿ )∈INTER), the inter-frame prediction parameters and motion vectors of the prediction block units variable-length-decoded by the variable-length decoding unit 31 are output to the motion compensation unit 35.

When the coding mode m( ^Bn ) variable-length-decoded by the variable-length decoding unit 31 is the intra coding mode (m( ^Bn )∈INTRA) (step ST25), the intra prediction unit 34 receives the intra prediction parameters for the prediction block output from the switch 33, and performs intra prediction processing on each prediction block P in the decoded block ^Bn using the intra prediction parameters through the same steps as those of the intra prediction unit ⁴ in FIG. 1 while referring to the decoded image stored in the intra prediction memory ₃₇ , thereby generating an intra prediction image P _INTRAi ⁿ (step ST26).

That is, when the index value of the intra prediction mode for the prediction ^block _Pin is 0 (vertical prediction), the intra prediction unit 34 calculates the prediction value of the pixels in the prediction ^block _Pin according to the above equation (1) to generate the intra ^prediction image _Pintran .

Furthermore, when the index value of the intra prediction mode for the prediction ^block _Pin is 1 (horizontal prediction), the prediction values of the pixels in the prediction ^block _Pin are calculated according to the above equation (2) to generate the ^intra prediction image _Pintrain .

However, when the block size of the vertical prediction using formula (1) and the horizontal prediction using formula (2) is limited, intra-frame prediction processing is performed in the prediction block Pin of a size other than the block size of the vertical prediction using ^formula (1) and the horizontal prediction using _formula (2) using the previous (MPEG-4AVC/H.264) vertical prediction and horizontal prediction.

When the coding mode m( ^Bn ) variable-length-decoded by the variable-length decoding unit 31 is an inter-frame coding mode (m( ^Bn )∈INTER) (step ST25), the motion compensation unit 35 receives the motion vector and inter-frame prediction parameters of the prediction block unit output from the switch 33, and performs inter-frame prediction processing on each prediction block P in the decoded block using the motion vector and inter-frame prediction parameters while referring to the filtered decoded image stored in the motion- ^{compensated prediction frame memory 39} _, thereby generating an inter-frame prediction image _PINTERi ⁿ (step ST27).

If the inverse quantization/inverse transform unit 32 receives compressed data and predicted differential coding parameters from the variable length decoding unit 31, it performs inverse quantization on the compressed data with reference to the predicted differential coding parameters through the same steps as the inverse quantization/inverse transform unit 8 in Figure 1, and performs inverse orthogonal transform processing on the transform coefficients of the inverse quantized compressed data with reference to the predicted differential coding parameters, and calculates a decoded predicted differential signal that is the same as the locally decoded predicted differential signal output from the inverse quantization/inverse transform unit 8 in Figure 1 (step ST28).

The addition unit 36 adds the decoded prediction differential signal calculated by the inverse quantization/inverse transform unit 32 to either the intra-frame prediction image P _INTRAi ⁿ generated by the intra-frame prediction unit 34 or the inter-frame prediction image P _INTERi ⁿ generated by the motion compensation unit 35 to calculate a decoded image, outputs it to the loop filter unit 38, and stores the decoded image in the intra-frame prediction memory 37 (step ST29).

This decoded image becomes a decoded image signal used in subsequent intra-frame prediction processing.

In the loop filter unit 38, if the processing of steps ST23 to ST29 for all decoding blocks ^Bn is completed (step ST30), the prescribed filtering processing is performed on the decoded image output from the addition unit 36, and the decoded image after filtering is stored in the motion compensation prediction frame memory 39 (step ST31).

The filtering process of the loop filter unit 38 may be performed in units of the largest decoded block or each decoded block of the input decoded image, or may be performed collectively for one picture after one picture of decoded images is input.

Examples of the predetermined filtering process include filtering of block boundaries to prevent discontinuity (block noise) at the coding block boundaries from becoming noticeable and filtering to compensate for distortion of a decoded image.

This decoded image becomes a reference image for motion compensation prediction and also becomes a reconstructed image.

As can be seen from the above, according to this embodiment 3, the intra-frame prediction unit 4 of the motion image encoding device is constructed to add a value proportional to the amount of change in the horizontal brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the left of the prediction block when the intra-frame prediction processing when generating the prediction image is a horizontal prediction processing, and to determine the added value as the prediction value of the prediction image; when the intra-frame prediction processing when generating the prediction image is a vertical prediction processing, add a value proportional to the amount of change in the vertical brightness value of the pixel adjacent to the left of the prediction block to the brightness value of the pixel adjacent to the upper side of the prediction block, and determine the added value as the prediction value of the prediction image, thereby achieving the effect of improving image quality by achieving high-precision prediction even when the signal value changes along the prediction direction.

In addition, according to the third embodiment, the intra-frame prediction unit 34 of the motion image decoding device is configured to, when the intra-frame prediction processing when generating the predicted image is a horizontal prediction processing, add a value proportional to the amount of change in the horizontal brightness value of the pixel adjacent to the upper side of the prediction block to the brightness value of the pixel adjacent to the prediction block, and determine the added value as the predicted value of the predicted image; when the intra-frame prediction processing when generating the predicted image is a vertical prediction processing, add a value proportional to the amount of change in the vertical brightness value of the pixel adjacent to the left side of the prediction block to the brightness value of the pixel adjacent to the upper side of the prediction block, and determine the added value as the predicted value of the predicted image. This has the effect of achieving high-precision prediction even when the signal value changes along the prediction direction, thereby improving image quality.

According to this embodiment 3, the scaling value 1/u used when the frame internal prediction units 4 and 34 perform horizontal prediction processing is configured so that the scaling value related to the row farther from the pixel adjacent to the top of the prediction block is set to a smaller value. Therefore, the farther the distance between the pixels adjacent to the top of the prediction block and the lower the correlation, the smaller the influence of the pixels adjacent to the top of the prediction block can be. As a result, the effect of high-precision prediction can be achieved.

In addition, regarding the scaling value 1/t used when the frame internal prediction units 4 and 34 implement vertical prediction processing, the scaling value related to the column that is farther away from the pixel adjacent to the left of the prediction block is set to a smaller value. Therefore, the farther the distance between the pixels adjacent to the left of the prediction block is, the lower the correlation becomes. The smaller the influence of the pixels adjacent to the left of the prediction block can be. As a result, the effect of high-precision prediction can be achieved.

In addition, in this embodiment 3, an example is shown in which the scaling value of the Nth row (the Nth row from the top of the prediction block) in the prediction block when the intra-frame prediction unit 4, 34 performs horizontal prediction processing is 1/ ^2N+1 (=1/2, 1/4, 1/8, 1/16,...), and the scaling value of the Mth column (the Mth column from the left end of the prediction block) in the prediction block when the intra-frame prediction unit 4, 34 performs vertical prediction processing is 1/ ^2M+1 (=1/2, 1/4, 1/8, 1/16,...). However, this is only an example, and any value may be used as long as the scaling value of the row farther from the top of the prediction block is smaller when the intra-frame prediction unit 4, 34 performs horizontal prediction processing, and the scaling value of the column farther from the left end of the prediction block is smaller when the intra-frame prediction unit 4, 34 performs vertical prediction processing.

Furthermore, the present invention can realize free combination of the various embodiments, modification of arbitrary components of the various embodiments, or omission of arbitrary components of the various embodiments within the scope of the present invention.

(Industrial Applicability)

The present invention is applicable to a moving picture encoding device that needs to encode moving pictures efficiently, and is also applicable to a moving picture decoding device that needs to decode efficiently encoded moving pictures.

Claims (5)

1. An image decoding device, characterized in that, The system includes an intra-frame prediction unit. When the coding mode associated with the coding block is an intra-frame coding mode, this unit treats the coding block or sub-blocks formed by dividing the coding block as processing blocks, and performs intra-frame prediction processing on each processing block to generate a predicted image. The intra-frame prediction unit performs vertical direction prediction processing based on the intra-frame prediction parameters. When the processing block is smaller than a specified size, the brightness values of the pixels adjacent to the top of the processing block are added together with a value proportional to the change in brightness value in the vertical direction of the pixels adjacent to the left of the processing block. This summed value is used as the predicted value of the predicted image. When the processing block is larger than a specified size, the brightness value of the pixel adjacent to the top of the processing block is used as the predicted value of the predicted image. 2. An image encoding device, characterized in that, The system includes an intra-frame prediction unit. When the coding mode associated with the coding block is an intra-frame coding mode, this unit treats the coding block or sub-blocks formed by dividing the coding block as processing blocks, and performs intra-frame prediction processing on each processing block to generate a predicted image. The intra-frame prediction unit performs vertical direction prediction processing based on the intra-frame prediction parameters. When the processing block is smaller than a specified size, the brightness values of the pixels adjacent to the top of the processing block are added together with a value proportional to the change in brightness value in the vertical direction of the pixels adjacent to the left of the processing block. This summed value is used as the predicted value of the predicted image. When the processing block is larger than a specified size, the brightness value of the pixel adjacent to the top of the processing block is used as the predicted value of the predicted image. 3. An image decoding method, characterized in that, The system includes an intra-frame prediction processing step. In this step, when the coding mode associated with the coding block is an intra-frame coding mode, the intra-frame prediction unit takes the coding block or a sub-block formed by dividing the coding block as a processing block, and performs intra-frame prediction processing on each processing block to generate a predicted image. In the intra-frame prediction processing step, when the intra-frame prediction parameters represent vertical direction prediction processing, When the processing block is smaller than a specified size, the brightness values of the pixels adjacent to the top of the processing block are added together with a value proportional to the change in brightness value in the vertical direction of the pixels adjacent to the left of the processing block. This summed value is used as the predicted value of the predicted image. When the processing block is larger than a specified size, the brightness value of the pixel adjacent to the top of the processing block is used as the predicted value of the predicted image. 4. An image encoding method, characterized in that, The system includes an intra-frame prediction processing step. In this step, when the coding mode associated with the coding block is an intra-frame coding mode, the intra-frame prediction unit takes the coding block or a sub-block formed by dividing the coding block as a processing block, and performs intra-frame prediction processing on each processing block to generate a predicted image. In the intra-frame prediction processing step, when the intra-frame prediction parameters represent vertical direction prediction processing, When the processing block is smaller than a specified size, the brightness values of the pixels adjacent to the top of the processing block are added together with a value proportional to the change in brightness value in the vertical direction of the pixels adjacent to the left of the processing block. This summed value is used as the predicted value of the predicted image. When the processing block is larger than a specified size, the brightness value of the pixel adjacent to the top of the processing block is used as the predicted value of the predicted image. 5. A recording medium storing a computer-readable program for recording encoded data encoded by an image encoding device, characterized in that, The program is executed by the processing device to perform the following processing: An image decoding device uses coded blocks or sub-blocks formed by dividing coded blocks as processing blocks, and performs prediction processing on each processing block to generate a predicted image. The encoded data includes: an encoding mode associated with the encoded block; intra-frame prediction parameters indicating the intra-frame prediction mode used when performing intra-frame prediction processing for each block, when the encoding mode is an intra-frame coding mode; information specifying the block size; and compressed data generated by compressing the difference between the image of the block and the predicted image. When the encoding mode is an intra-frame encoding mode and the intra-frame prediction parameters represent vertical prediction processing. Regarding the predicted image, the image decoding device adds the brightness values of the pixels adjacent to the top of the block to a value proportional to the change in brightness value in the vertical direction of the pixels adjacent to the left of the block. This summed value is used as the predicted value of the predicted image. When the processing block is larger than a specified size, the brightness value of the pixels adjacent to the top of the processing block is used as the predicted value of the predicted image, thereby generating the predicted image.