HK1233402B - Image encoding device and image encoding method - Google Patents

Description

Image encoding device and image encoding method

This application is a divisional application of application number 201280028658.2, application date June 5, 2012, and invention name “Image encoding device, image decoding device, image encoding method and image decoding method”.

Technical Field

The present invention relates to an image coding apparatus and an image coding method for compressing and coding an image and transmitting the image, and an image decoding apparatus and an image decoding method for decoding an image from coded data transmitted by the image coding apparatus.

Background Art

In the past, in international standard image coding methods such as MPEG and ITU-TH.26x, after the input image frame was divided into macroblock units consisting of 16×16 pixel blocks and motion compensation prediction was implemented, the predicted differential signal was orthogonally transformed/quantized on a block basis to compress information.

However, if the compression rate becomes high, there is a problem that compression efficiency is impaired due to a decrease in the quality of the prediction reference image used when performing motion compensation prediction.

Therefore, in the MPEG-4 AVC/H.264 encoding method (see Non-Patent Document 1), an in-loop blocking filter process is performed to remove block distortion of a prediction reference image that occurs with quantization of orthogonal transform coefficients.

Here, FIG16 is a structural diagram showing the image encoding device disclosed in Non-Patent Document 1. In FIG16 , FIG16 is a structural diagram showing the image encoding device disclosed in Non-Patent Document 1.

In this image coding apparatus, when a coding target image signal is input to the block division unit 101, the block division unit 101 divides the image signal into macroblock units and outputs the macroblock-based image signals as divided image signals to the prediction unit 102.

Upon receiving the divided image signal from the block division unit 101 , the prediction unit 102 predicts the image signal of each color component in the macroblock within a frame or between frames, and calculates a prediction difference signal.

In particular, when motion compensation prediction is performed between frames, a motion vector is searched for each macroblock itself or each subblock obtained by further dividing the macroblock.

Then, using the motion vector, motion compensation prediction is performed on the reference image signal stored in the memory 107 to generate a motion compensation prediction image, and the difference between the prediction signal representing the motion compensation prediction image and the segmented image signal is obtained to calculate the prediction difference signal.

Furthermore, the prediction unit 102 outputs the prediction signal generation parameters determined when the prediction signal is obtained to the variable-length coding unit 108 .

Furthermore, the parameters for prediction signal generation include, for example, information such as a motion vector indicating the amount of motion between frames.

Upon receiving the prediction difference signal from the prediction unit 102 , the compression unit 103 performs DCT (Discrete Cosine Transform) processing on the prediction difference signal to remove signal correlation, and then performs quantization to obtain compressed data.

Upon receiving the compressed data from the compression unit 103 , the local decoding unit 104 performs inverse DCT processing on the compressed data by inverse quantizing the compressed data, thereby calculating a prediction difference signal corresponding to the prediction difference signal output from the prediction unit 102 .

Upon receiving the prediction difference signal from the local decoding unit 104 , the adder 105 adds the prediction difference signal and the prediction signal output from the prediction unit 102 to generate a local decoded image.

The loop filter 106 removes block distortion superimposed on the local decoded image signal representing the local decoded image generated by the adder 105 , and stores the local decoded image signal after distortion removal in the memory 107 as a reference image signal.

Upon receiving the compressed data from the compression unit 103 , the variable-length coding unit 108 performs entropy coding on the compressed data and outputs a bit stream as a result of the coding.

Furthermore, when outputting a bit stream, the variable-length coding unit 108 multiplexes the prediction signal generation parameters output from the prediction unit 102 into the bit stream and outputs the resultant.

Here, in the method disclosed in non-patent document 1, the loop filter 106 determines the smoothing intensity (filtering intensity) for the surrounding pixels of the DCT block boundary based on information such as the coarseness of quantization, the coding mode, and the degree of dispersion of the motion vector, and implements filtering processing on the local decoded image, thereby seeking to reduce the distortion (block distortion) occurring at the block boundary.

This improves the quality of the reference image signal and makes it possible to enhance the efficiency of motion compensation prediction in subsequent encoding.

Non-Patent Document 1: MPEG-4 AVC (ISO/IEC 14496-10)/ITU-TH.264 Standard

Summary of the Invention

Conventional image coding devices configured as described above have a problem where, when loop filter 106 removes block distortion, the filtering strength applied to the luma signal component of the block is determined based on the coding mode, etc., while the filtering strength applied to the chroma signal components of the block is derived from the filtering strength applied to the luma signal components. Consequently, the filtering strength applied to the chroma signal components is not always appropriate, limiting the effect of image quality improvement.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide an image coding apparatus and an image coding method that can improve the accuracy of removing block distortion and thereby improve the quality of the coded image.

Another object of the present invention is to provide an image decoding device and an image decoding method that can improve the accuracy of removing block distortion and thereby improve the quality of decoded images.

The image coding device according to the present invention comprises: a block division unit for dividing an input image into blocks serving as processing units when performing prediction processing; a coding mode determination unit for determining a coding mode for the blocks divided by the block division unit; a prediction image generation unit for performing prediction processing on the blocks divided by the block division unit while referring to a local decoded image of an already coded block in accordance with the coding mode determined by the coding mode determination unit, thereby generating a prediction image; a difference image generation unit for generating a difference image between the blocks divided by the block division unit and the prediction image generated by the prediction image generation unit; an image compression unit for compressing the difference image generated by the difference image generation unit and outputting compressed data of the difference image; and a local decoded image generation unit for generating a prediction image. a unit for decompressing a differential image compressed by an image compression unit and adding the decompressed differential image to a prediction image generated by a prediction image generation unit to generate a local decoded image; a distortion removal unit for performing filtering processing on the local decoded image generated by the local decoded image generation unit to remove block distortion of the local decoded image; and a coding unit for encoding compressed data output from the image compression unit and a coding mode determined by a coding mode determination unit to generate a bit stream of coded data multiplexed with the compressed data and the coding mode, wherein the distortion removal unit sets the strength of the filter for removing the block distortion according to the signal component in accordance with the coding mode determined by the coding mode determination unit when removing the block distortion of the local decoded image.

According to the present invention, the present invention is configured to include: a block division unit that divides an input image into blocks that serve as processing units when performing prediction processing; a coding mode determination unit that determines a coding mode for the blocks divided by the block division unit; a prediction image generation unit that performs prediction processing on the blocks divided by the block division unit while referring to a local decoded image of an already coded block in accordance with the coding mode determined by the coding mode determination unit, thereby generating a prediction image; a difference image generation unit that generates a difference image between the blocks divided by the block division unit and the prediction image generated by the prediction image generation unit; an image compression unit that compresses the difference image generated by the difference image generation unit and outputs compressed data of the difference image; and a local decoded image generation unit that generates a difference image compressed by the image compression unit. Decompression is performed, and the decompressed differential image and the predicted image generated by the predicted image generating unit are added to generate a local decoded image; a distortion removal unit performs filtering processing on the local decoded image generated by the local decoded image generating unit to remove block distortion of the local decoded image; and an encoding unit encodes the compressed data output from the image compression unit and the encoding mode determined by the encoding mode determining unit to generate a bit stream of encoded data that multiplexes the compressed data and the encoding mode. When removing block distortion from the local decoded image, the distortion removal unit sets the strength of the filter for removing block distortion according to the signal component in accordance with the encoding mode determined by the encoding mode determining unit, thereby having the effect of improving the accuracy of removing block distortion and thus improving the quality of the encoded image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a diagram showing the structure of an image encoding apparatus according to Embodiment 1 of the present invention.

FIG2 is a flowchart showing the processing contents of the image encoding apparatus according to the first embodiment of the present invention.

FIG3 is a diagram showing the structure of the image decoding apparatus according to Embodiment 1 of the present invention.

FIG4 is a flowchart showing the processing contents of the image decoding apparatus according to Embodiment 1 of the present invention.

FIG. 5 is an explanatory diagram showing a case where a coding block of a maximum size is hierarchically divided into a plurality of coding blocks.

FIG6(a) is a diagram showing the distribution of partitions after division, and FIG6(b) is an explanatory diagram showing, by a quadtree graph, the allocation of coding modes m( ^Bn ) to partitions after hierarchical division.

FIG. 7 is an explanatory diagram showing the positions of filter-applied pixels within a coding block.

FIG. 8 is a flowchart showing the processing contents of the loop filter unit 11 .

FIG9 is a flowchart illustrating a method for determining filter strength.

FIG. 10 is an explanatory diagram showing the relationship between edge positions and pixel positions.

FIG. 11 is an explanatory diagram showing a unit of filtering processing for a vertical edge.

FIG. 12 is an explanatory diagram showing a unit of filtering processing for a horizontal edge.

FIG. 13 is an explanatory diagram showing the correspondence relationship between Q (the qP value of brightness) and the parameters β and Tc.

FIG14 is an explanatory diagram showing a bit stream generated by the variable-length coding unit 13. In FIG14 , as shown in FIG14 , a bit stream is ...

FIG15 is an explanatory diagram showing an example in which the size of the coding block ^Bn is ^Ln = ^kMn .

FIG16 is a diagram showing the structure of the image encoding device disclosed in Non-Patent Document 1. In FIG.

Code Description

1: Coding control unit (coding mode determination unit); 2: Block division unit (block division unit); 3: Switch (prediction image generation unit); 4: Intra-frame prediction unit (prediction image generation unit); 5: Motion compensation prediction unit (prediction image generation unit); 6: Subtraction unit (differential image generation unit); 7: Transformation/quantization unit (image compression unit); 8: Inverse quantization/inverse transformation unit (local decoded image); 9: Addition unit (local decoded image); 10: Memory for intra-frame prediction; 11: Loop filter unit (distortion removal unit); 12: Motion compensation prediction frame memory; 13: Variable length coding unit (coding unit); 21: Variable length decoding Coding unit (decoding unit); 22: switching switch (prediction image generation unit); 23: intra-frame prediction unit (prediction image generation unit); 24: motion compensation unit (prediction image generation unit); 25: inverse quantization/inverse transform unit (differential image generation unit); 26: adding unit (decoded image generation unit); 27: memory for intra-frame prediction; 28: loop filter unit (distortion removal unit); 29: motion compensation prediction frame memory; 101: block division unit; 102: prediction unit; 103: compression unit; 104: local decoding unit; 105: adder; 106: loop filter; 107: memory; 108: variable length coding unit.

DETAILED DESCRIPTION

Hereinafter, in order to explain the present invention in more detail, embodiments of the present invention will be described with reference to the accompanying drawings.

Implementation method 1.

In this embodiment 1, there is described: an image encoding device that inputs each frame image of an image, performs compression processing based on orthogonal transformation and quantization on the predicted differential signal obtained by performing motion compensation prediction between adjacent frames, and then performs variable-length encoding to generate a bit stream; and an image decoding device that decodes the bit stream output from the image encoding device.

The image encoding apparatus of this first embodiment is characterized in that it divides the image signal into regions of various sizes to perform intra-frame/inter-frame adaptive encoding in response to local changes in the spatial and temporal directions of the image signal.

In general, image signals exhibit locally varying signal complexity across space and time. When viewed spatially, within a given image frame, there are patterns with uniform signal characteristics across relatively wide image regions, such as the sky and walls, but also patterns with complex textures within smaller image regions, such as figures and paintings with detailed textures.

When observing over time, the local changes in the time direction of the sky and walls are small, but the outlines of moving people and objects undergo rigid/non-rigid motion over time, so the changes over time are large.

In the encoding process, a predicted differential signal with low signal power and entropy is generated through temporal/spatial prediction, thereby reducing the overall amount of code. However, if the parameters used for prediction can be evenly applied to the largest possible image signal area, the amount of code for the parameter can be reduced.

On the other hand, if the same prediction parameters are applied to an image signal pattern that varies greatly in time and space, the prediction error increases, and thus the code amount of the prediction difference signal cannot be reduced.

Therefore, it is desirable to reduce the area of the prediction target for image signal patterns that vary greatly in time and space, thereby increasing the amount of parameter data used for prediction but reducing the power and entropy of the prediction difference signal.

In order to perform such encoding suitable for the general properties of image signals, the image encoding device of embodiment 1 divides the image signal region into layers starting from a predetermined maximum block size, and performs prediction processing and prediction difference encoding processing on each divided region.

The image encoding device of this embodiment 1 processes image signals of arbitrary color spaces such as YUV signals composed of a luminance signal and two color difference signals, and RGB signals output from a digital imaging element, as well as arbitrary image signals such as monochrome image signals and infrared image signals whose image frames are composed of horizontal/vertical two-dimensional columns of digital samples (pixels).

The grayscale of each pixel can be 8 bits, 10 bits, 12 bits, etc.

However, in the following description, unless otherwise specified, the input video signal is assumed to be a YUV signal, and is assumed to be a 4:2:0 format signal in which the two color difference components U and V are subsampled relative to the luminance component Y.

The unit of processed data corresponding to each image frame is called a "picture." In this first embodiment, a "picture" is described as a signal of a progressively scanned image frame. However, if the image signal is an interlaced signal, a "picture" may also be a field image signal that constitutes a video frame. In the following description, a group of spatially continuous coding blocks may be referred to as a "slice."

FIG1 is a diagram showing the structure of an image encoding apparatus according to Embodiment 1 of the present invention.

In Figure 1, the encoding control unit 1 implements the following processing: determines the maximum size of the coding block that serves as the processing unit when performing motion compensation prediction processing (inter-frame prediction processing) or intra-frame prediction processing (intra-frame prediction processing), and determines the upper limit of the number of layers when the maximum-sized coding block is hierarchically divided.

The coding control unit 1 also performs processing to select a coding mode suitable for each hierarchically divided coding block from one or more available coding modes (one or more intra-frame coding modes and one or more inter-frame coding modes). Furthermore, the coding control unit 1 constitutes a coding mode determination unit.

The block division unit 2 performs the following processing: when a video signal representing an input image is input, the input image represented by the video signal is divided into coding blocks of the maximum size determined by the coding control unit 1, and the coding blocks are divided hierarchically until the upper limit of the number of layers determined by the coding control unit 1 is reached. The block division unit 2 also constitutes a block division unit.

The switching switch 3 implements the following processing: if the coding mode selected by the coding control unit 1 is the intra-frame coding mode, the coding blocks divided by the block division unit 2 are output to the intra-frame prediction unit 4; if the coding mode selected by the coding control unit 1 is the inter-frame coding mode, the coding blocks divided by the block division unit 2 are output to the motion compensation prediction unit 5.

The intra-frame prediction unit 4 performs the following processing: if a coding block divided by the block division unit 2 is received from the switching switch 3, it performs intra-frame prediction processing on the coding block while referring to the local decoded image (reference image) of the coded block stored in the intra-frame prediction memory 10, and uses the intra-frame prediction parameters output from the encoding control unit 1 to generate a predicted image.

The motion compensation prediction unit 5 performs the following processing: if a coding block divided by the block division unit 2 is received from the switching switch 3, motion search is performed and a motion vector is calculated by comparing the coding block with the local decoded image (reference image) of the coded block stored in the motion compensation prediction frame memory 12, and the inter-frame prediction processing (motion compensation prediction processing) is performed on the coding block using the motion vector and the inter-frame prediction parameters output from the encoding control unit 1 to generate a predicted image.

Furthermore, the switch 3 , the intra prediction unit 4 , and the motion compensation prediction unit 5 constitute a predicted image generation unit.

The subtraction unit 6 generates a difference image (= coded block - predicted image) by subtracting the predicted image generated by the intra prediction unit 4 or the motion compensation prediction unit 5 from the coded block divided by the block division unit 2. The subtraction unit 6 constitutes a difference image generation unit.

The transform/quantization unit 7 performs a transform process (e.g., a DCT (discrete cosine transform) or an orthogonal transform such as a KL transform pre-designed based on a specific learning sequence) on the difference image generated by the subtraction unit 6 in units of the transform block size included in the predicted difference coding parameters output from the coding control unit 1. Furthermore, the transform coefficients of the difference image are quantized using the quantization parameters included in the predicted difference coding parameters, and the quantized transform coefficients are output as compressed data of the difference image. Furthermore, the transform/quantization unit 7 constitutes image compression means.

The inverse quantization/inverse transform unit 8 performs the following processing: using the quantization parameter included in the predicted differential coding parameter output from the encoding control unit 1, the compressed data output from the transform/quantization unit 7 is inverse quantized, and according to the transform block size unit included in the predicted differential coding parameter, the inverse transform processing (such as inverse DCT (inverse discrete cosine transform), inverse KL transform and other inverse transform processing) of the inversely quantized compressed data is performed, thereby outputting the compressed data after the inverse transform processing as a locally decoded predicted differential signal (data representing the decompressed differential image).

The adding unit 9 performs the following processing: by adding the local decoded prediction differential signal output from the inverse quantization/inverse transform unit 8 and the prediction signal representing the predicted image generated by the intra-frame prediction unit 4 or the motion compensation prediction unit 5, a local decoded image signal representing the local decoded image is generated.

Furthermore, the inverse quantization/inverse transformation unit 8 and the addition unit 9 constitute a local decoded image generation unit.

The intra prediction memory 10 is a recording medium such as RAM that stores the local decoded image represented by the local decoded image signal generated by the adder 9 as an image to be used in the next intra prediction process by the intra prediction unit 4 .

The loop filter unit 11 performs a process of removing distortion (block distortion) occurring at a block boundary by performing a filtering process (loop filtering process) on the local decoded image signal generated by the adding unit 9 .

When removing block distortion from a local decoded image, the loop filter unit 11 sets the intensity of the filter for removing block distortion for each signal component (luminance signal component, color difference signal component) according to the coding mode (intra-frame coding mode, inter-frame coding mode) selected by the coding control unit 1.

In addition, the loop filter unit 11 constitutes a distortion removal unit.

The motion-compensated prediction frame memory 12 is a recording medium such as RAM that stores the local decoded image filtered by the loop filter unit 11 as a reference image to be used in the next motion-compensated prediction process by the motion-compensated prediction unit 5 .

The variable-length coding unit 13 performs variable-length coding on the compressed data output from the transform/quantization unit 7, the coding mode and prediction difference coding parameters output from the coding control unit 1, and the intra-frame prediction parameters output from the intra-frame prediction unit 4 or the inter-frame prediction parameters (including motion vectors) output from the motion-compensated prediction unit 5, thereby generating a bitstream of coded data multiplexed with the compressed data, coding mode, prediction difference coding parameters, and intra-frame prediction parameters/inter-frame prediction parameters. The variable-length coding unit 13 constitutes a coding unit.

In FIG1 , an example is assumed in which the components of the image coding apparatus, namely, the coding control unit 1, the block division unit 2, the switch 3, the intra-frame prediction unit 4, the motion-compensated prediction unit 5, the subtraction unit 6, the transform/quantization unit 7, the inverse quantization/inverse transform unit 8, the addition unit 9, the loop filter unit 11, and the variable-length coding unit 13, are each configured by dedicated hardware (e.g., a semiconductor integrated circuit or a single-chip microcomputer having a CPU mounted thereon). However, if the image coding apparatus is configured by a computer, a program describing the processing contents of the coding control unit 1, the block division unit 2, the switch 3, the intra-frame prediction unit 4, the motion-compensated prediction unit 5, the subtraction unit 6, the transform/quantization unit 7, the inverse quantization/inverse transform unit 8, the addition unit 9, the loop filter unit 11, and the variable-length coding unit 13 may be stored in a memory of the computer, and the CPU of the computer may execute the program stored in the memory.

FIG2 is a flowchart showing the processing contents of the image encoding apparatus according to the first embodiment of the present invention.

FIG3 is a diagram showing the structure of the image decoding apparatus according to Embodiment 1 of the present invention.

In FIG3 , the variable length decoding unit 21 performs the following processing: variable length decoding is performed from the coded data multiplexed in the bit stream to obtain compressed data, coding mode, prediction difference coding parameters, intra-frame prediction parameters/inter-frame prediction parameters (including motion vectors) related to each hierarchically divided coding block, and the compressed data and prediction difference coding parameters are output to the inverse quantization/inverse transform unit 25. The coding mode and intra-frame prediction parameters/inter-frame prediction parameters are output to the switch 22, and the coding mode is output to the loop filter unit 28. The variable length decoding unit 21 constitutes a decoding unit.

The switching switch 22 implements the following processing: when the coding mode related to the coding block output from the variable length decoding unit 21 is the intra-frame coding mode, the intra-frame prediction parameters output from the variable length decoding unit 21 are output to the intra-frame prediction unit 23; when the coding mode is the inter-frame coding mode, the inter-frame prediction parameters output from the variable length decoding unit 21 are output to the motion compensation unit 24.

The intra-frame prediction unit 23 performs the following processing: while referring to the decoded image (reference image) of the decoded block stored in the intra-frame prediction memory 27, it uses the intra-frame prediction parameters output from the switching switch 22 to perform intra-frame prediction processing on the coded block to generate a predicted image.

The motion compensation unit 24 performs the following processing: using the motion vector included in the inter-frame prediction parameters output from the switching switch 22 and the decoded image (reference image) of the decoded block stored in the motion compensation prediction frame memory 29, it performs inter-frame prediction processing on the coded block to generate a predicted image.

The switch 22 , the intra prediction unit 23 , and the motion compensation unit 24 constitute a predicted image generation unit.

The inverse quantization/inverse transform unit 25 performs the following processing: using the quantization parameter included in the prediction difference coding parameter output from the variable length decoding unit 21, it inversely quantizes the compressed data output from the variable length decoding unit 21, and performs inverse transform processing (e.g., inverse DCT (inverse discrete cosine transform) or inverse KL transform) on the inversely quantized compressed data in units of transform block size included in the prediction difference coding parameter, thereby outputting the inversely transformed compressed data as a decoded prediction difference signal (a signal representing the difference image before compression). The inverse quantization/inverse transform unit 25 also constitutes a difference image generation unit.

The addition unit 26 generates a decoded image signal representing a decoded image by adding the decoded prediction difference signal output from the inverse quantization/inverse transform unit 25 and a prediction signal representing a predicted image generated by the intra-frame prediction unit 23 or the motion compensation unit 24. The addition unit 26 constitutes a decoded image generation unit.

The intra prediction memory 27 is a recording medium such as RAM that stores the decoded image represented by the decoded image signal generated by the adder 26 as an image to be used in the next intra prediction process by the intra prediction unit 23 .

The loop filter unit 28 performs a process of removing distortion (block distortion) occurring at a block boundary by performing a filtering process (loop filtering process) on the decoded image signal generated by the adding unit 26 .

When removing block distortion from a decoded image, the loop filter unit 28 sets the intensity of the filter for removing block distortion according to the signal component (luminance signal component, color difference signal component) in accordance with the coding mode (intra-frame coding mode, inter-frame coding mode) output from the variable length decoding unit 21.

In addition, the loop filter unit 28 constitutes a distortion removal unit.

The motion compensation prediction frame memory 29 is a recording medium such as RAM that stores the decoded image filtered by the loop filter unit 28 as a reference image to be used in the next motion compensation prediction process by the motion compensation unit 24 .

In FIG3 , an example is illustrated in which the variable length decoding unit 21, the switch 22, the intra-frame prediction unit 23, the motion compensation unit 24, the inverse quantization/inverse transform unit 25, the addition unit 26, and the loop filter unit 28, which are components of the image decoding device, are each configured by dedicated hardware (e.g., a semiconductor integrated circuit or a single-chip microcomputer having a CPU mounted thereon). However, if the image decoding device is configured by a computer, a program describing the processing contents of the variable length decoding unit 21, the switch 22, the intra-frame prediction unit 23, the motion compensation unit 24, the inverse quantization/inverse transform unit 25, the addition unit 26, and the loop filter unit 28 may be stored in a memory of the computer, and the CPU of the computer may execute the program stored in the memory.

FIG4 is a flowchart showing the processing contents of the image decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be explained.

First, the processing contents of the image encoding apparatus in FIG1 will be described.

First, the encoding control unit 1 determines the maximum size of the coding block that serves as the processing unit when performing motion compensation prediction processing (inter-frame prediction processing) or intra-frame prediction processing (intra-frame prediction processing), and determines the upper limit number of layers when the maximum-sized coding block is divided into layers (step ST1 of Figure 2).

As a method for determining the maximum size of a coding block, for example, a method of determining a size corresponding to the resolution of an input image for all pictures may be considered.

Another possible method is to quantify the difference in complexity of local motion in the input image as a parameter, and determine the maximum size to be a small value in a picture with intense motion and a large value in a picture with less motion.

Regarding the upper limit of the number of layers, for example, a method of setting the number of layers may be considered such that the more intense the motion of the input image is, the deeper the number of layers is to detect finer motion, and the lower the number of layers is when the motion of the input image is small.

Furthermore, the encoding control unit 1 selects an encoding mode suitable for each hierarchically divided encoding block from one or more available encoding modes (M types of intra-frame encoding modes and N types of inter-frame encoding modes) (step ST2).

The method for selecting the coding mode performed by the coding control unit 1 is a well-known technology, so detailed description is omitted, but for example, there is the following method: using any available coding mode, implementing coding processing on the coding block to verify the coding efficiency, and selecting the coding mode with the best coding efficiency from the multiple available coding modes.

In the block division unit 2, if an image signal representing an input image is input, the input image represented by the image signal is divided into coding blocks of the maximum size determined by the coding control unit 1, and the coding blocks are divided hierarchically until the upper limit of the number of layers determined by the coding control unit 1 is reached (step ST3).

Here, FIG5 is an explanatory diagram showing a case where a coding block of a maximum size is hierarchically divided into a plurality of coding blocks.

In the example of FIG5 , the coding block of the largest size is the coding block B ⁰ of the 0th layer, which has a size of (L ⁰ , M ⁰ ) in terms of the luminance component.

In the example of FIG. 5 , the maximum-sized coding block B ⁰ is used as a starting point, and hierarchical division is performed to a predetermined depth determined separately using a quadtree structure, thereby obtaining coding blocks B ⁿ .

At depth n, a coding block ^Bn is an image region of size ( ^Ln , ^Mn ).

^Ln and ^Mn may be the same or different, but the example in FIG5 shows a case where ^Ln = ^Mn .

Hereinafter, the size of the coding block ^Bn is defined as the size ( ^Ln , ^Mn ) in the luma component of the coding block ^Bn .

Since the block dividing unit 2 performs quadtree division, (Ln ⁺¹ , Mn ⁺¹ ) = ( ^Ln /2, ^Mn /2) always holds.

However, in a color video signal (4:4:4 format) such as an RGB signal where all color components have the same number of samples, the size of all color components is ( ^Ln , ^Mn ). However, when processing the 4:2:0 format, the size of the coding block of the corresponding color difference component is ( ^Ln /2, ^Mn /2).

Hereinafter, the coding mode selectable in the coding block ^Bn of the nth layer will be expressed as m( ^Bn ).

In the case of a color video signal composed of multiple color components, the coding mode m( ^Bn ) may be configured to use a separate mode for each color component. However, unless otherwise specified, the coding mode m(Bn) is described below as referring to the coding mode for the luminance component of a coding block in the YUV signal 4:2:0 format.

Among the coding modes m( ^Bn ), there are one or more intra-frame coding modes (collectively referred to as "INTRA") and one or more inter-frame coding modes (collectively referred to as "INTER"). As described above, the coding control unit 1 selects the coding mode with the best coding efficiency for the coding block ^Bn from all coding modes available for the picture or a subset thereof.

As shown in FIG5 , the coding block ^Bn is further divided into one or more prediction processing units (partitions).

Hereinafter, the partitions belonging to the coding block ^Bn are referred to as _Pin ⁽ i: partition number in the nth layer).

How the partition _P in the coding block B ⁿ is ^divided is included as information in the coding mode m(B ⁿ ).

Regarding all partitions P _i ⁿ , prediction processing is performed according to the coding mode m(B ⁿ ), but individual prediction parameters can be selected for each partition P _i ⁿ .

The encoding control unit 1 generates a block division state as shown in FIG. 6 , for example, for the maximum-sized encoding block, and determines the encoding block B ⁿ .

The hatched portion of FIG6(a) shows the distribution of partitions after division, and FIG6(b) shows the allocation of coding modes m( ^Bn ) to partitions after hierarchical division using a quadtree diagram.

In FIG6(b), the nodes surrounded by □ represent nodes (coding blocks ^Bn ) to which coding mode m( ^Bn ) is assigned.

When the encoding control unit 1 selects the optimal encoding mode m( ^Bn ) for each partition _P in the ^encoding block ^Bn , if the encoding mode m( ^Bn ) is an intra-frame encoding mode (step ST4), the switching switch 3 outputs the ^partition _P in the encoding block ^Bn divided by the block division unit 2 to the intra-frame prediction unit 4.

On the other hand, if the coding mode m(B ⁿ ) is the inter-frame coding mode (step ST4 ), the partitions _P ⁱⁿ of the coding block B ⁿ divided by the block dividing unit 2 are output to the motion compensation prediction unit 5 .

Upon receiving ^the partition _Pin of the coding block ^Bn from the switch 3, the intra-frame prediction unit 4 performs intra-frame prediction processing on the partition Pin of the coding block ^Bn using the intra-frame prediction parameters corresponding to the coding mode m( ^Bn ) selected by the coding control unit 1, while referring to the local ^decoded image of the _coded block stored in the intra-frame prediction memory 10, thereby generating ^an intra-frame prediction image _Pin (step ST5).

After generating the intra-frame prediction image _Pin ^, the intra-frame prediction unit 4 outputs the intra-frame prediction image _Pin to the subtraction unit ⁶ and the addition unit 9. However, in order to enable the image decoding apparatus in FIG. 3 to also generate the same intra-frame prediction ^image _Pin , the intra-frame prediction parameters are output to the variable-length coding unit 13.

The intra prediction process performed by the intra prediction unit 4 complies with, for example, an algorithm defined in the AVC/H.264 standard (ISO/IEC 14496-10), but is not limited to this algorithm.

Upon receiving ^{the partition Pin} _of the coding block ^Bn from the switching switch 3, the motion-compensated prediction unit 5 performs a motion ^search and calculates a motion vector by comparing the partition _Pin of the coding block ^Bn with the local decoded image of the coded block stored in the motion-compensated prediction frame memory 12. Then, the motion vector and the inter-frame prediction parameters output from the encoding control unit 1 are used to perform inter-frame prediction processing on the coding block, thereby generating ^an inter-frame prediction image _Pin (step ST6).

After the motion-compensated prediction unit 5 generates ^the inter-frame prediction image _Pin , it outputs the inter-frame prediction image _Pin to the subtraction unit 6 and the addition unit 9. However, in order to enable the image decoding apparatus in FIG. 3 to also generate the same inter-frame prediction image _Pin ^, the inter-frame prediction parameters are output to the variable ^- length coding unit 13.

In addition, the inter-frame prediction parameters include the following information.

(1) Mode information describing the partitioning status of ^the partition _P in the coding block ^Bn

(2) Motion vector of ^partition _Pin

(3) When a plurality of local decoded images (reference images) are stored in the motion compensation prediction frame memory 12, reference image index information indicating which reference image is used to perform inter-frame prediction processing

(4) Index information indicating which motion vector prediction value is selected to be used when there are multiple motion vector prediction value candidates.

(5) In the case of multiple motion compensation interpolation filters, index information indicating which filter to use

(6) When the motion ^vector of the partition _Pin can represent multiple pixel precisions (half pixel, 1/4 pixel, 1/8 pixel, etc.), the selection information indicating which pixel precision to use

If the intra-frame prediction unit 4 or the motion-compensated prediction unit 5 generates a predicted image ( ^intra -frame predicted image _Pin , inter-frame predicted ^image _Pin ), the subtraction unit 6 subtracts the predicted image (intra-frame predicted image _Pin , inter-frame predicted image _Pin ) generated by the intra-frame prediction unit 4 or the motion-compensated prediction unit 5 from the ^{partition Pin} of the coding block ^Bn divided by the block division _unit ² , thereby generating a difference image, and outputs a predicted difference ^signal _ein representing the difference image to the transform/quantization unit 7 (step ST7).

Upon receiving the predicted differential signal _e ^indicative of the differential image from the subtraction unit 6, the transform/quantization unit 7 performs a transform process (e.g., a DCT (discrete cosine transform), an orthogonal transform such as a KL transform pre-designed based on a specific learning series) on the differential image in units of a transform block size included in the predicted differential coding parameters output from the encoding control unit 1, and quantizes the transform coefficients of the differential image using the quantization parameters included in the predicted differential coding parameters, thereby outputting the quantized transform coefficients as compressed data of the differential image to the inverse quantization/inverse transform unit 8 and the variable-length coding unit 13 (step ST8).

If the inverse quantization/inverse transform unit 8 receives the compressed data of the differential image from the transform/quantization unit 7, it uses the quantization parameter included in the predicted differential coding parameter output from the encoding control unit 1 to inverse quantize the compressed data of the differential image, and performs inverse transform processing (such as inverse DCT (inverse discrete cosine transform), inverse KL transform, and other inverse transform processing) on the inversely quantized compressed data according to the transform block size unit included in the predicted differential coding parameter, thereby outputting the compressed data after the inverse transform processing as a local decoding predicted differential signal to the addition unit 9 (step ST9).

Upon receiving the local decoded prediction difference signal from the inverse quantization/inverse transform unit 8, the adder 9 adds the local decoded prediction difference signal to a prediction signal representing a prediction image (intra-frame prediction image _Pin , inter-frame prediction image _Pin ) generated by the intra-frame prediction unit ⁴ or the motion compensation prediction unit ⁵ , thereby generating a local decoded partition image or a local decoded block image as a collection thereof, i.e., a local decoded image (step ST10).

When a local decoded image is generated in the adder 9 , a local decoded image signal representing the local decoded image is stored in the intra-frame prediction memory 10 , and the local decoded image signal is output to the loop filter 11 .

Upon receiving the local decoded image signal from the adding unit 9 , the loop filter unit 11 performs filtering processing on the local decoded image signal to remove distortion (block distortion) occurring at block boundaries (step ST11 ).

The details of the processing content in the loop filter unit 11 will be described later. However, when removing block distortion of the local decoded image, the intensity of the filter for removing block distortion is set according to the signal component (luminance signal component, color difference signal component) in accordance with the coding mode (intra-frame coding mode, inter-frame coding mode) selected by the coding control unit 1.

In addition, the filtering processing of the loop filter unit 11 can be performed according to the maximum coding block or each coding block unit of the local decoded image signal output from the addition unit 9, or it can be performed concentratedly on one picture after the local decoded image signal equivalent to the macroblock of one picture is output.

The processes of steps ST4 to ST10 are repeatedly performed until the processes for all the hierarchically divided coding blocks ^Bn are completed. When the processes for all the coding blocks ^Bn are completed, the process proceeds to step ST13 (step ST12).

The variable length coding unit 13 performs entropy coding on the compressed data output from the transform/quantization unit 7, the coding mode (including information indicating the segmentation status of the coding block) and the predicted differential coding parameters output from the coding control unit 1, and the intra-frame prediction parameters output from the intra-frame prediction unit 4 or the inter-frame prediction parameters (including motion vectors) output from the motion compensation prediction unit 5.

The variable-length coding unit 13 multiplexes the compressed data, which is the encoding result of the entropy encoding, the encoding mode, the prediction difference encoding parameter, and the encoded data of the intra-frame prediction parameter/inter-frame prediction parameter to generate a bit stream (step ST13).

The filtering process performed by the loop filter unit 11 will be described in detail below.

The loop filter unit 11 is a nonlinear smoothing filter that reduces block noise generated at the boundary between the partition, which is the prediction processing unit, and the transform block.

FIG. 7 is an explanatory diagram showing the positions of filter-applied pixels within a coding block.

In FIG. 7 , vertical edges and horizontal edges, and positions overlapping with block boundaries of partitions or transform blocks are targets of filtering.

In Figure 7, a KxK pixel grid is used to represent vertical and horizontal edges. The value of K can be fixed or set based on the maximum size of a coding block, the maximum size of a partition/transform block, etc.

FIG. 8 is a flowchart showing the processing contents of the loop filter unit 11 .

The filtering process of the loop filter unit 11 is performed for each coding block.

First, the loop filter unit 11 determines whether a vertical edge and a horizontal edge coincide with a boundary between partitions or transform blocks (step ST41 ).

When there is a vertical edge or a horizontal edge that coincides with a boundary between partitions or transform blocks, the loop filter unit 11 determines the filter strength of the portion that coincides with the boundary (steps ST42 and ST43). The method of determining the filter strength will be described later.

Once the filter strength is determined in the loop filter unit 11, filtering is performed while changing the final filter strength based on the filter strength determination result and the actual change in the pixel value to be filtered (steps ST44 and ST45). The filtering method will be described later.

The loop filter unit 11 repeatedly performs the processing of steps ST41 to 45 until the processing is completed for all coding blocks in the picture (step ST46 ).

Furthermore, identification information indicating whether or not to perform the loop filtering process on all coding blocks in a slice is multiplexed in the slice header, and the image encoding device is configured to determine the value of the identification information according to the situation and transmit the result to the image decoding device.

Next, the process of determining the filter strength of the loop filter unit 11 will be described.

FIG9 is a flowchart illustrating a method for determining filter strength.

The loop filter unit 11 obtains the filter strength bS for all pixels adjacent to vertical edges and horizontal edges according to the following conditions (step ST51 ).

In the following description, pixels near an edge are represented by symbols p _i (i=0, 1, 2, 3) and q _j (j=0, 1, 2, 3). However, the relationship between the edge position and the pixel position is defined as shown in FIG10 .

The coding block includes a block of a luminance signal component and a block of a color difference signal component. However, the loop filter unit 11 determines the filter strength according to the following conditions.

(1) Whether the coding mode of the coding block to be filtered is intra-frame coding mode or inter-frame coding mode

(2) Is the signal component to be filtered a luminance signal component or a color difference signal component?

(3) Whether the transform block including the pixel to be filtered contains non-zero transform coefficients

(4) Status of motion parameters in a partition including pixels to be filtered

The loop filter unit 11 determines the filter strength through the following procedure.

(Process 1)

If the coding mode is "intra coding mode" and the coding block includes p ₀ or q ₀ , the decision is as follows:

The signal component to be processed is the luminance signal component → bS = 2

The signal component to be processed is the color difference signal component → bS=4.

(Process 2)

If the conditions in step 1 are not met,

When the coding mode of the coding block including p ₀ or the coding block including q ₀ is "intra coding mode", the decision is as follows:

The signal component to be processed is the luminance signal component → bS = 1

The signal component to be processed is the color difference signal component → bS=3.

(Process 3)

If the conditions in steps 1 and 2 are not met,

In the case where _p0 or _q0 belongs to a transform block with non-zero orthogonal transform coefficients, the decision is as follows:

The signal component to be processed is the luminance signal component → bS = 2

The signal component to be processed is the color difference signal component → bS=2.

(Process 4)

If the conditions in steps 1 to 3 are not met, but any of the following conditions are met, the decision is as follows:

The signal component to be processed is the luminance signal component → bS = 1

The signal component to be processed is the color difference signal component → bS=1.

[condition]

The partition including p ₀ and the partition including q ₀ have different reference pictures or different numbers of motion vectors

In the partition including p ₀ and the partition including q ₀ , one motion vector is used, and the absolute value of the difference between the horizontal component or vertical component of the two motion vectors is greater than 4 at 1/4 pixel accuracy.

In each of the partitions including _p0 and _q0 , two motion vectors are used, and in at least one pair of motion vectors referring to the same reference picture (a motion vector in _p0 and a motion vector in _q0 ), the absolute value of the difference between the horizontal or vertical components of the two motion vectors is 4 or more at 1/4 pixel accuracy.

(Process 5)

If the conditions in steps 1 to 4 are not met (even if the edges other than the partition/transform block boundaries meet these conditions), the decision is as follows:

The signal component to be processed is the luminance signal component → bS = 0

The signal component to be processed is the color difference signal component → bS=0.

When the coding mode of the coding block of the processing object is the intra-frame coding mode, compared with the case of inter-frame prediction using prediction between frames, the prediction residual power of the luminance signal and the color difference signal are large, and the probability that the distribution of the quantized transform coefficients is significantly different for each signal component becomes higher.

The degree of block distortion is affected by how much effective transform coefficients are subjectively lost due to quantization. Therefore, in intra-frame coding in particular, it is preferable to be able to adjust the value of the filter strength as a measure of the degree of block distortion using luminance and chrominance.

In the previous loop filter 106 (refer to Figure 16), the filtering strength of the color difference signal component is always set to the same value as the filtering strength of the luminance signal component, but in this embodiment 1, the filtering strength is set according to the signal component (luminance signal component, color difference signal component) according to the conditions, so that the filtering strength that helps improve the image quality more than before is obtained.

In the loop filter unit 11 , when the filter strength is determined, filtering processing is performed in the order of vertical edges and horizontal edges based on the result of the filter strength determination.

First, the loop filter unit 11 obtains the maximum value of the filter strength bS for every K lines for vertical edges, and defines the maximum value as bSVer.

The loop filter unit 11 performs filtering processing on the K rows of pixels near the edge based on the maximum value bSVer.

Figure 11 illustrates the unit of filtering for vertical edges. K×K pixels centered on a vertical edge have the same bSVer. The final filter strength bS applied to each pixel is determined by the maximum bSVer and the change in pixel value at each pixel position.

Figure 12 shows the units of filtering processing for horizontal edges, which are the same as for vertical edges, except that the direction of processing changes from vertical to horizontal.

The following describes the filtering process performed on the processing target pixel.

The loop filter unit 11 performs filtering on the vertical edge direction of the luminance signal component according to the following procedure. The filtering on the horizontal edge direction of the luminance signal component is performed in the same manner, but instead of the maximum value bSVer, the maximum value bSHor of the filter strength bS for each K lines of horizontal edges is used.

[a] When bSVer = 0

No filtering is performed.

[b]bSVer≤2

(1) In FIG13 , the parameters β and Tc are determined when Q=(qP value of brightness).

FIG. 13 is an explanatory diagram showing the correspondence relationship between Q (the qP value of brightness) and the parameters β and Tc.

(2) Find d＝|p ₂ -2*p ₁ +p ₀ |+|q ₂ -2*q ₁ +q ₀ |+|p ₂ -2*p ₁ +p ₀ |+|q ₂ -2*q ₁ +q ₀ |,

Perform filtering operations according to the following conditions.

When d is smaller than (β>>2), |p ₃ - _{p 0} |+|q ₀ - _{q 3} | is smaller than (β>>2), and |p ₀ - q ₀ | is smaller than ((5*t _c +1))>>1),

p ₀ '＝Clipl _Y ((p ₂ +2*p ₁ +2*p ₀ +2*q ₀ +q ₁ +4)>>3)

p ₁ '＝Clipl _Y ((p ₂ +p ₁ +p ₀ +q ₀ +2)>>2)

p ₂ '＝Clipl _Y ((2*p ₃ +3*p ₂ +p ₁ +p ₀ +q ₀ +4)>>3)

q ₀ '＝C1ipl _Y ((p ₁ +2*p ₀ +2*q ₀ +2*q ₁ +q ₂ +4)>>3)

q ₁ '＝Clip _Y ((p ₀ +q ₀ +q ₁ +q ₂ +2)>>2)

q ₂ '＝Clipl _Y ((p ₀ +q ₀ +q ₁ +3*q ₂ +2*q ₃ +4)>>3)

In other cases,

Δ＝Clip3(-t _C , t _C , (13*(q ₀ -p ₀ )+4*(q ₁ -p ₁ )-5*(q ₂ -p ₀ )+16)>>5)

p ₀ '＝Clipl _Y (p ₀ +Δ)

q ₀ '＝Clipl _Y (q ₀ -Δ)

p ₁ '＝Clipl _Y (p ₁ +Δ/2)

q ₁ '＝Clipl _Y (q ₁ -Δ/2)

(3) The obtained p ₀ ' to p ₂ ' and q ₀ ' to q ₂ ' are replaced with the pixel values of p ₀ to p ₂ and q ₀ to q ₂ , and the replaced image is output as a decoded image to the motion compensation prediction frame memory 12 at the subsequent stage.

[c] bSVer＞2

13 , the parameter β is obtained when Q = (the qP value of luminance) and the parameter Tc is obtained when Q = (the qP value of luminance + 4). The subsequent processing is the same as that in the case of bSVer≤2.

The loop filter unit 11 performs filtering processing on the vertical edges of the color difference signal components according to the following procedure. The filtering processing on the horizontal edges of the color difference signal components is the same, but the maximum value bSHor is used instead of the maximum value bSVer. bSVer and bSVer are values calculated using the luminance at the same position.

[a] bSVer＞2

(1) Perform the following filtering operation.

Δ＝Clip3(-t _C , t _C , ((((q ₀ -p ₀ )＜＜2)+p ₁ -q ₁ +4)＞＞3))

p ₀ '＝Clipl _C (p ₀ +Δ)

q ₀ '＝Clipl _C (q ₀ -Δ)

(2) The obtained p ₀ ′ and q ₀ ′ are replaced with the pixel values of p ₀ and q ₀ , and the replaced image is output as a decoded image to the motion compensation prediction frame memory 12 at the subsequent stage.

[b]bSVer≤2

No filtering is performed.

The loop filter unit 11 of the image encoding device and the loop filter unit 28 of the image decoding device perform common processing, but the loop filter unit 11 of the image encoding device may be configured to set a control parameter for determining the filter strength.

For example, identification information indicating whether or not the value of the filter strength bS is signaled at a slice level may be multiplexed, thereby enabling the set value of the filter strength bS to be changed for each slice.

Regarding the change at this time, only the luminance signal component may be changed while the color difference signal component is fixed, or conversely, the luminance signal component may be fixed while only the color difference signal component is changed.

The setting value may be expressed as a signal, either as the value itself or as an offset from the default value of the filter strength bS. Alternatively, the filter strength of the color difference signal component may be expressed as an offset from the filter strength of the luminance signal component.

In particular, when the coding mode is the intra-frame coding mode, the setting value of the filter strength bS can be signaled as the value itself or an offset value. For example, the loop filter unit 11 can be configured to determine the filter strength according to the following procedure.

For coding blocks encoded in intra-frame coding mode, the filter strength value for the luma component is set to bSL, and the filter strength value for the chroma component is set to bSC. bSL and bSC are multiplexed into the bitstream as syntax information such as picture-level headers and slice headers, so that they can be shared between the encoding device and the decoding device. In this case,

(Process 1)

If the coding mode of the coding block including po or the coding block including qo is "intra coding mode" and the block is located at the edge of the coding block boundary, the following is determined:

The signal component to be processed is the luminance signal component → bS = max(4-bSL, 0)

The signal component to be processed is the color difference signal component → bS = max(4-bSC, 0)

Here, max(A, B) is a function that outputs the larger value of A and B.

(Process 2)

If the conditions in step 1 are not met,

When the coding mode of the coding block including po or the coding block including qo is "intra coding mode", the decision is as follows:

The signal component to be processed is the luminance signal component → bS = max(3-bSL, 0)

The signal component to be processed is the color difference signal component → bS = max(3 - bSC, 0).

When the coding mode is the intra-frame coding mode, there are the following situations: the situation where motion prediction cannot function effectively during the compression process and intra-frame coding is performed as a last resort; and the situation where intra-frame coding is performed periodically and intentionally from the perspective of error resistance and random access.

When intra-frame coding is performed as a last resort, distortion will overlap in accordance with the difficulty of coding. In contrast, when intra-frame coding is performed periodically and intentionally, since intra-frame coding is not used directly in relation to the difficulty of coding, the way in which distortion occurs in each block will differ.

Conventional loop filters do not have a means of determining the above situation and controlling the filtering strength.

Since periodic intra-frame interpolation is performed in units of slices or pictures, by controlling the filter strength in accordance with the application in these units, block distortion can be suppressed more effectively.

Furthermore, conversely, a configuration may be adopted in which the setting value of the filter strength bS when the coding mode is the inter-frame coding mode is signaled.

Here, FIG14 is an explanatory diagram showing a bit stream generated by the variable-length coding unit 13. In FIG14 , as shown in FIG14 , a bit stream generated by the variable-length coding unit 13 is generated.

The example of FIG. 14 shows a case where slice coded data is composed of a slice header and maximum coded block coded data corresponding to the number of blocks in the slice following the slice header.

Each maximum coding block's coded data includes a coding mode. Although not shown, the coded data for each maximum coding block includes prediction parameters such as motion vectors for each partition, prediction differential coding parameters such as transform block size, and prediction differential coding data (quantized transform coefficients).

The slice header includes: a loop filter ON/OFF flag, which is identification information of whether to perform loop filtering on all coding blocks in the slice; a filter strength information multiplexing flag, which is a flag indicating whether the set value of the filter strength bS is signaled; and filter strength information, which is multiplexed when the filter strength information multiplexing flag is "1".

The filter strength information multiplexing flag and the filter strength information may be configured to be multiplexed in a header information area defined in units such as pictures, sequences, and GOPs (Group of Pictures).

Next, the processing contents of the image decoding apparatus in FIG3 will be described.

If the variable length decoding unit 21 inputs the bit stream output from the image encoding device of Figure 1, it implements variable length decoding processing on the bit stream (step ST21 of Figure 4) to decode the information of the picture size (horizontal pixel number/vertical line number) specified according to the sequence unit or picture unit composed of more than one frame of pictures.

The variable length decoding unit 21 determines the maximum size of the coding block that serves as the processing unit when performing motion compensation prediction processing (inter-frame prediction processing) or intra-frame prediction processing (intra-frame prediction processing) in the same process as the encoding control unit 1 in Figure 1, and determines the upper limit number of layers when the maximum-sized coding block is hierarchically divided (step ST22).

For example, when the maximum size of a coding block is determined according to the resolution of an input image in an image coding apparatus, the maximum size of the coding block is determined according to the previously decoded frame size.

Furthermore, when information indicating the maximum size of a coding block and the upper limit of the number of layers is multiplexed in the bitstream, the information decoded from the bitstream is referenced.

When the bitstream structure is the structure shown in FIG. 14 , the variable length decoding unit 21 decodes the loop filter ON/OFF flag from the slice header before decoding the maximum coding block level.

The coding mode m(B ⁰ ) of the maximum-sized coding block B ⁰ multiplexed on the bitstream includes information indicating the division state of the maximum-sized coding block B ^0. Therefore, the variable-length decoding unit 21 decodes the coding mode m(B ⁰ ) of the maximum-sized coding block B ⁰ multiplexed on the bitstream and determines the hierarchically divided coding blocks B ⁿ (step ST23).

In the variable length decoding unit 21, once each coding block ^Bn is determined, the coding mode m( ^Bn ) of the coding block ^Bn is decoded, and the partition _{P in} belonging to the coding block ^Bn is determined based on the information of the partition ^P belonging to the coding mode m( ^{Bn )} .

In the variable length decoding unit 21, once ^the partition _Pin belonging to the coding block ^Bn is determined, the compressed data, coding mode, prediction difference coding parameters, intra-frame prediction parameters/inter-frame prediction parameters (including motion vectors) are decoded ^for each partition _Pin (step ST24).

When the coding mode m( ^Bn ) of the partition _Pin of the coding ^{block Bn} output from the variable length decoding unit 21 is the intra coding mode (step ST25), the switch 22 outputs the intra prediction parameters output from the variable length decoding unit 21 to the intra prediction unit 23.

On the other hand, when the coding mode m(B ⁿ ) of ^the partition _Pin is the inter coding mode (step ST25 ), the inter prediction parameters output from the variable length decoding unit 21 are output to the motion compensation prediction unit 24 .

Upon receiving the intra-frame prediction parameters from the switch 22, the intra-frame prediction unit 23 performs intra-frame prediction processing on the partition ^Pin of the coding block ^Bn using the intra-frame prediction parameters while referring to the decoded image (reference image) of the decoded block stored in the intra-frame prediction memory ₂₇ , thereby generating an intra-frame prediction image _Pin ⁽ step ST26).

Upon receiving the inter-frame prediction parameters output from the switching switch 22, the motion compensation unit 24 performs inter-frame prediction processing on the coded block using the motion vector included in the inter-frame prediction parameters and the decoded image (reference image) of the decoded block stored in the motion compensation prediction frame memory 29, thereby generating ^an intra-frame prediction image _Pin (step ST27).

The inverse quantization/inverse transformation unit 25 uses the quantization parameter included in the predicted differential coding parameter output from the variable length decoding unit 21 to inverse quantize the compressed data output from the variable length decoding unit 21, and performs inverse transformation processing (such as inverse DCT (inverse discrete cosine transform), inverse KL transform, and other inverse transformation processing) on the inversely quantized compressed data according to the transformation block size unit included in the predicted differential coding parameter, thereby outputting the compressed data after the inverse transformation processing as a decoded predicted differential signal (a signal representing the differential image before compression) to the addition unit 26 (step ST28).

If the adding unit 26 receives the decoded prediction differential signal from the inverse quantization/inverse transform unit 25, it adds the decoded prediction differential signal and the prediction signal representing the predicted image generated by the intra-frame prediction unit 23 or the motion compensation unit 24 to generate a decoded image, saves the decoded image signal representing the decoded image into the intra-frame prediction memory 27, and outputs the decoded image signal to the loop filter unit 28 (step ST29).

The processes of steps ST23 to ST29 are repeatedly performed until the processes for all the hierarchically divided coding blocks ^Bn are completed (step ST30).

If the loop filter unit 28 receives the decoded image signal from the addition unit 26, it performs filtering processing on the decoded image signal to remove the distortion (block distortion) occurring at the block boundary, and saves the decoded image represented by the decoded image signal after the distortion is removed into the motion compensation prediction frame memory 29.

The filtering process in the loop filter unit 28 is the same as the filtering process in the loop filter unit 11 of Figure 1. When removing block distortion of the decoded image, the intensity of the filtering for removing block distortion is set according to the signal component (luminance signal component, color difference signal component) in accordance with the coding mode (intra-frame coding mode, inter-frame coding mode) output from the variable-length decoding unit 21.

Furthermore, when the filter strength information multiplexing flag and the filter strength information are decoded from the slice header by the variable length decoding unit 21, filtering processing is performed using the filter strength bS indicated by the filter strength information.

As can be seen from the above description, according to this first embodiment, when removing block distortion from a decoded image, the loop filter unit 28 configured as an image decoding device sets the intensity of the filtering for removing block distortion according to the signal component (luminance signal component, color difference signal component) in accordance with the coding mode (intra-frame coding mode, inter-frame coding mode) output from the variable-length decoding unit 21. This has the effect of improving the accuracy of removing block distortion and the quality of the decoded image.

In the above description, an example of an image encoding device and an image decoding device that implement inter-frame motion compensation prediction processing (inter-frame prediction processing) is shown. However, even an image encoding device and an image decoding device that implement intra-frame prediction processing (intra-frame prediction processing) for all frames can be configured to remove block distortion by the loop filter units 11 and 28.

In an image encoding device and an image decoding device that implement a combination of intra-frame prediction processing (intra-frame prediction processing) and inter-frame motion compensation prediction processing (inter-frame prediction processing), when it is set to encode all frames through intra-frame prediction processing, it is also possible to control the operation of the loop filter units 11 and 28 not to be used.

In this first embodiment, an example is shown in which the size of the coding block ^Bn is ^Ln = ^Mn as shown in FIG5 . However, the size of the coding block ^Bn may be ^Ln ≠ ^Mn .

For example, as shown in FIG15 , consider the case where the size of the coding block ^Bn is ^Ln = ^kMn .

In the next division, (Ln ⁺¹ , Mn ⁺¹ ) = ( ^Ln , ^Mn ). Subsequent divisions may be similar to those shown in FIG5 or may be such that (Ln ⁺¹ , ^Mn+1 ) = ( ^Ln /2, ^Mn /2).

By setting M ⁰ = 16, for example, such a structure can define a maximum coding block that is a structure of horizontally concatenating macroblocks composed of 16×16 pixels, as in MPEG-2 (ISO/IEC 13818-2) and MPEG-4 AVC/H.264 (ISO/IEC 14496-10), which has the effect of facilitating the construction of an image coding device that maintains compatibility with existing methods.

Furthermore, even when ^Ln = ^kMn is not Ln=kMn but the system is vertically connected as in ^kLn = ^Mn , it is of course possible to perform division according to the same concept.

Furthermore, the present invention can realize modifications of arbitrary components of the embodiments or omissions of arbitrary components of the embodiments within the scope of the invention.

Industrial applicability

The image encoding device, image decoding device, image encoding method and image decoding method of the present invention have means for setting the intensity of filtering according to the signal component in accordance with the encoding mode, which can improve the accuracy of removing block distortion and improve the quality of the encoded image, so they can be applied to international standard image encoding methods such as MPEG and ITU-TH.26x.

Claims (2)

1. An image encoding apparatus for performing block-level encoding processing on an input image, characterized in that the image encoding apparatus comprises: The distortion removal unit performs filtering processing on a decoded image obtained by adding the predicted image to a decoded differential image obtained by decompressing compressed data (which is a difference between the input image and the predicted image related to the coded block), and removes block distortion generated in the boundaries of adjacent transform blocks used to transform the differential image in the decoded image. When removing block distortion from the decoded image, the distortion removal unit determines whether the vertical and horizontal edges are consistent with the boundaries of the partition or transform block. Based on the determination result, the information of the encoding mode of the encoding block (whether it is an intra-frame encoding mode or an inter-frame encoding mode) and whether the signal component to be filtered is a luminance signal component or a chrominance signal component, the filtering process is performed. The filtering intensity is set independently for each signal component, corresponding to the encoding mode of the encoding block. 2. An image encoding method, which performs encoding processing on an input image in block units, characterized in that the image encoding method comprises: The distortion removal process involves filtering the decoded image obtained by decompressing compressed data (obtained by compressing the input image and the prediction image as a difference related to the coded block) and the predicted image to remove block distortion generated at the boundaries of adjacent transform blocks used to transform the difference image in the decoded image. In the distortion removal process, when removing block distortion from the decoded image, it is determined whether the vertical and horizontal edges are consistent with the boundaries of the partition or transform block. Based on the determination result, the information of the encoding mode of the encoding block (whether it is an intra-frame encoding mode or an inter-frame encoding mode) and whether the signal component to be filtered is a luminance signal component or a chrominance signal component, filtering is performed. The filtering intensity is set independently for each signal component, corresponding to the encoding mode of the encoding block.