WO2019142764A1 - Moving-image coding device, moving-image decoding device, and moving-image filter device - Google Patents

Abstract

At a CTU boundary, a deblocking filter process cannot be performed until the pixel value of a CTU block processed later in a decoding sequence is generated and SAO carried out after the deblocking filter process cannot be processed in CTU units either. SAO is carried out before the deblocking filter and the EO processing region of SAO is limited, thereby SAO is processed in CTU units.

Description

Moving picture coding device, moving picture decoding device and moving picture filter device

One aspect of the present invention relates to a video decoding device, a video encoding device, and a video filter device.

In order to efficiently transmit or record moving pictures, a moving picture coding apparatus (image coding apparatus) that generates a coded stream by coding a moving picture, and decoding the coded stream A moving image decoding apparatus (image decoding apparatus) that generates a decoded image is used. Also, in the moving picture coding apparatus and the moving picture decoding apparatus, a filter device for enhancing the image quality of the moving picture is mounted.

As a specific moving picture coding method, for example, a method adopted or proposed in H.264 / AVC or High-Efficiency Video Coding (HEVC) may be mentioned.

In such a moving picture coding method, an image (picture) constituting a moving picture is a slice obtained by dividing the image, a coding tree unit obtained by dividing the slice (CTU: Coding Tree Unit) A coding unit (CU: Coding Unit, sometimes referred to as a coding unit) obtained by dividing a coding tree unit, and a prediction unit (block) obtained by dividing a coding unit It is managed by the hierarchical structure which consists of PU: Prediction Unit (TU) and transform unit (TU: Transform Unit), and is encoded / decoded per CU (nonpatent literature 1).

Also, in such a moving picture coding method, a predicted picture is usually generated based on a locally decoded picture obtained by coding / decoding an input picture, and the predicted picture is generated from the input picture (original picture). A prediction residual signal (sometimes referred to as a “residual signal”) obtained by subtraction is encoded (Non-Patent Document 1).

In addition, Non-Patent Document 1 discloses a deblocking filter that performs smoothing processing on adjacent pixels through block boundaries of an image to remove block distortion. Furthermore, an adaptive offset filter (SAO: Sample Adaptive Offset) is also disclosed, which is performed after the deblocking filter and adds an offset according to pixel classification for each pixel.

ITU-T H. 265 (04/2015) SERIES H: AUDIO VISUAL AND MULTIMEDIA SYSTEMS Infrastructure of audiovisual services-Coding of moving video High efficiency video coding

In Non-Patent Document 1 described above, SAO is performed after the deblocking filter. However, the deblocking filter performs the filtering process on two blocks across block boundaries. Therefore, at the CTU boundary, deblocking filtering can not be performed until pixel values (before deblocking filtering) of blocks of CTUs to be processed later in decoding order are generated. That is, the deblocking filter can not be processed on a CTU basis. Therefore, there is a problem that the SAO implemented after the deblocking filter processing can not be processed in CTU units.

The present invention aims to process SAO on a CTU basis by implementing SAO before deblocking filter.

An image filter device according to an aspect of the present invention is an image filter device that divides an image into blocks of a predetermined unit and adds an offset to each pixel value, and the block (target block) of a predetermined unit of an input image The first offset type that derives the category classification of the object pixel according to the magnitude relationship between the object pixel and the adjacent pixel, or the second offset that derives the category classification of the object pixel according to the size of the pixel value of the object pixel A means for assigning one of two offset types, ie, a type, a means for setting different filter target areas in the target block according to the offset type, and a category of each pixel are set using the input image and the offset type Means for deriving an offset, and the category corresponding to the category set according to the offset type. Means for performing filter processing on the filter target area of the target block by adding a set to each pixel, and the input image is a predicted image generated by intra prediction processing or inter prediction processing, and prediction It is an image obtained by adding the residual signal, and the means for setting the category and deriving the offset is specified by the target pixel value and the offset class when the offset type of the target block is the first offset type. A category is set using the two reference pixel values, an offset is derived for each category, and if the offset type of the target block is the second offset type, the category is selected from the band to which the target pixel value belongs. Set and derive the offset for each category.

According to the above configuration, since SAO is processed in CTU units, encoding / decoding processing can be performed without increasing delay.

It is the schematic which shows the structure of an image transmission system. It is a figure which shows the hierarchical structure of the data of a coding stream. It is an example of the syntax of a slice header and CTU. It is a figure which shows the structure of a moving image decoding apparatus. It is a figure explaining a de blocking filter. It is a figure explaining SAO. It is an example of the syntax of SAO. It is a block diagram of SAO. It is a flowchart explaining operation | movement of a category setting part. It is a figure explaining the field which performs the deblocking filter processing in CTU. FIG. 6 is a block diagram and a flowchart of a loop filter. It is a figure explaining the object field of conventional SAO. It is a figure explaining the object area | region of SAO of this invention. It is another figure explaining the object area | region of SAO of this invention. It is a block diagram of SAO. It is a flowchart of SAO. It is a figure which shows the structure of a moving image coding apparatus. It is a block diagram of SAO_E. It is a flowchart explaining operation | movement of an offset information calculation part. It is a flowchart explaining operation | movement of an offset information selection part. It is a block diagram of SAO_E. It is another example of the syntax of SAO. It is a flowchart explaining the operation | movement of an offset addition part. FIG. 1 is a diagram showing a configuration of a transmitting device equipped with a moving image encoding device according to the present embodiment and a receiving device equipped with a moving image decoding device. (a) shows a transmitting apparatus equipped with a moving picture coding apparatus, and (b) shows a receiving apparatus equipped with a moving picture decoding apparatus. FIG. 1 is a diagram showing a configuration of a recording apparatus equipped with a moving picture coding apparatus according to the present embodiment and a reproduction apparatus equipped with a moving picture decoding apparatus. (a) shows a recording apparatus equipped with a moving picture coding apparatus, and (b) shows a reproduction apparatus equipped with a moving picture decoding apparatus.

First Embodiment
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic view showing the configuration of an image transmission system 1 according to the present embodiment.

The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an image to be encoded, decodes the transmitted encoded stream, and displays an image. The image transmission system 1 includes a moving image coding device (image coding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41.

An image T is input to the moving picture coding device 11.

The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small area network (LAN), or a combination of these. The network 21 is not necessarily limited to a two-way communication network, and may be a one-way communication network for transmitting broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may be replaced by a storage medium recording an encoded stream Te such as a DVD (Digital Versatile Disc) or a BD (Blue-ray (registered trademark) Disc).

The video decoding device 31 decodes each of the encoded streams Te transmitted by the network 21 and generates one or more decoded images Td which are each decoded.

The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The form of the display may be stationary, mobile, HMD or the like.

<Operator>
The operators used herein are described below.

>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, and | = is an OR assignment operator.

|| indicates a disjunction.

X? Y: z is a ternary operator that takes y if x is true (other than 0) and z if x is false (0).

Clip3 (a, b, c) is a function that clips c to a value between a and b, and returns a if c <a, b if c> b, otherwise Is a function that returns c (where a <= b).

Abs (a) is a function that returns the absolute value of a.

Int (a) is a function that returns an integer value of a.

Floor (a) is a function that returns the largest integer less than or equal to a.

A / d represents the division of a by d (rounding down the decimal point).

Avg (a (i)) is a function for deriving N average values of a (0) to a (N-1).

Min (a (0), a (1),..., A (N-1)) is a function that returns the minimum value of a (0) to a (N-1).

<Structure of Coded Stream Te>
Prior to detailed description of the moving picture coding apparatus 11 and the moving picture decoding apparatus 31 according to the present embodiment, data of the coded stream Te generated by the moving picture coding apparatus 11 and decoded by the moving picture decoding apparatus 31 The structure will be described.

FIG. 2 is a diagram showing the hierarchical structure of data in the coded stream Te. The coded stream Te illustratively includes a sequence and a plurality of pictures forming the sequence. (A) to (f) in FIG. 2 respectively represent a coded video sequence defining the sequence SEQ, a coded picture defining the picture PICT, and a coding slice defining a slice S composed of a slice header SH and slice data SDATA. 3 is a diagram illustrating CTUs included in slice data and CUs included in CTUs. FIG.

(Encoded video sequence)
In the encoded video sequence, a set of data to which the moving picture decoding apparatus 31 refers in order to decode the target encoded stream Te is defined. As shown in FIG. 2A, the coded stream Te is a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, And, supplemental enhancement information SEI (Supplemental Enhancement Information) is included.

A video parameter set VPS is a set of coding parameters common to a plurality of moving images in a moving image composed of one or more layers, and one or more layers and individual layers included in the moving image. A set of related coding parameters is defined.

In the sequence parameter set SPS, a set of coding parameters to be referred to by the video decoding device 31 for decoding the target sequence is defined. For example, the width and height of the picture are defined. In addition, multiple SPS may exist. In that case, one of a plurality of SPSs is selected from PPS.

In the picture parameter set PPS, a set of coding parameters to be referred to by the video decoding device 31 for decoding each picture in the target sequence is defined. For example, a reference value of quantization width (pic_init_qp_minus 26) used for decoding a picture and a flag (weighted_pred_flag) indicating application of weighted prediction are included. In addition, multiple PPS may exist. In that case, one of a plurality of PPSs is selected from each slice header in the target sequence.

(Coded picture)
In the coded picture, a set of data to which the moving picture decoding apparatus 31 refers in order to decode the picture PICT to be processed is defined. The picture PICT includes slice 0 to slice NS-1 (NS is the total number of slices included in the picture PICT) as shown in (b) of FIG.

In the following, when it is not necessary to distinguish each of slice 0 to slice NS-1, the suffixes of reference numerals may be omitted and described. Further, the same is true for other data that is included in the encoded stream Te described below and that is suffixed.

(Coding slice)
In the coding slice, a set of data to which the moving picture decoding apparatus 31 refers in order to decode the slice S to be processed is defined. The slice S includes a slice header SH and slice data SDATA as shown in (c) of FIG.

The slice header SH includes a coding parameter group to which the video decoding device 31 refers in order to determine the decoding method of the target slice. The slice type specification information (slice_type) for specifying a slice type is an example of a coding parameter included in the slice header SH.

As slice types that can be designated by slice type designation information, (1) I slice using only intra prediction at the time of encoding, (2) P slice using unidirectional prediction at the time of encoding or intra prediction, (3) B-slice using uni-directional prediction, bi-directional prediction, or intra prediction at the time of encoding. Note that inter prediction is not limited to single prediction and bi prediction, and more reference pictures may be used to generate a predicted image. Hereinafter, when referred to as P and B slices, it refers to a slice including a block for which inter prediction can be used.

In addition, the slice header SH includes the SAO parameter SAOP_S referred to by the adaptive offset filter SAO provided in the video decoding device 31. FIG. 3A is a diagram showing syntax elements of the SAO parameter SAOP_S included as a part of slice header SH of the encoded stream according to the present embodiment (denoted as slice_segment_header () in FIG. 3A). is there. The SAO parameter SAOP_S is information indicating on / off etc. of the parameter of the SAO process, and when sllice_sao_luma_flag is 1, for the luminance (Y: color component cIdx == 0), SAO is turned on, 0 is turned off, and slice_sao_chroma_flag is In the case of 1, for the color difference (Cb: color component cIdx == 1 and Cr: color component cIdx == 2), the SAO is turned on, and when it is 0, it is turned off.

The slice header SH may include a reference (slice_pic_parameter_set_id) to the picture parameter set PPS.

In the slice data, a set of data to which the moving image decoding apparatus 31 refers in order to decode the slice data SDATA to be processed is defined. The slice data SDATA includes a coding tree unit CTU (CTU block) as shown in (d) of FIG. The CTU is a block of a fixed size (for example, 64 × 64) that configures a slice, and may also be referred to as a largest coding unit (LCU: Largest Coding Unit).

(Encoding tree unit)
In (d) of FIG. 2, a set of data to which the video decoding device 31 refers to to decode the coding tree unit CTU is defined. As (d) in FIG. 2 shows, the CTU includes a CTU header and a coding unit CU (CU block).

The CTU header includes a coding parameter to which the video decoding device 31 refers in order to determine the decoding method of the target CTU. Specifically, division information SP_CU specifying a division pattern to each CU of the target CTU, quantization parameter difference Δqp (qp_delta) specifying the size of quantization step, and SAO parameter SAOP_C referred to by SAO are included. included.

The division information SP_CU is information representing a coding tree for dividing a CTU, and more specifically, information specifying the shape, size, and position within the target CTU of each CU included in the target CTU. is there.

The quantization parameter difference Δqp is the difference qp−qp ′ between the quantization parameter qp in the target CTU and the quantization parameter qp ′ in the CTU encoded immediately before the target CTU.

In addition, the SAO parameter SAOP_C is a parameter referred to by the SAO included in the video decoding device 31. FIG. 3B is a diagram showing syntax elements of the SAO parameter SAOP_C included as part of the CTU (denoted as coding_tree_unit () in FIG. 3B) of the coded stream according to the present embodiment. . The SAO parameter SAOP_C includes, as a syntax element, SAO unit information sao_unit () described later.

In (e) of FIG. 2, a set of data to which the moving picture decoding apparatus 31 refers to to decode a target CTU is defined. The CTU is divided into a coding unit CU which is a basic unit of coding processing by recursive quadtree division (QT division) or binary tree division (BT division). A tree structure obtained by recursive quadtree division or binary tree division is called a coding tree (CT: Coding Tree), and nodes of the tree structure are called a coding node (CN: Coding Node). The intermediate nodes of the quadtree and binary tree are coding nodes, and the CTU itself is also defined as the highest coding node.

The CT includes, as CT information, a QT split flag (cu_split_flag) indicating whether or not to perform QT split, and a BT split mode (split_bt_mode) indicating a split method of BT split.

(Coding unit)
In (f) of FIG. 2, a set of data to which the moving picture decoding apparatus 31 refers to to decode the target CU is defined. Specifically, a CU is composed of a prediction tree, a transformation tree, and a CU header CUH. In the CU header, a prediction mode, a division method (PU division mode), and the like are defined.

In the prediction tree, prediction parameters (reference picture index, motion vector, etc.) of each prediction unit (PU: Prediction Unit) obtained by dividing a coding unit into one or more are defined. Put another way, a PU is one or more non-overlapping regions that make up a CU. Also, the prediction tree includes one or more PUs obtained by the above-mentioned division. In addition, below, the prediction unit which divided | segmented PU further is called a "subblock." The sub block is composed of a plurality of pixels. When PU and subblocks have the same size, there is one subblock in PU. If the PU is larger than the size of the subblock, the PU is divided into subblocks. For example, if the PU is 8x8 and the sub-blocks are 4x4, the PU is divided into four sub-blocks, which are horizontally divided into two and vertically divided into two.

The prediction process may be performed for each prediction unit (sub block).

Broadly speaking, there are two types of prediction in the prediction tree: intra prediction and inter prediction. Intra prediction is prediction in the same picture, and inter prediction refers to prediction processing performed between mutually different pictures (for example, between display times, between layer images).

In the case of intra prediction, there are 2Nx2N (the same size as the coding unit) and NxN as a division method.

Also, in the case of inter prediction, the division method is encoded according to the PU division mode (part_mode) of the coded stream, and 2Nx2N (same size as CU), 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and NxN There is. Note that 2NxN and Nx2N indicate 1: 1 symmetric division, and 2NxnU, 2NxnD and nLx2N and nRx2N indicate 1: 3 and 3: 1 asymmetric division. The PUs included in the CU are expressed as PU0, PU1, PU2, PU3 in order.

In addition, in the transformation tree, a CU is divided into one or more transformation units TU, and the position and size of each TU are defined. In other words, a TU is one or more non-overlapping regions that make up a CU. Also, the transformation tree includes one or more TUs obtained by the above-mentioned division.

Partitions in the transformation tree may be allocated as an area of the same size as a CU as TU, or by recursive quadtree partitioning, as in the case of CU partitioning described above.

The conversion process is performed for each TU.

(Prediction parameter)
The prediction image of the prediction unit PU is derived by the prediction parameters associated with the PU. The prediction parameters include intra prediction prediction parameters or inter prediction prediction parameters.

(Configuration of video decoding device)
FIG. 4 shows a moving picture decoding apparatus (image decoding apparatus) 31 according to the present invention.

The moving picture decoding apparatus 31 includes an entropy decoding unit 301, a prediction parameter decoding unit (predicted image decoding apparatus) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a predicted image generation unit (predicted image generation apparatus) 308, The configuration includes an inverse quantization / inverse transform unit 311 and an addition unit 312.

Further, the prediction parameter decoding unit 302 is configured to include an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.

Moreover, although the example which used CTU, CU, PU, and TU as a process unit below is described, you may process not only this example but CU unit instead of TU or PU unit. Alternatively, CTU, CU, PU, and TU may be replaced with blocks, and processing may be performed in units of blocks.

The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside to separate and decode individual codes (syntax elements). In entropy coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), which uses variable-length coding / decoding of syntax using a context (probability model) that is adaptively selected according to the kind of syntax and surrounding circumstances. And CAV (Context-based Adaptive Variable Length Coding) in which syntax is variable-length encoded using a predetermined table or a formula. The separated codes include prediction parameters for generating a prediction image and residual information for generating a residual signal.

The entropy decoding unit 301 outputs a part of the separated code to the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantized transform coefficient to the inverse quantization / inverse transform unit 311. The quantization transform coefficients are used to encode the residual signal by discrete cosine transform (DCT), discrete sine transform (DST), Karynen Loeve transform, and Karhunen Loeve transform in a coding process. Etc.) and is obtained by quantization. Further, the entropy decoding unit 301 decodes the SAO parameters SAOP_S and SAOP_C from the encoded stream, and supplies them to the adaptive offset filter (SAO) 3052.

The inter prediction parameter decoding unit 303 decodes the inter prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. Further, the inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308, and stores the inter prediction parameter in the prediction parameter memory 307.

The intra prediction parameter decoding unit 304 decodes the intra prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308, and stores it in the prediction parameter memory 307.

The loop filter 305 applies filters such as a deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adding unit 312. The loop filter 305 may not necessarily include the above three types of filters as long as the loop filter 305 is paired with the moving picture coding device 11. For example, the loop filter 305 may have only a deblocking filter. Details of the loop filter 305 will be described later.

The reference picture memory 306 stores the decoded image of the CU generated by the adding unit 312 in a predetermined position for each of the picture to be decoded and the CTU or CU.

The prediction parameter memory 307 stores prediction parameters in a predetermined position for each picture to be decoded and each prediction unit (or sub block, fixed size block, pixel). Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode separated by the entropy decoding unit 301.

The prediction image generation unit 308 receives the prediction mode predMode input from the entropy decoding unit 301, and also receives a prediction parameter from the prediction parameter decoding unit 302. Further, the predicted image generation unit 308 reads the reference picture from the reference picture memory 306.

When the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 performs block or inter prediction by using the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture (reference picture block). Generate a predicted image of subblocks.

When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs intra prediction using the intra prediction parameter input from the intra prediction parameter decoding unit 304 and the read reference pixel.

The prediction image generation unit 308 outputs the prediction image of the block generated by the inter prediction image generation unit 309 or the intra prediction image generation unit 310 to the addition unit 312.

The inverse quantization / inverse transform unit 311 inversely quantizes the quantized transform coefficient input from the entropy decoding unit 301 to obtain a transform coefficient. The inverse quantization / inverse transform unit 311 performs inverse frequency transform such as inverse DCT, inverse DST, and inverse KLT on the obtained transform coefficient to calculate a residual signal. The inverse quantization / inverse transform unit 311 outputs the calculated residual signal to the addition unit 312.

The addition unit 312 adds, for each pixel, the prediction image of the block input from the inter prediction image generation unit 309 or the prediction image generation unit 308 and the residual signal input from the inverse quantization / inverse conversion unit 311. Generate a decoded image of The addition unit 312 outputs the generated decoded image of the block to at least one of the deblocking filter 3051, the SAO 3052, and the ALF 3053.

(Loop filter 305)
FIG. 4B is a block diagram showing the configuration of the loop filter (image filter device) 305. As shown in FIG. 4B, the loop filter 305 includes a deblocking filter 3051, an SAO 3052, and an ALF 3053. The input to the loop filter is a decoded image in which the predicted image generated by the predicted image generation unit 308 and the residual signal derived by the inverse quantization / inverse conversion unit 311 are added by the addition unit 312 (pre-filter decoded image ).

(Deblocking filter 3051)
The deblocking filter 3051 determines that block distortion is present when the difference between the pixel values of the adjacent pixels through the block (CTU / CU / PU / TU) boundary is within a predetermined range. Then, the block boundary in the pre-deblocking filter image is subjected to deblocking processing to smooth the image in the vicinity of the block boundary. The image subjected to the deblocking processing by the deblocking filter 3051 is a deblocked decoded image P_DB. For example, the on / off determination of the deblocking filter is performed by the following equation.

abs (p20-2 * p10 + p00) + abs (p23-2 * p13 + p03) + abs (q20-2 * q10 + q00) + abs (q23-2 * q13 + q03) <β (Formula DB-1 )
Here, p2k, p1k, p0k, q0k, q1k and q2k are respectively a column or a row of pixels having a distance of 2, 1, 0, 0, 1, 2 from the block boundary as shown in FIG. p2k, p1k, and p0k are pixels included in the block P among the blocks P and Q bordering the boundary, and q0k, q1k, and q2k are pixels included in the block Q. k indicates the pixel number in the block boundary direction, and k> = 0. β is a threshold value derived from the average value QPavg of quantization parameters of the block P and the block Q, and pps_beta_offset_div2 and slice_beta_offset_div2 notified by the PPS or the slice header SH. If (Expression DB-1) is satisfied, the deblocking filter is turned on (implemented) for the boundary between block P and block Q. In addition, deblocking filter processing is performed by the following equation.

p2 '= Clip3 (p2-2 * tc, p2 + 2 * tc, (2 * p3 + 3 * p2 + p1 + p0 + q0 + 4) >> 3) (Expression DB-2)
p1 '= Clip 3 (p1-2 * tc, p1 + 2 * tc, (p2 + p1 + p0 + q0 + 2) >> 2)
p0 '= Clip 3 (p0-2 * tc, p0 + 2 * tc, (p2 + 2 * p1 + 2 * p0 + 2 * q0 + q1 + 4) >> 3)
q0 '= Clip 3 (q0-2 * tc, q0 + 2 * tc, (p1 + 2 * p0 + 2 * q0 + 2 * q1 + q2 + 4) >> 3)
q1 '= Clip 3 (q1-2 * tc, q1 + 2 * tc, (p0 + q0 + q1 + q2 + 2) >> 2)
q2 '= Clip3 (q2-2 * tc, q2 + 2 * tc, (p0 + q0 + q1 + 3 * q2 + 2 * q3 + 4) >> 3)
(Formula DB-2) is a process common to each k (k> = 0) in FIG. 5, so k is omitted. Each of p3, p2, p1, p0, q0, q1, q2 is a column or row of pixels having a distance of 3, 2, 1, 0, 0, 1, 2, 3 from the block boundary. tc is a variable for suppressing the filter processing, and is a threshold value derived from the average value QPavg of the quantization parameter of the block P and the block Q, pps_tc_offset_div2, slice_tc_offset_div2 notified by the PPS or the slice header SH, and the like.

As described above, the deblocking filter refers to pixel values of 4 pixels from the boundary and changes pixel values of 3 pixels from the boundary in two blocks bordering the boundary. That is, when the block boundary is a CTU boundary, the deblocking process requires pixels of blocks belonging to adjacent different CTUs. Therefore, deblocking processing can not be performed in CTU units.

(SAO3052)
SAO is a process using SAO parameters notified for each CTU, and has an effect of removing ringing distortion and quantization distortion. The SAO 3052 generates the SAO-completed decoded image P_SAO by performing SAO processing on the input image. The SAO is a filter that classifies pixel values into several categories, and adds / subtracts an offset in units of pixels for each category. The SAO has two types of offset types: edge offset (EO: Edge Offset, first offset) and band offset (BO: Band Offset, second offset), and the category classification method of pixel values in CTU is this offset. It depends on the type.

EO classifies the target pixel according to the magnitude relationship between the target pixel and the adjacent pixel (reference pixel, see FIG. 6A). The BO classifies the target pixel according to the size of the pixel value of the target pixel (see FIG. 6C). An offset defined for each category for both EO and BO is added to the pixel value of the target pixel.

EO refers to adjacent pixels in eight directions of the target pixel and changes the target pixel value. That is, at the CTU boundary, pixels of different CTUs adjacent to each other are required as reference pixels for EO processing. On the other hand, BO refers to the target pixel value itself and changes the target pixel value. That is, BO processing can be performed in CTU units.

(SAO unit)
FIG. 7 is a diagram showing the syntax of the SAO unit (denoted as sao_unit () in FIG. 7) of the coded stream according to the present embodiment. The SAO unit is notified for each CTU, and includes SAO offset information sao_offset () and merge control information (sao_merge_left_flag, sao_merge_up_flag) of SAO offset information as syntax elements. sao_merge_left_flag is a flag indicating whether or not SAO offset information of the target CTU is to be copied from the left adjacent CTU of the target CTU. sao_merge_up_flag is a flag indicating whether or not SAO offset information of the target CTU is to be copied from the upper adjacent CTU of the target CTU. SAO offset information is an offset type (sao_type_idx), an index (band offset position, sao_band_position) indicating the position of the band to which the offset is applied within the pixel value range, an offset class (reference pixel position of EO, sao_eo_class), and an offset value (sao_offset) including. sao_offset consists of the absolute value of the offset sao_offset_abs and the sign of the offset sao_offset_sign. sao_band_position is used when the offset type is BO. Note that the value of the offset in the coded stream may be encoded as a residual signal that derives the offset value using a quantized value or a prediction value such as linear prediction. The sao_type_idx and sao_offset will be described later. Note that “_luma” or “_chroma” following the syntax in FIG. 7 indicates that it is a syntax for luminance or chrominance, respectively.

The Descriptor shown in FIG. 3 and FIG. 7 will be described. ue (v) indicates that the syntax associated with this descriptor is an unsigned numeric value, and that the value is variable-length encoded. se (v) indicates that the syntax associated with this descriptor is a signed numeric value and is variable-length encoded. ae (v) indicates that the syntax associated with this descriptor is variable-length encoded using an arithmetic code. u (n) indicates that the syntax associated with this descriptor is an unsigned numeric value, and n-bit fixed-length coding is used. f (n) indicates that a fixed value bit pattern is used.

Next, the offset type sao_type_idx and the offset sao_offset will be described in more detail.

(Sao_type_idx)
sao_type_idx is an identifier representing an offset type. sao_type_idx is assigned to a predetermined unit (SAO unit, for example, CTU) for each color component. Therefore, the type for each predetermined unit is expressed as sao_type_idx [cIdx] [rx] [ry]. The subscript cIdx is an index indicating any of luminance, color difference Cr, and color difference Cb, and rx and ry respectively indicate x and y coordinates of the upper left position of a predetermined unit to be an object. In the present embodiment, sao_type_idx [cIdx] [rx] [ry] may be referred to simply as sao_type_idx or an offset type. Even when simply called sao_type_idx, the necessary individual values can be referred to as sao_type_idx [cIdx] [rx] [ry]. Hereinafter, although the predetermined unit is described as CTU, the predetermined unit is not limited to this.

There are two types of offset types: edge offset (EO) and band offset (BO). EO is processing for adding one of a plurality of offsets to the pixel value of the target pixel according to the difference between the pixel value of the target pixel and the pixel values of the peripheral pixels. Further, BO is a process of adding one of a plurality of offsets to the pixel value of the target pixel according to the target pixel value.

In the present embodiment, sao_type_idx takes an integer value from 0 to 2 and indicates what kind of filter processing is to be performed or no filter processing is to be performed on the SAO pre-decoded image. Specifically, it is as follows.
-Sao_type_idx = 0 indicates that SAO is not performed on the SAO pre-decoded image in the target CTU.
-Sao_type_idx = 2 indicates that EO is performed on the SAO pre-decoded image in the target CTU. In EO, sao_eo_class indicating the reference pixel position is encoded.
-Sao_type_idx = 1 indicates that BO is performed on the SAO pre-decoded image in the target CTU. In BO, sao_band_position which specifies the left end position of the pixel value band to be processed is encoded.

(SaoOffsetVal)
sao_offset_abs [cIdx] [rx] [ry] [i] and sao_offset_sign [cIdx] [rx] [ry] [i] are offsets added to the respective pixels included in the target CTU in the SAO process according to the present embodiment. It is a parameter used to derive a value. In the present embodiment, sao_offset_abs [cIdx] [rx] [ry] [i] and sao_offset_sign [cIdx] [rx] [ry] [i] may be simply referred to as sao_offset_abs, sao_offset_sign or an offset value. Even when simply called sao_offset_abs or sao_offset_sign, the necessary individual values can be referred to as sao_offset [cIdx] [rx] [ry] [i] and sao_offset_sign [cIdx] [rx] [ry] [i] It is.

sao_offset_abs [cIdx] [rx] [ry] [i] and sao_offset_sign [cIdx] [rx] [ry] [i] are i in addition to the above arguments cIdx, rx, ry, corresponding to the category cat described later Specified by The categories will be described later.

In the process of SAO in this embodiment, the absolute value sao_offset_abs of the offset and the code sao_offset_sign are included in the encoded stream, and the SAO 3052 derives and uses the offset value SaoOfftVal actually used from sao_offset_abs and sao_offset_sign.

SaoOffsetVal [cIdx] [rx] [ry] [i + 1] = (1-2 * sao_offset_sign [cIdx] [rx] [ry] [i]) * sao_offset_abs [cIdx] [rx] [ry] [i] <<log2OffsetScale (equation SAO-1)
Here, log2OffsetScale is a quantization parameter used when quantizing the offset, and typically uses a value such as 0, 1, 2, or the like.

Further, sao_offset_sign is encoded as a parameter for deriving the sign of the value of the offset to be added to each derived pixel. sao_offset_sign is encoded when the offset value to be added is other than zero.

Next, the range of values of sao_offset will be described.

When the offset type sao_type_idx is 2, that is, at the offset sao_offset of EO, the range is
From 0 to (1 << (min (bitDepth, bitDepthTh) -bitDepthD))-1
It is. Here, bitDepth is a value representing the bit depth of the pixel value to be subjected to SAO, bitDepthTh is the lower limit bit depth for quantizing the offset value, and bitDepthD is a value for adjusting the number of bits in the offset value range.

On the other hand, when the offset type sao_type_idx is 1, that is, in the offset sao_offset of BO, the range is
From-(1 << (min (bitDepth, bitDepthTh) -bitDepthD)) to (1 << (min (bitDepth, bitDepthTh) -bitDepthD) -1
It is. From the above, for example, when bitDepth = 8, bitDepthTh = 10, and bitDepthD = 5, the value range of the offset sao_offset is 0 to 7 in the case of EO and -8 to 7 in the case of BO. The derivation method of the offset type sao_type_idx, the band offset position sao_band_position, the offset sao_offset_abs, the offset code sao_offset_sign, and the offset class sao_eo_class differs depending on whether the merge control information (sao_merge_left_flag, sao_merge_up_flag) of the SAO parameter SAOP_C indicates merge. When the merge control information does not indicate merging, these parameters are decoded by the entropy decoding unit 301 and stored in the offset information storage unit 1002. When the merge control information indicates merge, the parameter of the adjacent CTU that has been decoded is read out from the offset information storage unit 1002 and set as a parameter of the target CTU. Specifically, when sao_merge_left_flag is 1, the parameter of the left adjacent CTU of the target CTU is set. When sao_merge_up_flag is 1, the parameter of the upper adjacent CTU of the target CTU is set.

More specifically, when sao_merge_up_flag = 1,
sao_offset_abs [cIdx] [rx] [ry] [i] = sao_offset_abs [cIdx] [rx] [ry-1] [i] (Expression SAO-2)
sao_offset_sign [cIdx] [rx] [ry] [i] = sao_offset_sign [cIdx] [rx] [ry-1] [i]
sao_type_idx [cIdx] [rx] [ry] [i] = sao_type_idx [cIdx] [rx] [ry-1] [i]
sao_band_position [cIdx] [rx] [ry] = sao_band_position [cIdx] [rx] [ry-1] [i]
sao_eo_class [cIdx] [rx] [ry] = sao_eo_class [cIdx] [rx] [ry-1] [i]
Set to If sao_merge_left_flag = 1, then
sao_offset_abs [cIdx] [rx] [ry] [i] = sao_offset_abs [cIdx] [rx-1] [ry] [i] (Expression SAO-3)
sao_offset_sign [cIdx] [rx] [ry] [i] = sao_offset_sign [cIdx] [rx-1] [ry] [i]
sao_type_idx [cIdx] [rx] [ry] [i] = sao_type_idx [cIdx] [rx-1] [ry] [i]
sao_band_position [cIdx] [rx] [ry] = sao_band_position [cIdx] [rx-1] [ry] [i]
sao_eo_class [cIdx] [rx] [ry] = sao_eo_class [cIdx] [rx-1] [ry] [i]
Set to Here, (rx, ry) is the upper left coordinates of the target CTU. When the band offset position sao_band_position is stored, in order to reduce the memory size of the offset information storage unit 1002, round processing may be performed to discard information of lower bits. Round processing is effective in reducing line memory. When the band offset position sao_band_position is read out from the offset information storage unit 1002, reverse round processing may be performed.

sao_band_position [cIdx] [rx] [ry] = ((sao_band_position [cIdx] [rx] [ry-1] [i]) >> shift) << shift (equation SAO-4)
sao_band_position [cIdx] [rx] [ry] = ((sao_band_position [cIdx] [rx-1] [ry] [i]) >> shift) << shift
The block diagram of SAO3052 is shown in FIG. The SAO 3052 includes a category setting unit 1001, an offset information storage unit 1002, and an offset addition unit 1003.

(Category setting unit 1001)
The category setting unit 1001 receives the SAO parameters (SAOP_S, SAOP_C) stored in the offset information storage unit 1002 and the SAO pre-decoded image rec. The category setting unit 1001 uses these to set the category of each pixel position corresponding to the offset type sao_type_idx. Further, the offset SaoOffsetVal of (Expression SAO-1) is derived, and the category and the offset are output to the offset addition unit 1003. When sao_type_idx is 0, the category is not calculated.

When sao_type_idx is 2 (EO), the category setting unit 1001 sets sao_eo_class to an offset class representing edge classification shown in FIG. 6A. The class corresponds to the reference pixel direction viewed from the target pixel, and is used to select the reference pixel. In the case of class = 0, the left and right pixels of the target pixel X are set in the reference pixels a and b, and in the case of class = 1, the pixels above and below the target pixel X are set in a, b. Upper left and lower right pixels of the target pixel X are set to a and b, and in the case of class = 3, lower left and upper right pixels of the target pixel x are set to a and b.

Next, the difference sign (rec [x] -rec [a]) and sign (rec [x] -rec [b]) of the pixel values of the target pixel x and the two reference pixels a and b designated by class are calculated. Use to derive edgeIdx.

edgeIdx = sign (rec [x] -rec [a]) + sign (rec [x] -rec [b]) + 2 (equation SAO-5)
sign (x) = 1 (x> 0)
= 0 (x = 0)
= -1 (x <0)
Here, rec [x] represents the decoded pixel value of the pixel x. The category cat is calculated from this edgeIdx.

cat = (edgeIdx = = 2)? 0: edgeIdx + 1 (edgeIdx <= 2) (Equation SAO-6)
= edgeIdx (edgeIdx> 2)
The magnitude relationship between the target pixel value x and the reference pixels a and b indicated by the category cat in the case of class = 0 is shown in FIG. Black circles represent the target pixel x, white circles on the left of x represent reference pixels a, white circles on the right of x represent reference pixels b, and the vertical direction represents the magnitude relationship of pixel values.

When sao_type_idx is 1 (BO), the category setting unit 1001 divides pixel values of 0 to 2 ^ bitDepth-1 into bands as shown in FIG. 6C. In the case of bit Depth = 8, the pixel value ranges from 0 to 255, and each band consists of eight consecutive pixel values (bandwidth band W = 8). When bitDepth = 10, the pixel value ranges from 0 to 1023, and each band consists of 32 consecutive pixel values (bandW = 32). The category setting unit 1001 sets four consecutive bands from the band indicated by the band offset position (sao_band_position) decoded by the entropy decoding unit 301 to 1 to 4 of the category cat.

cat = (rec [x] [y] >> bandShift) -sao_band_position + 1 (equation SAO-7)
if (cat <= 0 || cat> 4) cat = 0
Here, rec [x] [y] is the pixel value of the SAO pre-decoded image, and bandShift is log2 (bandW).

Furthermore, the offset SaoOffsetVal of (Expression SAO-1) is derived, and the category cat and the offset SaoOffsetVal are output to the offset addition unit 1003.

The operation of the category setting unit 1001 will be described with reference to the flowchart of FIG. The category setting unit 1001 checks the offset type sao_offset_type in S901, and proceeds to S902 in the case of EO, or proceeds to S905 in the case of BO. Steps S902 to S904 are processing of EO, and steps S905 to S906 are processing of BO. If the offset type sao_offset_type is EO, the category setting unit 1001 refers to sao_type_idx in S902 and sets adjacent pixels of the target pixel x as reference pixels a and b. The category setting unit 1001 sets the difference sign (rec [x] -rec [a]), sign (rec [x] -rec [b]) of the pixel values of the target pixel x and the two reference pixels a and b in S903. Derive edgeIdx using Next, the category setting unit 1001 calculates and sets a category cat from edgeIdx in S904.

If the offset type sao_offset_type is BO, the category setting unit 1001 divides the pixel value of 0 to 2 ^ bit Depth-1 into bands in S905. In step S 906, the category setting unit 1001 sets four consecutive bands from the band indicated by the band offset position (sao_band_position) to the categories 1 to 4 of the category cat. Then, when it is included in categories 1 to 4 in which the pixel value of the target pixel is included, this is set as the category of the target pixel.

In step S 907, the category setting unit 1001 derives the offset SaoOffsetVal using (Expression SAO-1).

The offset information storage unit 1002 stores the SAO parameters decoded by the entropy decoding unit 301. If the merge control information (sao_merge_left_flag, sao_merge_up_flag) is 1, the SAO parameter of the adjacent CTU is output to the category setting unit 1001, otherwise the SAO parameter decoded by the entropy decoding unit 301 is output.

The offset addition unit 1003 performs SAO using the category cat derived for each target pixel by the category setting unit 1001 and the offset SaoOffsetVal derived for each CTU. Assuming that the SAO pre-decoded pixel value is rec [x] [y], the output pixel value recsao [x] [y] of the offset addition unit 1003 is recsao [x] [y] = Clip 3 (0, (1 << bitDepth) -1, rec [x] [y] + SaoOffsetVal [cIdx] [rx] [ry] [cat]) (Expression SAO-8)
It is. Here, (rx, ry) is the upper left coordinates of the target CTU. recsao [x] [y] is output from the SAO 3052 as P_SAO.

(ALF3053)
The ALF 3053 performs ALF-processed decoded image P_ALF by performing filter processing on the input image using filter parameters (filter set number, filter coefficient group, region designation information, and on / off information) decoded from the encoded stream. Generate The ALF-completed decoded image P_ALF is output to the outside as a decoded moving image Td, and stored in the prediction parameter memory 307 in association with POC designation information decoded from the coded stream by the entropy decoding unit 301.

In the loop filter 305, the deblocking filter 3051 first performs smoothing processing on the pixels of the block boundary for the decoded image (pre-filter decoded pixel), and then the SAO 3052 removes ringing distortion for pixels in the CTU, Correction of pixel value deviation is performed. Finally, processing is performed to reduce the difference between the decoded image and the original image by ALF.

However, as described above, the deblocking filter can not process on a CTU basis, and after the unfiltered decoded image P of the right adjacent CTU and the lower adjacent CTU of the target CTU is generated, at the block boundary in the target CTU. The on / off determination and deblocking filter processing of all the pixels in contact can be completed. Therefore, SAO using the pixel value changed by the deblocking filter can not perform processing until the processing of the right adjacent CTU and the lower adjacent CTU of the target CTU is completed.

Since the deblocking filter has no coding parameter to be notified in CTU units, as shown in FIG. 10A, the deblocking filter can be a pixel (a white area surrounded by a broken line) that can perform deblocking filter processing only with pixels in the target CTU. Applies a deblocking filter during decoding of the target CTU. Next, if a pre-filter decoded image of the left end pixel of the right adjacent CTU of the target CTU is generated for the pixel in the target CTU that touches the right adjacent CTU (upper right-lower left hatched area surrounded by dashed line), Call. Furthermore, if a pre-filter decoded image of the top pixel of the lower adjacent CTU of the target CTU is generated for the pixels in contact with the lower adjacent CTU in the target CTU (upper left-lower right hatched area surrounded by dashed line), Call. By the above processing, it is possible to reduce the delay due to the deblocking filter.

However, there are SAO parameters that the SAO notifies on a CTU basis. For example, sao_merge_left_flag, sao_merge_up_flag, sao_type_idx, sao_offset_abs, sao_offset_sign, sao_band_position, sao_eo_class. If all the deblocking filtered pixels of the target CTU are not aligned, these parameters can not be derived at the video encoding device side. Also, as shown in FIG. 10 (b), the deblocking filtered pixels of the area subjected to the deblocking filter are derived using only the deblocking filtered pixels of the area subjected to the deblocking filter, or Although SAO parameters can be derived using the pre-deblocking filter pixel values in the unimplemented area, especially when the CTU size is small, the percentage of unperformed deblocking filter pixels increases, so the SAO parameters can be derived. Accuracy will be worse.

In this embodiment, an SAO process is performed before the deblocking filter, and a deblocking filter is performed on the SAO-decoded image to explain a technique for reducing delay while maintaining the accuracy of the derived SAO parameter. Do.

FIG. 11A is a block diagram of a loop filter 305D (in a moving image decoding apparatus) in the present embodiment. A loop filter 305D shown in FIG. 11A replaces the loop filter 305 in FIG. 4 and the input and output are also the same as the loop filter 305. That is, the input to the loop filter is a pre-filter decoded image, and the output from the loop filter is a loop-filtered decoded image. Further, in the loop filter 305D, the configurations of the deblocking filter 3051 and the ALF 3053 are the same as the configurations of the deblocking filter 3051 and the ALF 3053 in FIG. FIG. 11C is a flowchart showing the operation of the loop filter 305D. The loop filter 305D performs filtering processing in order of SAO, deblocking filter, and ALF.

In FIG. 11C, the loop filter 305D performs on / off determination of SAO (S1101). If the SAO is off (NO in S1101), the process advances to S1103. If the SAO is on (YES in S1101), the process proceeds to S1102. The SAO 3054 performs SAO processing (S1102).

Next, the loop filter 305D performs on / off determination of the deblocking filter (S1103). If the deblocking filter is off (NO in S1103), the process proceeds to S1105. If the deblocking filter is on (YES in step S1103), the process advances to step S1104. The deblocking filter 3051 performs deblocking filter processing (S1104).

Next, the loop filter 305D performs on / off determination of ALF (S1105). If the ALF is off (NO in S1105), the process ends. If the ALF is on (YES in S1105), the process advances to S1106. The ALF 3053 performs ALF processing (S1106), and ends the processing.

Next, the SAO 3054 will be described.

First, the conventional edge offset (EO) requires the adjacent pixels shown in FIG. 6 (a) to derive the category of the target pixel, and as shown in FIG. 12 (a), the target CTU (FIG. 12 (a)). An area adjacent to one pixel is used as a reference area outside the hatched area). In this case, one pixel at the right end and the lower end of the target CTU uses the SAO pre-decoded pixel (pre-filter decoded pixel) of the right adjacent CTU and the lower adjacent CTU of the target CTU. In other words, reference is made to the pixel of the subsequent CTU which is the CTU on the right side of the target CTU and the lower side. In the present invention, as shown in FIGS. 13 and 14, the filter target area (target area) of EO is limited to the hatched area in the figure which is located one pixel inside of the CTU boundary. That is, the filter target area is x = 0..CTUWidth-2 and y = 0..CTUHeight-2 (where CTUWIdth is the width of CTU and CTUHeight is the height of CTU). Although the details of these processes will be described later, EO can be performed in CTU units without waiting for the processing of the subsequent CTUs by not referring to the pixel values of the subsequent CTUs by limiting the target area of the EO. .

Next, since the band offset (BO) uses the pixel value of the target pixel itself for category derivation, as shown in FIG. 12 (b), additional adjacent to the target CTU (shaded area in FIG. 12 (b)) No space is required. Therefore, the BO can perform processing on a CTU basis without waiting for the subsequent CTU processing.

FIG. 15 is a block diagram of the SAO 3054 of the present invention. The difference from SAO 3052 in FIG. 8 is that a target area setting unit 1004 is added.

The target area setting unit 1004 sets pixels in the CTU (hatched area in FIG. 13A) excluding the right end one pixel and the lower end one pixel of the target CTU as a target area of EO. In this case, an area outside the target CTU of one pixel of the upper adjacent CTU of the target CTU and one pixel of the left adjacent CTU is used as a reference area. These are pixels in the already-decoded CTU (pixels of processed CTU), and are regions stored in the internal memory for use in intra prediction and deblocking filter in the target CTU. Therefore, when making the hatched area in FIG. 13A the target area of EO, there is no increase in additional memory and data transfer amount for SAO processing. Then, if a pre-filter decoded image obtained by adding the residual signal output from the inverse quantization / inverse transform unit 311 of the target CTU and the predicted image output from the predicted image generation unit 308 is input, the SAO process can be started. . Therefore, it is possible to reduce the processing delay of waiting for the processing of the subsequent CTU generated in the prior art to finish. Furthermore, by referring to the pixel values of the already decoded adjacent CTU as much as possible, it is possible to ease the restriction of the EO target area, and there is also an effect that the image quality can be maintained. The target area of BO (FIG. 13 (c)) is the entire area of CTU as in FIG. 12 (b).

The processes of the category setting unit 1001 and the offset addition unit 1003 are executed on the target area set by the target area setting unit 1004. The processes of the category setting unit 1001 and the offset addition unit 1003 are the same as those of the category setting unit 1001 and the offset addition unit 1003 in FIG.

FIG. 16 shows a flowchart of the SAO 3054.

The target area setting unit 1004 sets SAO target areas of EO and BO. The shaded area in FIG. 13A is set for EO, and the shaded area in FIG. 13C for BO (S1400).

The category setting unit 1001 checks the offset type sao_offset_type, and proceeds to S902 if EO, or proceeds to S905 if BO (S901). S902 to S904 are EO processing, S905 to S906 are BO processing, and S902 to S907 are the same operations as in FIG.

The offset addition unit 1003 adds the offset SaoOffsetVal corresponding to the category cat derived for each target pixel by the category setting unit 1001 to the target pixel, and carries out SAO (S1408).

As described above, the SAO can be processed in CTU units without using pixel values of subsequent CTUs, and delay can be reduced while maintaining image quality.

(Modification 1)
According to the first modification of the present invention, pixels in CTU (shaded area in FIG. 13B) excluding one pixel at the lower end of the target CTU are set as an EO target area. That is, the filter target area is x = 0..CTUWidth-1, y = 0 .. CTUHeight-2. In this case, a target CTU outside area corresponding to one pixel of the upper adjacent CTU of the target CTU, one pixel of the left adjacent CTU, and one pixel of the right adjacent CTU is required as a reference area. In order to refer to the pre-filter decoded pixel of the right adjacent CTU, a delay of 1 CTU occurs with respect to the target CTU. However, unlike the reference to the lower adjacent CTU where a delay corresponding to one CTU line occurs, the delay generated to reference the right adjacent CTU is small. Therefore, although the delay is increased by 1 CTU, the area of the right end of the target CTU is included in the target area of EO, so that the image quality is improved.

When the first modification is performed, the target area for EO set in the target area setting unit 1004 in FIG. 15 and S1400 in FIG. 16 is the hatched area in FIG. There is no other processing change.

By setting the SAO target area as in the first modification, although the delay is increased by 1 CTU compared to the case where the EO target area is set to FIG. 13A, EO can be performed on the pixel at the right end of the target CTU. . Therefore, the image quality can be maintained while the delay is reduced as compared with the prior art (FIG. 12A).

(Modification 2)
According to the second modification of the present invention, the pixels in the CTU (hatched area in FIG. 14A) excluding the left end one pixel and the right end pixel and the lower end pixel of the target CTU are set as the target area of EO. That is, the filter target area is x = 1..CTUWidth-2, y = 0..CTUHeight-2.

Alternatively, the CTU pixels (the shaded area in FIG. 14B) excluding the upper end one pixel and the right end pixel and the lower end pixel of the target CTU may be set as the EO target area. That is, the filter target area is x = 0..CTUWidth-2, y = 1..CTUHeight-2.

In these cases, the upper adjacent CTU of the target CTU or an area outside the target CTU for one pixel of the left adjacent CTU is required as a reference area. These areas are already stored in the internal memory. Therefore, when the hatched area in FIG. 14A or 14B is set as the target area, there is no increase in memory or data transfer amount, and no increase in delay. The target area of BO (FIG. 14D) is the entire area of CTU as in FIG. 12B.

When the second modification is performed, the target area for EO set in the target area setting unit 1004 in FIG. 15 and S1400 in FIG. 16 is the hatched area in FIG. 14A or 14B. There is no other processing change.

By setting the SAO target area as in the second modification, processing can be performed in CTU units without using pixel values of subsequent CTUs, and delay can be reduced.

(Modification 3)
The third modification of the present invention sets the EO target area to an area (shaded area in FIG. 14C) one pixel inward from the CTU boundary, and the BO target area to the entire CTU area (FIG. The hatched area shown in d) is set as in FIG. 12 (b)).

When the modification 3 is performed, the target area for EO set in the target area setting unit 1004 of FIG. 15 and S1400 of FIG. 16 is a hatched area of FIG. There is no other processing change.

By setting the SAO target area as in the third modification, the area required for the SAO including the reference area is limited to the inside of the CTU. Therefore, the delay can be reduced by simple processing without using pixel values of adjacent CTUs.

(Configuration of video coding device)
Next, the configuration of the moving picture coding apparatus 11 will be described. The configuration of the moving picture coding apparatus 11 will be described below as an example using FIG. The moving picture coding apparatus 11 includes a predicted image generation unit 101, a subtraction unit 102, a transform / quantization unit 103, an entropy coding unit 104, an inverse quantization / inverse transform unit 105, an addition unit 106, a loop filter 107, and prediction parameters. A memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, and a prediction parameter encoding unit 111 are included. The prediction parameter coding unit 111 includes an inter prediction parameter coding unit 112 and an intra prediction parameter coding unit 113.

The predicted image generation unit 101 generates, for each picture of the image T, a predicted image of PU for each coding unit CU, which is an area obtained by dividing the picture. Here, the predicted image generation unit 101 reads a decoded block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter coding unit 111. The prediction parameter input from the prediction parameter coding unit 111 is, for example, a motion vector in the case of inter prediction. The predicted image generation unit 101 reads a block at a position on a reference picture indicated by the motion vector starting from the target block, and generates a predicted image. In the case of intra prediction, the prediction parameter is, for example, an intra prediction mode. The predicted image generation unit 101 reads the pixel value of the adjacent block used in the intra prediction mode from the reference picture memory 109, and generates a predicted image. The predicted image generation unit 101 outputs the generated predicted image of the block to the subtraction unit 102 and the addition unit 106. The predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 described above.

The subtraction unit 102 subtracts the signal value of the prediction image of the block input from the prediction image generation unit 101 from the pixel value of the corresponding block position of the image T to generate a residual signal. The subtraction unit 102 outputs the generated residual signal to the transformation / quantization unit 103.

The transformation / quantization unit 103 performs frequency transformation on the residual signal input from the subtraction unit 102 to calculate transformation coefficients. The transform / quantization unit 103 quantizes the calculated transform coefficient to obtain a quantized transform coefficient. Transform / quantization section 103 outputs the obtained quantized transform coefficient to entropy coding section 104 and inverse quantization / inverse transform section 105.

The entropy coding unit 104 receives quantization transform coefficients from the transform / quantization unit 103 and receives prediction parameters from the prediction parameter coding unit 111. The entropy coding unit 104 entropy-codes the input division information, prediction parameters, quantized transform coefficients and the like to generate a coded stream Te, and outputs the generated coded stream Te to the outside. Also, the entropy coding unit 104 performs variable-length coding on the SAO parameters SAOP_S and SAOP_C supplied from the SAO_E 1072 and includes them in the coded stream.

The inverse quantization / inverse transform unit 105 is the same as the inverse quantization / inverse transform unit 311 (FIG. 4) in the video decoding device 31, and dequantizes the quantized transform coefficients input from the transform / quantization unit 103. Quantize to obtain transform coefficients. The inverse quantization / inverse transform unit 105 performs inverse transform on the obtained transform coefficient to calculate a residual signal. The inverse quantization / inverse transform unit 105 outputs the calculated residual signal to the addition unit 106.

The addition unit 106 adds the prediction image of the block input from the prediction image generation unit 101 and the residual signal input from the inverse quantization / inverse conversion unit 105 for each pixel to generate a decoded image. The addition unit 106 outputs the generated (pre-filter) decoded image to the loop filter 107.

The loop filter 107 applies a deblocking filter, a sample adaptive offset filter (SAO), and an adaptive loop filter (ALF) to the decoded image generated by the adding unit 106. The loop filter 107 may not necessarily include the three types of filters described above, and may have, for example, only a deblocking filter.

In the deblocking filter (E) 1071, SAO_E 1072, ALF (E) 1073, processing for determining filter parameters is added to the deblocking filter 3051, SAO 3052, ALF 3053, but the filter processing function is the same. The filter parameters are encoded and transmitted to the video decoding device 31.

(SAO in encoder)
In order to explain the operation of SAO in the image coding apparatus, a detailed block diagram of SAO_E 1072 in FIG. 17B is shown in FIG. The SAO_E 1072 includes an SAO information setting unit 1701 and an SAO 3052. The SAO 3052 is the same as in FIG. The SAO information setting unit 1701 includes an offset calculation unit 1702, an offset information selection unit 1703, and a merge information setting unit 1704.

First, the offset calculation unit 1702 will be described. The offset calculation unit 1702 calculates offsets of each offset class (0 to 3) of EO and each band of BO. When the offset type is EO, the category setting of each pixel is performed for each offset class, and the offset of the set category is calculated. In the case of BO, a band (category) belonging to each pixel is set, and the offset of the set category is calculated. A specific operation will be described using FIG.

The offset calculation unit 1702 initializes the offset class (class), count [class] [cat], and SAD [class] [cat] in S1801. Here, count [class] [cat] is a variable that counts the number of pixels per combination of (class, cat) in the CTU, and SAD [class] [cat] is decoding for each combination of (class, cat) in the CTU It is a variable that stores the absolute difference sum of the pixel value and the original pixel.

The offset calculation unit 1702 calculates the offset of EO in S1802 and S1803. In S1802, reference pixels a and b indicated by the offset class are set for the target pixel (x, y) to derive a category cat. Then, count [class] [cat] is incremented by one for the current offset class. Also, in the current offset class, the absolute difference between the SAO pre-decoded pixel value rec [x] [y] and the original pixel org [x] [y] is added to SAD [class] [cat].

SAD [class] [cat] + = abs (rec [x] [y] -org [x] [y]) (Formula SAO-9)
The above processing is performed on the target area in the CTU.

The offset calculation unit 1702 calculates offset offset [class] [cat] for categories 1 to 4 in S1803 using the following equation, and increments the offset class by one.

offset [class] [cat] = SAD [class] [cat] / count [class] [cat] (Expression SAO-10)
The category 0 offset is set to 0.

The offset calculation unit 1702 proceeds to S1805 if class = 4 (all EO ends) in S1804, and otherwise repeats S1802 and S1803.

The offset calculation unit 1702 calculates the offset of BO in S1805 and S1806. The band i to which the SAO pre-decoded pixel value rec [x] [y] of the target pixel belongs is obtained in S1805, and the difference sum SAD [class] [i] of the band i is rec [x] [y] and org [x] [ Add the difference of y]. Also, the count of band i is incremented by one.

i = rec [x] [y] >> bandShift (Expression SAO-11)
SAD [class] [i] + = (rec [x] [y] -org [x] [y])
count [class] [i] ++
Here, bandShift is log 2 (bandW) and bandW is the bandwidth of BO.

The offset calculation unit 1702 calculates offsets for 32 bands i (i = 0 to 31) in S1806.

offset [class] [i] = SAD [class] [i] / count [class] [i] (equation SAO-12)
As described above, the offset of each category (cat = 1 to 4) of EO (class = 0 to 3) and the offset of each band (i = 0 to 31) of BO (class = 4) were calculated.

The offset information selection unit 1703 uses the offset calculated by the offset calculation unit 1702 to select an offset type (EO / BO), an offset class in the case of EO, and a band position in the case of BO. FIG. 20 is a flowchart showing the operation of the offset information selection unit 1703. S1902 to S1904 are processes relating to EO, and S1905 to S1907 are processes relating to BO.

The offset information selection unit 1703 initializes the offset class class (0 to 3) and the absolute difference sum SAD [class] in S1901. The absolute difference sum SAD [class] is a variable that stores the absolute difference sum of the pixel value to which SAO is applied (to which the offset is added) and the original pixel value.

The absolute difference sum SAD [class] sets the reference pixels a and b indicated by the offset class in S1902, derives the category for each pixel of CTU by the method described above, and uses the offset assigned to each category Calculate the absolute difference sum SAD.

SAD [class] + = abs (rec [x] [y] + offset [class] [cat] -org [x] [y]) (Formula SAO-13)
The category 0 offset is set to 0. rec [x] [y] + offset [class] [cat] is the value of SAO at the time of combination of (class, cat), and the absolute difference between this and the original pixel value org [x] [y] is offset class Add to SAD every time. When the processing of the target area in the CTU is completed, the offset class is incremented by one.

The offset information selection unit 1703 checks in step S1903 whether the offset class is equal to four. If the offset class is not equal to 4, the processing of S1902 is performed until it is 4. If the offset class is equal to 4, processing of all offset classes of EO is completed, and the process advances to step S1904.

The offset information selection unit 1703 obtains i at which SAD [i] (i = 0 to 3) becomes minimum at S1904, and sets it as an EO offset class.

SAD _EO = min (SAD [i]), class = i that satisfies the left (Equation SAO-14)
In step S1905, the offset information selection unit 1703 calculates, for each band of CTUs, the sum of absolute differences between pixel values to which SAO is applied (offset is added) and original pixel values. Specifically, using the band i to which the target pixel (x, y) belongs and the offset offset [class] [i] of each band obtained by the offset calculation unit 1702, the band i has four consecutive bands (j to If it belongs to any of j + 3), the absolute difference sum of the pixel value rec [x] [y] + oft to which SAO is applied and the original pixel value org [x] [y] is calculated. here,
i = rec [X] >> bandShift (equation SAO-15)
oft = offset [class = 4] [j] (j <= i <= j + 3)
= 0 (otherwise)
SAD [i] = abs (rec [x] [y] + oft-org [x] [y])
I assume. After the above processing is performed on the CTU target area, j is incremented by one.

The offset information selection unit 1703 checks in step S1906 whether i <29. If j <29, there is a band that has not been processed, so S1905 is repeated until j = 29. If j = 29, processing has ended for all bands, and the process advances to step S1907.

In S1907, the offset information selection unit 1703 sets j (j = 0 to 31) that minimizes SAD [j] to the band position (sao_band_position) of the BO of the current CTU.

SAD _BO = min (SAD [j]), band_position = j that satisfies the left (Equation SAO-16)
In S1908, the offset information selection unit 1703 compares the magnitude relationship between SAO _EO and SAO _BO . If the SAO _BO is large, step S1909 follows and the offset type (type) of the current CTU is set to EO. If not, it proceeds to S1910 and sets the offset type (type) of the current CTU to BO.

The above is the processing performed by the offset information selection unit 1703.

The offset information (offset, offset type, offset class, band position) calculated by the SAO information setting unit 1701 is input to the SAO 3052. The operation of the SAO 3052 is the same as the operation of the SAO 3052 of the video decoding device 31, and the description thereof is omitted.

(Merge control information determination unit 1704)
If the SAO parameter of the target CTU derived by the offset calculation unit 1702 and the offset information selection unit 1703 is equal to the SAO parameter of the adjacent CTU already encoded, the merge control information (sao_merge_left_flag, sao_merge_up_flag) is set to 1. If the SAO parameter of the target CTU is not equal to the SAO parameter of the adjacent CTU, the merge control information is set to 0. More specifically, if the target CTU and the left adjacent CTU have the same SAO parameters, sao_merge_left_flag = 1, and if the target CTU and the upper adjacent CTU have the same SAO parameters, set sao_merge_up_flag = 1, otherwise Sets the merge control information to 0.

Next, SAO_E 1074 corresponding to SAO 3054 of the present invention will be described.

FIG. 11B is a block diagram of the loop filter 107E (in the moving picture coding apparatus) in the present embodiment. The loop filter 107E shown in FIG. 11B is a replacement of the loop filter 107 in FIG. 17, and the input and output are also the same as the loop filter 107. That is, the input to the loop filter is a pre-filter decoded image, and the output from the loop filter is a loop-filtered decoded image. In the loop filter 107E, the configurations of the deblocking filter 1071 and the ALF 1073 are the same as the configurations of the deblocking filter 1071 and the ALF 1073 in FIG. FIG. 11C is a flowchart showing the operation of the loop filter 107E. The flow of the process is the same as that of the loop filter 305D, so the description will be omitted.

Similar to the SAO 3054, the SAO_E 1074 sets the EO target area to the CTU pixels (hatched area in FIG. 13A) excluding the right end pixel and the lower end pixel of the target CTU. Further, the target region of BO is a hatched region in FIG. 13C, which is all pixels of the target CTU.

FIG. 21 is a block diagram of SAO_E 1074 of the present invention. The difference from SAO_E 1072 in FIG. 18 is that a target area setting unit 1705 is added. The target area setting unit 1705 sets a shaded area in FIG. 13A as a target area of SAO. The target area setting unit 1705 is the same as the target area setting unit 1004 in FIG. The offset calculation unit 1702, the offset information selection unit 1703, and the merge information setting unit 1704 are the same as the offset calculation unit 1702, the offset information selection unit 1703, and the merge information setting unit 1704 in FIG. Each process is performed on the target area of the SAO. The SAO 3054 is the same as the SAO 3054 in FIG.

By limiting the EO target area to CTU pixels excluding the right end pixel and the lower end pixel of the target CTU, SAO_E 1074 can derive SAO parameters SAOP_S and SAOP_C for each CTU for both EO and BO. . Therefore, the delay of the encoding process can be reduced. In addition, since the number of pixels that can be used for calculating the SAO parameter increases compared to the case where the deblocking filtered pixel is used, the accuracy of the SAO parameter can be improved.

(Modification 4)
SAO_E 1074 corresponding to SAO 3054 of Modification 1 of the present invention will be described in Modification 4 of the present invention.

When the modification 4 is performed, the target region for EO set by the target region setting unit 1705 in FIG. 21 is a CTU pixel (hatched region in FIG. 13B) excluding one lower end pixel of the target CTU. . There is no other processing change.

By setting the EO target area to pixels in the CTU excluding the lower end one pixel of the target CTU, a delay of 1 CTU occurs compared to the case where the EO target area is set to the hatched area in FIG. . However, since the EO can be performed on the rightmost pixel of the target CTU, the image quality is improved. The delay of 1 CTU is sufficiently smaller than the delay of 1 CTU line which occurs when the target area of EO is set as shown in FIG. 12A. Therefore, the delay can be reduced while maintaining the image quality as compared with the prior art (FIG. 12A).

(Modification 5)
SAO_E 1074 corresponding to SAO 3054 of Modification 2 of the present invention will be described in Modification 5 of the present invention.

When the modification 5 is performed, the target area for EO set by the target area setting unit 1705 in FIG. 21 is a pixel in CTU excluding one pixel at the left end and one pixel at the right end and one lower end of the target CTU (FIG. 14 The shaded area in (a)). Alternatively, the CTU pixels (the shaded area in FIG. 14B) excluding the upper end one pixel, the right end pixel, and the lower end pixel of the target CTU may be excluded. There is no other processing change. In these cases, the delay does not increase because the subsequent CTUs of the target CTU are not referenced.

By setting the EO target area as in the fifth modification, processing on a CTU basis can be enabled, and the delay can be reduced.

(Modification 6)
SAO_E 1074 corresponding to SAO 3054 of Modification 3 of the present invention will be described in Modification 6 of the present invention.

When the sixth modification is performed, the target region for EO set by the target region setting unit 1705 in FIG. 21 is a hatched region (hatched region in FIG. 14C) which is located one pixel inside from the CTU boundary. There is no other processing change.

By setting the EO target area as in the sixth modification, the area required for the SAO including the reference area is limited to the area inside the CTU. Therefore, the delay can be reduced by simple processing.

The prediction parameter memory 108 stores the prediction parameter generated by the coding parameter determination unit 110 in a predetermined position for each picture and CU to be coded.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 in a predetermined position for each picture and CU to be encoded.

The coding parameter determination unit 110 selects one of a plurality of sets of coding parameters. The coding parameter is the QT or BT division parameter or prediction parameter described above, or a parameter to be coded that is generated in association with these. The predicted image generation unit 101 generates a predicted image of a block using each of these sets of coding parameters.

The coding parameter determination unit 110 calculates an RD cost value indicating the size of the information amount and the coding error for each of the plurality of sets. The coding parameter determination unit 110 selects a set of coding parameters that minimize the calculated RD cost value. Thereby, the entropy coding unit 104 externally outputs the set of selected coding parameters as the coded stream Te, and does not output the set of non-selected coding parameters. The coding parameter determination unit 110 stores the determined coding parameters in the prediction parameter memory 108.

The prediction parameter coding unit 111 derives a format for coding from the parameters input from the coding parameter determination unit 110, and outputs the format to the entropy coding unit 104. Further, the prediction parameter coding unit 111 derives parameters necessary to generate a prediction image from the parameters input from the coding parameter determination unit 110, and outputs the parameters to the prediction image generation unit 101.

The inter prediction parameter coding unit 112 derives inter prediction parameters such as a difference vector based on the prediction parameters input from the coding parameter determination unit 110. The inter prediction parameter coding unit 112 is partially identical to the configuration in which the inter prediction parameter decoding unit 303 derives the inter prediction parameter, as a configuration for deriving a parameter necessary for generating a predicted image to be output to the predicted image generation unit 101. Includes configuration.

Further, the intra prediction parameter coding unit 113 derives the prediction parameters necessary for generating the prediction image to be output to the prediction image generation unit 101, the intra prediction parameter decoding unit 304 derives the intra prediction parameter, and Some include the same configuration.

Second Embodiment
In the present embodiment, another technique will be described in which the SAO process is performed before the deblocking filter, and the deblocking filter is performed on the SAO-decoded image.

In this embodiment, FIG. 22 is used instead of FIG. 7 as the syntax of SAO. In the syntax shown in FIG. 22, in EO, the positive / negative sign (sao_offset_sign) of the offset SaoOffsetVal derived for each CTU can be notified (SYN 2201). In the first embodiment, the sign of the EO offset SaoOffsetVal is not notified, and the sign is uniquely determined by the category. That is, in the case of the categories 1 and 2 shown in FIG. 6B, sao_offset_sign = 0 (indicating positive) is set, and in the case of the categories 3 and 4, sao_offset_sign = 1 (indicating negative) is set. Therefore, EO had the effect of removing ringing distortion, but had no edge enhancement effect. In the second embodiment, even in EO, sao_offset_sign is notified, and filtering is performed using positive and negative offsets, thereby providing EO with an edge emphasizing effect other than the ringing removal effect.

When the unfiltered decoded image obtained by adding the residual signal output from the inverse quantization / inverse transform unit 311 of the target CTU and the predicted image output from the predicted image generation unit 308 includes coding distortion There is. On the other hand, the deblocking filter changes the on / off and the filter strength depending on the pixel value of the block boundary. Therefore, in the case of edge enhancement by EO, if the output of SAO is affected by coding distortion, there is a possibility that the deblocking filter implemented after SAO may make an erroneous determination. For example, as a result of SAO edge-enhancing block distortion, the deblocking filter considers block distortion as an edge and determines that the filter is off. Moreover, when applying SAO of the image before a deblocking filter process, block distortion may be emphasized and a subjective image quality may fall.

Therefore, in the second embodiment, when EO is selected and the offset has an effect of emphasizing an edge, it is generated in the subsequent deblocking filter by not applying (turning off) SAO at the block boundary. Avoid possible image quality degradation. Here, the block is a unit to be processed by the deblocking filter, and is, for example, a PU or a TU.

The offset addition unit 10032 does not apply the filtering process to the target pixel when EO, and the target pixel is a block boundary pixel, (category 1 or 2), and sao_offset_sign = 1 (indicating negative). When the target pixel is a block boundary pixel, and categories 1 and 2 (category 3 or 4) and sao_offset_sign = 0 (indicating positive), the filtering process is not applied to the target pixel.

In other words, in the case of EO, the filtering process is applied to the target pixel when the target pixel is other than the block boundary pixel or ((category 1 or 2) and sao_offset_sign = 1). When the target pixel is a block boundary pixel or ((category 3 or 4) and sao_offset_sign = 0 other than 0), the filter processing is applied to the target pixel.

FIG. 23 is a flowchart showing the operation of the offset addition unit 10032 of this embodiment. The offset addition unit 10032 replaces the offset addition unit 1003 in FIGS. 8 and 15. Although the offset addition unit 1003 only performs the process of adding the offset as shown in S907 of FIG. 9 and S1408 of FIG. 16, the offset addition unit 10032 performs the process shown in FIG.

The offset addition unit S10032 determines in S2301 whether the offset type is EO. If the offset type is not EO (NO in S2301), the process proceeds to S2304. If the offset type is EO (YES in S2301), the process advances to S2302.

The offset addition unit S10032 determines Condition 1 (relationship between category and offset) in S2302. If (category 1 or 2) and sao_offset_sign = 1 (indicating a negative) or (category 3 or 4) and sao_offset_sign = 0 (indicating a positive) (YES in S2302), the process proceeds to S2303. Otherwise (NO in S2302), the process proceeds to S2304.

The offset addition unit S10032 determines in S2303 whether or not the target pixel is at a block boundary. If the target pixel is not at the block boundary (NO in S2303), the process proceeds to S2304. If the target pixel is at a block boundary (YES in S2303), the process ends without adding the offset to the target pixel. Here, whether or not the target pixel (x, y) is at the block boundary can be determined, for example, by the following equation.

if (x% CUWidth == 0 && y% CUHeight == 0) (Expression SAO-17)
Here, CUWidth and CUHeight are the width and height of the block. If (Equation SAO-17) is true, the target pixel (x, y) is at a block boundary, otherwise the target pixel (x, y) is not at a block boundary.

The offset addition unit S 10032 calculates the pixel value of the target pixel by (Expression SAO-8) in S 2304. The process of S2304 is the same as S907 of FIG. 9 and S1408 of FIG.

The determination of the target pixel position may not be a block boundary but may be a CTU boundary. In this case, the block width CUWidth and the block height CUHeight of (Expression SAO-17) are replaced by the CTU width CTUWidth and the CTU height CTUHeight.

As described above, in the present embodiment, when EO is selected and sao_offset_sign indicates a negative sign, image quality that may occur in the subsequent deblocking filter by turning off SAO at the block boundary Avoid degradation.

An image filter device according to an aspect of the present invention is an image filter device that divides an image into blocks of a predetermined unit and adds an offset to each pixel value, and the block (target block) of a predetermined unit of an input image The first offset type that derives the category classification of the target pixel according to the magnitude relationship between the target pixel and the adjacent pixel, or the second offset that classifies the category classification of the target pixel according to the size of the pixel value of the target pixel The category of each pixel is set using the means for assigning one of two offset types, ie, the type, the means for setting the filter target area for performing the filtering process of the target block, and the input image and the offset type; For each pixel, and an offset corresponding to a category set according to the offset. The input image is an image obtained by adding a prediction image generated by intra prediction processing or inter prediction processing and a prediction residual signal, and the category is set; The means for deriving the offset sets the category using the target pixel value and the two reference pixel values designated by the offset class when the target block is a predetermined first offset type, and the offset is set for each category. When the target block is a predetermined second offset type, a category is set from the band to which the target pixel value belongs, and an offset is derived for each category.

In the image filter device according to one aspect of the present invention, the means for setting the filter target area of the target block includes an area to be set when the target block is the first offset type, and an offset type of the target block. A different area is set to the area to be set in the case of

In the image filter device according to one aspect of the present invention, the means for setting the filter target area of the target block is a filter target area of the target block of the target block when the target block is the second offset type. Set to all areas.

In the image filter device according to one aspect of the present invention, the means for setting the filter target area of the target block is a filter target area of the target block, the target block boundary when the target block is the first offset type. It is set to an area that is inside by 1 pixel.

In the image filter device according to one aspect of the present invention, the means for setting the filter target area of the target block is a filter target area of the target block of the target block when the target block is the first offset type. The target block is set except for the right end one pixel and the lower end one pixel.

In the image filter device according to one aspect of the present invention, the means for setting the filter target area of the target block is a filter target area of the target block of the target block when the target block is the first offset type. The target block is set except for the left end one pixel, the right end one pixel, and the lower end one pixel.

In the image filter device according to one aspect of the present invention, the means for setting the filter target area of the target block is a filter target area of the target block of the target block when the target block is the first offset type. Set as the target block excluding the lower 1 pixel.

The image filter device according to an aspect of the present invention implements a deblocking filter on the output of the adaptive offset filter SAO.

The image filter device according to an aspect of the present invention performs a deblocking filter on the output of the adaptive offset filter SAO, and further performs an adaptive loop filter ALF on the output of the deblocking filter.

In an image filter device according to an aspect of the present invention, an image filter device that divides an image into blocks of a predetermined unit and adds an offset to each pixel value, and blocks (target block) a predetermined unit of an input image The first offset type that derives the category classification of the target pixel according to the magnitude relationship between the target pixel and the adjacent pixel, or the second offset that classifies the category classification of the target pixel according to the size of the pixel value of the target pixel Means for assigning one of two offset types, ie, type, setting the category of each pixel using the input image and the offset type, means for deriving the offset, and corresponding to the category set according to the offset Means for performing a filtering process by adding the offsets to each pixel. It is an image obtained by adding a prediction image generated by intra prediction processing or inter prediction processing and a prediction residual signal, and the means for performing the filter processing has an offset when the target block is a predetermined first offset type. If it is a negative value, filtering is turned off at the pixel of the block boundary of the target block.

In an image filter device according to an aspect of the present invention, an image filter device that divides an image into blocks of a predetermined unit and adds an offset to each pixel value, and blocks (target block) a predetermined unit of an input image The first offset type that derives the category classification of the target pixel according to the magnitude relationship between the target pixel and the adjacent pixel, or the second offset that classifies the category classification of the target pixel according to the size of the pixel value of the target pixel Means for assigning one of two offset types, ie, type, setting the category of each pixel using the input image and the offset type, means for deriving the offset, and corresponding to the category set according to the offset Means for performing a filtering process by adding the offsets to each pixel. It is an image obtained by adding a prediction image generated by intra prediction processing or inter prediction processing and a prediction residual signal, and the means for performing the filter processing is a predetermined one when the target block is a predetermined first offset type. If the condition is satisfied, the filtering process is turned off at the pixel at the subblock boundary of the target block, and the subblock is a processing unit of the deblocking filter.

In the image filter device according to one aspect of the present invention, the predetermined condition is that the offset is positive when the category is 1 or 2, or negative when the category is 3 or 4. is there.

(Example of software implementation)
Note that the moving picture coding apparatus 11 and a part of the moving picture decoding apparatus 31 in the embodiment described above, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, the dequantization Inverse transformation unit 311, addition unit 312, predicted image generation unit 101, subtraction unit 102, transformation / quantization unit 103, entropy encoding unit 104, inverse quantization / inverse transformation unit 105, loop filter 107, encoding parameter determination unit 110, the prediction parameter coding unit 111 may be realized by a computer. In that case, a program for realizing the control function may be recorded in a computer readable recording medium, and the computer system may read and execute the program recorded in the recording medium. Here, the “computer system” is a computer system built in any of the moving image encoding device 11 and the moving image decoding device 31, and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” means a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in a computer system. Furthermore, the “computer-readable recording medium” is one that holds a program dynamically for a short time, like a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line. In such a case, a volatile memory in a computer system serving as a server or a client may be included, which holds a program for a predetermined time. The program may be for realizing a part of the functions described above, or may be realized in combination with the program already recorded in the computer system.

In addition, part or all of the video encoding device 11 and the video decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the moving picture coding device 11 and the moving picture decoding device 31 may be individually processorized, or part or all may be integrated and processorized. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. In the case where an integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology, integrated circuits based on such technology may also be used.

As mentioned above, although one embodiment of this invention was described in detail with reference to drawings, a specific structure is not restricted to the above-mentioned thing, Various design changes etc. in the range which does not deviate from the summary of this invention It is possible to

[Application example]
The moving image encoding device 11 and the moving image decoding device 31 described above can be mounted and used in various devices that transmit, receive, record, and reproduce moving images. The moving image may be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like.

First, the fact that the above-described moving picture coding apparatus 11 and moving picture decoding apparatus 31 can be used for transmission and reception of moving pictures will be described with reference to FIG.

FIG. 24A is a block diagram showing the configuration of a transmitter PROD_A on which the moving picture coding apparatus 11 is mounted. As shown in FIG. 24 (a), the transmitter PROD_A modulates a carrier wave with an encoding unit PROD_A1 which obtains an encoded stream by encoding a moving image, and the encoding stream obtained by the encoding unit PROD_A1. The modulation unit PROD_A2 for obtaining the modulation signal by the above-mentioned and the transmission unit PROD_A3 for transmitting the modulation signal obtained by the modulation unit PROD_A2. The above-described moving picture coding apparatus 11 is used as the coding unit PROD_A1.

The transmission device PROD_A is a camera PROD_A4 for capturing a moving image, a recording medium PROD_A5 for recording the moving image, an input terminal PROD_A6 for externally inputting the moving image, and a transmission source of the moving image input to the encoding unit PROD_A1. , And may further include an image processing unit PRED_A7 that generates or processes an image. Although FIG. 24A illustrates the configuration in which the transmission device PROD_A includes all of the above, a part of the configuration may be omitted.

Note that the recording medium PROD_A5 may be a recording of a non-coded moving image, or a moving image encoded by a recording encoding method different from the transmission encoding method. It may be one. In the latter case, a decoding unit (not shown) may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1 for decoding the encoded stream read from the recording medium PROD_A5 according to the encoding scheme for recording.

FIG. 24 (b) is a block diagram showing the configuration of the reception device PROD_B on which the video decoding device 31 is mounted. As shown in FIG. 24 (b), the receiving device PROD_B receives the modulated signal, receives the modulated signal, demodulates the modulated signal received by the receiving unit PROD_B1, obtains the encoded stream, and demodulates the demodulated signal; And a decoding unit PROD_B3 for obtaining a moving image by decoding the encoded stream obtained by PROD_B2. The above-described moving picture decoding apparatus 31 is used as the decoding unit PROD_B3.

The receiving device PROD_B is a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. It may further comprise PROD_B6. Although FIG. 24 (b) exemplifies the configuration in which the receiving device PROD_B includes all of the above, a part of the configuration may be omitted.

Incidentally, the recording medium PROD_B5 may be for recording a moving image which has not been encoded, or is encoded by a recording encoding method different from the transmission encoding method. May be In the latter case, an encoding unit (not shown) may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5 to encode the moving image acquired from the decoding unit PROD_B3 according to the encoding method for recording.

The transmission medium for transmitting the modulation signal may be wireless or wired. Further, the transmission mode for transmitting the modulation signal may be broadcast (here, a transmission mode in which the transmission destination is not specified in advance), or communication (in this case, transmission in which the transmission destination is specified in advance) (Refer to an aspect). That is, transmission of the modulation signal may be realized by any of wireless broadcast, wired broadcast, wireless communication, and wired communication.

For example, a broadcasting station (broadcasting facility etc.) / Receiving station (television receiver etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B which transmits and receives a modulated signal by wireless broadcasting. A cable television broadcast station (broadcasting facility or the like) / receiving station (television receiver or the like) is an example of a transmitting device PROD_A / receiving device PROD_B which transmits and receives a modulated signal by cable broadcasting.

In addition, a server (such as a workstation) / client (television receiver, personal computer, smart phone, etc.) such as a VOD (Video On Demand) service or a video sharing service using the Internet is a transmitting device that transmits and receives modulated signals by communication. This is an example of PROD_A / receiving device PROD_B (Normally, in a LAN, either wireless or wired is used as a transmission medium, and in WAN, wired is used as a transmission medium). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. The smartphone also includes a multifunctional mobile phone terminal.

In addition to the function of decoding the encoded stream downloaded from the server and displaying it on the display, the client of the moving image sharing service has a function of encoding the moving image captured by the camera and uploading it to the server. That is, the client of the moving image sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, the fact that the above-described moving picture coding apparatus 11 and moving picture decoding apparatus 31 can be used for recording and reproduction of moving pictures will be described with reference to FIG.

FIG. 25A is a block diagram showing the configuration of a recording device PROD_C on which the above-described moving picture coding device 11 is mounted. As shown in FIG. 25A, the recording device PROD_C writes the encoded stream obtained by the encoding unit PROD_C1 for obtaining an encoded stream by encoding a moving image and the encoding unit PROD_C1 to the recording medium PROD_M. And a writing unit PROD_C2. The above-described moving picture coding apparatus 11 is used as the coding unit PROD_C1.

The recording medium PROD_M may be (1) a type incorporated in the recording device PROD_C, such as a hard disk drive (HDD) or a solid state drive (SSD), or (2) an SD memory. It may be of a type connected to the recording device PROD_C, such as a card or Universal Serial Bus (USB) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray (registered trademark) A disc such as a registered trademark) may be loaded into a drive device (not shown) built in the recording device PROD_C.

In addition, the recording device PROD_C is a camera PROD_C3 for capturing a moving image as a supply source of the moving image input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting the moving image from the outside, and a reception for receiving the moving image The image processing unit PROD_C5 may further include an image processing unit PROD_C6 that generates or processes an image. Although FIG. 25A exemplifies the configuration in which the recording apparatus PROD_C includes all of the above, a part of the configuration may be omitted.

Note that the receiving unit PROD_C5 may receive a non-coded moving image, and receives a coded stream coded by a transmission coding scheme different from the recording coding scheme. It may be In the latter case, it is preferable to interpose a transmission decoding unit (not shown) that decodes the coded stream coded by the transmission coding scheme between the reception unit PROD_C5 and the coding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, etc. (In this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images). . In addition, a camcorder (in this case, the camera PROD_C3 is the main supply source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main supply source of moving images), a smartphone (this In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images) and the like are also examples of such a recording device PROD_C.

FIG. 25 (b) is a block showing the configuration of the playback device PROD_D on which the above-described moving picture decoding device 31 is mounted. As shown in FIG. 25B, the playback device PROD_D obtains a moving image by decoding a coded stream read by the reading unit PROD_D1 that reads the coded stream written in the recording medium PROD_M and the reading unit PROD_D1. And a decryption unit PROD_D2. The above-described moving picture decoding apparatus 31 is used as the decoding unit PROD_D2.

The recording medium PROD_M may be (1) a type incorporated in the playback device PROD_D such as an HDD or an SSD, or (2) such as an SD memory card or a USB flash memory. It may be of a type connected to the playback device PROD_D, or (3) it may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.

In addition, the playback device PROD_D is a display PROD_D3 that displays a moving image as a supply destination of the moving image output by the decoding unit PROD_D2, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image. It may further comprise PROD_D5. Although FIG. 25 (b) exemplifies the configuration in which the playback device PROD_D includes all of these, part of the configuration may be omitted.

The transmitting unit PROD_D5 may transmit an uncoded moving image, or transmit an encoded stream encoded by a transmission coding scheme different from the recording coding scheme. It may be In the latter case, an encoding unit (not shown) may be interposed between the decoding unit PROD_D2 and the transmission unit PROD_D5 for encoding moving pictures according to a transmission encoding scheme.

As such a playback device PROD_D, for example, a DVD player, a BD player, an HDD player, etc. may be mentioned (in this case, the output terminal PROD_D4 to which a television receiver etc. is connected is the main supply destination of moving images) . In addition, television receivers (in this case, the display PROD_D3 is the main supply destination of moving images), digital signage (also referred to as an electronic signboard or electronic bulletin board, etc.), the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images. First, desktop type PC (in this case, output terminal PROD_D4 or transmission unit PROD_D5 is the main supply destination of moving images), laptop type or tablet type PC (in this case, display PROD_D3 or transmission unit PROD_D5 is moving image) The main supply destination of the image), the smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the moving image), and the like are also examples of such a reproduction device PROD_D.

(Hardware realization and software realization)
In addition, each block of the above-described moving picture decoding apparatus 31 and moving picture coding apparatus 11 may be realized as hardware by a logic circuit formed on an integrated circuit (IC chip), or CPU (Central Processing). It may be realized as software using Unit).

In the latter case, each of the devices described above includes a CPU that executes instructions of a program that implements each function, a read only memory (ROM) that stores the program, a random access memory (RAM) that develops the program, the program, and various data. And a storage device (recording medium) such as a memory for storing the The object of the embodiment of the present invention is to record computer program readable program codes (execution format program, intermediate code program, source program) of control programs of the above-mentioned respective devices which are software for realizing the functions described above. The present invention can also be achieved by supplying a medium to each of the above-described devices, and a computer (or a CPU or an MPU) reading and executing a program code recorded on a recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, CDs (Compact Disc Read-Only Memory) / MO disks (Magneto-Optical disc). / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (Blu-ray (registered trademark) Disc: registered trademark), etc. Disc including optical disc, IC card (memory) Cards) / cards such as optical cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory (registered trademark)) / semiconductor memory such as flash ROM, or PLD Logic circuits such as programmable logic device (FPGA) and field programmable gate array (FPGA) can be used.

Further, each device may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. This communication network is not particularly limited as long as the program code can be transmitted. For example, the Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / CableTelevision) communication network, Virtual Private Network (Virtual Private Network) A telephone network, a mobile communication network, a satellite communication network, etc. can be used. Also, the transmission medium that constitutes this communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even if wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared such as IrDA (Infrared Data Association) or remote control, BlueTooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (registered trademark) (Digital Living Network Alliance (registered trademark)), cellular phone network, satellite link, digital terrestrial broadcasting It can also be used by wireless such as networks. The embodiment of the present invention may also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

Embodiments of the present invention are not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. That is, an embodiment obtained by combining technical means appropriately modified within the scope of the claims is also included in the technical scope of the present invention.

An embodiment of the present invention is suitably applied to a video decoding apparatus that decodes a coded stream obtained by coding image data, and a video coding apparatus that generates a coded stream obtained by coding image data. be able to. Further, the present invention can be suitably applied to the data structure of a coded stream generated by the moving picture coding apparatus and referred to by the moving picture decoding apparatus.

41 Moving image display device
31 Moving picture decoding device
11 Video Coder

Claims (9)

In an image filter apparatus that divides an image into blocks of predetermined units and adds an offset to each pixel value,
The first offset type that derives the category classification of the target pixel according to the magnitude relationship between the target pixel and the adjacent pixel in a predetermined unit block (target block) of the input image, or the size of the pixel value of the target pixel And means for assigning one of two offset types, a second offset type for deriving the category classification of the target pixel,
Means for setting different filter target areas in the target block according to the offset type;
Means for setting the category of each pixel using the input image and the offset type, and deriving the offset;
Means for performing filtering on the filter target area of the target block by adding to each pixel an offset corresponding to a category set according to the offset type;
The input image is an image obtained by adding a prediction image generated by intra prediction processing or inter prediction processing and a prediction residual signal,
The means for setting the category and deriving the offset is
If the offset type of the target block is the first offset type, a category is set using the target pixel value and two reference pixel values specified in the offset class, and the offset is derived for each category,
An image filter apparatus which sets a category from a band to which a target pixel value belongs and derives an offset for each category when the offset type of the target block is the second offset type. The means for setting the filter target area of the target block sets the filter target area of the target block to the entire area of the target block when the offset type of the target block is the second offset type. The image filter device according to claim 1, characterized in that: When the offset type of the target block is the first offset type, the means for setting the filter target region of the target block is such that the filter target region of the target block is one pixel inside of the target block boundary. The image filter device according to claim 1, wherein the image filter device is set in an area in which When the offset type of the target block is the first offset type, the means for setting the filter target region of the target block includes the filter target region of the target block and the right end one pixel of the target block. 2. The image filter device according to claim 1, wherein the block is set to a block excluding one pixel at the lower end. When the offset type of the target block is the first offset type, the means for setting the filter target region of the target block is the filter target region of the target block for the left end one pixel of the target block, The image filter device according to claim 1, wherein the block is set to a block excluding one pixel at the right end and one pixel at the lower end. When the offset type of the target block is the first offset type, the means for setting the filter target region of the target block, the filter target region of the target block, the lower 1 pixel of the target block The image filter apparatus according to claim 1, wherein the image filter is set to a block to be excluded. In an image filter apparatus that divides an image into blocks of predetermined units and adds an offset to each pixel value,
The first offset type that derives the category classification of the target pixel according to the magnitude relationship between the target pixel and the adjacent pixel in a predetermined unit block (target block) of the input image, or the size of the pixel value of the target pixel And means for assigning one of two offset types, a second offset type for deriving the category classification of the target pixel,
Means for setting the category of each pixel using the input image and the offset type, and deriving the offset;
Means for performing filtering by adding an offset corresponding to a category set according to the offset type to each pixel,
The input image is an image obtained by adding a prediction image generated by intra prediction processing or inter prediction processing and a prediction residual signal,
When the target block is a predetermined first offset type, the means for performing the filter processing turns off the filter processing at the pixel of the block boundary of the target block if the predetermined condition is satisfied. Filter device. An image decoding apparatus comprising the image filter apparatus according to any one of claims 1 to 7. An image coding apparatus comprising the image filter apparatus according to any one of claims 1 to 7.

Images