[go: up one dir, main page]

WO2012081706A1 - Dispositif de filtre d'image, dispositif de filtre, décodeur, codeur et structure de données - Google Patents

Dispositif de filtre d'image, dispositif de filtre, décodeur, codeur et structure de données Download PDF

Info

Publication number
WO2012081706A1
WO2012081706A1 PCT/JP2011/079232 JP2011079232W WO2012081706A1 WO 2012081706 A1 WO2012081706 A1 WO 2012081706A1 JP 2011079232 W JP2011079232 W JP 2011079232W WO 2012081706 A1 WO2012081706 A1 WO 2012081706A1
Authority
WO
WIPO (PCT)
Prior art keywords
filter
unit
image
region
characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2011/079232
Other languages
English (en)
Japanese (ja)
Inventor
知宏 猪飼
隆紀 山崎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sharp Corp
Original Assignee
Sharp Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Corp filed Critical Sharp Corp
Publication of WO2012081706A1 publication Critical patent/WO2012081706A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing

Definitions

  • the present invention relates to an image filter device and a filter device that perform image filtering.
  • the present invention also relates to an image filter device and an encoding device and a decoding device including the filter device.
  • the present invention also relates to a data structure of encoded data decoded by such a decoding device.
  • a moving image encoding device that generates encoded data by encoding the moving image, and decoding by decoding the encoded data
  • a video decoding device (decoding device) that generates an image is used.
  • As a specific moving picture encoding method for example, H.264 is used. H.264 / MPEG-4.
  • AVC Non-Patent Document 1
  • VCEG Video Coding Expert Group
  • KTA Key Integrity Expert Group
  • TMuC Test Model Under Consideration
  • an image (picture) that constitutes a moving image is a slice obtained by dividing the image, a maximum coding unit (LCU: Largegest Coding Unit) obtained by dividing the slice, It is managed by a coding unit (CU: Coding Unit) obtained by dividing the coding unit and a hierarchical structure consisting of blocks and partitions obtained by dividing the coding unit. In many cases, the block is minimized. Encoded as a unit.
  • a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a difference image (“residual” between the predicted image and the input image is generated.
  • difference image or “prediction residual”.
  • inter prediction inter-screen prediction
  • intra-screen prediction intra-screen prediction
  • inter prediction a prediction image in a prediction target frame is generated for each prediction unit by applying motion compensation using a motion vector to a reference image in a reference frame (decoded image) obtained by decoding the entire frame. Is done.
  • intra prediction based on locally decoded images in the same frame, predicted images in the frame are sequentially generated.
  • Patent Document 1 for each pixel value of a decoded image, the decoded image is classified into a plurality of regions based on the degree of activity (Activity) indicating the randomness of the image around the pixel.
  • An adaptive loop filter (hereinafter also simply referred to as “adaptive filter”) that generates a filtered decoded image by performing filter processing while switching a filter coefficient group for each region is disclosed.
  • the filter coefficient group used for the adaptive filter is adaptively determined so as to minimize an error between the encoding target image and the filtered decoded image obtained by applying the adaptive filter to the decoded image. It has been done.
  • the encoding device and decoding device provided with such an adaptive filter can improve prediction accuracy and encoding efficiency by generating a prediction image with reference to the filtered decoded image.
  • Non-Patent Document 3 divides a (local) decoded image into a plurality of regions, and according to the degree of activity (Activity) indicating local randomness of the (local) decoded image in each region,
  • An adaptive loop filter (Adaptive Loop ⁇ ⁇ Filter) that performs filter processing while switching filter coefficient groups for each region is disclosed.
  • the filter coefficient group for each region is determined by the encoding device so as to minimize the error between the filtered image in the region and the encoding target image.
  • An encoding device and a decoding device provided with such an adaptive filter generate a prediction image with reference to a filtered decoded image obtained by performing filter processing on the (local) decoded image using the adaptive filter. As a result, it is possible to improve the prediction accuracy and the encoding efficiency.
  • the inventor has obtained an experimental result that, for an area having a certain range of activity, the coding efficiency is rather lowered by applying an adaptive filter.
  • a method of switching the filter coefficient group for each region based on region characteristics other than the activity in addition to the activity may be considered.
  • the filter coefficient group is switched for each region based on the two region characteristics, there is a problem that the calculation cost on the encoding device side and the decoding device side becomes high.
  • the present invention has been made in view of the above-described problems, and an object of the present invention is to provide an image filter device capable of improving encoding efficiency as compared with the prior art, and a decoding device including such an image filter device, and It is to realize an encoding device.
  • the inventor has obtained knowledge that encoding efficiency can be improved by switching on / off of filter processing according to local image characteristics in a decoded image.
  • an image filter device includes a filter unit that operates on an input image composed of a plurality of unit regions, and an image characteristic of the input image for each of the plurality of unit regions.
  • a characteristic value calculating means for calculating a characteristic value representing a filter on area in which each of the unit areas is made to act on the filter means in accordance with the value of the characteristic value for the unit area calculated by the characteristic value calculating means,
  • a classification unit that classifies the filter unit into any one of the filter-off regions where the filter unit does not act is provided.
  • the classification means determines each unit area constituting the input image according to the value of the characteristic value representing the image characteristic of the input image in the unit area, and Classify into any of the filter-off areas.
  • the filter means acts on the input image in the unit area belonging to the filter-on area, but does not act on the input image in the unit area belonging to the filter-off area.
  • the image filter device configured as described above, it is possible to switch on / off of the filter process according to the local image characteristics of the input image. Therefore, by using the output image obtained by the image filter device acting on the decoded image while switching on / off of the filtering process as a reference image for generating the predicted image, the prediction accuracy and the coding efficiency are improved as compared with the conventional case. Can be improved.
  • the unit region is, for example, a unit for generating a predicted image. It may be a certain prediction unit (partition), may be a conversion unit that is a unit of frequency conversion, or may be other than those.
  • the unit area may be a unit composed of only one pixel.
  • the data structure of the encoded data according to the present invention includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas.
  • the characteristic value calculation means for calculating, and the filter-on region in which the filter means is activated or the filter means is not activated in accordance with the value of the characteristic value for each unit area calculated by the characteristic value calculation means.
  • a data structure of encoded data that is referred to by an image filter device including classification means for classifying into any of the filter-off regions, and includes a filter coefficient group used by the filter means It is a feature.
  • the decoding apparatus that refers to the encoded data configured as described above represents filter means that operates on an input image composed of a plurality of unit regions, and represents the image characteristics of the input image for each of the plurality of unit regions.
  • a characteristic value calculating means for calculating a characteristic value, and a filter-on area or the filter means for causing the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculating means.
  • classifying means for classifying it into any one of the filter-off regions that do not act on the filter.
  • the filter means performs filter processing using a filter coefficient group included in the encoded data. Therefore, such a decoding apparatus can generate a prediction image with high prediction accuracy.
  • the encoded data referred to by such a decoding apparatus does not need to include a flag for switching on / off of the filtering process, the code amount of the encoded data is reduced. Therefore, the encoded data is encoded data with high encoding efficiency.
  • a calculation unit that calculates an average of pixel values of the input image, and each unit region is calculated by the calculation unit.
  • the classification means for classifying into any of a plurality of types, and the pixel value of each pixel in each unit area are set for the type including the pixel And filtering means for calculating using a filter coefficient group.
  • each unit area is classified into one of a plurality of types according to the average of the pixel values for the unit area calculated by the calculation unit,
  • the pixel value of each pixel in the unit area is calculated using the filter coefficient group set for the type including the pixel. Therefore, according to the above configuration, the filter processing can be performed using the filter coefficient group set for each type, so that the encoding efficiency is improved. Further, according to the above configuration, each unit area is classified according to a characteristic value with a small calculation processing amount, ie, an average of pixel values, so that the processing amount can be reduced.
  • a filter device includes a region characteristic for calculating a region characteristic value of each unit region in a filter device including a linear filter in which filter coefficients are adaptively set.
  • the linear filter includes an offset that depends on the region characteristic for performing a filter operation on each pixel belonging to each unit region, and the region characteristic of the unit region calculated by the region characteristic calculating unit This is performed by using an offset corresponding to the value of and a filter coefficient group that does not depend on the region characteristics.
  • a filter device includes a linear filter in which filter coefficient groups are adaptively set, and includes a target pixel in an output image of the linear filter. Truncation means is provided for truncating the change amount, which is the difference between the pixel value and the pixel value of the target pixel in the input image of the linear filter.
  • a filter device including a linear filter in which filter coefficients are adaptively set
  • at least one first filter coefficient group depending on the first region characteristic is associated.
  • the first filter coefficient table recorded in association with the value of the first area characteristic and the second filter coefficient group depending on the second area characteristic are recorded in association with the value of the corresponding second area characteristic.
  • Decoding means for decoding the second filter coefficient table, and area characteristic calculation means for calculating the values of the first area characteristic and the second area characteristic of each unit area, and the linear filter includes each of the unit areas belonging to each unit area.
  • a filter operation using the second filter coefficient group corresponding to the value of the region characteristic is executed.
  • each unit area can be obtained without decoding a filter coefficient table in which a plurality of filter coefficient groups depending on both the first area characteristic and the second area characteristic are recorded in association with the corresponding area characteristic pair.
  • the filter operation according to both the first region characteristic and the second region characteristic can be performed. That is, according to the above configuration, a decoded image close to the original image can be generated with high accuracy without causing a significant increase in the code amount. As a result, higher encoding efficiency than conventional can be realized.
  • the decoding device and the encoding device provided with the filter device are also included in the scope of the present invention.
  • the image filter device includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas.
  • a characteristic value calculation means for calculating the filter area, and a filter-on area that causes the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculation means, or the filter Classification means for classifying the filter into any one of the filter-off regions in which the means is not operated.
  • the encoding efficiency is improved as compared with the conventional case.
  • FIG. 1 It is a block diagram which shows the structure of the adaptive filter with which the moving image decoding apparatus which concerns on the 1st Embodiment of this invention is provided. It is a figure which shows the data structure of the encoding data which are produced
  • (A) shows the configuration of the picture layer of the encoded data
  • (D) shows the configuration of the leaf CU included in the CU layer
  • (e) shows the configuration of the inter prediction information for the leaf CU
  • ( f) shows the configuration of the intra prediction information for the leaf CU
  • (g) shows the configuration of the filter parameter included in the slice header.
  • (A) shows a characteristic value calculation reference region CRR composed of pixels included in the target unit region UR and pixels adjacent to the target unit region UR, and (b) is included in the target unit region UR.
  • a characteristic value calculation reference region CRR composed of pixels is shown. It is a graph which shows the experimental data obtained by the inventor. (A) shows the square error reduction amount with respect to the activity for each region for a plurality of sequences, and (b) shows the average (“average” for the plurality of sequences for the square error reduction amount at each activity. )) And the total number of unit areas (“count”) having each degree of activity. It is a table
  • surface which shows the example of a classification process by the adaptive filter which concerns on the 1st Embodiment of this invention.
  • FIG. 6A is a diagram illustrating unit areas set by the adaptive filter according to the first embodiment of the present invention, and FIG.
  • FIG. 6A shows the position and size of a unit area included in a target coding unit according to a CU partition flag.
  • (B) is a diagram showing a case where a unit area included in the target unit area has a predetermined size. It is a figure for demonstrating the filter process by the adaptive filter which concerns on the 1st Embodiment of this invention, Comprising: (a) calculates the pixel value of the filter object pixel in the object unit area
  • FIG. 8 is a diagram illustrating a filter reference region R that is a set of pixels referred to for the purpose and a filter reference range RA that is defined as a union of the filter reference regions R for each filter target pixel.
  • FIG. 4 is a diagram showing filter coefficients assigned to each pixel included in a reference region R. It is a table
  • surface which shows the example of a classification process by the adaptive filter which concerns on the 1st modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 2nd modification of the 1st Embodiment of this invention. It is a graph which shows the experimental data obtained by the inventor. (A) shows a square error reduction amount with respect to an average pixel value for each region for a plurality of sequences, and (b) shows an average (for a plurality of sequences) for a square error reduction amount at each average pixel value.
  • (A) to (d) show each coding unit to be subjected to on / off control in the maximum coding unit when the value of the syntax alf_cu_control_max_depth is 0 to 3, together with a branch diagram representing a hierarchical structure. ing. It is a figure for demonstrating the classification process by the adaptive filter which concerns on the 4th modification of the 1st Embodiment of this invention.
  • (A) shows an example of classification processing by the adaptive filter, and (b) shows the position and size of the unit area included in the maximum coding unit target coding unit hierarchically specified by the CU partitioning flag.
  • (C) shows a case where the unit area included in the target unit area has a predetermined size.
  • FIG. 1 It is a block diagram which shows the structure of the adaptive filter with which the moving image encoder which concerns on the 1st Embodiment of this invention is provided. It is a block diagram which shows the structure of the adaptive filter which concerns on the 2nd modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 5th modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 6th modification of the 1st Embodiment of this invention. It is a block diagram which shows the 1st structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention.
  • FIG. 27 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of FIG. 26 is mounted on a decoding device. It is a block diagram which shows the 2nd structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the adaptive filter suitable for mounting in an encoding apparatus, when mounting the adaptive filter of FIG. 28 in a decoding apparatus. It is a block diagram which shows the 3rd structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention. [Fig. 31] Fig. 31 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of Fig.
  • Fig. 30 is mounted on a decoding device. It is a block diagram which shows the 4th structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention.
  • Fig. 30 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of Fig. 29 is mounted on a decoding device. It is a block diagram which shows the 1st structural example of the adaptive filter which concerns on the 3rd Embodiment of this invention. It is a figure which shows the reference area of the 1st spatial filter with which the adaptive filter of FIG. 34 is provided. It is a figure which shows the reference area of the 2nd spatial filter with which the adaptive filter of FIG. 34 is provided.
  • FIG. 43 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of FIGS. 34 and 42 is mounted on a decoding device. It is a block diagram which shows the 2nd structural example of the adaptive filter which concerns on the 3rd Embodiment of this invention.
  • Fig. 41 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of Fig. 40 is mounted on a decoding device.
  • FIG. 43 is a diagram illustrating a reference region of an enlarged linear filter included in the adaptive filter of FIG. 42.
  • FIG. 43 is a diagram illustrating another reference region of the enlarged linear filter included in the adaptive filter of FIG. 42.
  • FIG. 46 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of FIG. 45 is mounted on a decoding device. It is a figure which shows the data structure of the coding data produced
  • FIG. 49 shows the structure for the moving image decoding apparatus suitable for mounting of the adaptive filter which concerns on each embodiment. It is the block diagram which showed the structure of the moving image encoder suitable for mounting of the adaptive filter which concerns on each embodiment. It is a figure which shows the modification of the coding data shown in FIG. It is the figure shown about the structure of the transmitter which mounts the said moving image encoder, and the receiver which mounts the said moving image decoder.
  • A shows a transmitting apparatus equipped with a moving picture coding apparatus
  • (b) shows a receiving apparatus equipped with a moving picture decoding apparatus.
  • Embodiment 1 A first embodiment of a decoding device and an encoding device according to the present invention will be described below with reference to the drawings.
  • the decoding apparatus according to the present embodiment decodes a moving image from encoded data. Therefore, hereinafter, this is referred to as “moving image decoding apparatus”.
  • the encoding device according to the present embodiment generates encoded data by encoding a moving image. Therefore, in the following, this is referred to as a “video encoding device”.
  • the scope of application of the present invention is not limited to this. That is, as will be apparent from the following description, the features of the present invention can be realized without assuming a plurality of frames. That is, the present invention can be applied to a general decoding apparatus and a general encoding apparatus regardless of whether the target is a moving image or a still image.
  • the encoded data # 1 has a hierarchical structure including a sequence layer, a GOP (Group Of Pictures) layer, a picture layer, a slice layer, and a maximum coding unit (LCU: Large Coding Unit) layer.
  • GOP Group Of Pictures
  • LCU Large Coding Unit
  • FIG. 2 shows the hierarchical structure below the picture layer in the encoded data # 1.
  • FIGS. 2A to 2F are a picture layer P, a slice layer S, an LCU layer LCU, a leaf CU included in the LCU (denoted as CUL in FIG. 2D), and inter prediction (inter-screen prediction), respectively. It is a figure which shows the structure of inter prediction information PI_Inter which is the prediction information PI about a partition, and intra prediction information PI_Intra which is the prediction information PI about an intra prediction (prediction in a screen) partition.
  • PI_Inter is the prediction information PI about a partition
  • intra prediction information PI_Intra which is the prediction information PI about an intra prediction (prediction in a screen) partition.
  • the picture layer P is a set of data that is referenced by the video decoding device 1 in order to decode a target picture that is a processing target picture. As shown in FIG. 2A, the picture layer P includes a picture header PH and slice layers S1 to SNs (Ns is the total number of slice layers included in the picture layer P).
  • the picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture.
  • the encoding mode information (entropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.
  • Each slice layer S included in the picture layer P is a set of data referred to by the video decoding device 1 in order to decode a target slice that is a slice to be processed.
  • the slice layer S includes a slice header SH and LCU layers LCU1 to LCUn (Nc is the total number of LCUs included in the slice S).
  • the slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice.
  • Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.
  • I slice that uses only intra prediction at the time of encoding (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
  • the slice header SH includes a filter parameter FP that is referred to by an adaptive filter included in the video decoding device 1.
  • the configuration of the filter parameter FP will be described later and will not be described here.
  • Each LCU layer LCU included in the slice layer S is a set of data that the video decoding device 1 refers to in order to decode the target LCU that is the processing target LCU.
  • the LCU layer LCU is composed of a plurality of coding units (CU: Coding Units) obtained by hierarchically dividing the LCU into a quadtree.
  • the LCU layer LCU is a coding unit corresponding to the highest level in a hierarchical structure that recursively includes a plurality of CUs.
  • each CU included in the LCU layer LCU has a hierarchical structure that recursively includes a CU header CUH and a plurality of CUs obtained by dividing the CU into quadtrees. is doing.
  • each CU excluding the LCU is half the size of the CU to which the CU directly belongs (that is, the CU one layer higher than the CU), and the size that each CU can take is encoded data # 1.
  • a CU that is not further divided is called a leaf CU.
  • the CU header CUH includes a coding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target CU. Specifically, as shown in FIG. 2C, a CU division flag SP_CU that specifies whether or not the target CU is further divided into four subordinate CUs is included. When the CU division flag SP_CU is 0, that is, when the CU is not further divided, the CU is a leaf CU.
  • CU leaf A CU (CU leaf) that is not further divided is handled as a prediction unit (PU: Prediction Unit) and a transform unit (TU: Transform Unit).
  • PU Prediction Unit
  • TU Transform Unit
  • the leaf CU (denoted as CUL in FIG. 2 (d)) includes (1) PU information PUI that is referred to when the moving image decoding apparatus 1 generates a predicted image, and (2) The TU information TUI that is referred to when the residual data is decoded by the moving picture decoding apparatus 1 is included.
  • the skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target PU.
  • the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target leaf, PU information PUI and TU information TUI in the leaf CU are omitted. Note that the skip flag SKIP is omitted for the I slice.
  • the PU information PUI includes a skip flag SKIP, prediction type information PT, and prediction information PI as shown in FIG.
  • the prediction type information PT is information that specifies whether intra prediction or inter prediction is used as a predicted image generation method for the target leaf CU (target PU).
  • the prediction information PI includes intra prediction information PI_Intra or inter prediction information PI_Inter depending on which prediction method is specified by the prediction type information PT.
  • a PU to which intra prediction is applied is also referred to as an intra PU
  • a PU to which inter prediction is applied is also referred to as an inter PU.
  • the PU information PUI includes information specifying the shape and size of each partition included in the target PU and the position in the target PU.
  • the partition is one or a plurality of non-overlapping areas constituting the target leaf CU, and the generation of the predicted image is performed in units of partitions.
  • the TU information TUI specifies a quantization parameter difference ⁇ qp (tu_qp_delta) that specifies the magnitude of the quantization step, and a division pattern for each block of the target leaf CU (target TU).
  • TU partition information SP_TU and quantized prediction residuals QD1 to QDNT are included.
  • the quantization parameter difference ⁇ qp is a difference qp ⁇ qp ′ between the quantization parameter qp in the target TU and the quantization parameter qp ′ in the TU encoded immediately before the TU.
  • TU partition information SP_TU is information that specifies the shape and size of each block included in the target TU and the position in the target TU.
  • Each TU can be, for example, a size from 64 ⁇ 64 pixels to 2 ⁇ 2 pixels.
  • the block is one or a plurality of non-overlapping areas constituting the target leaf CU, and encoding / decoding of the prediction residual is performed in units of blocks.
  • Each quantized prediction residual QD is encoded data generated by the moving image encoding apparatus 2 performing the following processes 1 to 3 on a target block that is a processing target block.
  • Process 1 DCT transform (Discrete Cosine Transform) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image.
  • Process 2 The DCT coefficient obtained in Process 1 is quantized.
  • Process 3 The DCT coefficient quantized in Process 2 is variable length encoded.
  • the inter prediction information PI_Inter includes a coding parameter that is referred to when the video decoding device 1 generates an inter prediction image by inter prediction. As shown in FIG. 2 (e), the inter prediction information PI_Inter includes inter PU partition information SP_Inter that specifies a partition pattern for each partition of the target PU, and inter prediction parameters PP_Inter1 to PP_InterNe (Ne for each partition). The total number of inter prediction partitions included in the target PU).
  • the inter-PU partition information SP_Inter is information for designating the shape and size of each inter prediction partition included in the target PU (inter PU) and the position in the target PU.
  • the inter PU is composed of four symmetric splittings of 2N ⁇ 2N pixels, 2N ⁇ N pixels, N ⁇ 2N pixels, and N ⁇ N pixels, and 2N ⁇ nU pixels, 2N ⁇ nD pixels, and nL ⁇ 2N. It is possible to divide into 8 types of partitions in total by four asymmetric splits of pixels and nR ⁇ 2N pixels.
  • the specific value of N is defined by the size of the CU to which the PU belongs, and the specific values of nU, nD, nL, and nR are determined according to the value of N.
  • an inter PU of 128 ⁇ 128 pixels is 128 ⁇ 128 pixels, 128 ⁇ 64 pixels, 64 ⁇ 128 pixels, 64 ⁇ 64 pixels, 128 ⁇ 32 pixels, 128 ⁇ 96 pixels, 32 ⁇ 128 pixels, and 96 ⁇ It is possible to divide into 128-pixel inter prediction partitions.
  • the inter prediction parameter PP_Inter includes a reference image index RI, an estimated motion vector index PMVI, and a motion vector residual MVD.
  • the motion vector residual MVD is encoded data generated by the moving image encoding device 2 executing the following processes 4 to 6.
  • Process 4 Select an encoded / decoded locally decoded image (more precisely, an image obtained by performing deblocking processing and adaptive filtering on the encoded / decoded local decoded image)
  • the motion vector mv for the target partition is derived with reference to the selected encoded / decoded local decoded image (hereinafter also referred to as “reference image”).
  • Process 5 An estimation method is selected, and an estimated value (hereinafter also referred to as “estimated motion vector”) pmv of the motion vector mv assigned to the target partition is derived using the selected estimation method.
  • Process 6 The motion vector residual MVD obtained by subtracting the estimated motion vector pmv derived in Process 5 from the motion vector mv derived in Process 4 is encoded.
  • the reference image index RI designates the locally decoded image (reference image) that has been encoded / decoded selected in the process 4.
  • the estimated motion vector index PMVI described above is the estimation method selected in the process 5. Is specified.
  • the estimation methods that can be selected in the processing 5 include: (1) a locally decoded image being encoded / decoded (more precisely, a region that has already been decoded in a locally decoded image being encoded / decoded).
  • a median of a motion vector allocated to a partition adjacent to the target partition hereinafter also referred to as “adjacent partition” is used as an estimated motion vector pmv.
  • a motion vector assigned to a partition (often referred to as a “collocated partition”) occupying the same position as the target partition is used as an estimated motion vector pmv, etc. Is mentioned.
  • the prediction parameter PP related to the partition for which unidirectional prediction is performed includes one reference image index RI, one estimated motion vector index PMVI, and one motion vector residual MVD.
  • the prediction parameters PP for a partition that performs bi-directional prediction include two reference image indexes RI1 and RI2, two estimated motion vector indexes PMVI1 and PMVI2, and two motion vector residuals MVD1. And MVD2.
  • the intra prediction information PI_Intra includes an encoding parameter that is referred to when the video decoding device 1 generates an intra predicted image by intra prediction.
  • the intra prediction information PI_Intra includes intra PU partition information SP_Intra that specifies a partition pattern of the target PU (intra PU) into each partition, and intra prediction parameters PP_Intra1 to PP_IntraNa for each partition. (Na is the total number of intra prediction partitions included in the target PU).
  • the intra-PU partition information SP_Intra is information that specifies the shape and size of each intra-predicted partition included in the target PU, and the position in the target PU.
  • the intra PU split information SP_Intra includes an intra split flag (intra_split_flag) that specifies whether or not the target PU is split into partitions. If the intra partition flag is 1, the target PU is divided symmetrically into four partitions. If the intra partition flag is 0, the target PU is not divided and the target PU itself is one partition.
  • N 2 n , n is an arbitrary integer of 1 or more.
  • a 128 ⁇ 128 pixel intra PU can be divided into 128 ⁇ 128 pixel and 64 ⁇ 64 pixel intra prediction partitions.
  • the intra prediction parameter PP_Intra includes an estimation flag MPM and a residual prediction mode index RIPM.
  • the intra prediction parameter PP_Intra is a parameter for designating an intra prediction method (prediction mode) for each partition.
  • the estimation flag MPM is a flag indicating whether or not the prediction mode estimated based on the prediction mode allocated to the peripheral partition of the target partition that is the processing target is the same as the prediction mode for the target partition.
  • examples of partitions around the target partition include a partition adjacent to the upper side of the target partition and a partition adjacent to the left side of the target partition.
  • the residual prediction mode index RIPM is an index included in the intra prediction parameter PP_Intra when the estimated prediction mode and the prediction mode for the target partition are different, and is an index for designating a prediction mode assigned to the target partition. It is.
  • the slice header SH includes the filter parameter FP that is referred to by the adaptive filter included in the video decoding device 1.
  • the adaptive filter included in the video decoding device 1 classifies each of one or a plurality of unit areas included in the target slice or the target LCU into one of a plurality of types, and the unit area Is switched according to the type to which the unit area belongs. Further, the adaptive filter process is performed on the unit area for which the adaptive filter process is set to ON using the filter coefficient group selected according to the type to which the unit area belongs.
  • the unit region is any of a slice, an LCU, a CU (other than a leaf CU), a leaf CU (a prediction unit PU, a transform unit TU), a partition that configures a prediction unit PU, and a block that configures a transform unit TU. It may be a block of 4 ⁇ 4 pixels, 8 ⁇ 8 pixels, etc. different from the CU, etc., or may be an area composed of only one pixel, or other than those. May be.
  • Fig. 2 (g) shows the data structure of the filter parameter FP.
  • the filter parameter FP can be configured to include area designation information for designating the position and size of each unit area in the target slice or the target LCU.
  • the adaptive filter included in the moving image decoding apparatus 1 can specify the position and size of each unit region by referring to the region designation information.
  • the filter parameter FP may be configured not to include such region designation information, and the moving image encoding device 2 and the moving image decoding device 1 may use a unit region having a predetermined size.
  • the unit area described above includes a slice, an LCU, a CU (other than a leaf CU), a leaf CU (a prediction unit PU, a transform unit TU), a partition constituting the prediction unit PU, and a block constituting the transform unit TU.
  • the adaptive filter included in the video decoding device 1 refers to the CU partition flag SP_CU, inter-PU partition information SP_Inter, intra-PU partition information SP_Intra, TU partition information SP_TU, etc.
  • the position and size of the unit area can be specified hierarchically.
  • NT-1 (NT is the total number of filter coefficients included in the filter coefficient group I) and (3) an offset oI.
  • filter coefficient groups with different filter numbers are used for the filter processing of unit areas classified into different types.
  • the filter coefficient group 0 and the filter coefficient group 1 are used for the filter processing of the unit area classified into type 1 and the unit area classified into type 2 different from type 1, respectively.
  • the moving picture decoding apparatus 1 includes H.264 as a part thereof.
  • Decoding device including technology adopted in KTA software which is a codec for joint development in AVC and VCEG (Video Coding Expert Group) and technology adopted in TMuC (Test Model Under Consistration) software which is the successor codec It is.
  • FIG. 3 is a block diagram showing a configuration of the moving picture decoding apparatus 1.
  • the moving picture decoding apparatus 1 includes a variable length code decoding unit 11, a predicted image generation unit 12, an inverse quantization / inverse conversion unit 13, an adder 14, a frame memory 15, a deblocking filter 16, and The adaptive filter 17 is provided.
  • the predicted image generation unit 12 includes a motion vector restoration unit 12a, an inter predicted image generation unit 12b, an intra predicted image generation unit 12c, and a prediction method determination unit 12d.
  • the moving picture decoding apparatus 1 is an apparatus for generating moving picture # 2 by decoding encoded data # 1.
  • variable-length code decoding unit 11 decodes the prediction parameter PP related to each partition from the encoded data # 1, and supplies the decoded prediction parameter PP to the predicted image generation unit 12. Specifically, for the inter prediction partition, the variable length code decoding unit 11 receives the inter prediction parameter PP_Inter including the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD from the encoded data # 1. These are decoded and supplied to the motion vector restoration unit 12a.
  • the intra prediction parameter PP_Intra including the estimation flag MPM, the residual index RIPM, and the additional index AI is decoded from the encoded data # 1, and these are supplied to the intra prediction image generation unit 12c.
  • variable length code decoding unit 11 decodes the prediction type information PT for each partition from the encoded data # 1, and supplies this to the prediction method determination unit 12d. Further, the variable length code decoding unit 11 decodes the quantization prediction residual QD for each block and the quantization parameter difference ⁇ qp for the TU including the block from the encoded data # 1, and performs inverse quantization / inverse This is supplied to the conversion unit 13. Further, the variable length code decoding unit 11 decodes the filter parameter FP from the encoded data # 1 and supplies this to the adaptive filter 17.
  • the predicted image generation unit 12 identifies whether each partition is an inter prediction partition for performing inter prediction or an intra prediction partition for performing intra prediction based on the prediction type information PT for each partition. In the former case, the inter prediction image Pred_Inter is generated, and the generated inter prediction image Pred_Inter is supplied to the adder 14 as the prediction image Pred. In the latter case, the intra prediction image Pred_Intra is generated, The generated intra predicted image Pred_Intra is supplied to the adder 14. Note that, when the skip mode is applied to the processing target PU, the predicted image generation unit 12 omits decoding of other parameters belonging to the PU.
  • the motion vector restoration unit 12a restores the motion vector mv related to each inter prediction partition from the motion vector residual MVD related to that partition and the restored motion vector mv ′ related to another partition. Specifically, (1) the estimated motion vector pmv is derived from the restored motion vector mv ′ according to the estimation method specified by the estimated motion vector index PMVI, and (2) the derived estimated motion vector pmv and the motion vector remaining are derived. The motion vector mv is obtained by adding the difference MVD. It should be noted that the restored motion vector mv ′ relating to other partitions can be read from the frame memory 15. The motion vector restoration unit 12a supplies the restored motion vector mv to the inter predicted image generation unit 12b together with the corresponding reference image index RI.
  • the inter prediction image generation unit 12b generates a motion compensated image mc related to each inter prediction partition by inter-screen prediction. Specifically, using the motion vector mv supplied from the motion vector restoration unit 12a, the motion compensation image mc from the filtered decoded image P_ALF ′ designated by the reference image index RI supplied from the motion vector restoration unit 12a. Is generated.
  • the filtered decoded image P_ALF ′ is an image obtained by performing deblocking processing by the deblocking filter 16 and filtering processing by the adaptive filter 17 on the decoded image that has already been decoded for the entire frame.
  • the inter-predicted image generation unit 12b can read out the pixel value of each pixel constituting the filtered decoded image P_ALF ′ from the frame memory 15.
  • the motion compensated image mc generated by the inter predicted image generation unit 12b is supplied to the prediction method determination unit 12d as an inter predicted image Pred_Inter.
  • the intra predicted image generation unit 12c generates a predicted image Pred_Intra related to each intra prediction partition. Specifically, first, a prediction mode is specified based on the intra prediction parameter PP_Intra supplied from the variable length code decoding unit 11, and the specified prediction mode is assigned to the target partition in, for example, raster scan order.
  • the estimation flag MPM is decoded, and the estimation flag MPM indicates that the prediction mode for the target partition to be processed is the same as the prediction mode assigned to the peripheral partition of the target partition. If it is, the prediction mode assigned to the partition around the target partition is assigned to the target partition.
  • the estimation flag MPM indicates that the prediction mode for the target partition to be processed is not the same as the prediction mode assigned to a partition around the target partition, the remaining The prediction mode index RIPM is decoded, and the prediction mode indicated by the residual prediction mode index RIPM is assigned to the target partition.
  • the intra predicted image generation unit 12c generates a predicted image Pred_Intra from the (local) decoded image P by intra prediction according to the prediction method indicated by the prediction mode assigned to the target partition.
  • the intra predicted image Pred_Intra generated by the intra predicted image generation unit 12c is supplied to the prediction method determination unit 12d.
  • the intra predicted image generation unit 12c may be configured to generate the predicted image Pred_Intra from the filtered decoded image P_ALF by intra prediction.
  • the prediction method determination unit 12d determines whether each partition is an inter prediction partition that should perform inter prediction or an intra prediction partition that should perform intra prediction based on the prediction type information PT about the PU to which each partition belongs. To do. In the former case, the inter prediction image Pred_Inter generated by the inter prediction image generation unit 12b is supplied to the adder 14 as the prediction image Pred. In the latter case, the inter prediction image generation unit 12c generates the inter prediction image Pred_Inter. The intra predicted image Pred_Intra that has been processed is supplied to the adder 14 as the predicted image Pred.
  • the inverse quantization / inverse transform unit 13 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transform on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 14.
  • the inverse quantization / inverse transform unit 13 derives the quantization step QP from the quantization parameter difference ⁇ qp supplied from the variable length code decoding unit 11.
  • the generation of the prediction residual D by the inverse quantization / inverse transform unit 13 is performed in units of blocks obtained by dividing TUs or TUs.
  • the adder 14 generates the decoded image P by adding the prediction image Pred supplied from the prediction image generation unit 12 and the prediction residual D supplied from the inverse quantization / inverse conversion unit 13.
  • the generated decoded image P is stored in the frame memory 15.
  • the deblocking filter 16 determines the block boundary in the decoded image P or the block boundary in the decoded image P when the difference between the pixel values of pixels adjacent to each other via the block boundary or the partition boundary in the decoded image P is smaller than a predetermined threshold value. By performing a deblocking process on the partition boundary, the block boundary or an image near the partition boundary is smoothed.
  • the image subjected to the deblocking process by the deblocking filter 16 is stored in the frame memory 15 as a deblocked decoded image P_DB.
  • the adaptive filter 17 (1) sets one or a plurality of unit areas constituting the deblocked decoded image P_DB to any of a plurality of types according to the image characteristics of the deblocked decoded image P_DB in the unit area. (2) ON / OFF of adaptive filter processing for the unit area is switched according to the type to which the unit area belongs. Further, (3) adaptive filter processing is performed on the deblocked decoded image P_DB in the unit region for which the adaptive filter processing is turned on using a filter coefficient group selected according to the type to which the unit region belongs. .
  • FIG. 1 is a block diagram showing the configuration of the adaptive filter 17.
  • the adaptive filter 17 includes a region characteristic value calculation unit 171, a region classification unit 172, and a filter processing unit 173.
  • the region characteristic value calculation unit 171 reads each pixel value of the deblocked decoded image P_DB in the characteristic value calculation reference region including the target unit region to be processed from the frame memory 15, and based on these pixel values, the target unit A characteristic value (also referred to as an area characteristic value) representing the image characteristic of the deblocked decoded image P_DB in the area is calculated.
  • the characteristic value calculation reference region refers to a region formed of pixels referred to for calculating the characteristic value of the deblocked decoded image P_DB in the target unit region.
  • the characteristic value calculation reference area is composed of only the target unit area, or is composed of the target unit area and an area around the target unit area.
  • the region characteristic value calculation unit 171 is a configuration for calculating a characteristic value related to the target unit region, and can also calculate the characteristic value using another image having a correlation with the target unit region. Therefore, the region characteristic value calculation unit 171 is not limited to the deblocked decoded image P_DB, and may calculate the characteristic value with reference to another image, for example, the decoded image P. Also in this case, the pixel value located in the coordinate area corresponding to the target unit area is used.
  • the adaptive filter 17 having the deblocked decoded image P_DB as an input is described. However, the adaptive filter 17 may be configured to have a plurality of inputs. In this way, when the adaptive filter 17 has a plurality of different input images, the characteristic value may be calculated using one of these input images.
  • the region characteristic value calculation unit 171 includes an activity calculation unit 171a.
  • the activity calculation unit 171a calculates an activity ACT as the characteristic value of the deblocked decoded image P_DB in the target unit region.
  • the activity is an index indicating the randomness of the image, and whether there is a large high-frequency component in the conversion coefficient obtained when the DCT conversion is performed on the region including the divided region.
  • the calculation is based on the pixel values of adjacent pixels. The greater the difference in pixel value between adjacent pixels, the greater the activity.
  • the calculated activity ACT for each divided region is supplied to the region classification unit 172.
  • the activity calculation unit 171a uses, for example, the following formula (1a): Thus, the activity ACT can be calculated.
  • Formula (1a) corresponds to evaluating the activity using the sum of absolute values of adjacent pixel values (Sum of Absolute Difference, SAD) as an index.
  • the activity calculation unit 171a may be configured to calculate the activity ACT by using the following mathematical formula (1b).
  • Formula (1b) corresponds to the evaluation of activity using the sum of modified Laplacians (Sum-ModifiedModLaplacian, SML) of the pixel values of adjacent pixels as an index.
  • the activity calculation unit 171a applies not only to the pixel S (i, j) in the target unit region, but also to the target unit region. Reference is also made to adjacent pixels S ( ⁇ 1, j), S (Bx, j), S (i, ⁇ 1), and S (i, By). That is, the characteristic value calculation reference region when using the formula (1a) or (1b) is the pixel S (i, j) in the target unit region and the pixel S ( ⁇ 1, j) adjacent to the target unit region. , S (Bx, j), S (i, ⁇ 1), and S (i, By).
  • FIG. 4A shows the characteristic value calculation reference region CRR when the activity calculation unit 171a calculates the activity ACT for the target unit region UR of 4 ⁇ 4 pixels using the formula (1a) or (1b). Shown in
  • the pixel value of the deblocked decoded image P_DB may not be referred to for some pixels in the characteristic value calculation reference area.
  • the pixel value of the undecoded LCU cannot be referred to.
  • the activity calculation unit 171a calculates the activity ACT without referring to those pixel values that cannot be referred to.
  • the activity calculation unit 171a obtains the following obtained by partially limiting the range of the sum of i and j in Equation (1a) What is necessary is just to calculate activity ACT using Numerical formula (1a ').
  • the activity calculation unit 171a refers to only the pixel value in the target unit region for the activity of the target unit region regardless of whether the target unit region is in contact with the slice boundary or the LCU boundary. It can also be configured to calculate. For example, the activity calculation unit 171a can also calculate the activity ACT by using the following formula (1c).
  • Equation (1c) corresponds to evaluating the activity of the pixel value of the adjacent pixel using the SAD of the pixel value of the adjacent pixel as an index, as in Equation (1a). Differently, the degree of activity is calculated by referring to only the pixels included in the target unit area.
  • the activity calculation unit 171a may be configured to calculate the activity ACT by using the following formula (1d).
  • Equation (1d) corresponds to the evaluation of the activity using the SML of the pixel value of the adjacent pixel as an index, but unlike Equation (1d), In other words, the degree of activity is calculated by referring only to the included pixels.
  • the characteristic value calculation reference is made.
  • the region CRR is the same region as the target unit region UR.
  • a variance VP which will be described later, is also an example of the degree of activity that can be calculated with reference to only pixel values in the target unit region.
  • the activity level calculation unit 171a calculates the activity level for the target unit area by referring only to the pixel values in the target unit area
  • the activity level calculation unit 171a compares the pixel value other than the target unit area with reference to the pixel value.
  • the number of pixels read from the frame memory 15 is reduced, and the data transmission amount from the frame memory 15 to the adaptive filter 16 is reduced. Further, since the number of pixels referred to calculate the activity level is small and exception processing is unnecessary, the processing amount for calculating the activity level is reduced.
  • the activity calculation part 171a which calculates activity ACT by using Formula (1c) can also be expressed as follows. That is, the activity calculation unit 171a determines the activity ACT (i, j) for the pixel (i, j) in the vicinity of the coordinates (i, j) ⁇ (i, j), (i + 1, j), (i , J + 1) ⁇ with reference to the pixel value of each pixel, and ACT (i in the inner region ⁇ (i, j)
  • the activity calculation unit 171a that calculates the activity ACT by using the mathematical formula (1d) can be expressed as follows. That is, the activity calculation unit 171a determines the activity ACT (i, j) for the pixel (i, j) in the vicinity of the coordinates (i, j) ⁇ (i, j), (i + 1, j), (i , J + 1), (i ⁇ 1, j), (i, j ⁇ 1) ⁇ with reference to the pixel value of each pixel, the inner region ⁇ (i, j)
  • the ACT for the target unit region is calculated using the sum of ACT (i, j) in Bx-2, 1 ⁇ j ⁇ By-2 ⁇ .
  • the activity calculation unit 171a that calculates the activity with reference to only the pixels included in the target unit region has the entire vicinity of the target unit region as an internal region including pixels included in the target unit region.
  • the degree of activity of each target pixel region is calculated from the degree of activity of each pixel belonging to the internal area after deriving the degree of activity of each pixel belonging with reference to the pixel values of the pixels included in the vicinity of the pixel. It can also be expressed as a thing.
  • the configuration for calculating the degree of activity with reference to only the pixels included in the target unit region is not limited to the above example, and the target unit region is a pixel whose entire neighborhood is included in the target unit region. Is divided into a peripheral region consisting of pixels that are not included in the target unit region, and the activity of each pixel belonging to the internal region is referred to the pixel value of the pixel in the vicinity of the pixel. And the activity of each pixel belonging to the peripheral region is derived by referring to the pixel value of the pixel in the vicinity of the pixel and in the unit region, and then belonging to the unit region A configuration may be employed in which the degree of activity for the unit region is calculated from the degree of activity of each pixel.
  • the region classification unit 172 classifies the target unit region into one of a plurality of types based on the activity ACT calculated by the region characteristic value calculation unit 171 as the characteristic value for the target unit region. Subsequently, according to the type, the target region is classified into either a region where the filter process is off (filter off region) or a region where the filter process is on (filter on region). As will be described later, the relationship between the type and the region may be fixed in advance, or a flag indicating which type is classified into which region may be encoded.
  • a type classified into the filter-on region is referred to as a filter-on type
  • a type classified into the filter-off region is referred to as a filter-off type.
  • the target unit area may be classified into the filter-on area and the filter-off area from the direct characteristic value (activity ACT) without classifying the target unit area into a type.
  • the region classification unit 172 is a filter coefficient group included in the filter parameter FP supplied from the variable length code decoding unit 11, and includes a type to which the target unit region belongs. Are assigned to the target unit area. On the other hand, when the target unit region is classified as the filter-off region, the region classification unit 172 does not assign a filter coefficient group to the target unit region.
  • the area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on area or a filter off area, and (2) the target unit area is a filter on area.
  • the filter coefficient group allocated to the target unit area is supplied to the filter processing unit 173.
  • the on / off information U indicates on / off of the filter processing for the target unit region, and can therefore be expressed by 1 bit. Since the classification of the target unit area into the filter-on area / filter-off area is equivalent to the classification of the target unit area into the filter-on type / filter-off type, in the following description, the filter-on area / filter-off area is the same value. Handles the OFF area and the filter ON type / filter OFF type without distinction.
  • FIG. 5A is a graph showing experimental data obtained by the inventor, and a plurality of sequences (“BasketballDrill_q32”, “PartyScene_q32”, “RaceHorses_q32”, “BQSquare_q32”) whose quantization parameter value is 32. ) For each unit region, and a square error reduction amount obtained when adaptive filter processing is performed for each unit region.
  • the square error reduction amount has a positive correlation with the coding efficiency, and indicates that the larger the square error reduction amount, the better the coding efficiency.
  • the unit area in which the square error reduction amount is negative has a negative contribution to the coding efficiency, that is, a contribution to reduce the coding efficiency.
  • the square error reduction amount is negative for any sequence.
  • ⁇ For the sequence “BasketballDrill_q32”, the range of activity is about 5 or less ⁇
  • the range of activity is about 8 or less ⁇
  • the range of activity is about 3 or less ⁇
  • Sequence Regarding “BQSquare_q32” the square error reduction amount is negative in the range where the activity is approximately 5 or more and approximately 20 or less and in each range where the activity is approximately 3 or less.
  • FIG. 5B is a graph showing the average (“average”) for a plurality of sequences and the total number of unit areas (“count”) having each activity, with respect to the square error reduction amount at each activity. .
  • the value of the graph indicating the average of a plurality of sequences is negative in the range where the activity is approximately 8 or less.
  • the area classification unit 172 can be configured to perform any of the classification processes among the following (classification process example 1-1) to (classification process example 1-7). Further, among the following (Classification Processing Example 1-1) to (Classification Processing Example 1-7), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 1-1) to (Classification process example 1-7).
  • type 0 is a filter-off type
  • types 1 to Non are filter-on types.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Th1 represents a predetermined threshold value.
  • Th1 5 for the sequence “BasketballDrill_q32” -For the sequence "PartyScene_q32”
  • Th1 8
  • Th1 3 for the sequence “RaceHorses_q32” -For the sequence "BQSquare_q32”
  • Th1 20 May be used.
  • a unit region whose activity is equal to or less than a predetermined threshold as a type region 0, that is, a unit region that is not subjected to adaptive filter processing, the negative contribution to the coding efficiency can be reduced. Therefore, the encoding efficiency is improved.
  • the value of the threshold Th1 is preferably set according to a specific calculation formula for calculating the activity.
  • the value of the threshold Th1 that is preferably used in the configuration that calculates the activity ACT using the formula (1b) may be different from the value of the threshold Th1 that is used in the configuration that uses the formula (1a).
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Th2 indicates a predetermined threshold value, and it is assumed that Th1 ⁇ Th2.
  • a specific value of the threshold Th2 may be determined in advance so that the encoding efficiency is improved.
  • a unit area whose activity is equal to or less than a predetermined threshold is set to type 0, that is, a unit area for which adaptive filter processing is not performed. Therefore, the negative contribution to the coding efficiency can be reduced, so that the coding efficiency is improved.
  • a predetermined value as the specific value of the threshold Th2 in this processing example so that the coding efficiency is improved.
  • the area classification unit 172 classifies the target unit area as follows as shown in FIG.
  • the value of the threshold Th2 may be determined in advance by experiments or the like so that the processing amount is further reduced while maintaining high coding efficiency.
  • the experimental data shown in FIG. 5 indicates that when the activity value is approximately 90 or more, even if adaptive filter processing is performed, the coding efficiency is hardly improved. Therefore, if the threshold value Th2 is set to about 90, the processing amount can be reduced while maintaining high coding efficiency.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • the specific value of the threshold Th2 in this processing example is set to a value of about 90 so that the processing amount is further reduced while maintaining high coding efficiency, as in the classification processing example 1-3. You just have to.
  • the encoding efficiency can be effectively improved by using such threshold values Th1 and Th2.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th1 or Th2 ⁇ ACT ⁇
  • Type 2 Th1 ⁇ ACT ⁇ Th2
  • the specific value of the threshold value Th1 in the present processing example may be determined in the same manner as in the classification processing example 1-1.
  • the specific value of the threshold value Th2 in this processing example may be determined in the same manner as in the classification processing example 1-3.
  • the encoding efficiency can be effectively improved by using such threshold values Th1 and Th2.
  • the region classification unit 172 classifies the target unit region as follows, as illustrated in FIG.
  • Type 1 ACT ⁇ Th1 ⁇
  • Type 2 Th1 ⁇ ACT ⁇ Th2 ⁇
  • Type 3 Th2 ⁇ ACT
  • the specific values of the threshold values Th1 and Th2 in this processing example may be determined in advance so as to improve the encoding efficiency.
  • the region characteristic value calculation unit 171 refers to only the pixel value in the target unit region when calculating the activity ACT for the target unit region.
  • the number of pixels read from the frame memory 15 is reduced and the amount of data transmission from the frame memory 15 to the adaptive filter 16 is reduced compared to a configuration that also refers to pixel values other than the target unit region.
  • the processing amount for calculating the activity level is reduced.
  • the filter parameter FP includes on / off designation information V for designating the on / off of the filter processing for each type
  • the area classification unit 172 refers to the on / off designation information V, thereby performing the filter processing for each type.
  • On / off may be determined.
  • the on / off designation information V is information for designating the on / off of the filtering process for each type, and therefore the bit length depends on the total number of types.
  • the filter parameter FP includes 4-bit on / off designation information V as information for designating on / off of the filter processing for each type.
  • the present processing example is not limited to this, and is generally applicable when the total number of types is Ntp and the filter parameter FP includes Ntp bit on / off designation information V.
  • FIG. 6G is a table showing the classification processing in this processing example.
  • the region classification unit 172 classifies the target unit region as follows.
  • Type C 2 ⁇ Th ⁇ ACT ⁇ 3 ⁇ Th ⁇
  • Type D 3 ⁇ Th ⁇ ACT
  • the region classification unit 172 refers to the on / off designation information V included in the filter parameter FP, and determines the on / off of the filter processing for each type. For example, when the on / off designation information V is “0101” in binary expression, type A and type C are set to the filter off type, and type B and type D are set to the filter on type.
  • a specific value of the threshold Th is preferably a predetermined value so that the encoding efficiency is improved.
  • the region classification unit 172 assigns a filter coefficient group included in the filter parameter FP to the target unit region when the target unit region is classified into the filter on type.
  • a filter coefficient group included in the filter parameter FP when a plurality of filter coefficient groups are included in the filter parameter FP, the same selection rule as the selection rule used in the moving image encoding device 2 that generates the encoded data # 1 is used. Any one of the plurality of filter coefficient groups may be selected and assigned to the target unit area.
  • the region classification unit 172 does not assign a filter coefficient group to the target unit region.
  • the area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on type or a filter off type, and (2) the target unit area is classified as a filter on type. If it is, the filter coefficient group assigned to the target unit area is supplied to the filter processing unit 173.
  • the region classification unit 172 sets each unit region as a CU partition flag SP_CU, inter-PU partition information SP_Inter, intra-PU partition information SP_Intra, TU partition information SP_TU, and the like included in the encoded data # 1. May be set so as to have a size and position specified hierarchically, or may be set to have a size and position specified by the area specifying information included in the encoded data # 1, It may be set to have a predetermined size (the same applies hereinafter).
  • FIG. 7A shows an example of a unit area that is hierarchically specified by the CU division flag SP_CU included in the encoded data # 1.
  • the maximum hierarchical depth for the target LCU is 2, and the positions and sizes of the unit areas UR0 to UR6 included in the target LCU are hierarchically specified by the CU division flag SP_CU. .
  • FIG. 7B an example of a unit area having a predetermined size is shown in FIG. 7B.
  • the target LCU for example, 16 ⁇ 16 pixels
  • unit areas UR0 to UR15 having a predetermined size (for example, 4 ⁇ 4 pixels).
  • Each unit area can be identified from each other by an index assigned in the raster scan order as shown in FIG. 7B, for example.
  • the CU partition flag SP_CU, inter-PU partition information SP_Inter, intra-PU partition information SP_Intra, and TU partition information SP_TU included in the encoded data # 1 The processing amount can be reduced as compared with the case of referring to the above.
  • the filter processing unit 173 When the on / off information U supplied from the region classification unit 172 indicates the filter on type, the filter processing unit 173 (1) each pixel value of the deblocked decoded image P_DB in the filter reference range including the target unit region (2) A filtered decoded image P_ALF in the target unit region is generated by performing filter processing using the filter coefficient group supplied from the region classification unit 172 on these pixels. To do.
  • the generated filtered decoded image P_ALF is stored in the frame memory 15.
  • the filter reference range refers to a range that is referred to in order to generate the filtered decoded image P_ALF in the target unit region, and includes a target unit region and a region around the target unit region.
  • the filter processing unit 173 does not perform the filtering process on the target unit region.
  • the filter coefficient group supplied from the region classification unit 172 is the filter coefficient group I (I is any integer satisfying 1 ⁇ I ⁇ Nf, and Nf is the total number of filter coefficient groups included in the filter parameter FP) ).
  • the pixel value of the filter target pixel in the filtered decoded image P_ALF (also referred to as “filtered image”) in the target unit region is represented as SF (x ′, y ′), and the deblocked decoded image in the filter reference range.
  • the pixel value of P_DB (also referred to as “pre-filter image”) is represented as S (x, y).
  • the filter processing unit 173 calculates the pixel value SF (x ′, y ′) by the following formula (2).
  • the coordinates (x, y) may be the same coordinates as the coordinates (x ′, y ′), or may be different coordinates as long as they have a one-to-one correspondence.
  • AI (i, j) represents a filter coefficient to be multiplied by the pixel value S (x + i, y + j) of the pre-filter image among the filter coefficients included in the filter coefficient group I, and oI represents the filter coefficient group.
  • I represents an offset included in I.
  • filter reference region R represents a region (also referred to as “filter reference region R”) that is referred to in order to calculate the pixel value of the filter target pixel, and is set according to the position of the filter target pixel.
  • filter reference range RA also referred to as “filter reference range RA”
  • the filter reference range RA can also be expressed as a set of pixels required for calculating all pixel values of the filtered image in the target unit region.
  • FIG. 8A shows a filter reference region R and a filter reference range RA when the target unit region UR is 8 ⁇ 8 pixels and the filter reference region is 5 ⁇ 5 taps.
  • the hatched pixels indicate the filter target pixels S (x ′, y ′).
  • FIG. 8B shows an example of how to assign filter coefficients to each pixel included in the filter reference region R.
  • the hatched pixels indicate the filter target pixels S (x ′, y ′).
  • each filter coefficient can be assigned to each pixel included in the filter reference region R so as to have a rotational symmetry of 180 degrees.
  • the present embodiment is not limited to this, and the assignment of each filter coefficient to each pixel value may not have rotational symmetry.
  • the filter reference region R may be a rhombus region composed of pixels having a city area distance from the filter target pixel of Ncb or less in units of pixels, or regions having other shapes. Good.
  • the method of assigning the filter coefficient to each pixel included in the filter reference region R and the shape of the filter reference region R may be appropriately set according to the configuration of the moving picture encoding device that generates the encoded data # 1. Good.
  • the activity calculation unit 171a included in the region characteristic value calculation unit 171 uses the vertical activity ACT_V and the horizontal activity as the characteristic value of the deblocked decoded image P_DB in the target unit region instead of the activity ACT described above.
  • the degree ACT_H may be calculated, and the region classification unit 172 may classify the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V.
  • the horizontal direction activity ACT_H and the vertical direction activity ACT_V can be calculated by, for example, the first term and the second term of the mathematical formula (1a) described above.
  • the horizontal direction activity ACT_H and the vertical direction activity ACT_V may be calculated by the first term and the second term of any one of the above formulas (1b) to (1d), respectively. it can.
  • the region classification unit 172 classifies the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V calculated by the activity calculation unit 171a.
  • the region classification unit 172 may be configured to perform any of the classification processes from the following (Classification process example 2-1) to (Classification process example 2-5). Also, among the following (Classification Processing Example 2-1) to (Classification Processing Example 2-5), classification processing designation information for designating which classification processing is performed is included in the encoded data # 1, The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 2-1) to (Classification process example 2-5).
  • type 0 is a filter-off type
  • types 1 to Non are filter-on types.
  • the area classification unit 172 classifies the target unit area as follows as shown in FIG.
  • Type 0 ACT_H ⁇ ThHor1 and ACT_V ⁇ ThVer1
  • Type 1 ThHor1 ⁇ ACT_H and ACT_V ⁇ ThVer1
  • Type 2 ACT_H ⁇ ThHor1 and ThVer1 ⁇ ACT_V
  • Type 3 ThHor1 ⁇ ACT_H and ThVer1 ⁇ ACT_V
  • values of the thresholds ThHor1 and ThVer1 values that are set in advance so as to improve the encoding efficiency may be used.
  • the target unit area is classified into one of a plurality of types including type 0 according to each value of the horizontal direction activity ACT_H and the vertical direction activity ACT_V, so that the deblocked decoded image P_DB
  • the classification process can be performed more appropriately according to the image characteristics along the horizontal and vertical directions of the image. Therefore, according to this processing example, encoding efficiency can be improved more effectively.
  • the region classification unit 172 classifies the target unit region as follows as shown in FIG. 9B.
  • Type 1 ACT_H ⁇ ThHor1 and ACT_V ⁇ ThVer1
  • Type 2 ACT_H ⁇ ThHor1 and ThVer1 ⁇ ACT_V
  • Type 3 ThHor1 ⁇ ACT_H and ACT_V ⁇ ThVer1
  • Type 4 ThHor1 ⁇ ACT_H and ThVer1 ⁇ ACT_V
  • values of the thresholds ThHor1 and ThVer1 values that are set in advance so as to improve the coding efficiency may be used, as in the classification processing example 2-2.
  • the target unit area is classified into one of a plurality of types not including type 0 according to each value of the horizontal activity ACT_H and the vertical activity ACT_V.
  • the region characteristic value calculation unit 171 refers to only the pixel value in the target unit region when calculating the activity ACT for the target unit region.
  • the number of pixels read from the frame memory 15 is reduced and the amount of data transmission from the frame memory 15 to the adaptive filter 16 is reduced compared to a configuration that also refers to pixel values other than the target unit region.
  • the processing amount for calculating the activity is reduced.
  • the area classification unit 172 classifies the target unit area as follows as illustrated in FIG.
  • Type 0 (ACT_H ⁇ ThHor1 and ACT_V ⁇ ThVer1), or ThHor2 ⁇ ACT_H, or ThVer2 ⁇ ACT_V Type 1: ThHor1 ⁇ ACT_H ⁇ ThHor2 and ACT_V ⁇ ThVer1 Type 2: ACT_H ⁇ ThHor1 and ThVer1 ⁇ ACT_V ⁇ ThVer2 Type 3: ThHor1 ⁇ ACT_H ⁇ ThHor2 and ThVer1 ⁇ ACT_V ⁇ ThVer2
  • values of the thresholds ThHor1, ThVer1, ThHor2, and ThVer2 values that are set in advance so as to improve the encoding efficiency may be used.
  • the threshold values ThHor1 and ThVer1 are used to perform a finer classification process, thereby further improving the encoding efficiency.
  • one configuration of the present invention is a combination of a plurality of characteristic values (here, the horizontal activity ACT_H and the vertical activity) as the characteristic values used for the classification of the filter-on region and the filter-off region. This includes the case of using ACT_V).
  • one configuration of the present invention includes a case where a combination of a plurality of characteristic values is used as a characteristic value used for assigning a filter coefficient group to the filter-on region.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th1 or Th2 ⁇ ACT Type 1: Th1 ⁇ ACT ⁇ Th2 and ACT_H ⁇ ACT_V Type 2: Th1 ⁇ ACT ⁇ Th2 and ACT_V ⁇ ACT_H
  • ACT indicates the above-described activity ACT
  • the activity calculation unit 171a is configured to individually calculate the horizontal activity ACT_H and the vertical activity ACT_V as in the present modification.
  • ACT ACT_H + ACT_V Can be obtained.
  • the filter coefficient group is switched according to the magnitude of the horizontal direction activity level ACT_H and the vertical direction activity level ACT_V.
  • ACT_H ⁇ ACT_V that is, when the vertical activity ACT_V is equal to or higher than the horizontal activity ACT_H, there is a high possibility that a vertical component change such as a horizontal edge exists, and when ACT_H> ACT_V, the vertical edge, etc. There is a high probability that there will be horizontal changes.
  • the directionality of the image in the target unit region is calculated according to the magnitudes of the characteristic values ACT_H and ACT_V, and a filter is selected according to the directionality.
  • ACT_H ⁇ ACT_V that is, the directionality when the change in the vertical direction is larger
  • ACT_V ⁇ ACT_H that is, the directionality when the change in the horizontal direction is larger.
  • 0 and 1 are indexes (also referred to as directionality indexes) for distinguishing the directionality of images from each other.
  • the region classification unit 172 selects a filter type according to a direction index that is a value indicating the direction in the target unit region.
  • the characteristic value used for the classification of the filter-on region and the filter-off region (here, the activity ACT), and the characteristic value used for assigning the filter coefficient group to the filter-on region (here, the horizontal direction activity) ACT_H and vertical activity ACT_V) may be different. That is, the present invention has, as one configuration, a means for classifying the filter on area and the filter off area based on a certain characteristic value, and a means for assigning a filter coefficient group to the filter on area based on a characteristic value different from the certain characteristic value.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th1, or (Th2 ⁇ ACT and 2 ⁇ ACT_H ⁇ ACH_H), or (Th1 ⁇ ACT ⁇ Th2 and 2 ⁇ ACT_H ⁇ ACT_V and 2 ⁇ ACT_V ⁇ ACT_H)
  • Type 1 Th1 ⁇ ACT ⁇ Th2 and 2 ⁇ ACT_H ⁇ ACT_V
  • Type 2 Th1 ⁇ ACT ⁇ Th2 and 2 ⁇ ACT_V ⁇ ACT_H
  • Type 3 (Th2 ⁇ ACT and 2 ⁇ ACT_H ⁇ ACT_V), or (Th2 ⁇ ACT and 2 ⁇ ACT_H ⁇ ACT_V and 2 ⁇ ACT_V ⁇ ACT_H)
  • ACT represents the above-described activity ACT.
  • the directionality in the case of 2 ⁇ ACT_H ⁇ ACT_V is expressed as directionality 0 using the directionality indexes 0 to 2, and the directionality in the case of 2 ⁇ ACT_V ⁇ ACT_H is set to the direction.
  • the directionality in the case of 2 ⁇ ACT_H ⁇ ACT_V and 2 ⁇ act_v ⁇ act_h is expressed as directionality 2.
  • Directionality 0, 1, and 2 correspond to the cases where there are horizontal edges, vertical edges, and diagonal edges, respectively.
  • the region classifying unit 172 selects the filter type for the target unit region in accordance with the directionality index that is a value indicating the directionality of the image in the target unit region.
  • the filter is turned off when the activity is low.
  • the degree of activity is medium
  • the diagonal filter is turned off, and the filters 1 and 2 are assigned to the vertical and horizontal directions.
  • the horizontal filter is turned off and the filter 3 is assigned in a direction other than the horizontal direction.
  • the filter 0 is set when the activity is medium and the directionality is oblique. In the sequence to which this case is applied, there are few oblique lines in the region of activity, and encoding is performed by assigning filters in the oblique direction. This is because the efficiency is not improved. Note that this example is an example, and appropriate allocation differs depending on sequence dependency.
  • the adaptive filter 17 causes the region characteristic value calculation unit 171 to use the activity ACT (or the horizontal activity ACT_H and the vertical activity ACT_V as the characteristic value of the deblocked decoded image P_DB in the target unit region. ) And the region classification unit 172 classifies the target unit region into one of a plurality of types based on the activity ACT (or horizontal activity ACT_H and vertical activity ACT_V). Although mentioned, this embodiment is not limited to this.
  • an average of pixel values (also referred to as an average pixel value) is calculated as the characteristic value of the deblocked decoded image P_DB in the target unit region, and the target unit region is selected from a plurality of types based on the calculated average pixel value. A configuration to be classified will be described.
  • FIG. 10 is a block diagram showing the configuration of the adaptive filter 17 according to this modification.
  • the region characteristic value calculation unit 171 in the adaptive filter 17 according to this modification includes a pixel value reference unit 171 b.
  • the pixel value reference unit 171b calculates the average pixel value MP by reading each pixel value of the deblocked decoded image P_DB in the target unit region to be processed from the frame memory 15 and taking the average of the pixel values.
  • the pixel value reference unit 171b uses, for example, the following formula (3):
  • the average pixel value MP can be calculated by taking a simple average of the pixel values.
  • the pixel value reference unit 171b may be configured to calculate the average pixel value MP by taking a weighted average using a predetermined weight coefficient for each pixel value instead of taking a simple average of each pixel value. .
  • the region classification unit 172 classifies the target unit region into one of a plurality of types based on the average pixel value MP calculated by the region characteristic value calculation unit 171 as the characteristic value for the target unit region.
  • the target unit area is further classified into a filter-on area or a filter-off area depending on the type.
  • the region classification unit 172 is a filter coefficient group included in the filter parameter FP supplied from the variable length code decoding unit 11, and includes a type to which the target unit region belongs. Are assigned to the target unit area.
  • the region classification unit 172 does not assign a filter coefficient group to the target unit region.
  • the area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on type or a filter off type, and (2) the target unit area is a filter on type. In the case of the classification, the filter coefficient group allocated to the target unit area is supplied to the filter processing unit 173.
  • FIG. 11A is a graph showing experimental data obtained by the inventor, and is for a plurality of sequences (“PartyScene_q32”, “RaceHorses_q32”, “BQSquare_q32”) having a quantization parameter value of 32. It is a graph which shows the average pixel value for every unit area
  • the square error reduction amount has a positive correlation with the coding efficiency, and indicates that the larger the square error reduction amount, the better the coding efficiency. Further, it is indicated that the unit area in which the square error reduction amount is negative has a negative contribution to the coding efficiency, that is, a contribution to reduce the coding efficiency.
  • the square error reduction amount is negative for any sequence.
  • the sequence “PartyScene_q32” a range in which the average pixel value is approximately 30 or less, and a range in which the average pixel value is 230 or more.
  • the sequence “RaceHorses_q32” a range in which the average pixel value is approximately 40 to 50, and
  • the square error reduction amount is negative in the range where the average pixel value is about 30 or less and the range where the average pixel value is about 235 or more. ing.
  • FIG. 11B is a graph showing the average (“average”) for a plurality of sequences and the total number of unit areas (“count”) having each average pixel value for the square error reduction amount at each average pixel value. It is.
  • the value of the graph indicating the average of a plurality of sequences is negative in the range where the average pixel value is about 30 or less and in the range where the average pixel value is about 230 or more.
  • the region classification unit 172 can be configured to perform any of the classification processes among the following (classification process examples 3-1) to (classification process examples 3-7).
  • classification processing designation information for designating which classification processing is performed is included in the encoded data # 1, and the region
  • the classification unit 172 may be configured to selectively perform a classification process designated by the classification process designation information among (Classification process examples 3-1) to (Classification process examples 3-7).
  • type 0 is a filter-off type
  • types 1 to Non are filter-on types.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • ThM1 indicates a predetermined threshold value.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • ThM2 indicates a predetermined threshold value, and it is assumed that ThM1 ⁇ ThM2.
  • a specific value of the threshold ThM2 may be determined in advance so that encoding efficiency is improved.
  • a unit area whose average pixel value is equal to or less than a predetermined threshold is set to type 0, that is, a unit area for which no adaptive filter processing is performed. Therefore, the negative contribution to the coding efficiency can be reduced, so that the coding efficiency is improved.
  • the encoding efficiency can be further improved.
  • a specific value of the threshold value ThM2 in the present processing example is preferably a value determined in advance so that the encoding efficiency is improved.
  • the area classification unit 172 classifies the target unit area as follows as shown in FIG.
  • ThM2 indicates a predetermined threshold value.
  • ThM2 a value predetermined for each sequence may be used in view of the experimental data shown in FIG.
  • Th2M 230 -For the sequence “RaceHorses_q32”
  • Th2M 225 -For the sequence “BQSquare_q32”
  • Th2M 235 May be used.
  • ThM2 230 may be used when the threshold Th2M is determined without depending on the sequence.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • a specific value of the threshold ThM2 may be determined in the same manner as in the classification processing example 3-3.
  • a specific value of the threshold ThM1 may be determined in advance so that encoding efficiency is improved. By using such threshold values ThM1 and ThM2, the encoding efficiency can be effectively improved.
  • the area classification unit 172 classifies the target unit area as follows as shown in FIG.
  • Type 0 MP ⁇ ThM1 or ThM2 ⁇ MP ⁇
  • Type 2 ThM1 ⁇ MP ⁇ ThM2
  • the specific value of the threshold ThM1 in this processing example may be determined in the same manner as in the classification processing example 3-1.
  • the specific value of the threshold ThM2 in this processing example may be determined in the same manner as in the classification processing example 3-3.
  • the encoding efficiency can be effectively improved by using such threshold values ThM1 and ThM2.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 1 ACT ⁇ ThM1 ⁇
  • Type 2 ThM1 ⁇ ACT ⁇ ThM2 ⁇
  • Type 3 ThM2 ⁇ ACT
  • the specific values of the threshold values ThM1 and ThM2 in this processing example may be determined in advance so as to improve the encoding efficiency.
  • the filter coefficient group can be switched according to the value of the average pixel value in the target unit region, so that the encoding efficiency is improved.
  • the filter parameter FP includes on / off designation information V for designating the on / off of the filter processing for each type
  • the area classification unit 172 refers to the on / off designation information V, thereby performing the filter processing for each type.
  • On / off may be determined.
  • the on / off designation information V is information for designating the on / off of the filtering process for each type, and therefore the bit length depends on the total number of types.
  • the filter parameter FP includes 4-bit on / off designation information V as information for designating on / off of the filter processing for each type.
  • the present processing example is not limited to this, and is generally applicable when the total number of types is Ntp and the filter parameter FP includes Ntp bit on / off designation information V.
  • FIG. 12G is a diagram illustrating a classification process in this process example.
  • the region classification unit 172 classifies the target unit region as follows.
  • Type C 2 ⁇ ThM ⁇ MP ⁇ 3 ⁇ ThM ⁇
  • Type D 3 ⁇ ThM ⁇ MP
  • the region classification unit 172 refers to the on / off designation information V included in the filter parameter FP, and determines the on / off of the filter processing for each type. For example, when the on / off designation information V is “0101” in binary expression, type A and type C are set to the filter off type, and type B and type D are set to the filter on type.
  • the region classification unit 172 assigns a filter coefficient group included in the filter parameter FP to the target unit region.
  • the same selection rule as the selection rule used in the moving image encoding device 2 that generates the encoded data # 1 is used. Any one of the plurality of filter coefficient groups may be selected and assigned to the target unit area.
  • the region classification unit 172 does not assign a filter coefficient group to the target unit region.
  • the area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on type or a filter off type, and (2) the target unit area is classified as a filter on type. If it is, the filter coefficient group assigned to the target unit area is supplied to the filter processing unit 173.
  • the filter processing can be switched on and off for each range of the average pixel value, the encoding efficiency is improved.
  • the pixel value reference unit 171b included in the region characteristic value calculation unit 171 takes the average of the pixel values as the characteristic value of the deblocked decoded image P_DB in the target unit region to be processed.
  • the average pixel value MP is calculated, and the region classification unit 172 classifies the target unit region into one of a plurality of types based on the average pixel value MP. It is not limited to this.
  • the pixel value reference unit 171b included in the region characteristic value calculation unit 171 calculates the average pixel value MP and the variance VP of each pixel value as the characteristic value of the deblocked decoded image P_DB in the target unit region.
  • the area classification unit 172 classifies the target unit area into one of a plurality of types based on the average pixel value MP and the variance VP.
  • the target unit area is a rectangle of Bx pixels ⁇ By pixels
  • the coordinates of each pixel in the target unit area are (i, j) (i and j are 0 ⁇ i ⁇ Bx ⁇ 1 and 0 ⁇ j, respectively).
  • ⁇ By ⁇ 1 the pixel value of the deblocked decoded image P_DB at the coordinates is represented by S (i, j)
  • the pixel value reference unit 171b may, for example, ) Can be used to calculate the variance VP.
  • variance VP calculated by the pixel value reference unit 171b functions as an index indicating the randomness of the deblocked decoded image P_DB in the target unit region, similarly to the activity ACT described above.
  • the pixel value reference unit 171b calculates the average pixel value MP using, for example, Equation (3).
  • the area classification unit 172 sets the target unit area to one of a plurality of types based on the average pixel value MP and the variance VP calculated by the area characteristic value calculation unit 171 as the characteristic values for the target unit area.
  • the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type).
  • the region classification unit 172 is a filter coefficient group included in the filter parameter FP supplied from the variable length code decoding unit 11, and corresponds to the type to which the target unit region belongs.
  • the assigned filter coefficient group is assigned to the target unit area.
  • the region classification unit 172 does not assign a filter coefficient group to the target unit region.
  • the area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on type or a filter off type, and (2) the target unit area is a filter on type. In the case of the classification, the filter coefficient group allocated to the target unit area is supplied to the filter processing unit 173.
  • the region classification unit 172 may be configured to perform any of the classification processes from the following (Classification process example 4-1) to (Classification process example 4-2). Further, among the following (Classification Processing Example 4-1) to (Classification Processing Example 4-2), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 4-1) to (Classification process example 4-2).
  • type 0 is a filter-off type
  • types 1 to Non are filter-on types.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 VP ⁇ ThV1 Type 1: ThV1 ⁇ VP and MP ⁇ ThM1 Type 2: ThV1 ⁇ VP and ThM1 ⁇ MP
  • the threshold values ThM1 and ThV1 may be determined so that the coding efficiency is improved.
  • the distribution unit VP that is, the target unit region in which the randomness of the image is equal to or less than the threshold value ThV1 is set as a unit region that is not subjected to the filtering process. Therefore, the encoding efficiency is improved as in the configuration in which the unit area whose activity ACT is equal to or less than the predetermined threshold is set as a unit area that is not subjected to the filtering process.
  • the filter coefficient group is switched according to the magnitude of the average pixel value, so that the coding efficiency can be improved more effectively.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 VP ⁇ ThV1, or Th2 ⁇ VP, or ThM2 ⁇ MP
  • Type 1 ThV1 ⁇ VP ⁇ ThV2 and MP ⁇ ThM1
  • Type 2 ThV1 ⁇ VP ⁇ ThV2 and ThM1 ⁇ MP ⁇ ThM1
  • the threshold values ThM1, ThV1, and ThM2 may be determined so that the coding efficiency is improved. Further, the value of the threshold ThV2 may be determined so as to reduce the processing amount while maintaining high coding efficiency.
  • the unit region where the variance VP is equal to or less than the threshold value ThV1 and the unit region where the variance VP is greater than the threshold value ThV2 are set as the unit region where the filter process is not performed. Further, since the filter coefficient group is switched according to the average pixel value for the unit area to be subjected to the filter process, the encoding efficiency can be effectively improved.
  • ⁇ Modification 4> As described above, the characteristic value of the deblocked decoded image P_DB in the target unit area is calculated, and the adaptive filter that switches on / off the filter process for the target unit area based on the calculated characteristic value has been described.
  • the form is not limited to this.
  • FIG. 14 is a diagram illustrating a part of syntax included in the filter parameter FP (denoted as alf_param in FIG. 14) of the encoded data # 1 according to the present modification.
  • the configuration of the encoded data # 1 according to this modification is substantially the same as the configuration of the encoded data # 1 described above, but differs in the following points. That is, as shown in FIG. 14, in the filter parameter FP of the encoded data # 1 according to the present modification, (1) syntax adaptive_loop_filter_flag that specifies whether or not to perform adaptive filter processing on the target slice, (2) Syntax alf_cu_control_flag that specifies whether or not to turn on / off the adaptive filter processing for each coding unit, (3) Maximum division depth (maximum split) from the maximum coding unit for the coding unit that is the target of on / off control (depth) specifying syntax (hierarchy specifying information) alf_cu_control_max_depth, (4) syntax alf_length_cu_control_info specifying the number of coding units to be on / off controlled, and (5) coding unit to be on / off controlled Syntax (on / off flag) alf_cu_flag [i] that specifies on / off of adaptive filter
  • FIGS. 15 (a) to 15 (d) show respective coding units to be subjected to on / off control in the maximum coding unit LCU when the value of alf_cu_control_max_depth is 0 to 3, together with a branch diagram showing a hierarchical structure. Show.
  • the filter parameter FP includes an on / off flag alf_cu_flag [0] for the entire maximum coding unit LCU.
  • the filter parameter FP includes on / off flags alf_cu_flag [0] to alf_cu_flag for each of the encoding units 0 to 3 one layer lower than the maximum encoding unit LCU. [3] is included.
  • the filter parameter FP includes encoding unit 0 and encoding unit 7 that are one layer lower than the maximum encoding unit LCU, and maximum encoding unit LCU.
  • On / off flags alf_cu_flag [0] to alf_cu_flag [7] for each of the encoding units 1 to 6 in the lower hierarchy are included.
  • the filter parameter FP includes coding unit 0 and coding unit 10 that are one layer lower than the maximum coding unit LCU, and two layers from the maximum coding unit LCU.
  • On-off flags alf_cu_flag [0] to alf_cu_flag [10] are included for each of the lower encoding units 1 to 4, the encoding unit 9, and the encoding units 5 to 8 that are three layers lower than the maximum encoding unit LCU. .
  • FIG. 16A shows an activity ACT as a characteristic value for the target unit region and an on / off flag decoded from the encoded data # 1 by the region classification unit 172 included in the adaptive filter 17 according to this modification.
  • An example of processing for classifying the target unit region into either type 0 or type 1 with reference to the on / off flag alf_cu_flag_ [i] for the coding unit including the target unit region is shown.
  • the on / off flag alf_cu_flag_ [i] for the coding unit including the target unit region is 0, that is, the on / off flag alf_cu_flag [i] performs the filtering process on the coding unit.
  • the area classification unit 172 classifies the target unit area as type 0 regardless of the magnitude of the characteristic value. Therefore, the filter processing unit 173 included in the adaptive filter 17 does not perform filter processing on such a target unit region.
  • the filter processing unit 173 included in the adaptive filter 17 performs the filter process only on the target unit region whose on / off flag alf_cu_flag_ [i] is 1 and classified into a type other than type 0.
  • the activity is exemplified as the characteristic value.
  • the present modification is not limited to this, and an average pixel value or directionality may be used as the characteristic value.
  • the present modification is classified based on the characteristic values described in the classification processing examples 1-1 to 1-7, 2-1 to 2-4, 3-1 to 3-7, and 4-1 to 4-2. Can be widely applied to a classification method obtained by combining any of the above and classification based on the on / off flag alf_cu_flag [i].
  • unit areas classified into types other than type 0, that is, unit areas that are actually subjected to filter processing by the filter processing unit 173 are hatched.
  • FIG. 16B illustrates a case where each unit area has a size and a position that are hierarchically specified by the CU division flag SP_CU included in the encoded data # 1.
  • the present modification is not limited to this, and each unit area may have a predetermined size.
  • the unit area for calculating characteristics is equal to or less than the minimum CU (minimum coding unit).
  • the minimum CU minimum coding unit
  • the unit area for calculating the characteristics is preferably a unit area composed of four or more pixels (2 ⁇ 2 pixels, 1 ⁇ 4 pixels, 4 ⁇ 1 pixels).
  • the processing amount can be reduced.
  • the unit area for calculating the characteristic is small, the memory size for temporarily storing the calculated characteristic value until the filtering process is increased.
  • the memory size can be reduced by using four or more pixels. Can be reduced.
  • each unit area includes the CU partitioning flag SP_CU included in the encoded data # 1 as in the example illustrated in FIG. It is also possible to have a size and position specified hierarchically by.
  • the adaptive filter 17 switches on / off of the filter process in units smaller than the encoding unit corresponding to the maximum division depth specified by alf_cu_control_max_depth. be able to. That is, by setting the unit area for calculating the characteristics to a small unit that does not depend on the hierarchy designation information, even when the control area designated by the hierarchy designation information is large, the on / off control can be performed in a smaller unit. Therefore, since it is possible to perform fine on / off control while suppressing the code amount of the on / off flag, encoding efficiency is improved.
  • the target unit area for performing characteristic calculation is determined according to the control unit determined by the hierarchy designation information, it is necessary to perform characteristic calculation processing according to the structure of the control unit. In this case, the processing becomes complicated with many branches, and the amount of calculation increases, and the scale of software and circuits increases. As described above, when the target unit area for performing characteristic calculation is determined without depending on the hierarchy designation information, the complexity of characteristic value calculation is reduced.
  • the area characteristic value calculation unit 171 in the present modification includes a directionality calculation unit 171c in addition to the activity calculation unit 171a.
  • the directionality calculation unit 171c calculates the edge angle Theta as the directionality of the deblocked decoded image P_DB in the target unit region.
  • the directionality calculation unit 171c supplies the calculated edge angle Theta to the region classification unit 172.
  • the region classification unit 172 selects a filter type according to the activity ACT and the edge angle Theta.
  • the directionality calculation unit 171c calculates the edge angle Theta of the deblocked decoded image P_DB in the target unit region.
  • the directionality calculation processing by the directionality calculation unit 171c will be described for each of the case where the target unit region includes only one pixel and the case where the target unit region includes a plurality of pixels.
  • the directionality calculation unit 171c sets a 3 ⁇ 3 pixel reference region centered on the target pixel, and for each pixel in the reference region,
  • the pixel values Sobel_x and Sobel_y after the Sobel filter are calculated by applying the Sobel filter matrices SFM_x and SFM_y defined by the following equations.
  • the directionality calculating unit 171c calculates the edge angle Theta by the following mathematical formula.
  • Theta arctan (Sobel_x / Sobel_y)
  • arctan represents an inverse function of the trigonometric function tan.
  • the angle Theta increases clockwise and is expressed in a range from 0 degrees to 180 degrees (0 ° ⁇ Theta ⁇ 180 °).
  • the directionality calculation unit 171c first sets a reference region of 3 ⁇ 3 pixels for each pixel included in the target unit region, and performs a source search for each pixel by the method described above. Pixel values Sobel_x and Sobel_y after the bell filter are calculated.
  • the directionality calculation unit 171c calculates the edge strength ES for each pixel by the following mathematical formula, and uses the pixel values Sobel_x and Sobel_y after the Sobel filter for the pixel having the largest edge strength ES.
  • the obtained edge angle Theta is supplied to the region classification unit 172.
  • the directionality calculation unit 171c may be configured to output a number (direction index) corresponding to the edge direction without calculating the angle of the edge.
  • the direction index can be generated as follows by dividing the case according to the magnitude relation between Sobel_y and Sobel_x.
  • Direction index 0
  • Directionality index 1
  • and the signs of Sobel_y and Sobel_x are equal
  • index of index 2
  • Directionality index 3
  • four directional indexes are calculated. However, two directional indexes or eight directional indexes may be calculated.
  • the area classification unit 172 may be configured to perform any of the classification processes among the following (classification process example 5-1) to (classification process example 5-2). Further, among the following (Classification Processing Example 5-1) to (Classification Processing Example 5-2), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification Process Example 5-1) to (Classification Process Example 5-2).
  • type 0 is a filter-off type
  • types 1 to Non are filter-on types.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th
  • Type 1 ACT> Th and (0 ° ⁇ Theta ⁇ 45 ° or 135 ° ⁇ Theta ⁇ 180 °)
  • Type 2 ACT> Th and 45 ° ⁇ Theta ⁇ 135 °
  • a target unit region whose activity ACT is equal to or less than a predetermined threshold Th is set as a unit region that is not subjected to filter processing. Further, when the activity ACT is larger than the threshold Th, the target unit region is classified into one of two types according to the edge angle.
  • the region classification unit 172 classifies the target unit region as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th1
  • Type 1 Th1 ⁇ ACT ⁇ Th2 and (0 ° ⁇ Theta ⁇ 22.5 ° or 157.5 ° ⁇ Theta ⁇ 180 °)
  • Type 2 Th1 ⁇ ACT ⁇ Th2 and 22.5 ° ⁇ Theta ⁇ 67.5 °
  • Type 3 Th1 ⁇ ACT ⁇ Th2 and 67.5 ° ⁇ Theta ⁇ 112.5 °
  • Type 4 Th1 ⁇ ACT ⁇ Th2 and 112.5 ° ⁇ Theta ⁇ 157.5 ° ⁇
  • Type 5 ACT> Th2
  • the target unit area whose activity ACT is equal to or less than the predetermined threshold Th1 is set as a unit area that is not subjected to the filter process.
  • the target unit region is classified into one of four types (type 1 to type 4) according to the edge angle.
  • the target unit area is classified as type 5.
  • the classification process in the present modification is not limited to the above example.
  • the target unit region is selected from any of a plurality of types according to the edge angle. It is good also as a structure classified into these.
  • the region characteristic value calculation unit 171 in this modification includes both an activity calculation unit 171a and a pixel value reference unit 171b.
  • the region classification unit 172 selects a filter type according to the activity ACT and the average pixel value MP. Since the activity calculation processing by the activity calculation unit 171a and the average pixel value calculation processing by the pixel value reference unit 171b have already been described, description thereof will be omitted here.
  • the region classification unit 172 may be configured to perform any of the classification processes from the following (Classification process example 6-1) to (Classification process example 6-2). Further, among the following (Classification Processing Example 6-1) to (Classification Processing Example 6-2), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 6-1) to (Classification process example 6-2).
  • type 0 is a filter-off type
  • types 1 to Non are filter-on types.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th1, or ACT> Th2, or (Th1 ⁇ ACT ⁇ Th2 and MP ⁇ ThM1)
  • Type 1 Th1 ⁇ ACT ⁇ Th2 and ThM1 ⁇ MP ⁇ ThM2
  • Type 2 Th1 ⁇ ACT ⁇ Th2 and ThM2 ⁇ MP
  • the target unit area whose activity ACT is equal to or smaller than the predetermined threshold Th1 and the target unit area whose activity ACT is larger than the predetermined threshold Th2 are set as unit areas not subjected to the filter process.
  • the target unit area whose activity ACT is larger than the threshold Th1 and equal to or smaller than the threshold Th2 and whose average pixel value MP is equal to or smaller than the predetermined threshold ThM1 is also set as a unit area not subjected to the filtering process. Further, when the activity ACT is larger than the threshold Th1 and equal to or smaller than the threshold Th2, and the average pixel value MP is larger than the threshold ThM1, it is classified into either type 1 or type 2 according to the average pixel value MP. To do.
  • the area classification unit 172 classifies the target unit area as follows, as shown in FIG.
  • Type 0 ACT ⁇ Th1, or (Th1 ⁇ ACT ⁇ Th2 and MP ⁇ ThM1), or (Th2 ⁇ ACT and MP ⁇ ThM1)
  • Type 1 Th1 ⁇ ACT ⁇ Th2 and ThM1 ⁇ MP ⁇ ThM2
  • Type 2 Th1 ⁇ ACT ⁇ Th2 and ThM2 ⁇ MP
  • Type 3 Th2 ⁇ ACT and ThM1 ⁇ MP
  • the target unit area whose activity ACT is equal to or less than the predetermined threshold Th1 is set as a unit area that is not subjected to the filter process.
  • the target unit region whose average pixel value MP is equal to or less than the predetermined threshold ThM1 is also set as a unit region that is not subjected to the filtering process.
  • the activity ACT is larger than the threshold Th1 and equal to or smaller than the threshold Th2, and the average pixel value MP is larger than the threshold ThM1, it is classified into either type 1 or type 2 according to the average pixel value MP. To do. Further, when the activity ACT is larger than the threshold value Th2 and the average pixel value MP is larger than the threshold value ThM1, it is classified as type 3.
  • the moving image encoding apparatus 2 includes H.264 as a part thereof.
  • FIG. 21 is a block diagram showing a configuration of the moving picture encoding apparatus 2.
  • the moving image encoding device 2 includes a predicted image generation unit 21, a transform / quantization unit 22, an inverse quantization / inverse transform unit 23, an adder 24, a frame memory 25, a deblocking filter 26, An adaptive filter 27, a variable length code encoding unit 28, and a subtractor 29 are provided.
  • the predicted image generation unit 21 includes an intra predicted image generation unit 21a, a motion vector detection unit 21b, an inter predicted image generation unit 21c, a prediction method control unit 21d, and a motion vector redundancy deletion unit. 21e.
  • the moving image encoding device 2 is a device that generates encoded data # 1 by encoding moving image # 10 (encoding target image).
  • the predicted image generation unit 21 recursively divides the processing target LCU into one or a plurality of lower-order CUs, further divides each leaf CU into one or a plurality of partitions, and uses an inter-screen prediction for each partition.
  • a predicted image Pred_Inter or an intra predicted image Pred_Intra using intra prediction is generated.
  • the generated inter prediction image Pred_Inter and intra prediction image Pred_Intra are supplied to the adder 24 and the subtractor 29 as the prediction image Pred.
  • the prediction image generation unit 21 omits encoding of other parameters belonging to the PU for the PU to which the skip mode is applied. Also, (1) the mode of division into lower CUs and partitions in the target LCU, (2) whether to apply the skip mode, and (3) which of the inter predicted image Pred_Inter and the intra predicted image Pred_Intra for each partition Whether to generate is determined so as to optimize the encoding efficiency.
  • the intra predicted image generation unit 21a generates a predicted image Pred_Intra for each partition by intra prediction. Specifically, (1) a prediction mode used for intra prediction is selected for each partition, and (2) a prediction image Pred_Intra is generated from the decoded image P using the selected prediction mode. The intra predicted image generation unit 21a supplies the generated intra predicted image Pred_Intra to the prediction method control unit 21d.
  • the intra predicted image generation unit 21a determines an estimated prediction mode for the target partition from the prediction modes assigned to the peripheral partitions of the target partition, and the estimated prediction mode and the prediction mode actually selected for the target partition Are supplied as a part of the intra prediction parameter PP_Intra to the variable length code encoding unit 27 via the prediction scheme control unit 21d, and the variable length code encoding unit 27
  • the flag MPM is included in the encoded data # 1.
  • the intra predicted image generation unit 21a sets the residual prediction mode index RIPM indicating the prediction mode for the target partition. Then, as a part of the intra prediction parameter PP_Intra, the variable length code encoding unit 27 supplies the residual prediction mode index RIPM to the encoded data # 1 via the prediction scheme control unit 21d. It is assumed to be included
  • the intra predicted image generation unit 21a may be configured to generate a predicted image Pred_Intra from the filtered decoded image P_ALF by intra-screen prediction.
  • the motion vector detection unit 21b detects a motion vector mv regarding each partition. Specifically, (1) the filtered decoded image P_ALF ′ used as the reference image is selected, and (2) the target partition is searched by searching for the region that best approximates the target partition in the selected filtered decoded image P_ALF ′. Detects a motion vector mv.
  • the filtered decoded image P_ALF ′ is an image obtained by performing deblocking processing by the deblocking filter 26 and filtering processing by the adaptive filter 27 on the decoded image that has already been decoded for the entire frame.
  • the motion vector detection unit 21b can read out the pixel value of each pixel constituting the filtered decoded image P_ALF ′ from the frame memory 25.
  • the motion vector detection unit 21b supplies the detected motion vector mv to the inter predicted image generation unit 21c and the motion vector redundancy deletion unit 21e together with the reference image index RI that specifies the filtered decoded image P_ALF ′ used as the reference image. To do.
  • the inter prediction image generation unit 21c generates a motion compensated image mc related to each inter prediction partition by inter-screen prediction. Specifically, the motion compensation image mc is obtained from the filtered decoded image P_ALF ′ designated by the reference image index RI supplied from the motion vector detection unit 21b, using the motion vector mv supplied from the motion vector detection unit 21b. Generate. Similar to the motion vector detection unit 21b, the inter prediction image generation unit 21c can read out the pixel value of each pixel constituting the filtered decoded image P_ALF ′ from the frame memory 25. The inter prediction image generation unit 21c supplies the generated motion compensated image mc (inter prediction image Pred_Inter) together with the reference image index RI supplied from the motion vector detection unit 21b to the prediction method control unit 21d.
  • inter prediction image Pred_Inter inter prediction image Pred_Inter
  • the prediction scheme control unit 21d compares the intra predicted image Pred_Intra and the inter predicted image Pred_Inter with the encoding target image, and selects whether to perform intra prediction or inter prediction.
  • the prediction scheme control unit 21d supplies the intra prediction image Pred_Intra as the prediction image Pred to the adder 24 and the subtractor 29, and also uses the intra prediction parameter PP_Intra supplied from the intra prediction image generation unit 21a. This is supplied to the variable length code encoder 28.
  • the prediction scheme control unit 21d supplies the inter prediction image Pred_Inter as the prediction image Pred to the adder 24 and the subtractor 29, and the reference image index RI and motion vector redundancy described later.
  • the estimated motion vector index PMVI and the motion vector residual MVD supplied from the deletion unit 21e are supplied to the variable length code encoding unit 27 as an inter prediction parameter PP_Inter.
  • the motion vector redundancy deletion unit 21e deletes redundancy in the motion vector mv detected by the motion vector detection unit 21b. Specifically, (1) an estimation method used for estimating the motion vector mv is selected, (2) an estimated motion vector pmv is derived according to the selected estimation method, and (3) the estimated motion vector pmv is subtracted from the motion vector mv. As a result, a motion vector residual MVD is generated. The motion vector redundancy deleting unit 21e supplies the generated motion vector residual MVD to the prediction method control unit 21d together with the estimated motion vector index PMVI indicating the selected estimation method.
  • the transform / quantization unit 22 performs (1) DCT transform (Discrete Cosine Transform) for each block (transform unit) on the prediction residual D obtained by subtracting the predicted image Pred from the encoding target image, and (2) obtains the DCT transform.
  • the obtained DCT coefficients are quantized, and (3) the quantized prediction residual QD obtained by the quantization is supplied to the variable length code encoding unit 27 and the inverse quantization / inverse transform unit 23.
  • the transform / quantization unit 22 (1) selects a quantization step QP to be used for quantization for each TU, and (2) sets a quantization parameter difference ⁇ qp indicating the size of the selected quantization step QP.
  • the inverse quantization / inverse transform unit 23 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 24.
  • the quantization prediction residual QD is inversely quantized, the quantization step QP supplied from the transform / quantization unit 22 is used.
  • the prediction residual D output from the inverse quantization / inverse transform unit 23 is obtained by adding a quantization error to the prediction residual D input to the transform / quantization unit 22. Common names are used for this purpose.
  • the adder 24 adds the predicted image Pred selected by the prediction scheme control unit 21d to the prediction residual D generated by the inverse quantization / inverse transform unit 23, thereby obtaining the (local) decoded image P. Generate.
  • the (local) decoded image P generated by the adder 24 is supplied to the deblocking filter 27 and stored in the frame memory 25, and is used as a reference image in intra prediction.
  • variable-length code encoding unit 28 (Variable-length code encoding unit 28) The variable-length code encoding unit 28 (1) the quantization prediction residual QD and ⁇ qp supplied from the transform / quantization unit 22, and (2) the quantization parameter PP (inter prediction) supplied from the prediction scheme control unit 21d. The parameter PP_Inter and the intra prediction parameter PP_Intra) and (3) the filter parameter FP supplied from the adaptive filter 27 is variable-length-encoded to generate encoded data # 1.
  • CABAC Context-based Adaptive Binary Arithmetic Coding
  • CAVLC Context-based Adaptive VLC
  • the subtracter 29 generates the prediction residual D by subtracting the prediction image Pred selected by the prediction method control unit 21d from the encoding target image.
  • the prediction residual D generated by the subtractor 29 is DCT transformed / quantized by the transform / quantization unit 22.
  • Deblocking filter 26 When the difference between the pixel values of pixels adjacent to each other via a block boundary or partition boundary in the decoded image P is smaller than a predetermined threshold, the deblocking filter 26 By performing a deblocking process on the partition boundary, the block boundary or an image near the partition boundary is smoothed.
  • the image subjected to the deblocking process by the deblocking filter 26 is stored in the frame memory 25 as a deblocked decoded image P_DB.
  • the adaptive filter 27 (1) sets one or a plurality of unit areas constituting the deblocked decoded image P_DB to any of a plurality of types according to the image characteristics of the deblocked decoded image P_DB in the unit area. (2) ON / OFF of adaptive filter processing for the unit area is switched according to the type to which the unit area belongs. Also, (3) filter coefficients for each type so that the error between the deblocked decoded image P_DB and the encoding target image (moving image # 10) in the unit area where the adaptive filter processing is set to ON is minimized. (4) Using the determined filter coefficient group, adaptive filter processing is performed on the deblocked decoded image P_DB.
  • FIG. 22 is a block diagram showing the configuration of the adaptive filter 27. As illustrated in FIG. 22, the adaptive filter 27 includes a region characteristic value calculation unit 271, a region classification unit 272, and a filter processing unit 273.
  • the region characteristic value calculation unit 271 reads each pixel value of the deblocked decoded image P_DB in the characteristic value calculation reference region including the target unit region to be processed from the frame memory 15, and based on the pixel value, the target unit A characteristic value (also referred to as an area characteristic value) representing the image characteristic of the deblocked decoded image P_DB in the area is calculated.
  • the characteristic value calculation reference region refers to a region formed of pixels referred to for calculating the characteristic value of the deblocked decoded image P_DB in the target unit region.
  • the characteristic value calculation reference area is composed of only the target unit area, or is composed of the target unit area and an area around the target unit area.
  • the region characteristic value calculation unit 271 includes an activity calculation unit 271a.
  • the activity calculation unit 271a calculates an activity ACT as the characteristic value of the deblocked decoded image P_DB in the target unit region. Further, the calculated activity ACT for each divided region is supplied to the region classification unit 272.
  • the specific method for calculating the activity level by the activity level calculation unit 271a is the same as the method for calculating the activity level by the activity level calculation unit 171a included in the moving image decoding apparatus 1, and thus the description thereof is omitted here.
  • the region classification unit 272 classifies the target unit region into one of a plurality of types based on the activity ACT calculated by the region characteristic value calculation unit 271 as the characteristic value for the target unit region.
  • the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type).
  • the region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.
  • the area classification unit 272 may be configured to perform any one of the following classification processing examples 1-1 ′ to (classification processing examples 1-6 ′).
  • the region classification unit 272 is configured to selectively perform any one of the following (classification processing example 1-1 ′) to (classification processing example 1-6 ′), It is good also as a structure which includes the classification process designation
  • type 0 is a filter-off type
  • (Classification example 1-7 ′) Substantially the same processing as (classification processing example 1-7) by the region classification unit 172 is performed. However, when the region classification unit 172 performs the main classification process, the filter processing unit 273 has higher encoding efficiency for on / off of the filter processing for each type (for example, types A to D in FIG. 6G). The on / off designation information V indicating the determined on / off is supplied to the region classification unit 172.
  • the filter processing unit 273 classifies each unit region included in the target slice or the target LCU based on the type information supplied from the region classifying unit 272, and outputs the deblocked decoded image P_DB and the encoding target image (moving image). A filter coefficient group is determined for each type so that the error from # 10) is minimized.
  • an image made up of unit regions classified as type I in the deblocked decoded image P_DB in the target slice or the target LCU is represented as an image I, and coordinates of pixels included in the image I are (x,
  • the pixel value of the pixel that is y) is expressed as SI (x, y)
  • the pixel value of the pixel whose coordinates are (x, y) among the pixels included in the encoding target image in the target slice or the target LCU is ST ( x, y)
  • the filter processing unit 273 derives, for example, a filter coefficient aI (i, j) and an offset oI that minimize the square error E given by the following equation (5). .
  • the filter processing unit 273 derives a filter coefficient group I including the filter coefficient aI (i, j) and the offset oI for all unit areas classified into types other than type 0 by the area classifying unit 272. For example, when the target slice or the target LCU is configured from unit areas classified into type 0, type 1, and type 2, the filter processing unit 273 performs processing for the unit areas classified into type 1.
  • a filter coefficient group 1 and a filter coefficient group 2 for a unit region classified as type 2 are derived.
  • the filter processing unit 273 performs filtering on the deblocked decoded image P_DB in each unit region included in the target slice or the target LCU by performing filter processing using the filter coefficient group derived for each type.
  • a decoded image P_ALF is generated. For example, when the target slice or the target LCU is composed of unit areas classified into type 0, type 1, and type 2, the filter processing unit 273 performs the deblocked decoding in the target slice or the target LCU.
  • a filter process using the filter coefficient group 1 is performed on a unit area classified as type 1 in the image P_DB, and a filter process using the filter coefficient group 2 is performed on a unit area classified as type 2.
  • the filter coefficient group for each type derived by the filter processing unit 273 is supplied to the variable length code encoding unit 28 as a part of the filter parameter FP.
  • the activity calculation unit 271a included in the region characteristic value calculation unit 271 uses, as the characteristic value of the deblocked decoded image P_DB in the target unit region, the vertical activity ACT_V and the horizontal activity instead of the activity ACT described above.
  • the degree ACT_H may be calculated, and the region classification unit 272 may classify the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V.
  • the horizontal direction activity ACT_H and the vertical direction activity ACT_V can be calculated by, for example, the first term and the second term of the mathematical formula (1a) described above.
  • the horizontal direction activity ACT_H and the vertical direction activity ACT_V may be calculated by the first term and the second term of any one of the above formulas (1b) to (1d), respectively. it can.
  • the region classification unit 272 classifies the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V calculated by the activity calculation unit 271a.
  • the region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.
  • the classification processing by the region classification unit 272 according to the present modification is the same as the classification processing by the region classification unit 172 described in ⁇ Modification 1>, and thus description thereof is omitted here.
  • the adaptive filter 27 calculates an average of pixel values (also referred to as an average pixel value) as a characteristic value of the deblocked decoded image P_DB in the target unit region, and sets the target unit region to a plurality of types based on the calculated average pixel value. It is good also as a structure classified into either.
  • FIG. 23 is a block diagram showing a configuration of the adaptive filter 27 according to this modification.
  • the region characteristic value calculation unit 271 in the adaptive filter 27 according to this modification includes a pixel value reference unit 271b.
  • the pixel value reference unit 271b calculates the average pixel value MP by reading each pixel value of the deblocked decoded image P_DB in the target unit area to be processed from the frame memory 25 and taking the average of the pixel values.
  • the region classification unit 272 classifies the target unit region into one of a plurality of types based on the average pixel value MP calculated by the region characteristic value calculation unit 271 as the characteristic value for the target unit region.
  • the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type).
  • the region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.
  • the area classification unit 272 can be configured to perform any of the following classification processing examples (classification processing example 3-1 ′) to (classification processing example 3-7 ′).
  • the region classification unit 272 is configured to selectively perform any one of the following (classification processing examples 3-1 ′) to (classification processing examples 3-7 ′). It is good also as a structure which includes the classification process designation
  • the pixel value reference unit 271b included in the region characteristic value calculation unit 271 calculates the average pixel value MP and the variance VP of each pixel value as the characteristic value of the deblocked decoded image P_DB in the target unit region, and the region classification unit 272
  • the target unit area may be classified into one of a plurality of types based on the average pixel value MP and the variance VP.
  • the calculation process of the variance VP by the pixel value reference unit 271b according to this modification is the same as the calculation process of the variance VP by the pixel value reference unit 171b described in ⁇ Modification 3>, and thus the description thereof is omitted here.
  • the region classification unit 272 sets the target unit region to one of a plurality of types based on the average pixel value MP and the variance VP calculated by the region characteristic value calculation unit 271 as the characteristic value for the target unit region.
  • the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type).
  • the region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.
  • the classification processing by the region classification unit 272 according to the present modification is the same as the classification processing by the region classification unit 172 described in ⁇ Modification 3>, and thus description thereof is omitted here.
  • the adaptive filter 27 determines whether or not to perform adaptive filter processing on the target slice, and supplies a syntax adaptive_loop_filter_flag indicating the determined result to the variable-length code encoding unit 28; (2) a function for determining whether to control on / off of the adaptive filter processing for each coding unit, and supplying a syntax alf_cu_control_flag indicating the determined result to the variable-length code encoding unit 28; and (3) on / off control.
  • the maximum division depth (maximum split depth) from the maximum coding unit is determined for the coding unit to be processed, and the syntax (hierarchy designation information) alf_cu_control_max_depth indicating the determined result is input to the variable-length code encoding unit 28
  • the function to be supplied and (4) the number of coding units to be subjected to on / off control is determined, and the syntax alf_length_cu_control_info indicating the determined result is variable (5)
  • a syntax (on / off flag) alf_cu_flag [i] indicating the result of determining the on / off of the adaptive filter processing for each coding unit to be subjected to on / off control ] To the variable-length code encoding unit 28.
  • the adaptive filter 27 according to this modification can be configured to perform the determinations (1) to (5) so that the encoding efficiency is further improved.
  • the target unit area for performing characteristic calculation is determined according to the control unit determined by the hierarchy designation information, it is necessary to perform characteristic calculation processing according to the structure of the control unit. In this case, the processing becomes complicated with many branches, and the amount of calculation increases, and the scale of software and circuits increases. As described above, when the target unit area for performing characteristic calculation is determined without depending on the hierarchy designation information, the complexity of characteristic value calculation is reduced.
  • a video encoding device it is desirable to test encoding in a plurality of hierarchy designation information, and to perform optimization such as selecting and coding the optimum hierarchy.
  • characteristic calculation is performed depending on the hierarchy designation information. When the position and size of the unit area to be changed are changed, it is necessary to redo the characteristic calculation every time encoding in different hierarchy designation information is tested. Such recalculation is not appropriate because it increases the amount of calculation.
  • the region classification unit 272 refers to the characteristic value for the target unit region and the on / off flag alf_cu_flag [i] for the encoding unit including the target unit region, and sets the target unit region to a plurality of types. It classifies in either.
  • the classification processing by the region classification unit 272 according to the present modification is the same as the classification processing by the region classification unit 172 described in ⁇ Modification 4>, and thus description thereof is omitted here.
  • the region characteristic value calculation unit 271 includes a directionality calculation unit 271c in addition to the activity calculation unit 271a.
  • the directionality calculation unit 271c calculates the edge direction Theta in the same manner as the directionality calculation unit 171c described in ⁇ Modification 5>.
  • the classification process by the area classification unit 272 according to this modification is the same as the classification process by the area classification unit 172 described in ⁇ Modification 5>.
  • the region characteristic value calculation unit 271 according to the present modification includes a pixel value reference unit 271b in addition to the activity calculation unit 271a.
  • the classification process by the area classification unit 272 according to the present modification is the same as the classification process by the area classification unit 172 described in ⁇ Modification 6>.
  • the image filter device includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas.
  • a characteristic value calculation means for calculating the filter area, and a filter-on area that causes the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculation means, or the filter And classifying means for classifying the filter into any one of the filter-off regions in which the means is not operated.
  • the classification means determines each unit area constituting the input image according to the value of the characteristic value representing the image characteristic of the input image in the unit area, and Classify into any of the filter-off areas.
  • the filter means acts on the input image in the unit area belonging to the filter-on area, but does not act on the input image in the unit area belonging to the filter-off area.
  • the image filter device configured as described above, it is possible to switch on / off of the filter process according to the local image characteristics of the input image. Therefore, by using the output image obtained by the image filter device acting on the decoded image while switching on / off of the filtering process as a reference image for generating the predicted image, the prediction accuracy and the coding efficiency are improved as compared with the conventional case. Can be improved.
  • the unit region is, for example, a unit for generating a predicted image. It may be a certain prediction unit (partition), may be a conversion unit that is a unit of frequency conversion, or may be other than those.
  • the unit area may be a unit composed of only one pixel.
  • the classifying unit further classifies each unit region belonging to the filter-on region into one of a plurality of types according to the value of the characteristic value for the unit region calculated by the characteristic value calculating unit.
  • the filter means preferably calculates the pixel value of each pixel in each unit region belonging to the filter-on region using a filter coefficient group set for a type including the pixel.
  • the classification unit assigns each of the unit areas belonging to the filter-on area to any one of a plurality of types according to the value of the characteristic value for the unit area calculated by the characteristic value calculation unit.
  • the filter means calculates the pixel value of each pixel in each unit region belonging to the filter-on region using a filter coefficient group set for the type including the pixel. Therefore, according to the above configuration, even when the image characteristics of the input image are different for each unit region, it is possible to perform the filtering process using the filter coefficient group set for each type. That is, even when the image characteristics of the input image are different for each unit region, an output image that is more suitable for generating a predicted image can be generated. Therefore, prediction accuracy and encoding efficiency can be improved by using an output image generated by the image filter device according to the present invention configured as described above as a reference image for generating a predicted image.
  • the characteristic value calculation means calculates the activity of the input image in the unit area as a characteristic value for each unit area.
  • the characteristic value calculating means calculates the activity as a characteristic value representing the image characteristic of the input image in each unit area. Therefore, according to the above configuration, an output image that is more suitable for generating a predicted image can be generated even when the input image has a different degree of activity for each unit region.
  • the activity level can also be expressed as an index indicating the randomness of the image.
  • the characteristic value calculation means calculates the activity of each pixel belonging to the internal region, the entire neighborhood of which is composed of pixels included in the unit region, as the pixel value of the pixel included in the vicinity of the pixel. It is preferable to calculate the activity of the unit region from the activity of each pixel belonging to the inner region after being derived with reference to FIG.
  • the activity of each pixel belonging to the internal region whose entire neighborhood is composed of pixels included in the unit region is referred to the pixel value of the pixel included in the vicinity of the pixel. Then, the activity for the unit region is calculated from the activity of each pixel belonging to the internal region. Therefore, according to the above configuration, the degree of activity for each unit region can be calculated by referring to only the pixels in the unit region. In other words, the degree of activity for a certain unit area can be calculated without referring to pixels that are not included in the unit area.
  • the characteristic value calculation means calculates an average of pixel values of the input image in the unit area as a characteristic value for each unit area.
  • the characteristic value calculating means calculates an average of the pixel values in the unit area as a characteristic value representing the image characteristic of the input image in each unit area. Therefore, according to the above configuration, even when the input image has an average of different pixel values for each unit region, an output image that is more suitable for generating a predicted image can be generated.
  • the characteristic value calculation means calculates a plurality of characteristic values having different calculation methods for each unit region.
  • the characteristic value calculation means calculates a plurality of characteristic values having different calculation methods for each unit region. Therefore, according to the above configuration, each unit region can be classified more appropriately, so that the encoding efficiency can be further improved.
  • the characteristic value calculation means calculates an average of pixel values of the input image in the unit area and a variance of pixel values of the input image in the unit area as characteristic values for each unit area. Is preferred.
  • the characteristic value calculation means includes, as characteristic values for each unit region, an average of the pixel values of the input image in the unit region, and a variance of the pixel values of the input image in the unit region. Is calculated. Therefore, according to the above configuration, each unit region can be classified more appropriately, so that the encoding efficiency can be further improved.
  • the dispersion of pixel values can also be regarded as an index indicating the randomness of the image.
  • the variance of the pixel values is obtained by taking the difference between the square average of the pixel values and the square of the average of the pixel values.
  • the average of the pixel values used for calculating the variance of the pixel values the average value of the pixel values calculated as one of the characteristic values can be used as it is, while suppressing an increase in the processing amount. Thus, each unit area can be classified more appropriately.
  • the characteristic value calculation means calculates the vertical activity of the input image in the unit region and the horizontal activity of the input image in the unit region as the characteristic value for each unit region. Is preferable.
  • the characteristic value calculation means uses the vertical activity of the input image in the unit region and the horizontal activity of the input image in the unit region as the characteristic value for each unit region. Calculate the degree. Therefore, according to said structure, according to the directionality of the activity of an input image, each unit area
  • the classification unit obtains an on / off flag for designating on / off of the filtering process for each predetermined unit including one or more unit areas from outside, and the unit area included in the unit for which the on / off flag designates on Is preferably classified into either the filter-on region or the filter-off region.
  • the unit area included in the unit in which the on / off flag designates on is classified into either the filter on area or the filter off area. Therefore, according to the above configuration, the code amount of data to be referred to is reduced as compared with the configuration in which the on / off of the filtering process is switched only by the on / off flag.
  • the predetermined unit is, for example, an encoding / decoding unit.
  • a coding unit (CU: Coding Unit) can be used.
  • the predetermined unit related to the on / off flag is hierarchically specified by hierarchical specification information acquired from the outside, and the unit region is composed of pixels that are equal to or smaller than the minimum encoding unit and equal to or larger than four pixels. It is preferable that the area is determined regardless of the hierarchy designation information.
  • the predetermined unit is hierarchically specified by hierarchical designation information acquired from the outside, and the unit area is determined without depending on the hierarchical designation information. Therefore, according to the above configuration, it is possible to switch the filtering process on and off for each finer unit while reducing the code amount of the hierarchy designation information. In addition, the complexity of the characteristic value calculation is reduced as compared with the case of changing with the hierarchy designation information. In addition, in the encoding device, it is not necessary to recalculate the characteristic value when determining the optimum hierarchy designation information, so that the amount of calculation is reduced. Moreover, according to said structure, the said unit area
  • the decoding device is a decoding device that decodes encoded data and generates a decoded image after filtering, wherein the image filter device and the predicted image in each unit region are represented by the image filter device.
  • a prediction image generation unit that generates a reference image by referring to the generated output image, and the image filter device includes a prediction image generated by the prediction image generation unit, a residual image decoded from the encoded data, and It is characterized in that the decoded image obtained by adding is used as an input image, and the decoded image after filtering is generated as an output image.
  • the image filter device included in the decoding device includes the prediction image generated by the prediction image generation unit and the residual decoded from the encoded data. Since the decoded image obtained by adding the difference image is used as the input image, and the filtering process is switched on / off for each unit area according to the characteristic value representing the image characteristic of the input image (decoded image), the input image Even when the image characteristics are different for each unit region, an output image suitable as an image to be referred to for generating a predicted image can be generated. Therefore, according to the above configuration, the prediction accuracy of the predicted image is improved.
  • the encoding device having the configuration corresponding to the above configuration it is possible to improve the prediction accuracy of the predicted image even when the image characteristics of the encoding target image in each unit region are different for each unit region. Therefore, encoded data with high encoding efficiency can be generated. Moreover, according to the decoding apparatus having the above configuration, it is possible to appropriately decode such encoded data with high encoding efficiency.
  • An encoding apparatus is an encoding apparatus that generates encoded data by encoding a residual image between an encoding target image and a prediction image, the image filter device, and each unit Prediction image generation means for generating the prediction image in the region with reference to the output image generated by the image filter device, and the image filter device includes the prediction image generated by the prediction image generation means and the prediction image generation device.
  • a decoded image obtained by adding the residual image is used as an input image, and an output image is generated.
  • the image filter apparatus included in the encoding apparatus includes a prediction image generated by the prediction image generation unit, an encoding target image, and a prediction image. Since the decoded image obtained by adding the residual images is used as the input image, and the filtering process is switched on and off for each unit area according to the characteristic value representing the image characteristics of the input image (decoded image), the image of the input image Even when the characteristics are different for each unit region, it is possible to generate an output image suitable as an image to be referred to for generating a predicted image. Therefore, according to the above configuration, the prediction accuracy of the predicted image is improved. Since the prediction accuracy is improved, the encoding device can generate encoded data with high encoding efficiency.
  • the filter coefficient group used by the filter means minimizes the difference between the encoding target image and the output image in each unit region included in the filter-on region. It is preferable that it is determined to be.
  • the filter coefficient group used by the filter means minimizes the difference between the encoding target image and the output image in each unit area included in the filter-on area for the filter-on area. Therefore, the encoding device including the filter unit can generate a prediction image with high prediction accuracy. Thereby, encoded data with high encoding efficiency can be generated.
  • the filter coefficient group used by the filter unit is an encoding target image and the output image. It is determined to minimize the difference for each type.
  • the data structure of the encoded data according to the present invention includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas.
  • the characteristic value calculation means for calculating, and the filter-on region in which the filter means is activated or the filter means is not activated in accordance with the value of the characteristic value for each unit area calculated by the characteristic value calculation means.
  • a data structure of encoded data that is referred to by an image filter device including classification means for classifying into any of the filter-off regions, and includes a filter coefficient group used by the filter means It is a feature.
  • the decoding apparatus that refers to the encoded data configured as described above represents filter means that operates on an input image composed of a plurality of unit regions, and represents the image characteristics of the input image for each of the plurality of unit regions.
  • a characteristic value calculating means for calculating a characteristic value, and a filter-on area or the filter means for causing the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculating means.
  • classifying means for classifying it into any one of the filter-off regions that do not act on the filter.
  • the filter means performs filter processing using a filter coefficient group included in the encoded data. Therefore, such a decoding apparatus can generate a prediction image with high prediction accuracy.
  • the encoded data referred to by such a decoding apparatus does not need to include a flag for switching on / off of the filtering process, the code amount of the encoded data is reduced. Therefore, the encoded data is encoded data with high encoding efficiency.
  • a calculation unit that calculates an average of pixel values of the input image, and each unit region is calculated by the calculation unit.
  • the classification means for classifying into any of a plurality of types, and the pixel value of each pixel in each unit area are set for the type including the pixel And filtering means for calculating using a filter coefficient group.
  • each unit area is classified into one of a plurality of types according to the average of the pixel values for the unit area calculated by the calculation unit,
  • the pixel value of each pixel in the unit area is calculated using the filter coefficient group set for the type including the pixel. Therefore, according to the above configuration, the filter processing can be performed using the filter coefficient group set for each type, so that the encoding efficiency is improved. Further, according to the above configuration, each unit area is classified according to a characteristic value with a small calculation processing amount, ie, an average of pixel values, so that the processing amount can be reduced.
  • Each block of the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central Processing Unit). You may implement
  • IC chip integrated circuit
  • CPU Central Processing Unit
  • each device includes a CPU that executes instructions of a program that realizes each function, a ROM (Read (Memory) that stores the program, a RAM (Random Memory) that expands the program, the program, and various types
  • a storage device such as a memory for storing data is provided.
  • An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of each of the above devices, which is software that realizes the above-described functions, is recorded so as to be readable by a computer. This can also be achieved by supplying to each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).
  • Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R.
  • IC cards including memory cards
  • semiconductor memories such as mask ROM / EPROM / EEPROM / flash ROM, or PLD (Programmable logic device) or FPGA (Field Programmable Gate Array) Logic circuits can be used.
  • each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network.
  • the communication network is not particularly limited as long as it can transmit the program code.
  • the Internet intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like can be used.
  • the transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type.
  • wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by radio such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA), mobile phone network, satellite line, and digital terrestrial network.
  • wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by radio such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA), mobile phone network, satellite line, and digital terrestrial network.
  • NFC High Data Rate
  • NFC Near Field Communication
  • DLNA Digital Living Network Alliance
  • a moving image encoding device that generates encoded data by encoding the moving image, and decoding by decoding the encoded data
  • a video decoding device (decoding device) that generates an image
  • a specific moving picture encoding method for example, H.264 is used.
  • H.264 / MPEG-4. Examples thereof include a method adopted in KTA software which is a codec for joint development in AVC and VCEG (Video Coding Expert Group).
  • an image (picture) constituting a moving image is obtained by dividing a slice obtained by dividing an image, a macroblock obtained by dividing the slice, and a macroblock. It is managed by a hierarchical structure consisting of blocks to be encoded, and is usually encoded for each block.
  • a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and difference data between the predicted image and the input image is encoded. It becomes.
  • methods for generating a predicted image methods called inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction) are known.
  • intra prediction predicted images in a corresponding frame are sequentially generated based on a locally decoded image in the same frame.
  • one prediction direction is selected from prediction directions included in a predetermined prediction direction (prediction mode) group for each prediction unit (for example, block), and A prediction pixel value on the prediction target region is generated by extrapolating the pixel value of the reference pixel in the locally decoded image in the selected prediction direction.
  • inter prediction by applying motion compensation using a motion vector to a reference image in a reference frame (decoded image) in which the entire frame is decoded, a predicted image in a prediction target frame is converted into a prediction unit ( For example, it is generated for each block).
  • Non-Patent Document 3 divides a (local) decoded image into a plurality of regions, and determines each region according to the level of activity indicating the local randomness of the (local) decoded image in each region.
  • the filter coefficient group for each region is determined by the encoding device so as to minimize the error between the filtered image in the region and the encoding target image.
  • An encoding device and a decoding device provided with such an adaptive filter generate a prediction image with reference to a filtered decoded image obtained by performing filter processing on the (local) decoded image using the adaptive filter. As a result, it is possible to improve the prediction accuracy and the encoding efficiency. (Task) However, the conventional adaptive filter still has room for improving the encoding efficiency.
  • a method of switching the filter coefficient group for each region based on region characteristics other than the activity in addition to the activity may be considered.
  • the filter coefficient group is switched for each region based on the two region characteristics, there is a problem that the calculation cost on the encoding device side and the decoding device side becomes high.
  • a filter device includes a region characteristic calculation unit that calculates a value of a region characteristic of each unit region in a filter device including a linear filter in which filter coefficients are adaptively set.
  • the linear filter is an offset that depends on the region characteristic for performing a filter operation on each pixel belonging to each unit region, and the value of the region characteristic of the unit region calculated by the region characteristic calculating unit This is performed using an offset according to the above and a filter coefficient group that does not depend on the region characteristics.
  • the region characteristic has a value corresponding to a color difference component type, and the linear filter performs the filter operation using an offset corresponding to the color difference component type. preferable.
  • the optimum offset value (maximizing encoding efficiency) is different. Therefore, according to the above configuration, higher encoding efficiency can be achieved than when the filter operation is performed using an offset that does not depend on the type of the color difference signal.
  • a filter device is a filter device including a linear filter in which filter coefficient groups are adaptively set, and a pixel value of a target pixel in an output image of the linear filter And truncating means for truncating the change amount, which is the difference between the pixel value of the target pixel in the input image of the linear filter.
  • the filter device includes a flag decoding unit that decodes a flag indicating whether or not to truncate the change amount, and outputs an output image of the linear filter by truncating the change amount or outputting the change amount. It is desirable to further include a switch for switching whether to output without truncation according to the value of the flag decoded by the flag decoding means.
  • the filter device is used to truncate the change amount when executing a region characteristic calculation unit that calculates a region characteristic of each unit region and a filter process that uses a pixel belonging to each unit region as an output pixel. It is desirable that the apparatus further comprises threshold value determining means for determining the threshold value in the unit area according to the area characteristic calculated by the area characteristic calculating means.
  • the threshold value determination means may determine the threshold value based on a correspondence relationship between the predetermined region characteristic and the threshold value, or may determine the region characteristic determined on the encoding device side.
  • the threshold value may be determined based on a correspondence relationship with the region characteristic.
  • a configuration further including a threshold table decoding unit that decodes a threshold table recorded by associating the region characteristics with the threshold is employed.
  • a filter device including a linear filter in which filter coefficients are adaptively set
  • at least one first filter coefficient group depending on the first region characteristic corresponds to the first filter coefficient group.
  • the first filter coefficient table recorded in association with the value of the region characteristic and the second filter coefficient group depending on the second region characteristic are recorded in association with the corresponding second region characteristic value.
  • each unit area can be obtained without decoding a filter coefficient table in which a plurality of filter coefficient groups depending on both the first area characteristic and the second area characteristic are recorded in association with the corresponding area characteristic pair.
  • the filter operation according to both the first region characteristic and the second region characteristic can be performed. That is, according to the above configuration, a decoded image close to the original image can be generated with high accuracy without causing a significant increase in the code amount. As a result, higher encoding efficiency than conventional can be realized.
  • the filter device includes: a flag decoding unit that decodes a flag indicating whether or not to use the second filter coefficient group for a filter operation in the linear filter; the first filter coefficient group in the linear filter; Whether to output an output image obtained by a filter operation using the second filter coefficient group to the outside or to output an output image obtained by a filter operation using only the first filter coefficient group in the linear filter to the outside It is desirable to further include a switch that switches according to the value of the flag decoded by the flag decoding means.
  • a decoding device and an encoding device including the filter device are also included in the scope of the present invention. (Effect of the above configuration) According to the present invention, it is possible to realize an adaptive filter having higher encoding efficiency than the conventional one without causing a significant increase in calculation cost.
  • Embodiment 2 A second embodiment of the present invention will be described with reference to FIGS.
  • the adaptive filter 100 shown in FIG. 26 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 110 and a filter calculation unit 120.
  • the filter setting unit 110 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 111.
  • the filter coefficient group w decoded by the filter coefficient decoding unit 111 is supplied to the filter calculation unit 120.
  • the filter calculation unit 120 is a means for executing a filter calculation, and includes a linear filter 121, a change amount calculation unit 122, a change amount restriction unit 123, a pixel value change unit 124 (truncating means), and a threshold value determination unit 125. ing.
  • the linear filter 121 has pixel values s [1], s [2],..., S [N] of reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (A1). From this, the pixel value s ′ of the target pixel in the output image is calculated.
  • s ′ w [1] s [1] + w [2] s [2] +... + w [N] s [N] + c ... (A1)
  • w [1], w [2],..., W [N], c are elements of the filter coefficient group w decoded by the filter coefficient decoding unit 121.
  • the calculated pixel value s ′ is supplied to the subsequent change amount limiting unit 123.
  • the change amount calculation unit 122 subtracts the pixel value s ′ of the target pixel in the output image of the linear filter 121 from the pixel value s of the same pixel (a pixel occupying the same position as the target pixel) in the input image of the linear filter 121.
  • the calculated difference ⁇ s s ⁇ s ′ is supplied to the change amount limiting unit 123 at the subsequent stage. In the present specification, this difference ⁇ s is described as “amount of change”.
  • the change amount limiting unit 123 truncates the change amount ⁇ s according to the equation (A2).
  • the truncated change amount ⁇ s ′ is supplied to the subsequent pixel value changing unit 124.
  • the change amount limiting unit 123 may truncate the change amount ⁇ s according to the equation (A2 ′).
  • the pixel value changing unit 124 subtracts the truncated change amount ⁇ s ′ from the pixel value s in the input image of the linear filter 121.
  • the calculated difference s ′′ s ⁇ s ′ is output to the outside of the filter calculation unit 120.
  • the threshold determination unit 125 determines the threshold th used for truncating the change amount ⁇ s.
  • a predetermined threshold th0 is used as the threshold th used to truncate the change amount ⁇ s.
  • the threshold value determination unit 125 reads the threshold value th0 from a memory (not shown) and notifies the change amount restriction unit 123 described above.
  • the filter calculation part 120 should just obtain the same calculation result as the calculation result mentioned above, and does not need to follow the calculation procedure same as the calculation procedure mentioned above.
  • the difference is calculated by the change amount calculation unit 122.
  • the elements of the filter coefficient group w decoded by the filter coefficient decoding unit 121 may be quantized filter coefficients.
  • the linear filter 121 may be configured such that (a) the quantized filter coefficient is dequantized and then the filter operation using the dequantized filter coefficient is performed, and (b) A configuration in which the filter operation using the filter coefficient before inverse quantization is performed and then the result of the filter operation is shifted to the right may be employed.
  • the filter coefficient after quantization is 256 times the filter coefficient before quantization, so by shifting the filter operation result to the right by 8 bits
  • the same calculation result can be obtained except for calculation errors such as roundoff error and the case of performing filter calculation using filter coefficients before dequantization.
  • the coding apparatus may be equipped with the adaptive filter 100 'shown in FIG. As shown in FIG. 27, in the adaptive filter 100 ′ on the encoding device side, the filter coefficient decoding unit 111 in the adaptive filter 100 on the decoding device side is replaced with a filter coefficient determination unit 111a and a filter coefficient encoding unit 111b. Configuration is taken.
  • the filter coefficient determination unit 111a minimizes the square error with respect to the original image (encoding target image) of the output image of the linear filter 121 using the least square method.
  • the filter coefficient encoding unit 111b encodes the filter coefficient group w determined by the filter coefficient determination unit 111a.
  • the other configuration of the adaptive filter 100 ′ on the encoding device side is the same as that of the adaptive filter 100 on the decoding device side, and the description thereof is omitted here.
  • the adaptive filter 100 shown in FIG. 28 is an improvement of the adaptive filter 100 according to the first configuration example, and is suitable for mounting in a decoding device, like the adaptive filter 100 according to the first configuration example. .
  • the differences between the adaptive filter 100 according to this configuration example and the adaptive filter 100 according to the first configuration example are the following two points.
  • a restriction flag decoding unit 112 is added to the filter setting unit 110.
  • the restriction flag decoding unit 112 is means for decoding the restriction flag from the encoded data.
  • the restriction flag refers to a flag that specifies whether to output the pre-truncated pixel value s ′′ or the post-truncated pixel value s ′ as the pixel value of the target pixel in the output image.
  • a switch 126 is added to the filter calculation unit 120.
  • the switch 126 determines whether to output the post-truncated pixel value s ′′ or the pre-truncated pixel value s ′ as the pixel value of the target pixel in the output image. It is means for switching according to the value of the flag.
  • the encoding apparatus may be equipped with the adaptive filter 100 ′ shown in FIG. 29.
  • the restriction flag decoding unit 112 of the adaptive filter 100 on the decoding device side instead of the restriction flag decoding unit 112 of the adaptive filter 100 on the decoding device side, a restriction flag determining unit 112a and a restriction flag encoding unit 112b are provided. It is added to the filter setting unit 110.
  • the restriction flag determination unit 112a determines whether or not the change amount truncation is valid for each unit area, and then the value of the restriction flag according to the determination result.
  • the restriction flag encoding unit 112b encodes the value of the restriction flag determined by the restriction flag determination unit 112a. For example, in the unit area in which the truncation of the change amount is determined to be valid, the limit flag determination unit 112a sets the first value (eg, “1”) as the limit flag and determines that the truncation of the change amount is not valid. In the unit area, the restriction flag is set to a second value (for example, “0”) different from the first value.
  • Other configurations of the adaptive filter 100 ′ on the encoding device side are the same as those shown in the first configuration example.
  • an arbitrary area obtained by dividing a frame can be used as a determination unit for determining whether or not the change amount truncation is effective.
  • M ⁇ N block M represents the width of the block, for example, takes a value such as 1, 2, 4, or 8.
  • N represents the height of the block, for example 1, 2, 4 or the like. Or a value such as 8, etc.
  • the restriction flag encoding unit 112b encodes a restriction flag for each slice and embeds the code in each slice header.
  • the switch 126 is switched for each slice.
  • the change amount truncation As a method for determining whether or not the change amount truncation is valid, for example, (1) an error when truncating the change amount (error with respect to the original image of the output image of the pixel value changing unit 124), and the change amount And (2-1) error when truncating the change amount, error when not truncating the change amount, and calculating the error when the change amount is not truncated (error relative to the original image of the output image of the linear filter 121) If the change amount is smaller than that, it is determined that the change amount truncation is effective. (2-2) If not, the change amount truncation is determined not to be effective.
  • the encoding efficiency when the change amount is truncated and the encoding efficiency when the change amount is not truncated are calculated, and (2-1) the change amount is truncated.
  • the change amount truncation is effective.
  • An adaptive filter 100 shown in FIG. 30 is an improvement of the adaptive filter 100 according to the second configuration example, and is suitable for mounting in a decoding device, like the adaptive filter 100 according to the second configuration example. .
  • the differences between the adaptive filter 100 according to this configuration example and the adaptive filter 100 according to the second configuration example are the following two points.
  • a region characteristic calculation unit 127 is added to the filter calculation unit 120.
  • the area characteristic calculation unit 127 calculates the area characteristic k of each unit area based on the input image (decoded image or decoded image_DB after deblocking).
  • the threshold value determination unit 125 determines the value of the threshold value th used for the filter calculation for pixels belonging to each unit area. More specifically, the threshold value th (k) associated with the region characteristic k calculated by the region characteristic calculation unit 127 is specified in the threshold value table TH described later.
  • the calculation unit of the region characteristic k may be any of a slice, macroblock, block, LCU, CT, CU, PT, PU, TT, TU, or M ⁇ N block.
  • the threshold value determination unit 125 sets the region of the macro block as the threshold value th used for the filter calculation for pixels belonging to the macro block for each macro block.
  • the value th (k) corresponding to the characteristic k is specified.
  • the region characteristic k that can be used to determine the threshold th includes, for example, activity.
  • the activity is a publicly known index indicating the local randomness of the decoded image.
  • the region characteristic calculation unit 127 calculates, for example, the activity k of each unit region according to the equation (A3).
  • (i, j) represents coordinates with the upper left corner of the target unit region as the origin, and S (i, j) represents a pixel (i in the input image (decoded image or decoded image_DB after deblocking)). , J), and ⁇ represents the sum over all the pixels constituting the target unit region.
  • Expression (A3) corresponds to evaluating the degree of activity using the sum of absolute values of differences from adjacent pixel values (Sum of Absolute Difference, SAD) as an index.
  • the activity is not limited to the one defined by the equation (A3) as long as it is an index of local randomness of the decoded image.
  • a method for determining the threshold th according to the activity k for example, a method of referring to the threshold table TH can be mentioned.
  • the threshold table TH for example, a table in which the activity k and the threshold th (k) corresponding to the activity k are recorded in association with each other can be used.
  • the threshold determination unit 125 specifies the threshold th (k) associated with the activity k calculated by the activity calculation unit 127 in the threshold table TH.
  • An example of the threshold table TH is shown in Table 1.
  • the value of the threshold th is decreased as the activity decreases. This is to prevent linking that may occur when a filter process with a large change amount ⁇ s is applied to a flat portion with low activity.
  • the threshold table TH referred to for determining the threshold th may be (1) created in advance or (2) created on the encoding device side based on the original image. Alternatively, (3) it may be created on the decoding device side based on the decoded image, but in this configuration example, a threshold table TH created in advance is referred to. That is, the threshold value determination unit 125 reads out and uses a predetermined threshold value table TH from the memory.
  • the adaptive filter 100 ′ shown in FIG. 31 may be installed in the encoding apparatus.
  • a region characteristic calculation unit 127 is added to the filter calculation unit 120 as in the adaptive filter 100 on the decoding device side. Similar to the region characteristic calculation unit 127 in the adaptive filter 100 on the decoding device side, the region characteristic calculation unit 127 in the adaptive filter 100 ′ on the encoding device side applies an input image (local decoded image or local decoded image_DB after deblocking) to the input image. Based on this, the region characteristic k of each unit region is calculated.
  • Other configurations of the adaptive filter 100 ′ on the encoding device side are the same as those shown in the second configuration example.
  • the threshold value th was demonstrated based on the area
  • region characteristic k for example, activity
  • the adaptive filter 100 shown in FIG. 29A is an improvement of the adaptive filter 100 according to the third configuration example, and is suitable for mounting in a decoding device, like the adaptive filter 100 according to the third configuration example. Is.
  • the differences between the adaptive filter 100 according to this configuration example and the adaptive filter 100 according to the third configuration example are the following four points.
  • a threshold table decoding unit 113 is added to the filter setting unit 110.
  • the threshold table decoding unit 113 decodes the threshold table TH from the encoded data.
  • the threshold value table TH refers to a table in which the area characteristic k and the threshold value th (k) corresponding to the area characteristic k are recorded in association with each other.
  • the threshold value determination unit 125 determines the threshold value th (k) used in the filter calculation for the pixels belonging to each unit area. More specifically, the threshold value th (k) associated with the region characteristic k calculated by the region characteristic calculation unit 127 is specified in the threshold table TH decoded by the threshold table decoding unit 113.
  • the filter coefficient decoding unit 111 in the adaptive filter 100 according to the third configuration example is replaced with a filter coefficient table decoding unit 114.
  • the filter coefficient table decoding unit 114 decodes the filter coefficient table W from the encoded data.
  • the filter coefficient table W refers to a table in which the region characteristic k and the filter coefficient group w (k) corresponding to the region characteristic k are recorded in association with each other.
  • a filter coefficient determination unit 128 is added to the filter calculation unit 120.
  • the filter coefficient determination unit 128 determines a filter coefficient group w (k) used in the filter calculation for pixels belonging to each unit region. More specifically, in the filter coefficient table W decoded by the filter coefficient table decoding unit 114, the filter coefficient group w (k) associated with the region characteristic k calculated by the region characteristic calculation unit 127 is specified.
  • the unit for calculating the region characteristic k may be any of a slice, macroblock, block, LCU, CT, CU, PT, PU, TT, TU, or M ⁇ N block.
  • the threshold value determination unit 125 determines a threshold value th (k) corresponding to the region characteristic k of the macro block for each macro block, and the filter coefficient determination unit 128.
  • a filter coefficient group w (k) corresponding to the region characteristic k of the macroblock is determined.
  • the coding apparatus may be equipped with the adaptive filter 100 'shown in FIG.
  • the adaptive filter 100 ′ on the encoding device side (1) instead of the threshold table decoding unit 113 in the adaptive filter 100 on the decoding device side, a threshold table creation unit 113 a and a threshold table encoding unit 113 b include a filter setting unit 110.
  • a filter coefficient table creation unit 114a and a filter coefficient table encoding unit 114b are added to the filter setting unit 110. .
  • the threshold value table creating unit 113a creates the threshold value table TH
  • the threshold value table coding unit 113b is created by the threshold value table creating unit 113a.
  • the table TH is encoded.
  • the threshold table creation unit 113a determines each element of the threshold table TH by the following processing.
  • the threshold value table creating unit 113a uses the filter coefficient table W created by the filter coefficient table creating unit 114a and threshold values TH1 to THNt (Nt is the total number of threshold values) prepared in advance, for each area characteristic k. Filter processing is performed when a threshold value of a type is used. A square error between the image after filtering and the encoding target image is calculated. Subsequently, a value that minimizes the square error among Nt square errors is set as the threshold th (k) for each region characteristic k.
  • the filter coefficient table creation unit 114a creates the filter coefficient table W
  • the filter coefficient table coding unit 114b performs the filter coefficient table creation unit.
  • the filter coefficient table W created by 114a is encoded.
  • the filter coefficient table creation unit 114a determines a filter coefficient table W that minimizes a square error with respect to the original image (encoding target image) of the output image of the linear filter 121 using the least square method.
  • the configuration in which the threshold table TH itself and the filter coefficient table W itself are encoded is shown, but the configuration of the adaptive filter 100 on the encoding device side is not limited to this.
  • the threshold table TH / filter coefficient table W is selected from a plurality of predetermined threshold tables TH / filter coefficient tables W, the index of the selected threshold table TH / filter coefficient table W is encoded.
  • the code obtained by encoding the index of the threshold table TH / filter coefficient table W is shorter than the code obtained by encoding the threshold table TH / filter coefficient table W itself. Therefore, if such a configuration is adopted, encoding efficiency can be improved.
  • Embodiment 3 A third embodiment of the present invention will be described with reference to FIGS.
  • the adaptive filter 200 shown in FIG. 34 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 210 and a filter calculation unit 220.
  • the filter setting unit 210 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 211.
  • the filter coefficient decoding unit 211 decodes the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 from the encoded data.
  • the first filter coefficient table W1 is a table in which a first region characteristic k1 described later and a first filter coefficient group w1 (k1) are recorded in association with each other
  • the second filter coefficient table W2 is a first filter coefficient table W2 described later. It is the table which recorded 2 area
  • the filter calculation unit 220 is a means for executing a filter calculation, and includes a first spatial filter 221, an offset addition unit 222, a second linear filter 223, an addition unit 224, and a region characteristic calculation unit 225.
  • the first spatial filter 221 uses the pixel values s1 [1], s1 [2],..., S1 [] of the reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (B1). N1], the pixel value s1 ′ of the target pixel in the output image is calculated.
  • w1 (k1) [1], w1 (k1) [2],..., W1 (k1) [N1] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later.
  • W1 (k1) [N1] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later.
  • W1 (k1) [1] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later.
  • W1 (k1) [N1] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later.
  • W1 (k1) [N1] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later.
  • the offset addition unit 222 is a means for adding the offset c to s1 ′ calculated by the first spatial filter 221.
  • c is an offset decoded by the filter coefficient decoding unit 221.
  • the sum s ⁇ b> 1 ′ + c calculated by the offset addition unit 222 is supplied to the subsequent addition unit 224.
  • the second spatial filter 223 uses the pixel values s2 [1], s2 [2],..., S2 [] of the reference pixel in the input image (decoded image or decoded image after deblocking) according to the equation (B2). N2], the pixel value s2 ′ of the target pixel in the output image is calculated.
  • s2 ′ w2 (k2) [1] s2 [1] + w2 (k2) [2] s2 [2] +. + W2 (k2) [N2] s2 [N2] (B2)
  • w2 (k2) [1], w2 (k2) [2],..., W2 (k2) [N2] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later.
  • This is an element of the second filter coefficient group w2 (k2) corresponding to the second region characteristic k2. More specifically, in the second filter coefficient table W2 decoded by the filter coefficient decoding unit 221, the second filter coefficient group w2 (k2) associated with the second region characteristic k2 calculated by the region characteristic calculation unit 225. Elements.
  • the pixel value s2 ′ calculated by the second spatial filter 223 is supplied to the subsequent addition unit 224.
  • the addition unit 224 adds s2 ′ calculated by the second spatial filter 223 to the sum s1 ′ + c calculated by the offset addition unit 222.
  • the sum s 1 ′ + c + s 2 ′ calculated by the adder 224 is output to the outside of the adaptive filter 200.
  • the region characteristic calculation unit 225 calculates a first region characteristic k1 and a second region characteristic k2 for each unit region.
  • the first region characteristic k1 and the second region characteristic k2 are different region characteristics, for example, activity and edge direction.
  • Arbitrary areas obtained by dividing a frame for example, slices, macroblocks, blocks, LCUs, CTs, CUs, PTs, PUs, TTs, TUs, or M ⁇ N blocks, a first area characteristic k1 and a second area
  • the calculation unit of the characteristic k2 can be used. For example, when the calculation unit of the first region characteristic k1 and the second region characteristic k2 is a macro block, the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) are updated for each macro book. Will be.
  • FIG. 35 is a diagram illustrating a setting example of the reference area of the first spatial filter 221.
  • the center pixel corresponds to the target pixel
  • the black pixel corresponds to the reference pixel.
  • white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1, the notation of “(k1)” is omitted in FIG.
  • the center pixel corresponds to the target pixel
  • the black pixel corresponds to the reference pixel.
  • white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the second region characteristic k2, the notation of “(k2)” is omitted in FIG.
  • the encoding device cannot calculate the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2).
  • the first filter coefficient group w1 (k1 ) and the second filter coefficient group w2 (k2) cannot be calculated.
  • FIG. 37 is a diagram illustrating another setting example of the reference region of the first spatial filter 221.
  • the center pixel corresponds to the target pixel
  • the black pixel corresponds to the reference pixel.
  • white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1, the notation of “(k1)” is omitted in FIG.
  • the center pixel corresponds to the target pixel
  • the black pixel corresponds to the reference pixel.
  • white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the second region characteristic k2, the notation of “(k2)” is omitted in FIG.
  • the reference region of the first spatial filter 221 is different from the reference region of the second spatial filter 223. Therefore, there is no possibility that the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) cannot be calculated in the encoding device.
  • the adaptive filter 200 includes (1) the first filter coefficient table W1 in which a plurality of first filter coefficient groups w1 (k1) are recorded in association with the corresponding first region characteristics k1, And a filter coefficient decoding unit 211 for decoding the second filter coefficient table W2 in which a plurality of second filter coefficient groups w2 (k2) are recorded in association with the corresponding second area characteristic k2, and (2) each unit area An area characteristic calculation unit 225 for calculating the first area characteristic k1 and the second area characteristic k2, and (3) a filter operation using each pixel belonging to each unit area as a target pixel, which is recorded in the first filter coefficient table W1.
  • the filter coefficient group w1 (k1) corresponding to the first area characteristic k1 of the unit area calculated by the area characteristic calculation unit 225
  • the filter coefficient group w2 (k2) corresponding to the second area characteristic k2 of the unit area calculated by the area characteristic calculation unit 225
  • a filter calculation unit 220 that executes a filter calculation using the above.
  • the adaptive filter 200 when the adaptive filter 200 according to this configuration example is used, a plurality of filter coefficient groups w (k1, k2) depending on both the first region characteristic k1 and the second region characteristic k2 are converted into corresponding region characteristic pairs (k1 , K2), the filter operation corresponding to both the first region characteristic k1 and the second region characteristic k2 of each unit region can be performed without decoding the filter coefficient table W recorded in association with the unit region. That is, if the adaptive filter according to this configuration example is used, a decoded image close to the original image can be accurately reproduced without causing a significant increase in the code amount.
  • the elements of the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) decoded by the filter coefficient decoding unit 221 may be quantized filter coefficients.
  • the offset c decoded by the filter coefficient decoding unit 221 may also be a quantized offset.
  • the linear filter comprising the first spatial filter and the offset adding unit 222 performs (a) a filter operation using the filter coefficient after the inverse quantization after dequantizing the quantized filter coefficient and the offset. Or (b) a configuration in which a filter operation using a filter coefficient and an offset before inverse quantization is performed, and then a result of the filter operation is shifted to the right.
  • the linear filter including the second spatial filter may be configured to perform a filter operation using the filter coefficient after inverse quantization after dequantizing the quantized filter coefficient
  • b) A configuration in which the filter operation using the pre-quantization filter coefficient is performed and then the result of the filter operation is shifted to the right may be employed.
  • the filter coefficient after quantization is 256 times the filter coefficient before quantization, so by shifting the filter operation result to the right by 8 bits The same calculation result as that of the configuration (a) is obtained.
  • the encoding apparatus may be equipped with the adaptive filter 200 'shown in FIG.
  • the filter coefficient decoding unit 211 in the adaptive filter 200 on the decoding device side is replaced with a filter coefficient table creation unit 211a and a filter coefficient table encoding unit 211b.
  • the replacement configuration is taken.
  • the filter coefficient table creation unit 211a creates the first filter coefficient table W1 and the second filter coefficient table W2, and (2) the filter coefficient table coding unit 211b. Encodes the first filter coefficient table W1 and the second filter coefficient table W2 created by the filter coefficient table creation unit 211aa.
  • the filter coefficient table creation unit 211a uses the least square method to minimize the square error with respect to the original image of the filter output image (the output image of the addition unit 224) and the second filter coefficient table W1.
  • the table W2 is determined.
  • 40 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 210 and a filter calculation unit 220.
  • the filter setting unit 210 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 211 and a second region characteristic use flag decoding unit 212.
  • the filter coefficient decoding unit 211 decodes the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 from the encoded data, as in the first configuration example.
  • the second region characteristic use flag decoding unit 212 decodes the second region characteristic use flag enable_dir from the encoded data.
  • the second region characteristic use flag enable_dir is a 1-bit specifying whether or not to perform a filter operation using the filter coefficient group w2 (k2) determined according to the second region characteristic k2 in the filter operation for each unit region.
  • the flag is “1” when the filter operation using the filter coefficient group w2 (k2) is to be performed, and the value “0” when the filter operation using the filter coefficient group w2 (k2) is not to be performed.
  • the setting unit of the second region characteristic use flag enable_dir is arbitrary, and may or may not be the same as the calculation unit of the first region characteristic k1 and the second region characteristic k2.
  • the filter calculation unit 220 is a means for performing a filter calculation, and includes a first spatial filter 221, an offset addition unit 222, a second linear filter 223, an addition unit 224, a region characteristic calculation unit 225, and a switch 226. ing.
  • the first spatial filter 221 has pixel values s1 [1] and s1 of reference pixels in the input image (decoded image or decoded image after deblocking) according to the above-described equation (B1). From [2],..., S1 [N1], the pixel value s1 ′ of the target pixel in the output image is calculated.
  • the offset addition unit 222 adds the offset c to s1 ′ calculated by the first spatial filter 221 as in the first configuration example. However, in this configuration example, the sum s1 ′ + c calculated by the offset adding unit 222 is supplied to the subsequent adding unit 224 and the switch 226. Note that the output image of the offset adding unit 222 is hereinafter also referred to as “one filter output image”.
  • the second spatial filter 223 has pixel values s2 [1] and s2 of reference pixels in the input image (decoded image or decoded image after deblocking) according to the above-described equation (B2). From [2],..., S2 [N2], the pixel value s2 ′ of the target pixel in the output image is calculated. The pixel value s ⁇ b> 2 ′ calculated by the second spatial filter 223 is supplied to the subsequent adding unit 224. However, when the value of the second region characteristic use flag relating to the unit region including the target pixel (the setting unit of the second region characteristic use flag) is “0”, the filter calculation shown in the above-described equation (B2) is executed. do not do.
  • the adding unit 224 adds s2 ′ calculated by the second spatial filter 223 to the sum s1 ′ + c calculated by the offset adding unit 222.
  • the sum s 1 ′ + c + s 2 ′ calculated by the adder 224 is supplied to the switch 226 at the subsequent stage.
  • the output image of the adding unit 224 is hereinafter also referred to as “two-filter output image”.
  • the region characteristic calculation unit 225 calculates the first region characteristic k1 and the second region characteristic k2 for each unit region (region characteristic calculation unit). For example, when the unit for calculating the first region characteristic k1 and the second region characteristic k2 is a macro block, the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) are provided for each macro book. Will be updated.
  • the switch 226 determines whether to output the one-filter output image or the two-filter output image to the value of the second region characteristic use flag enable_dir decoded by the second region characteristic use flag decoding unit 212. Switch accordingly. More precisely, the sum s1 ′ + c calculated by the offset adder 222 or the sum s1 ′ + c + s2 ′ calculated by the adder 225 is output as the pixel value of the target pixel in the output image. Is switched according to the value of the second region characteristic use flag enable_dir decoded by the second region characteristic use flag decoding unit 212.
  • the sum s1 ′ + c calculated by the offset adding unit 222 is output, and the second region characteristic use flag enable_dir is set.
  • the sum s1 ′ + c + s2 ′ calculated by the adder 225 is output.
  • the adaptive filter 200 according to the present configuration example (1) the second region characteristic indicating whether or not to perform the filter operation using the second filter coefficient group w2 (k2) corresponding to the second region characteristic k2.
  • a second region characteristic use flag decoding unit 212 that decodes the use flag is further provided.
  • the filter calculation unit 120 corresponds to the first region characteristic k1 according to the value of the second region characteristic use flag.
  • the filter operation using only one filter coefficient group w1 (k1) is executed, or the first filter coefficient group w1 (k1) corresponding to the first area characteristic k1 and the second filter corresponding to the second area characteristic k2 It is configured to switch whether to perform a filter operation using both of the coefficient group w2 (k2).
  • the adaptive filter 200 according to the present configuration example when used, only the first filter coefficient group w1 (k1) is applied to a unit region that does not need to be subjected to filter operation using the second filter coefficient group w2 (k2). The filter operation using is executed. Therefore, if the adaptive filter 200 according to this configuration example is used, it is possible to further reduce the calculation cost required for the filter operation on the unit region that does not need to be subjected to the filter operation using the second filter coefficient group w2 (k2).
  • the encoding apparatus may be equipped with the adaptive filter 200 'shown in FIG. As shown in FIG. 41, in the adaptive filter 200 ′ on the encoding device side, instead of the second region characteristic use flag decoding unit 212 in the adaptive filter 200 on the decoding device side, a second region characteristic use flag determining unit 212a and A second region characteristic use flag encoding unit 212b is added.
  • the filter coefficient table creation unit 211a creates the first filter coefficient table W1 and the second filter coefficient table W2, and (2) the filter coefficient table coding unit 211b. Encodes the first filter coefficient table W1 and the second filter coefficient table W2 created by the filter coefficient table creation unit 211aa.
  • the filter coefficient table creation unit 211a uses the least square method to minimize the square error with respect to the original image (encoding target image) of (a) 1 filter output image (output image of the offset addition unit 222).
  • a filter coefficient table pair (W1, W2) comprising a first filter coefficient table W1 and (b) a second filter coefficient table W2 that minimizes a square error with respect to the original image of the two-filter output image (the output image of the adder 224) ) And create.
  • the filter coefficient table encoding unit 211b generates the first filter coefficient generated by the filter coefficient table generating unit 211a when the least square error with respect to the original image is smaller in the 1 filter output image than in the 2 filter output image.
  • the first filter coefficient table W1 and the first filter coefficient table W1 created by the filter coefficient table creation unit 211a The 2 filter coefficient table W2 is encoded.
  • the second region characteristic use flag determination unit 212a performs the minimum square error with respect to the original image of the one filter output image and the minimum with respect to the original image of the two filter output image. After comparing with the square error, the value of the second region characteristic use flag is determined according to the comparison result.
  • the second region characteristic use flag encoding unit 212b is a second region characteristic use flag determining unit. The value of the second region characteristic use flag determined by 212a is encoded.
  • the necessary number of executions of the least squares method is six times (when the second filter coefficient group w2 (k2) is determined depending on the first filter coefficient group w1 (k1)) or four times (first filter coefficient group).
  • w1 (k1) is determined depending on the second filter coefficient group w2 (k2)). That is, according to this configuration example, when the filter coefficient group w (k1, k2) determined depending on both the first region characteristic k1 and the second region characteristic k2 is used as in the conventional case, the minimum two times of 24 times are used. Where multiplication is required, the number of times can be significantly reduced.
  • the 42 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 210 and a filter calculation unit 220.
  • the filter setting unit 210 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 211 and an expanded filter coefficient setting unit 213.
  • the filter coefficient decoding unit 211 decodes the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 from the encoded data, as in the second configuration example.
  • the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 decoded by the filter coefficient decoding unit 211 are supplied to the enlarged filter coefficient calculation unit 213.
  • the enlarged filter coefficient setting unit 213 creates an enlarged filter coefficient table W based on the first filter coefficient table W1, the offset c, and the second filter coefficient table W2.
  • the region characteristic pair (k1, k2) is a pair of the first region characteristic k1 and the second region characteristic k2, and the filter coefficient group w (k1, k2) is a filter coefficient of the expanded linear filter 227 described later. It is used as a group.
  • the expanded filter coefficient setting unit 213 includes each filter coefficient group w (k1, k2) included in the expanded filter coefficient table W so that the expanded linear filter 227 becomes a filter equivalent to the filter operation unit 220 according to the first configuration example. ) Is
  • the filter calculation unit 220 includes an enlarged linear filter 227 and a region characteristic calculation unit 225.
  • the magnifying linear filter 227 performs pixel values s [1], s [2],..., S [N] of reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (B3). ], The pixel value s ′ of the target pixel in the output image is calculated.
  • s ′ w (k1, k2) [1] s [1] + w (k1, k2) [2] s [2] +. + W (k1, k2) [N] s [N] + c (B3)
  • w (k1, k2) [1], w (k1, k2) [2],..., W (k1, k2) [N] are target pixels calculated by the region characteristic calculation unit 225. Is an element of the enlarged filter coefficient group w (k1, k2) corresponding to the first region characteristic k1 and the second region characteristic k2 of the unit region including.
  • an area characteristic pair (k1) composed of the first area characteristic k1 and the second area characteristic k2 calculated by the area characteristic calculation unit 225. , K2) are elements of the enlarged filter coefficient group w (k1, k2).
  • the area characteristic calculation unit 225 calculates the first area characteristic k1 and the second area characteristic k2 for each unit area, as in the first configuration example. For example, when the unit for calculating the first region characteristic k1 and the second region characteristic k2 is a macro block, the enlarged filter coefficient group w (k1, k2) is set for each macro book.
  • FIG. 43 is an enlarged view equivalent to the filter operation unit 220 including the first spatial filter 221 in which the reference area is set as shown in FIG. 35 and the second spatial filter 223 in which the reference area is set as shown in FIG.
  • FIG. 10 is a diagram illustrating a setting example of a reference region of the linear filter 227.
  • the center pixel corresponds to the target pixel
  • the black pixel corresponds to the reference pixel.
  • white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1 and the second region characteristic k2, the notation of “(k1, k2)” is omitted in FIG.
  • FIG. 44 is an enlargement equivalent to the filter operation unit 220 including the first spatial filter 221 in which the reference region is set as shown in FIG. 37 and the second spatial filter 223 in which the reference region is set as shown in FIG.
  • FIG. 10 is a diagram illustrating a setting example of a reference region of the linear filter 227.
  • the center pixel corresponds to the target pixel
  • the black pixel corresponds to the reference pixel.
  • white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1 and the second region characteristic k2, the notation of “(k1, k2)” is omitted in FIG.
  • the adaptive filter 200 according to the configuration example also includes (1) the first filter coefficient group w1 (k1) that depends on the first region characteristic k1, At least one first filter coefficient table W1 recorded in association with the corresponding first area characteristic k1 and at least one second filter coefficient group w2 (k2) depending on the second area characteristic k2 correspond to each other.
  • a filter coefficient decoding unit 211 that decodes the second filter coefficient table W2 recorded in association with the area characteristic k2, and (2) an area characteristic calculation unit 225 that calculates the first area characteristic k1 and the second area characteristic k2 of each unit area.
  • the adaptive filter 200 when the adaptive filter 200 according to this configuration example is used, a plurality of filter coefficient groups w (k1, k2) depending on both the first region characteristic k1 and the second region characteristic k2 are converted into corresponding region characteristic pairs (k1 , K2), the filter operation corresponding to both the first region characteristic k1 and the second region characteristic k2 of each unit region can be performed without decoding the filter coefficient table W recorded in association with the unit region. That is, if the adaptive filter according to this configuration example is used, a decoded image close to the original image can be accurately reproduced without causing a significant increase in the code amount.
  • FIG. ⁇ Embodiment 4 A fourth embodiment of the present invention will be described with reference to FIGS. 45 to 46.
  • FIG. 45 to 46 A fourth embodiment of the present invention will be described with reference to FIGS. 45 to 46.
  • 45 is an adaptive filter suitable for mounting on a decoding device, and includes a filter setting unit 310 and a filter calculation unit 320.
  • the filter setting unit 310 is a means for setting filter characteristics, and includes a spatial filter coefficient decoding unit 311 and an offset table decoding unit 312.
  • the spatial filter coefficient group w decoded by the spatial filter coefficient decoding unit 311 is supplied to a spatial filter 321 described later.
  • the offset table decoding unit 312 decodes the offset table C from the encoded data.
  • the offset table C refers to a table in which at least one offset c (k ′) depending on the region characteristic k ′ is recorded in association with the corresponding region characteristic k ′.
  • the offset table C decoded by the offset table decoding unit 312 is supplied to an offset addition unit 324 described later.
  • the filter calculation unit 320 is a means for executing a filter calculation, and includes a spatial filter 321, a region characteristic calculation unit 322, an offset determination unit 323, and an offset addition unit 324.
  • the spatial filter 321 has pixel values s [1], s [2],..., S [N] of reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (C1). From this, the pixel value s ′ of the target pixel in the output image is calculated.
  • s ′ w [1] s [1] + w [2] s [2] +... + w [N] s [N] ... (C1)
  • w [1], w [2],..., W [N] are elements of the spatial filter coefficient group w decoded by the spatial filter coefficient decoding unit 321.
  • the calculated pixel value s ′ is supplied to the subsequent offset addition unit 324.
  • the region characteristic calculation unit 322 calculates a region characteristic k ′ of each unit region in the input image (decoded image or decoded image after deblocking).
  • a region characteristic k ′ in the present configuration example for example, a luminance magnitude (band offset), an edge type (edge offset), or the like can be used.
  • k ′ Y / 16 (a value obtained by quantizing the luminance value Y of the target pixel) may be set.
  • the region characteristic k ′ calculated by the region characteristic calculation unit 322 is supplied to the offset determination unit 323.
  • the offset determination unit 323 determines the value of the offset c used for the filter calculation for the pixels belonging to each unit area. More specifically, in the offset table C decoded by the offset table decoding unit 312, the offset c (k ′) associated with the region characteristic k ′ of the unit region calculated by the region characteristic calculation unit 322 is specified. . The offset c (k ′) determined by the offset determination unit 323 is supplied to the offset addition unit 324.
  • the offset addition unit 324 adds the offset c (k ′) determined by the offset determination unit 323 to the pixel value s ′ calculated by the spatial filter 321.
  • the sum s ′ + c (k ′) calculated by the offset adding unit 324 is output to the outside of the adaptive filter 300.
  • An arbitrary area obtained by dividing the frame can be used as a unit for calculating the area characteristic k ′.
  • a slice, a macro block, a block, an LCU, a CT, a CU, a PT, a PU, a TT, a TU, or an M ⁇ N block can be used as a unit for calculating the region characteristic k ′.
  • an offset c (k ′) corresponding to the region characteristic k ′ of the macroblock is used for the filter calculation for pixels belonging to each macroblock. be able to.
  • the adaptive filter 300 performs the filter operation on each pixel belonging to each unit region by using the offset c (k ′) depending on the region characteristic k ′ of the unit region and the unit.
  • a filter operation unit 320 is provided that executes using a filter coefficient group w that does not depend on the region characteristic k ′ of the region. Therefore, it is possible to achieve higher encoding efficiency than a conventional adaptive filter that performs a filter operation using a certain offset c.
  • the spatial filter coefficient w (k ′) corresponding to the region characteristic k ′ is set.
  • the calculation cost on both the encoding device side and the decoding device side can be reduced and the code amount of the encoded data can be reduced as compared with the case where the filter operation is executed by using. That is, higher encoding efficiency can be realized at a lower calculation cost than the conventional adaptive filter.
  • the filter operation is performed using a certain spatial filter coefficient group w.
  • the present invention is not limited to this. That is, the filter coefficient group w only needs to be independent of the area characteristic k ′, and the filter coefficient group w (k ′′) corresponding to the area characteristic k ′′ different from the area characteristic k ′ is used for filtering.
  • s ′ w (k ′′) [1] s [1] + w (k ′′) [2] s [2] +. + W (k ′′) [N] s [N] + c (k ′) (C2)
  • a configuration may be adopted in which a filter operation is performed using an offset c (k ′, k ′′) corresponding to these two region characteristics k ′, k ′′. That is, you may perform the filter calculation according to (C3) Formula.
  • s ′ w (k ′′) [1] s [1] + w (k ′′) [2] s [2] +. + W (k ′′) [N] s [N] + c (k ′, k ′′) (C3)
  • a configuration may be adopted in which a filter operation is performed using a spatial filter coefficient group w (k ′, k ′′) corresponding to these two region characteristics k ′, k ′′. That is, you may perform the filter calculation according to (C4) Formula.
  • s ′ w (k ′, k ′′) [1] s [1] + w (k ′, k ′′) [2] s [2] +. + W (k ′, k ′′) [N] s [N] + c (k ′) (C4)
  • the activity can be used as the second region characteristic k ′′.
  • the magnitude of the luminance can be used as the second region characteristic k ′′.
  • the amount calculated from the pixel values of the pixels existing in and / or around the unit region is used as the region characteristic k ′ of each unit region.
  • the present invention is not limited to this. is not. In other words, any amount can be used as the region characteristic k ′ of the unit region as long as it is an amount indicating the characteristic of the unit region (in other words, an amount that can be used for classification of the unit region).
  • an encoding parameter relating to a unit region or an encoding parameter relating to a unit including the unit region (for example, a macro block including a block) can be used as the region characteristic k ′ of the unit region.
  • the intra prediction mode and the inter prediction mode are examples of encoding parameters that can be used as the region characteristic k ′. Further, as will be described later, the type of image to which the unit area belongs may be used as the area characteristic k ′ of the unit area. The same can be said for the region characteristic k ′′ described above.
  • a plurality of different offsets c (k) corresponding to each of a plurality of different values k ′ k ′ (0), k ′ (1),..., K ′ (M) of the region characteristic k ′. '(0)), c (k' (0)),..., C (k '(M)), when encoding each offset c (k' (t)) (or its Instead of encoding the quantized value as it is, a difference value (or a quantized value thereof) from the encoded offset c (k ′ (t ⁇ 1)) may be encoded.
  • the offset table decoding unit 312 After decoding the difference value ⁇ (k ′ (t)), the offset value c (k ′ (t)) is obtained according to (C5) using the decoded offset value c (k ′ (t ⁇ 1)).
  • c (k ′ (t)) c (k ′ (t ⁇ 1)) + ⁇ (k ′ (t)) (C6)
  • c (k ′ (0)), c (k ′ (0)),..., C (k ′ (M)) one offset serving as a starting point of encoding / decoding, For example, c (k ′ (0)) is encoded / decoded regardless of other offsets.
  • the configuration for executing the filter operation using the offset c (k ′) corresponding to the region characteristic k ′ has been described, but the configuration using the offset c corresponding to the target to be subjected to the filter processing may also be used. Conceivable. Specifically, an offset c having different values is used when a filter operation is performed on the color difference signal Cb and when a filter operation is performed on the color difference signal Cr.
  • the adaptive filter is based on the principle that the spatial correlation between pixels existing in the original picture is reproduced by filtering. Since the color difference component Cb and the color difference component Cr have edges in the same direction at substantially the same location, the inter-pixel correlation between a certain pixel and its surrounding pixels is often similar to each other.
  • the common use of the spatial filter w (k ′′) determined in relation to the feature of the inter-pixel correlation between the color difference component Cb and the color difference component Cr particularly increases the code amount for the filter coefficient group of the adaptive filter. This is particularly appropriate at a relatively low bit rate, which is difficult to apply, however, because the offset between the decoded image and the original image cannot be said to be close between the color difference component Cb and the color difference component Cr. It is appropriate to use the offset c having different values for the color difference component Cb and the color difference component Cr.
  • a spatial filter coefficient group w (k ′′) corresponding to various region characteristics k ′′ may be used.
  • a spatial filter coefficient group w having different values may be used when the filter operation is executed.
  • the chrominance component Cb and the chrominance component Cr have edges in the same direction at substantially the same place, but their strengths are not necessarily close. Therefore, when encoding at a relatively high bit rate, the chrominance component Cb and the chrominance component It is appropriate to use a spatial filter coefficient group w different from Cr.
  • the encoding device may be installed with the adaptive filter 300 ′ shown in FIG. 46.
  • the spatial filter coefficient decoding unit 311 in the adaptive filter 300 on the decoding device side is replaced with an adaptive filter coefficient determination unit 3111 and a spatial filter coefficient encoding unit 311b.
  • the offset table decoding unit 312 in the adaptive filter 300 on the decoding device side is replaced with an offset table encoding unit 312a.
  • the adaptive filter coefficient determination unit 3111 uses the least square method to calculate a square error with respect to the original image (encoding target image) of the output image of the filter operation unit 320.
  • the spatial filter coefficient group w and the offset table C to be minimized are determined.
  • the objective function to be minimized by the least square method is a function having the order N of the spatial filter coefficient group and the order of the offset table C (for example, N2) as variables, and becomes 0 by partially differentiating the objective function with respect to the N + N2 variable. It can be calculated by solving simultaneous equations.
  • the spatial filter coefficient group w and the offset table C that minimize the error are determined. That is, when the spatial filter coefficient group w (k ′, k ′′) is determined according to the two region characteristics k ′ and k ′′, the least square method is calculated for each region characteristic pair (k ′, k ′′). If the spatial filter coefficient group w (k ′′) is determined according to one characteristic k ′′, the least square method is calculated for each region characteristic k ′′.
  • the filter coefficient group w is determined to be one, one least square method is calculated. Also, (2) the spatial filter coefficient encoding unit 311b encodes the spatial filter coefficient group w determined by the adaptive filter coefficient determination unit 311a, and (3) the offset table encoding unit 312a performs the adaptive filter coefficient determination unit 311a.
  • the offset table C created by is encoded.
  • Encoded data / decoding device / encoding device A video encoding device (encoding device) and a video decoding device (decoding device) on which the adaptive filters according to Embodiments 2 to 4 described above are mounted will be described. A structure of encoded data generated by the moving image encoding device and decoded by the moving image decoding device will also be described.
  • the encoded data # 1 generated by the moving image encoding device 2 and decoded by the moving image decoding device 1 includes a sequence layer, a GOP (Group Of Pictures) layer, a picture layer, a slice layer, and a macroblock layer. Has a hierarchical structure.
  • FIG. 47 shows the structure of encoded data for the layers below the picture layer.
  • 47A to 47F show the structures of the picture layer P, slice layer S, macroblock layer MB, macroblock header MBH, inter prediction partition prediction parameter PP, and intra prediction partition prediction parameter PP, respectively.
  • the picture layer P is a set of data referred to by the video decoding device 1 in order to decode the target picture. As shown in FIG. 47A, the picture layer P includes a picture header PH and slice layers S 1 to S Ns (Ns is the total number of slice layers included in the picture layer P).
  • the picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture.
  • the encoding mode information (entoropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.
  • Each slice layer S included in the picture layer P is a set of data referred to by the video decoding device 1 in order to decode the target slice.
  • the slice layer S includes a slice header SH and macroblock layers MB 1 to MB Nm (Nm is the total number of macroblocks included in the slice S).
  • the slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice.
  • Slice type designation information (slice_type) for designating a slice type
  • POC designation information (pic_order_cnt_lbs, delta_pic_order_cnt, etc.) for designating the display order (POC: Picture Order Count) of a picture including the target slice
  • POC Picture Order Count
  • the weighting factor designation information pred_weight_table
  • pred_weight_table that designates the weighting factor used when the moving image coding apparatus 2 performs the coding is an example of a coding parameter included in the slice header SH.
  • slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, and (2) P using unidirectional prediction or intra prediction at the time of encoding. Slice, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding.
  • the slice header SH includes a filter parameter FP that is referred to by the adaptive filter included in the video decoding device 1.
  • Each macroblock layer MB included in the slice layer S is a set of data referred to by the video decoding device 1 in order to decode the target macroblock.
  • the macroblock layer MB includes a skip flag SKIP, a macroblock header MBH, prediction parameters PP 1 to PP NP , and quantized prediction errors QD 1 to QD Nb .
  • Np represents the number of partitions (prediction units) included in the target macroblock
  • Nb represents the number of blocks (conversion units) included in the target macroblock.
  • the value of the skip flag SKIP is 1, that is, when the target macroblock is a skip block, the macroblock header MBH, the prediction parameters PP 1 to PP NP , and the quantized prediction errors QD 1 to QD in that macroblock layer Nb is omitted.
  • the macroblock header MBH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target macroblock.
  • macroblock type designation information MBT (mb_type) for designating the macroblock type of the target macroblock
  • CBP coded_block_pattern
  • quantization step Quantization parameter difference ⁇ qp (mb_qp_delta) that specifies the size of.
  • the macroblock type designation information MBT includes prediction unit designation information PT and conversion unit designation information TT.
  • the prediction unit designation information PT includes a partition pattern (that is, the size of each partition included in the target macroblock and a position in the target macroblock) into the partition (prediction unit) of the target macroblock, and video coding.
  • the prediction method (L0 unidirectional prediction, L1 unidirectional prediction, bidirectional prediction, intra prediction, etc.) used when the apparatus 2 generates a predicted image in each partition is designated.
  • the partition size is, for example, 16 ⁇ 16 pixels, 8 ⁇ 8 pixels, 4 ⁇ 4 pixels, 16 ⁇ 8 pixels, 8 ⁇ 16 pixels, 8 ⁇ 4 pixels, 4 ⁇ 8 pixels, 8 ⁇ 1 pixels, and 1 X 8 pixels can be selected.
  • each partition has M ⁇ N pixels (M is an integer equal to or smaller than the number of pixels in the horizontal direction of the target macroblock, and N is an integer equal to or smaller than the number of pixels in the vertical direction of the target macroblock).
  • M is an integer equal to or smaller than the number of pixels in the horizontal direction of the target macroblock
  • N is an integer equal to or smaller than the number of pixels in the vertical direction of the target macroblock.
  • the conversion unit designation information TT designates the division pattern (that is, the size of each block included in the target macroblock and the position in the target macroblock) into the block (conversion unit) of the target macroblock.
  • Each block is 16 ⁇ 16 pixels, 8 ⁇ 8 pixels, 4 ⁇ 4 pixels, 16 ⁇ 8 pixels, 8 ⁇ 16 pixels, 8 ⁇ 4 pixels, 4 ⁇ 8 pixels, 8 ⁇ 1 pixels, and 1 ⁇ 8 pixels.
  • each block has M ′ ⁇ N ′ pixels (M ′ is an integer less than or equal to the number of pixels on one side of the target macroblock, and N ′ is less than or equal to the number of pixels on the other side of the target macroblock. (Which is an integer).
  • the partition when the macro block is a square area of 16 ⁇ 16 pixels is exemplified, but the present invention is not limited to this.
  • a 64 ⁇ 64 pixel macroblock a 64 ⁇ 64 pixel or 32 ⁇ 32 pixel square area, or 64 ⁇ 32 pixel, 32 ⁇ 64 pixel, 32 ⁇ 16 pixel, or 16 ⁇ 32 pixel
  • the rectangular area is also allowed as a partition or block.
  • the quantization parameter difference ⁇ qp is a difference qp ⁇ qp ′ between the quantization parameter qp in the target macroblock and the quantization parameter qp ′ in the macroblock encoded immediately before the macroblock.
  • Each quantized prediction residual QD n included in the macroblock layer MB is encoded data generated by the moving image encoding apparatus 2 performing the following processes 1 to 3 on the target block.
  • Process 1 DCT transform (Discrete Cosine Transform) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image.
  • Process 2 The DCT coefficient obtained in Process 1 is quantized.
  • Process 3 The DCT coefficient quantized in Process 2 is variable length encoded.
  • the prediction parameter PP related to the inter prediction partition in which the prediction image is generated by the inter prediction includes a reference image index RI, an estimated motion vector, as shown in FIG.
  • the index PMVI and the motion vector residual MVD are included.
  • the motion vector residual MVD is encoded data generated by the moving image encoding device 2 executing the following processes 4 to 6.
  • Process 4 Select an encoded / decoded locally decoded image (more precisely, an image obtained by performing deblocking processing and adaptive filtering on the encoded / decoded local decoded image)
  • the motion vector mv for the target partition is derived with reference to the selected encoded / decoded local decoded image (hereinafter also referred to as “reference image”).
  • Process 5 An estimation method is selected, and an estimated value (hereinafter also referred to as “estimated motion vector”) pmv of the motion vector mv assigned to the target partition is derived using the selected estimation method.
  • Process 6 The motion vector residual MVD obtained by subtracting the estimated motion vector pmv derived in Process 5 from the motion vector mv derived in Process 4 is encoded.
  • the reference image index RI described above specifies the locally decoded image (reference image) that has been encoded / decoded selected in the process 4, and the estimated motion vector index PMVI described above is selected in the process 5.
  • the estimation methods that can be selected in the processing 5 include: (1) a locally decoded image being encoded / decoded (more precisely, a region that has already been decoded in a locally decoded image being encoded / decoded).
  • a median of a motion vector allocated to a partition adjacent to the target partition hereinafter also referred to as “adjacent partition” is used as an estimated motion vector pmv.
  • a motion vector assigned to a partition (often referred to as a “collocated partition”) occupying the same position as the target partition is used as an estimated motion vector pmv, etc. Is mentioned.
  • the prediction parameter PP related to the partition for which unidirectional prediction is performed includes one each of the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD.
  • the prediction parameters PP for a partition that performs bi-directional prediction include two reference image indexes RI1 and RI2, two estimated motion vector indexes PMVI1 and PMVI2, and two motion vector residuals MVD1. And MVD2.
  • the prediction parameter PP related to the intra prediction partition for which a prediction image is generated by intra prediction includes a prediction index PI as shown in FIG. 47 (f).
  • the prediction index PI is an index for designating an intra prediction method (prediction mode) for the target partition. Note that since the types of prediction modes that can be selected for the target partition differ depending on the size of the target partition, the range of values that the prediction index can take also differs depending on the size of the target partition.
  • the data structure of the encoded data # 1 is not limited to that shown in FIG.
  • a data structure as shown in FIG. 50 obtained by changing a part of a layer structure below a picture in a moving image may be adopted.
  • FIG. 50 is a diagram showing a modification of the data structure of the encoded data # 1.
  • (a) is a picture layer PCT corresponding to a picture
  • (b) is a slice layer S corresponding to a slice obtained by dividing the picture
  • (c) is a tree obtained by dividing the slice.
  • Tree block layers TBLK and (d) corresponding to blocks (Tree blocks) indicate CU layer CUs corresponding to CUs (Coding Units) obtained by dividing the tree blocks.
  • the picture layer PICT includes various types of information that the video decoding device 1 refers to in order to decode a picture (also referred to as a target picture). Specifically, as shown in FIG. 50A, a picture header PH and slice layers S 1 to S NS (NS is the total number of slices included in the target picture) are included.
  • the picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture.
  • the encoding mode information (entropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.
  • entropy_coding_mode_flag When entropy_coding_mode_flag is 0, the target picture is encoded by CAVLC (Context-based “Adaptive” Variable “Length” Coding). When entropy_coding_mode_flag is 1, the target picture is encoded by CABAC (Context-based Adaptive Binary Arithmetic Coding).
  • picture header PH is also referred to as a picture parameter set (PPS).
  • PPS picture parameter set
  • the slice layer S includes various types of information that the video decoding device 1 refers to in order to decode a slice (also referred to as a target slice). Specifically, as shown in FIG. 50B, a slice header SH and tree block layers TBLK 1 to TBLK NC (NC is the total number of tree blocks included in the target slice) are included.
  • the slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice.
  • the slice type designation information (slice_type) that designates the slice type of the target slice is an example of an encoding parameter included in the slice header SH.
  • I slice that uses only intra prediction at the time of encoding (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.
  • the filter parameter FP that defines the filter coefficient group of the adaptive filter that acts on the target slice is also included in the slice header SH.
  • the tree block layer TBLK includes various types of information that the video decoding device 1 refers to in order to decode a tree block (also referred to as a target tree block). Specifically, as shown in FIG. 50C, a tree block header TBLKH and CU information CUI 1 to CUI NL (NL is the total number of CUs included in the target tree block) are included.
  • the CU is obtained by recursively dividing the tree block into quadtrees.
  • Recursive quadtree partitioning refers to a process in which a tree block is divided into four equal parts and then a step of dividing the partition obtained by the previous partitioning into four equal parts is repeated. At this time, only a part of the partition obtained by the previous division is allowed to be divided into four equal parts. That is, CUs having different sizes can coexist in the tree block.
  • the tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree.
  • the tree block corresponds to the root of the coding tree
  • the CU corresponds to the leaf of the coding tree.
  • a leaf of the coding tree is a node corresponding to the CU, and thus may be referred to as a coding node (coding node).
  • the size that each CU can take depends on the size of the tree block and the maximum hierarchical depth (maximummaximhierarchical depth) included in the sequence parameter set SPS of the encoded data # 1. For example, when the size of the tree block is 64 ⁇ 64 pixels and the maximum hierarchical depth is 3, there are three types of sizes, that is, CUs of 64 ⁇ 64 pixels, 32 ⁇ 32 pixels, and 16 ⁇ 16 pixels. Can exist.
  • the tree block header TBLKH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target tree block.
  • the tree block division information SP_TBLK that specifies the division pattern of the target tree block into each CU, the quantization parameter difference ⁇ qp (qp_delta) that specifies the size of the quantization step, and the like are encoded parameters included in the tree block header TBLKH. It is an example.
  • the tree block division information SP_TBLK is information indicating a division pattern when the target tree block is divided into CUs, in other words, information representing a coding tree.
  • the tree block division information SP_TBLK can be realized, for example, by a set of flags (split_coding_unit_flag) indicating whether or not each node is divided into four in the coding tree.
  • the quantization parameter difference ⁇ qp is a difference qp ⁇ qp ′ between the quantization parameter qp in the target tree block and the quantization parameter qp ′ in the tree block encoded immediately before the target tree block.
  • At least one prediction block is obtained by dividing the CU.
  • the CU is covered by at least one prediction block that does not overlap each other.
  • a tree having a node corresponding to a CU as a root and a node corresponding to a prediction block as a leaf is called a prediction tree (PT).
  • a prediction block is an execution unit of prediction processing, and is also referred to as PU (prediction unit).
  • intra prediction there are roughly two types of division in the prediction tree: intra prediction and inter prediction.
  • intra prediction there are 2N ⁇ 2N (the same size as the encoding node) and N ⁇ N division methods.
  • inter prediction there are 2N ⁇ 2N (the same size as the encoding node), 2N ⁇ N, N ⁇ 2N, N ⁇ N, and the like.
  • At least one conversion block is obtained by dividing the CU.
  • the CU is covered by at least one transform block that does not overlap each other.
  • a tree having a node corresponding to a CU as a root and a node corresponding to a prediction block as a leaf is referred to as a transform tree (TT).
  • the transformation block is another meaning of execution of DCT transformation, and is also called TU (transform unit).
  • the division in the transformation tree includes the one in which an area having the same size as the encoding node is assigned as the transformation block, and the one in the recursive quadtree division as in the above-described division of the tree block.
  • the CU information CUI includes various types of information that the video decoding device 1 refers to in order to decode a CU (hereinafter also referred to as a target CU). Specifically, as shown in FIG. 50 (d), a skip flag SKIP, PT information PTI, and TT information TTI are included.
  • the skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target CU.
  • the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, the CU The PT information PTI and the TT information TTI in the information CUI are omitted. Note that the skip flag SKIP is omitted for the I slice.
  • PT information PTI is information regarding the prediction tree PT having the target CU as a root. Specifically, as shown in FIG. 50 (d), the prediction type information PType and the prediction information PInfo are included.
  • Prediction type information PType is information that specifies whether intra prediction or inter prediction is used to generate a predicted image in the target CU.
  • PInfo PT division information (not shown) for specifying a division pattern for dividing the target CU into PUs, and a prediction parameter referred to by the video decoding device 1 to generate a prediction image of each PU (Not shown) is included.
  • the prediction parameters of a PU for which a predicted image is generated by inter prediction include a reference image index, an estimated motion vector index, a motion vector residual, and the like.
  • the prediction parameters related to the intra PU for which the prediction image is generated by the intra prediction include a prediction index for specifying the prediction mode.
  • the TT information TTI is information related to a conversion tree having the target CU as a root. Specifically, as shown in FIG. 50 (d), TT division information SP_TT and quantization prediction residuals QD 1 to QD NT (NT is the total number of blocks included in the target CU) are included.
  • TT division information SP_TT is information expressing a division pattern when dividing the target CU into TUs, in other words, information expressing a tree structure of a conversion tree having the target CU as a root.
  • the TT division information SP_TT can be configured with, for example, a flag (split_transform_unit_flag) indicating whether or not to perform division of the target CU and information (trafoDepth) indicating the division depth. For example, when the size of the CU is 64 ⁇ 64, each TU obtained by the division can take a size from 32 ⁇ 32 pixels to 2 ⁇ 2 pixels.
  • the quantized prediction residuals QD 1 to QD NT are codes obtained by performing the following processing on each TU.
  • DCT transform Discrete Cosine Transform
  • the transform coefficient obtained in Process 1 is quantized.
  • the transform coefficient quantized in Process 2 is variable length encoded.
  • the moving image decoding apparatus 1001 includes, as a part thereof, H.264. H.264 / MPEG-4. It is a decoding device including AVC and its successor technology.
  • FIG. 48 is a block diagram showing a configuration of the moving picture decoding apparatus 1001.
  • the video decoding device 1001 is a device for generating video # 1002 by decoding the encoded data # 1001, and as shown in FIG. 48, a variable length code decoding unit 1013, a motion vector restoration unit 1014, A buffer memory 1015, an inter prediction image generation unit 1016, an intra prediction image generation unit 1017, a prediction method determination unit 1018, an inverse quantization / inverse transformation unit 1019, an adder 1020, a deblocking filter 1041, and an adaptive filter 1042 are provided. Yes.
  • the adaptive filter 100, 200, 300 according to Embodiments 2 to 4 described above is used as the adaptive filter 1042.
  • the variable length code decoding unit 1013 decodes the prediction parameter PP related to each partition from the encoded data # 1001. That is, for the inter prediction partition, the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD are decoded from the encoded data # 1001 and supplied to the motion vector restoration unit 1014.
  • the intra prediction partition (1) size designation information for designating the size of the partition and (2) prediction index designation information for designating the prediction index are decoded from the encoded data # 1001, and this is decoded into the intra prediction image. It supplies to the production
  • variable length code decoding unit 1013 decodes the macroblock type MBT from the encoded data, and supplies this to the prediction method determination unit 1018 (not shown). Furthermore, the variable length code decoding unit 1013 decodes the quantization prediction residual QD for each block and the quantization parameter difference ⁇ qp for the macroblock including the block from the encoded data # 1001, and dequantizes and decodes them. This is supplied to the inverse transform unit 1019. Also, the variable length code decoding unit 1013 supplies the filter parameter FP included in the encoded data # 1001 to the adaptive filter 1042.
  • the motion vector restoration unit 1014 restores the motion vector mv related to each inter prediction partition from the motion vector residual MVD related to that partition and the restored motion vector mv ′ related to other partitions. Specifically, (1) the estimated motion vector pmv is derived from the restored motion vector mv ′ according to the estimation method specified by the estimated motion vector index PMVI, and (2) the derived estimated motion vector pmv and the motion vector remaining are derived. The motion vector mv is obtained by adding the difference MVD. It should be noted that the restored motion vector mv ′ related to other partitions can be read from the buffer memory 1015.
  • the motion vector restoration unit 1014 supplies the restored motion vector mv to the inter predicted image generation unit 1017 together with the corresponding reference image index RI. For the inter prediction partition that performs bi-directional prediction (weighted prediction), the restored two motion vectors mv1 and mv2 are supplied to the inter prediction image generation unit 1017 together with the corresponding reference image indexes RI1 and RI2.
  • the inter prediction image generation unit 1016 generates a motion compensation image mc related to each inter prediction partition. Specifically, using the motion vector mv supplied from the motion vector restoration unit 1014, the motion compensated image mc from the filtered decoded image P_FL ′ designated by the reference image index RI also supplied from the motion vector restoration unit 1014. Is generated.
  • the filtered decoded image P_FL ′ is obtained by performing deblocking processing by the deblocking filter 1041 and adaptive filtering processing by the adaptive filter 1042 on the decoded image that has already been decoded.
  • the inter prediction image generation unit 1016 is an image, and can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 1015.
  • the motion compensated image mc generated by the inter predicted image generating unit 1016 is supplied to the prediction method determining unit 1018 as an inter predicted image Pred_Inter.
  • a motion compensated image mc1 is generated from the filtered decoded image P_FL1 ′ specified by the reference image index RI1 using the motion vector mv1.
  • a motion compensated image mc2 is generated from the filtered decoded image P_FL2 ′ specified by the reference image index RI2 using the motion vector mv2, and (3) weighting between the motion compensated image mc1 and the motion compensated image mc2
  • An inter predicted image Pred_Inter is generated by adding an offset value to the average.
  • the intra predicted image generation unit 1017 generates a predicted image Pred_Intra related to each intra prediction partition. Specifically, first, referring to the correspondence table recorded in its own memory, the prediction mode is specified based on the size designation information and the prediction index designation information supplied from the variable length code decoding unit 1013, The identified prediction mode is assigned to the target partition in, for example, raster scan order. Subsequently, a predicted image Pred_Intra is generated from the decoded image P according to the prediction method indicated by the prediction mode. The intra predicted image Pred_Intra generated by the intra predicted image generation unit 1017 is supplied to the prediction method determination unit 1018. In addition, since the specific production
  • the prediction method determination unit 1018 determines whether each partition is an inter prediction partition that should perform inter prediction or an intra prediction partition that should perform intra prediction, based on the macroblock type MBT. In the former case, the inter predicted image Pred_Inter generated by the inter predicted image generation unit 1016 is supplied to the adder 1020 as the predicted image Pred. In the latter case, the intra predicted image generation unit 1017 generates the inter predicted image Pred_Inter. The intra predicted image Pred_Intra that has been processed is supplied to the adder 1020 as the predicted image Pred.
  • the inverse quantization / inverse transform unit 1019 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 1020.
  • the inverse quantization / inverse transform unit 1019 derives a quantization step QP from the quantization parameter difference ⁇ qp supplied from the variable length code decoding unit 1013.
  • the generation of the prediction residual D by the inverse quantization / inverse transform unit 1019 is performed in units of blocks (transform units).
  • the adder 1020 generates a decoded image P by adding the prediction image Pred supplied from the prediction method determination unit 1018 and the prediction residual D supplied from the inverse quantization / inverse transformation unit 1019.
  • the deblocking filter 1041 determines the block boundary in the decoded image P when the difference between the pixel values of the pixels adjacent to each other via the block boundary in the decoded image P or the macroblock boundary is smaller than a predetermined threshold, or By performing deblocking processing on the macroblock boundary, an image near the block boundary or the macroblock boundary is smoothed.
  • the image subjected to the deblocking process by the deblocking filter 1041 is output to the adaptive filter 42 as a deblocked decoded image P_DB.
  • the adaptive filter 1042 subjects the deblocked decoded image P_DB supplied from the deblocking filter 1041 to a filtering process using a filter coefficient obtained by decoding the filter parameter FP, so that the filtered decoded image P_FL Is generated.
  • the image subjected to the filter processing by the adaptive filter 1042 is output to the outside as the filtered decoded image P_FL, and is associated with the POC designation information decoded from the encoded data by the variable length code decoding unit 1013 in the buffer memory 1015. Stored.
  • the structure of the moving image decoding apparatus 1001 corresponding to the case where encoded data # 1001 has the data structure shown in FIG. 47 was demonstrated here, the structure of the moving image decoding apparatus 1001 is not limited to this. That is, the video decoding apparatus 1 can be made to correspond to the encoded data # 1001 having the data structure shown in FIG.
  • the moving image encoding apparatus 1002 includes, as a part thereof, an H.264 format. H.264 / MPEG-4. It is a decoding device including AVC and its successor technology.
  • FIG. 49 is a block diagram showing a configuration of the moving picture encoding apparatus 1002.
  • the moving image encoding apparatus 1002 is an apparatus that generates encoded data # 1001 by encoding moving image # 1010 (encoding target image).
  • the adaptive folders 100 ′, 200 ′, and 300 ′ according to Embodiments 2 to 4 described above are used as the adaptive folder 1034.
  • the transform / quantization unit 1021 performs (1) DCT transform (Discrete Cosine Transform) for each block on the prediction residual D obtained by subtracting the predicted image Pred from the encoding target image, and (2) DCT coefficients obtained by the DCT transform. (3) The quantized prediction residual QD obtained by the quantization is supplied to the variable-length code encoding unit 22 and the inverse quantization / inverse transform unit 1023.
  • the transform / quantization unit 1021 selects (1) a quantization step QP to be used for quantization for each macroblock, and (2) a quantization parameter difference ⁇ qp indicating the size of the selected quantization step QP.
  • the difference value obtained by subtracting the value of '.
  • the variable length code encoding unit 1022 includes (1) a quantization prediction residual QD and ⁇ qp supplied from the transform / quantization unit 1021, and (2) a quantization parameter PP supplied from a prediction scheme control unit 1028 described later. (3) The encoded data # 1001 is generated by variable-length encoding the filter parameter FP supplied from the adaptive filter 1034 described later.
  • the inverse quantization / inverse transform unit 1023 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 1031.
  • the quantization step QP supplied from the transform / quantization unit 1021 is used.
  • the prediction residual D output from the inverse quantization / inverse transform unit 1023 is obtained by adding a quantization error to the prediction residual D input to the transform / quantization unit 1021. Common names are used for this purpose.
  • the intra predicted image generation unit 1025 generates a predicted image Pred_Intra related to each partition. Specifically, (1) a prediction mode used for intra prediction is selected for each partition, and (2) a prediction image Pred_Intra is generated from the decoded image P using the selected prediction mode. The intra predicted image generation unit 1025 supplies the generated intra predicted image Pred_Intra to the prediction method control unit 1028.
  • the intra predicted image generation unit 1025 refers to the correspondence table stored in its own memory, and specifies the prediction index PI for each partition from the prediction mode selected for each partition and the size of each partition.
  • the prediction index designation information indicating the prediction index PI for each partition is supplied to the prediction method control unit 28.
  • generation part 1025 can be performed by the process similar to the process demonstrated in (the production
  • the motion vector detection unit 1027 detects a motion vector mv related to each partition. Specifically, (1) the filtered decoded image P_FL ′ to be used as a reference image is selected, and (2) the target partition is searched by searching for the region that best approximates the target partition in the selected filtered decoded image P_FL ′. Detects a motion vector mv.
  • the filtered decoded image P_FL ′ is obtained by performing deblocking processing by the deblocking filter 1033 and adaptive filtering processing by the adaptive filter 1034 on the decoded image that has already been decoded.
  • the motion vector detection unit 1027 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 1024.
  • the motion vector detection unit 1027 supplies the detected motion vector mv to the inter prediction image generation unit 1026 and the motion vector redundancy deletion unit 1029 together with the reference image index RI that specifies the filtered decoded image P_FL ′ used as the reference image.
  • the reference image index RI specifies the filtered decoded image P_FL ′ used as the reference image.
  • two filtered decoded images P_FL1 ′ and P_FL2 ′ are selected as reference images, and each of the two filtered decoded images P_FL1 ′ and P_FL2 ′ is selected.
  • Corresponding motion vectors mv1 and mv2 and reference image indexes RI1 and RI2 are supplied to the inter predicted image generation unit 1026 and the motion vector redundancy deletion unit 1029.
  • the inter prediction image generation unit 1026 generates a motion compensation image mc related to each inter prediction partition. Specifically, using the motion vector mv supplied from the motion vector detection unit 1027, the motion compensation image mc is obtained from the filtered decoded image P_FL ′ specified by the reference image index RI supplied from the motion vector detection unit 1027. Generate. Similar to the motion vector detection unit 1027, the inter predicted image generation unit 1026 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 1024. The inter prediction image generation unit 1026 supplies the generated motion compensated image mc (inter prediction image Pred_Inter) to the prediction scheme control unit 1028 together with the reference image index RI supplied from the motion vector detection unit 1027.
  • inter prediction image Pred_Inter inter prediction image Pred_Inter
  • the motion compensated image mc1 is generated from the filtered decoded image P_FL1 ′ specified by the reference image index RI1 using the motion vector mv1, and (2 )
  • a motion compensated image mc2 is generated from the filtered reference image P_FL2 ′ specified by the reference image index RI2 using the motion vector mv2, and (3) an offset value is added to the weighted average of the motion compensated image mc1 and the motion compensated image mc2. Is added to generate the inter predicted image Pred_Inter.
  • the prediction scheme control unit 1028 compares the intra predicted image Pred_Intra and the inter predicted image Pred_Inter with the encoding target image, and selects whether to perform intra prediction or inter prediction.
  • intra prediction the prediction scheme control unit 1028 supplies the intra prediction image Pred_Intra as the prediction image Pred to the adder 1031 and the subtractor 1032 and also predicts the prediction index PI supplied from the intra prediction image generation unit 1025.
  • the parameter PP is supplied to the variable length code encoding unit 1022.
  • the prediction scheme control unit 1028 supplies the inter prediction image Pred_Inter as the prediction image Pred to the adder 1031 and the subtracter 1032 and the reference image index supplied from the inter prediction image generation unit 1026.
  • RI and an estimated motion vector index PMVI and a motion vector residual MVD supplied from a motion vector redundancy deletion unit 1029 are supplied as prediction parameters PP to a variable length code encoding unit.
  • the prediction residual D is generated by the subtracter 1032 by subtracting the prediction image Pred selected by the prediction method control unit 1028 from the encoding target image.
  • the prediction residual D generated by the subtracter 1032 is DCT transformed / quantized by the transform / quantization unit 1021 as described above.
  • the adder 1031 by adding the prediction image Pred selected by the prediction method control unit 1028 to the prediction residual D generated by the inverse quantization / inverse transformation unit 1023, the adder 1031 generates a local decoded image P. Generated.
  • the local decoded image P generated by the adder 1031 passes through the deblocking filter 1033 and the adaptive filter 1034, and is then stored in the buffer memory 1024 as a filtered decoded image P_FL and used as a reference image in inter prediction.
  • the motion vector redundancy deletion unit 1029 deletes the redundancy in the motion vector mv detected by the motion vector detection unit 1027. Specifically, (1) an estimation method used for estimating the motion vector mv is selected, (2) an estimated motion vector pmv is derived according to the selected estimation method, and (3) the estimated motion vector pmv is subtracted from the motion vector mv. As a result, a motion vector residual MVD is generated. The motion vector redundancy deletion unit 1029 supplies the generated motion vector residual MVD to the prediction method control unit 1028 together with the estimated motion vector index PMVI indicating the selected estimation method.
  • the deblocking filter 1033 determines the block boundary in the decoded image P or the block boundary in the decoded image P when the difference between the pixel values of pixels adjacent to each other via the block boundary or the macroblock boundary in the decoded image P is smaller than a predetermined threshold value. By performing deblocking processing on the macroblock boundary, an image near the block boundary or the macroblock boundary is smoothed.
  • the image that has been deblocked by the deblocking filter 1033 is output to the adaptive filter 1034 as a deblocked decoded image P_DB.
  • the adaptive filter 1034 generates a filtered decoded image P_FL by performing an adaptive filter process on the deblocked decoded image P_DB supplied from the deblocking filter 1033.
  • the filtered decoded image P_FL that has been filtered by the adaptive filter 1034 is stored in the buffer memory 1024.
  • the configuration of moving image encoding device 1002 corresponding to the case where encoded data # 1001 has the data structure shown in FIG. 47 has been described, but the configuration of moving image encoding device 1002 is not limited to this. That is, by appropriately changing the processing unit of various processes, the moving picture encoding apparatus 1002 can be made to correspond to the encoded data # 1001 having the data structure shown in FIG. ⁇ Application example >>
  • the above-described moving image encoding devices 2 and 1002 and moving image decoding devices 1 and 1001 can be used by being mounted on various devices that transmit, receive, record, and reproduce moving images.
  • the moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.
  • the moving image encoding device 2 and the moving image decoding device 1 will be described as an example, but the same applies to the moving image encoding device 1002 and the moving image decoding device 1001.
  • moving picture encoding apparatus 2 and moving picture decoding apparatus 1 can be used for transmission and reception of moving pictures.
  • FIG. 51A is a block diagram illustrating a configuration of a transmission device PROD_A in which the moving image encoding device 2 is mounted.
  • the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image and the encoded data obtained by the encoding unit PROD_A1.
  • a modulation unit PROD_A2 that obtains a modulation signal
  • a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided.
  • the moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.
  • the transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1.
  • An image processing unit A7 that generates or processes an image may be further provided. In FIG. 51A, a configuration in which all of these are provided in the transmission device PROD_A is illustrated, but a part may be omitted.
  • the recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.
  • FIG. 51B is a block diagram illustrating a configuration of a receiving device PROD_B in which the moving image decoding device 1 is mounted.
  • the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulator.
  • a decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2.
  • the moving picture decoding apparatus 1 described above is used as the decoding unit PROD_B3.
  • the receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3.
  • PROD_B6 may be further provided.
  • FIG. 51B illustrates a configuration in which all of these are provided in the receiving device PROD_B, but some of them may be omitted.
  • the recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.
  • the transmission medium for transmitting the modulation signal may be wireless or wired.
  • the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.
  • a terrestrial digital broadcast broadcasting station (broadcasting equipment or the like) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting.
  • a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.
  • a server workstation etc.
  • Client television receiver, personal computer, smart phone etc.
  • VOD Video On Demand
  • video sharing service using the Internet is a transmitting device for transmitting and receiving modulated signals by communication.
  • PROD_A / reception device PROD_B usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN.
  • the personal computer includes a desktop PC, a laptop PC, and a tablet PC.
  • the smartphone also includes a multi-function mobile phone terminal.
  • the video sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.
  • the moving picture encoding apparatus 2 and the moving picture decoding apparatus 1 described above can be used for recording and reproduction of moving pictures.
  • FIG. 52 (a) is a block diagram showing a configuration of a recording apparatus PROD_C in which the above-described moving picture encoding apparatus 2 is mounted.
  • the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M.
  • the moving image encoding apparatus 2 described above is used as the encoding unit PROD_C1.
  • the recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) Or a drive device (not shown) built in the recording device PROD_C.
  • HDD Hard Disk Drive
  • SSD Solid State Drive
  • SD memory such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) Or a drive device (not shown) built in the recording device PROD_C.
  • the recording device PROD_C is a camera PROD_C3 that captures moving images as a supply source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and reception for receiving moving images.
  • the unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided.
  • FIG. 52A illustrates a configuration in which the recording apparatus PROD_C includes all of these, but a part of the configuration may be omitted.
  • the receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.
  • Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images).
  • a camcorder in this case, the camera PROD_C3 is a main source of moving images
  • a personal computer in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images
  • a smartphone in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images
  • the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images
  • FIG. 52 (b) is a block diagram showing a configuration of a playback device PROD_D in which the above-described video decoding device 1 is mounted.
  • the playback device PROD_D reads the moving image by decoding the read data PROD_D1 for reading the encoded data written on the recording medium PROD_M and the encoded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained.
  • the moving picture decoding apparatus 1 described above is used as the decoding unit PROD_D2.
  • the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.
  • the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2.
  • PROD_D5 may be further provided.
  • FIG. 52B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but some of the configurations may be omitted.
  • the transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.
  • Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images).
  • a television receiver in this case, the display PROD_D3 is a main supply destination of moving images
  • a digital signage also referred to as an electronic signboard or an electronic bulletin board
  • the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images.
  • Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image)
  • a smartphone which is a main image supply destination
  • a smartphone in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination
  • the like are also examples of such a playback device PROD_D.
  • the adaptive filter 100 including the linear filter 121 in which filter coefficients are adaptively set, the pixel value of the target pixel in the output image of the linear filter 121 and the pixel value of the target pixel in the input image of the linear filter 121 Truncating means for truncating the change amount ⁇ s, which is the difference between the two, is provided.
  • LCU Large Coding Unit
  • HEVC High Efficiency Video Coding
  • a leaf CU is a CU (Coding Unit, coding) It is also called the leaf of the tree).
  • PU and TU in the said embodiment are respectively equivalent to the prediction tree (Prediction Tree) and transformation tree (transform tree) in HEVC.
  • the partition of PU in the said embodiment is corresponded to PU (Prediction Unit) in HEVC.
  • a block obtained by dividing a TU corresponds to a TU (Transformation Unit) in HEVC.
  • the present invention can be suitably used for an image filter device that filters an image. Further, the present invention can be suitably used for a decoding device that decodes encoded data and an encoding device that generates encoded data. Further, it can be suitably used for an image encoding device such as a moving image encoding device and an image decoding device such as a moving image decoding device.
  • Video decoding device 12 Predictive image generation unit (predictive image generation means) 17 Adaptive filter (image filter device) 171 Area characteristic value calculation unit (characteristic value calculation means) 171a Activity calculation unit 171b Pixel value reference unit 172 Region classification unit (classification means) 173 Filter processing unit (filter means) 2 Video encoding device (encoding device) 21 Predicted image generating unit (predicted image generating means) 27 Adaptive filter (image filter device) 271 Area characteristic value calculation unit (characteristic value calculation means) 271a Activity calculation unit 271b Pixel value reference unit 272 Region classification unit (classification means) 273 Filter processing unit (filter means) 100, 200, 300, 42 Adaptive filter (filter device on the decoding device side) 110, 210, 310 Filter setting unit 120, 220, 320 Filter operation unit 100 ', 200', 300 ', 34 Adaptive filter (filter device on the encoding device side) 110 ′, 210 ′, 310 ′ Filter setting unit 120

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un filtre adaptatif (17) comprenant : une unité de calcul de caractéristiques de région (171) qui calcule des valeurs de caractéristiques représentant les caractéristiques d'image des images décodées débloquées (P_DB) pour chaque région unitaire ; et une unité de classement de région (172) qui classe chaque zone unitaire en soit une région avec filtre actif dans laquelle une unité de traitement de filtre (173) est rendue opérationnelle, soit une unité avec filtre inactif dans laquelle l'unité de traitement de filtre (173) n'est pas rendue opérationnelle, selon les valeurs de caractéristiques de la région unitaire.
PCT/JP2011/079232 2010-12-17 2011-12-16 Dispositif de filtre d'image, dispositif de filtre, décodeur, codeur et structure de données Ceased WO2012081706A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2010-282195 2010-12-17
JP2010282195 2010-12-17
JP2011053610 2011-03-10
JP2011-053610 2011-03-10

Publications (1)

Publication Number Publication Date
WO2012081706A1 true WO2012081706A1 (fr) 2012-06-21

Family

ID=46244798

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/079232 Ceased WO2012081706A1 (fr) 2010-12-17 2011-12-16 Dispositif de filtre d'image, dispositif de filtre, décodeur, codeur et structure de données

Country Status (1)

Country Link
WO (1) WO2012081706A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014123251A (ja) * 2012-12-21 2014-07-03 Nippon Hoso Kyokai <Nhk> ディジタル信号処理装置およびプログラム

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH03270378A (ja) * 1990-03-19 1991-12-02 Victor Co Of Japan Ltd データ圧縮装置
JP2006101397A (ja) * 2004-09-30 2006-04-13 Victor Co Of Japan Ltd 動画像符号化装置及び動画像符号化プログラム
WO2008084745A1 (fr) * 2007-01-09 2008-07-17 Panasonic Corporation Appareil de codage et de décodage d'image
WO2009110559A1 (fr) * 2008-03-07 2009-09-11 株式会社 東芝 Dispositif de codage/décodage dynamique d'image
WO2009110160A1 (fr) * 2008-03-07 2009-09-11 株式会社 東芝 Procédé et dispositif de codage/décodage dynamique d'une image

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH03270378A (ja) * 1990-03-19 1991-12-02 Victor Co Of Japan Ltd データ圧縮装置
JP2006101397A (ja) * 2004-09-30 2006-04-13 Victor Co Of Japan Ltd 動画像符号化装置及び動画像符号化プログラム
WO2008084745A1 (fr) * 2007-01-09 2008-07-17 Panasonic Corporation Appareil de codage et de décodage d'image
WO2009110559A1 (fr) * 2008-03-07 2009-09-11 株式会社 東芝 Dispositif de codage/décodage dynamique d'image
WO2009110160A1 (fr) * 2008-03-07 2009-09-11 株式会社 東芝 Procédé et dispositif de codage/décodage dynamique d'une image

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Test Model under Consideration Output Document (draft007), Joint Collaborative Team on Video Coding", (JCT-VC) OF ITU-T SG16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-B205, 2ND MEETING, October 2010 (2010-10-01), GENEVA, CH, pages 1 - 12, 168 *
MARTA KARCZEWICZ ET AL.: "Video coding technology proposal by Qualcomm Inc.", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG16 WP3 AND ISO/IEC JTC1/SC29/WG11, JCTVC-A121, 1ST MEETING, April 2010 (2010-04-01), DRESDEN, DE, pages 1 - 25 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014123251A (ja) * 2012-12-21 2014-07-03 Nippon Hoso Kyokai <Nhk> ディジタル信号処理装置およびプログラム

Similar Documents

Publication Publication Date Title
JP7200320B2 (ja) 画像フィルタ装置、フィルタ方法および動画像復号装置
US12149735B2 (en) Image decoding device, image encoding device, and image decoding method
JP6190361B2 (ja) 算術復号装置、画像復号装置、算術符号化装置、および画像符号化装置
CN103858430B (zh) 图像解码装置、图像解码方法及图像编码装置
WO2012081609A1 (fr) Dispositif de filtre d&#39;image, décodeur, codeur et structure de données
JP7139144B2 (ja) 画像フィルタ装置
WO2019065488A1 (fr) Dispositif de décodage d&#39;image, et dispositif de codage d&#39;image
WO2012077719A1 (fr) Dispositif de décodage d&#39;images et dispositif de codage d&#39;images
WO2013046990A1 (fr) Appareil de décodage de décalage, appareil de codage de décalage, appareil de filtre d&#39;image et structure de données
JP2013141094A (ja) 画像復号装置、画像符号化装置、画像フィルタ装置、および符号化データのデータ構造
WO2012121352A1 (fr) Dispositif de décodage de vidéo, dispositif de codage de vidéo et structure de données
JP2013223050A (ja) フィルタ装置、復号装置、および符号化装置
WO2012090962A1 (fr) Dispositif de décodage d&#39;image, dispositif de codage d&#39;image, structure de données des données codées, dispositif de décodage arithmétique et dispositif de codage arithmétique
WO2012137890A1 (fr) Appareil de filtrage d&#39;image, appareil décodeur, appareil codeur et structure de données
WO2012081706A1 (fr) Dispositif de filtre d&#39;image, dispositif de filtre, décodeur, codeur et structure de données
WO2013018737A1 (fr) Dispositif de filtrage d&#39;image, dispositif de décodage, dispositif de codage et structure de données
JP2013251827A (ja) 画像フィルタ装置、画像復号装置、画像符号化装置、およびデータ構造
WO2012043676A1 (fr) Dispositif de décodage, dispositif de codage et structure de données
WO2012043766A1 (fr) Dispositif de décodage d&#39;image, dispositif de codage d&#39;image et structure de données pour données codées

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11848609

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11848609

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP