[go: up one dir, main page]

WO2013154026A1 - Appareil et procédé de traitement d'image - Google Patents

Appareil et procédé de traitement d'image Download PDF

Info

Publication number
WO2013154026A1
WO2013154026A1 PCT/JP2013/060362 JP2013060362W WO2013154026A1 WO 2013154026 A1 WO2013154026 A1 WO 2013154026A1 JP 2013060362 W JP2013060362 W JP 2013060362W WO 2013154026 A1 WO2013154026 A1 WO 2013154026A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
offset
image
size
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2013/060362
Other languages
English (en)
Japanese (ja)
Inventor
佐藤　数史
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of WO2013154026A1 publication Critical patent/WO2013154026A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present disclosure relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method capable of reducing information to be sent to a decoding side and improving encoding efficiency.
  • MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image encoding system, and is a standard that covers both interlaced scanning images and progressive scanning images, as well as standard resolution images and high-definition images.
  • MPEG2 is currently widely used in a wide range of applications for professional and consumer applications.
  • a code amount (bit rate) of 4 to 8 Mbps is assigned to an interlaced scanned image having a standard resolution of 720 ⁇ 480 pixels.
  • a high resolution interlaced scanned image having 1920 ⁇ 1088 pixels is assigned a code amount (bit rate) of 18 to 22 Mbps.
  • bit rate code amount
  • MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but it did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the image coding system, the standard was approved as an international standard in December 1998 as ISO / IEC 14496-2.
  • the standardization schedule is H.03 in March 2003. H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as AVC).
  • Non-Patent Document 1 a method having an FIR filter in a motion compensation loop has been proposed (see Non-Patent Document 1, for example).
  • this FIR filter coefficient is obtained by Wiener Filter so as to minimize the error from the input image, thereby minimizing deterioration in the reference image and encoding efficiency of the image compression information to be output. It is possible to improve.
  • HEVC High Efficiency Video (Coding) by JCTVC (Joint Collaboration (Team-Video Coding)), which is a joint standardization organization of ITU-T and ISO / IEC, for the purpose of further improving encoding efficiency than H.264 / AVC. Standardization of the encoding method is underway. Regarding the HEVC standard, CommitteeCommitdraft, which is the first draft version specification, was issued in February 2012 (see Non-Patent Document 2, for example).
  • a coding unit (Coding Unit) is defined as an encoding unit similar to a macroblock in AVC. Further, one coding unit can be divided into one or more prediction units (Prediction Unit: PU) that mean a unit of prediction processing. Then, intra prediction or inter prediction is performed for each prediction unit.
  • Prediction Unit Prediction Unit: PU
  • one coding unit can be divided into one or more transform units (Transform Unit: TU) which means a unit of orthogonal transform. Then, for each transform unit, orthogonal transform from image data to transform coefficient data and quantization of transform coefficient data are performed.
  • transform Unit Transform Unit
  • Non-Patent Document 3 a short-distance intra prediction method that enables selection of a relatively small non-square prediction unit (for example, a linear or rectangular prediction unit) in the intra prediction mode.
  • a method called "suggestion" is proposed.
  • the shape of the conversion unit may also be a non-square according to the shape of the prediction unit.
  • HEVC a method called an adaptive offset filter proposed in Non-Patent Document 4 is adopted.
  • the adaptive offset filter is provided between the deblocking filter and the adaptive loop filter.
  • band offsets There are two types of adaptive offsets called band offsets and six types called edge offsets, and it is also possible not to apply offsets. Then, the image can be divided into quad-trees, and the type of adaptive offset described above can be selected for each divided region. By using this method, encoding efficiency can be improved.
  • whether or not the block includes an edge has a correlation with the TU size and the PU size. That is, for example, for a flatter region, a larger TU size tends to be selected. For example, for a region including an edge, a smaller TU size tends to be selected.
  • Non-Patent Document 3 it is determined by calculating a cost function value or the like which of the edge offset and the band offset is better without using such a correlation with the TU size. It was. Therefore, the amount of calculation is large, and information indicating whether an edge offset or a band offset is selected must be sent to the decoding side.
  • the present disclosure has been made in view of such a situation, and reduces information to be transmitted to the decoding side and improves encoding efficiency.
  • An image processing apparatus includes a decoding unit that generates an image by decoding an encoded stream that is encoded in units having a hierarchical structure, and a block size of an image generated by the decoding unit, or An offset setting unit that sets an offset type for adaptive offset processing according to the area, and the adaptive offset processing for the image generated by the decoding unit with the offset type set by the offset setting unit. And an adaptive offset processing unit to perform.
  • the offset setting unit sets a band offset for the block when the size or area of the block is large, and sets an edge offset for the block when the size or area of the block is small. Can do.
  • the block is a TU (Transform Unit).
  • a receiver that receives the encoded stream and on / off information indicating whether the adaptive offset processing is on or off; and the decoder generates the image by decoding the encoded stream received by the receiver.
  • the adaptive offset processing unit when the on / off information received by the receiving unit is on of the adaptive offset processing, is an image generated by the decoding unit with the type of offset set by the offset setting unit. The adaptive offset processing can be performed on the target.
  • the offset setting unit sets a band offset when the size or area of the block is equal to or larger than the first size or the first area, and the block size or area is the first size or the first area. If it is smaller, an edge offset can be set.
  • the offset setting unit may set the adaptive offset processing to be off when the size or area of the block is equal to or larger than the first size or the second size or the second area larger than the first area. it can.
  • the offset setting unit can set an edge offset for the block when NSQT (Non-Square-Quadtree Transform) is applied to the block.
  • NSQT Non-Square-Quadtree Transform
  • the offset setting unit sets the type of offset of the adaptive offset processing according to the size or area of the short side of the block. Can do.
  • the block is an LCU (Largest Coding Unit), and the offset setting unit can set the type of offset of the adaptive offset processing according to the integration of the area of the sub-blocks included in the LCU.
  • LCU Large Coding Unit
  • an image processing device generates an image by decoding an encoded stream encoded in a unit having a hierarchical structure, and sets the size or area of a block of the generated image. Accordingly, the type of offset of the adaptive offset process is set, and the adaptive offset process is performed on the generated image with the set type of offset.
  • An image processing apparatus includes an offset setting unit that sets an offset type of adaptive offset processing according to a size or area of a block of an image subjected to local decoding processing when an image is encoded, With the type of offset set by the offset setting unit, using the image as the target, the adaptive offset processing unit that performs the adaptive offset processing, and the image that has been subjected to the adaptive offset processing by the adaptive offset processing unit, An encoding unit that encodes the image in a unit having a hierarchical structure.
  • the offset setting unit sets a band offset for the block when the size or area of the block is large, and sets an edge offset for the block when the size or area of the block is small. Can do.
  • the block is a TU (Transform Unit).
  • a transmission unit that transmits an image encoded by the encoding unit; the adaptive offset processing unit determines whether the adaptive offset processing is on or off; and when the adaptive offset processing is on, the offset setting The adaptive offset processing is performed on the image with the type of offset set by the unit, and the transmission unit can transmit on / off information indicating whether the adaptive offset processing is on or off.
  • the offset setting unit sets a band offset when the size or area of the block is equal to or larger than the first size or the first area, and the block size or area is the first size or the first area. If it is smaller, an edge offset can be set.
  • the offset setting unit may set the adaptive offset processing to be off when the size or area of the block is equal to or larger than the first size or the second size or the second area larger than the first area. it can.
  • the offset setting unit can set an edge offset for the block when NSQT (Non-Square-Quadtree Transform) is applied to the block.
  • NSQT Non-Square-Quadtree Transform
  • the offset setting unit sets the type of offset of the adaptive offset processing according to the size or area of the short side of the block. Can do.
  • the block is an LCU (Largest Coding Unit), and the offset setting unit can set the type of offset of the adaptive offset processing according to the integration of the area of the sub-blocks included in the LCU.
  • LCU Large Coding Unit
  • An image processing method sets an offset type of adaptive offset processing according to a size or area of an image block subjected to local decoding processing when an image processing device encodes an image. Then, the adaptive offset processing is performed on the image with the set offset type, and the image is encoded in units having a hierarchical structure using the image on which the adaptive offset processing has been performed.
  • an image is generated by decoding an encoded stream that is encoded in units having a hierarchical structure, and an offset of adaptive offset processing is performed according to the size or area of the block of the generated image Is set. Then, the adaptive offset processing is performed on the generated image with the set offset type.
  • the type of offset of the adaptive offset process is set according to the size or area of the block of the image that has been locally decoded when the image is encoded. Then, the adaptive offset processing is performed on the image with the set offset type, and the image is encoded in a unit having a hierarchical structure using the image subjected to the adaptive offset processing.
  • the above-described image processing apparatus may be an independent apparatus, or may be an internal block constituting one image encoding apparatus or image decoding apparatus.
  • an image can be decoded.
  • the information sent to the decoding side can be reduced and the encoding efficiency can be improved.
  • an image can be encoded.
  • the information sent to the decoding side can be reduced and the encoding efficiency can be improved.
  • FIG. 1 is a block diagram illustrating a main configuration example of an AVC image encoding device.
  • FIG. It is a block diagram which shows the main structural examples of the image decoding apparatus of an AVC system. It is a block diagram which shows the main structural examples of the image coding apparatus to which an adaptive loop filter is applied. It is a block diagram which shows the main structural examples of the image decoding apparatus to which an adaptive loop filter is applied. It is a figure explaining the structural example of a coding unit. It is a figure explaining Non-Square
  • FIG. 20 is a block diagram illustrating a main configuration example of a computer. It is a block diagram which shows an example of a schematic structure of a television apparatus. It is a block diagram which shows an example of a schematic structure of a mobile telephone. It is a block diagram which shows an example of a schematic structure of a recording / reproducing apparatus. It is a block diagram which shows an example of a schematic structure of an imaging device.
  • FIG. 1 illustrates a configuration of an embodiment of an image encoding apparatus that encodes an image using an H.264 and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC (Advanced Video Coding)) encoding method.
  • H.M The H.264 and MPEG encoding methods are referred to as AVC methods.
  • the image encoding device 1 includes an A / D conversion unit 11, a screen rearrangement buffer 12, a calculation unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, an accumulation buffer 17, An inverse quantization unit 18, an inverse orthogonal transform unit 19, and a calculation unit 20 are included.
  • the image encoding device 1 is also configured to include a deblock filter 21, a frame memory 22, a selection unit 23, an intra prediction unit 24, a motion prediction / compensation unit 25, a predicted image selection unit 26, and a rate control unit 27. Has been.
  • the A / D converter 11 A / D converts the input image data, outputs it to the screen rearrangement buffer 12, and stores it.
  • the screen rearrangement buffer 12 rearranges the stored frame images in the display order in the order of frames for encoding in accordance with the GOP (Group of Picture) structure.
  • the screen rearrangement buffer 12 supplies the image with the rearranged frame order to the arithmetic unit 13.
  • the screen rearrangement buffer 12 also supplies the image in which the frame order has been rearranged to the intra prediction unit 24 and the motion prediction / compensation unit 25.
  • the calculation unit 13 subtracts the prediction image supplied from the intra prediction unit 24 or the motion prediction / compensation unit 25 via the prediction image selection unit 26 from the image read from the screen rearrangement buffer 12, and the difference information Is output to the orthogonal transform unit 14.
  • the calculation unit 13 subtracts the prediction image supplied from the intra prediction unit 24 from the image read from the screen rearrangement buffer 12. For example, in the case of an image on which inter coding is performed, the calculation unit 13 subtracts the prediction image supplied from the motion prediction / compensation unit 25 from the image read from the screen rearrangement buffer 12.
  • the orthogonal transform unit 14 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference information supplied from the computation unit 13 and supplies the transform coefficient to the quantization unit 15.
  • the quantization unit 15 quantizes the transform coefficient output from the orthogonal transform unit 14.
  • the quantization unit 15 sets a quantization parameter based on the information regarding the target value of the code amount supplied from the rate control unit 27, and performs quantization.
  • the quantization unit 15 supplies the quantized transform coefficient to the lossless encoding unit 16.
  • the lossless encoding unit 16 performs lossless encoding such as variable length encoding and arithmetic encoding on the quantized transform coefficient. Since the coefficient data is quantized under the control of the rate control unit 27, the code amount becomes a target value set by the rate control unit 27 (or approximates the target value).
  • the lossless encoding unit 16 acquires information indicating intra prediction from the intra prediction unit 24, and acquires information indicating inter prediction mode, motion vector information, and the like from the motion prediction / compensation unit 25.
  • information indicating intra prediction is hereinafter also referred to as intra prediction mode information.
  • information indicating an information mode indicating inter prediction is hereinafter also referred to as inter prediction mode information.
  • the lossless encoding unit 16 encodes the quantized transform coefficient, and converts various information such as a filter coefficient, intra prediction mode information, inter prediction mode information, and a quantization parameter into one piece of header information of encoded data. Part (multiplex).
  • the lossless encoding unit 16 supplies the encoded data obtained by encoding to the accumulation buffer 17 for accumulation.
  • the lossless encoding unit 16 performs lossless encoding processing such as variable length encoding or arithmetic encoding.
  • variable length coding examples include H.264.
  • CAVLC Context-Adaptive Variable Length Coding
  • arithmetic coding examples include CABAC (Context-Adaptive Binary Arithmetic Coding).
  • the accumulation buffer 17 temporarily holds the encoded data supplied from the lossless encoding unit 16.
  • the accumulation buffer 17 stores the accumulated encoded data at a predetermined timing in an H.264 format.
  • the transform coefficient quantized by the quantization unit 15 is also supplied to the inverse quantization unit 18.
  • the inverse quantization unit 18 inversely quantizes the quantized transform coefficient by a method corresponding to the quantization by the quantization unit 15.
  • the inverse quantization unit 18 supplies the obtained transform coefficient to the inverse orthogonal transform unit 19.
  • the inverse orthogonal transform unit 19 performs inverse orthogonal transform on the supplied transform coefficient by a method corresponding to the orthogonal transform process by the orthogonal transform unit 14.
  • the inversely orthogonal transformed output (restored difference information) is supplied to the calculation unit 20.
  • the calculation unit 20 is supplied from the intra prediction unit 24 or the motion prediction / compensation unit 25 to the inverse orthogonal transform result supplied from the inverse orthogonal transform unit 19, that is, the restored difference information, via the predicted image selection unit 26. Predicted images are added to obtain a locally decoded image (decoded image).
  • the calculation unit 20 adds the prediction image supplied from the intra prediction unit 24 to the difference information.
  • the calculation unit 20 adds the predicted image supplied from the motion prediction / compensation unit 25 to the difference information.
  • the addition result is supplied to the deblock filter 21 or the frame memory 22.
  • the deblocking filter 21 removes block distortion of the decoded image by appropriately performing deblocking filter processing.
  • the deblocking filter 21 supplies the filter processing result to the frame memory 22. Note that the decoded image output from the arithmetic unit 20 can be supplied to the frame memory 22 without going through the deblocking filter 21. That is, the deblocking filter process of the deblocking filter 21 can be omitted.
  • the frame memory 22 stores the supplied decoded image, and outputs the stored decoded image as a reference image to the intra prediction unit 24 or the motion prediction / compensation unit 25 via the selection unit 23 at a predetermined timing. .
  • the frame memory 22 supplies the reference image to the intra prediction unit 24 via the selection unit 23.
  • the frame memory 22 supplies the reference image to the motion prediction / compensation unit 25 via the selection unit 23.
  • the selection unit 23 supplies the reference image to the intra prediction unit 24 when the reference image supplied from the frame memory 22 is an image to be subjected to intra coding.
  • the selection unit 23 supplies the reference image to the motion prediction / compensation unit 25 when the reference image supplied from the frame memory 22 is an image to be inter-encoded.
  • the intra prediction unit 24 performs intra prediction (intra-screen prediction) that generates a prediction image using the pixel value in the processing target picture supplied from the frame memory 22 via the selection unit 23.
  • the intra prediction unit 24 performs this intra prediction in a plurality of modes (intra prediction modes) prepared in advance.
  • an intra 4 ⁇ 4 prediction mode, an intra 8 ⁇ 8 prediction mode, and an intra 16 ⁇ 16 prediction mode are defined for the luminance signal.
  • a prediction mode independent of the luminance signal can be defined for each macroblock.
  • intra 4 ⁇ 4 prediction mode one intra prediction mode is defined for each 4 ⁇ 4 luminance block, and for intra 8 ⁇ 8 prediction mode, for each 8 ⁇ 8 luminance block. become.
  • intra 16 ⁇ 16 prediction mode and the color difference signal one prediction mode is defined for each macroblock.
  • the intra prediction unit 24 generates prediction images in all candidate intra prediction modes, evaluates the cost function value of each prediction image using the input image supplied from the screen rearrangement buffer 12, and selects the optimum mode. select. When the optimal intra prediction mode is selected, the intra prediction unit 24 supplies the prediction image generated in the optimal mode to the calculation unit 13 and the calculation unit 20 via the predicted image selection unit 26.
  • the intra prediction unit 24 supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the lossless encoding unit 16 as appropriate.
  • the motion prediction / compensation unit 25 uses the input image supplied from the screen rearrangement buffer 12 and the reference image supplied from the frame memory 22 via the selection unit 23 for the image to be inter-coded, Perform motion prediction (inter prediction).
  • the motion prediction / compensation unit 25 performs a motion compensation process according to the detected motion vector, and generates a prediction image (inter prediction image information).
  • the motion prediction / compensation unit 25 performs such inter prediction in a plurality of modes (inter prediction modes) prepared in advance.
  • the motion prediction / compensation unit 25 generates prediction images in all candidate inter prediction modes, evaluates the cost function value of each prediction image, and selects an optimal mode.
  • the motion prediction / compensation unit 25 supplies the generated predicted image to the calculation unit 13 and the calculation unit 20 via the predicted image selection unit 26.
  • the motion prediction / compensation unit 25 supplies the inter prediction mode information indicating the adopted inter prediction mode and the motion vector information indicating the calculated motion vector to the lossless encoding unit 16.
  • the predicted image selection unit 26 supplies the output of the intra prediction unit 24 to the calculation unit 13 and the calculation unit 20 in the case of an image to be subjected to intra coding, and in the case of an image to be subjected to inter coding, the motion prediction / compensation unit 25.
  • the output is supplied to the calculation unit 13 and the calculation unit 20.
  • the rate control unit 27 controls the quantization operation rate of the quantization unit 15 based on the compressed image stored in the storage buffer 17 so that overflow or underflow does not occur.
  • FIG. 2 is a block diagram illustrating a main configuration example of an image decoding apparatus that realizes image compression by orthogonal transformation such as discrete cosine transformation or Karhunen-Labe transformation and motion compensation.
  • An image decoding device 31 shown in FIG. 2 is a decoding device corresponding to the image encoding device 1 of FIG.
  • the encoded data encoded by the image encoding device 1 is supplied to an image decoding device 31 corresponding to the image encoding device 1 via an arbitrary path such as a transmission path or a recording medium, and is decoded. .
  • the image decoding device 31 includes a storage buffer 41, a lossless decoding unit 42, an inverse quantization unit 43, an inverse orthogonal transform unit 44, a calculation unit 45, a deblock filter 46, a screen rearrangement buffer 47, And a D / A converter 48. Further, the image decoding device 31 includes a frame memory 49, a selection unit 50, an intra prediction unit 51, a motion compensation unit 52, and an image selection unit 53.
  • the accumulation buffer 41 receives and accumulates the transmitted encoded data. That is, the accumulation buffer 41 is also a receiving unit for the transmitted encoded data. This encoded data is encoded by the image encoding device 1.
  • the lossless decoding unit 42 decodes the encoded data read from the accumulation buffer 41 at a predetermined timing by a method corresponding to the encoding method of the lossless encoding unit 16 in FIG.
  • intra prediction mode information is stored in the header portion of the encoded data.
  • the lossless decoding unit 42 also decodes the intra prediction mode information and supplies the information to the intra prediction unit 51.
  • motion vector information is stored in the header portion of the encoded data.
  • the lossless decoding unit 42 also decodes the motion vector information and supplies the information to the motion compensation unit 52.
  • the inverse quantization unit 43 inversely quantizes the coefficient data (quantization coefficient) obtained by decoding by the lossless decoding unit 42 by a method corresponding to the quantization method of the quantization unit 15 in FIG. That is, the inverse quantization unit 43 performs inverse quantization of the quantization coefficient by the same method as the inverse quantization unit 18 of FIG.
  • the inverse quantization unit 43 supplies the inversely quantized coefficient data, that is, the orthogonal transform coefficient, to the inverse orthogonal transform unit 44.
  • the inverse orthogonal transform unit 44 is a method corresponding to the orthogonal transform method of the orthogonal transform unit 14 in FIG. 1 (the same method as the inverse orthogonal transform unit 19 in FIG. 1), and inverse orthogonal transforms the orthogonal transform coefficient to obtain an image code.
  • the decoding residual data corresponding to the residual data before being orthogonally transformed in the encoding apparatus 1 is obtained. For example, fourth-order inverse orthogonal transform is performed.
  • the decoded residual data obtained by the inverse orthogonal transform is supplied to the calculation unit 45.
  • a prediction image is supplied to the calculation unit 45 from the intra prediction unit 51 or the motion compensation unit 52 via the image selection unit 53.
  • the computing unit 45 adds the decoded residual data and the predicted image, and obtains decoded image data corresponding to the image data before the predicted image is subtracted by the computing unit 13 of the image encoding device 1.
  • the arithmetic unit 45 supplies the decoded image data to the deblock filter 46.
  • the deblock filter 46 removes block distortion of the supplied decoded image, and then supplies it to the screen rearrangement buffer 47.
  • the screen rearrangement buffer 47 rearranges images. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 12 in FIG. 1 is rearranged in the original display order.
  • the D / A converter 48 D / A converts the image supplied from the screen rearrangement buffer 47, and outputs and displays it on a display (not shown).
  • the output of the deblock filter 46 is further supplied to the frame memory 49.
  • the frame memory 49, the selection unit 50, the intra prediction unit 51, the motion compensation unit 52, and the image selection unit 53 are the frame memory 22, the selection unit 23, the intra prediction unit 24, and the motion prediction / compensation unit 25 of the image encoding device 1. , And the predicted image selection unit 26, respectively.
  • the selection unit 50 reads the image to be interprocessed and the image to be referenced from the frame memory 49 and supplies them to the motion compensation unit 52. In addition, the selection unit 50 reads an image used for intra prediction from the frame memory 49 and supplies the image to the intra prediction unit 51.
  • the information indicating the intra prediction mode obtained by decoding the header information is appropriately supplied from the lossless decoding unit 42 to the intra prediction unit 51.
  • the intra prediction unit 51 generates a prediction image from the reference image acquired from the frame memory 49 based on this information, and supplies the generated prediction image to the image selection unit 53.
  • the motion compensation unit 52 acquires information (prediction mode information, motion vector information, reference frame information, flags, various parameters, and the like) obtained by decoding the header information from the lossless decoding unit 42.
  • the motion compensation unit 52 generates a prediction image from the reference image acquired from the frame memory 49 based on the information supplied from the lossless decoding unit 42 and supplies the generated prediction image to the image selection unit 53.
  • the image selection unit 53 selects the prediction image generated by the motion compensation unit 52 or the intra prediction unit 51 and supplies the selected prediction image to the calculation unit 45.
  • FIG. 3 is a block diagram illustrating a configuration example of an image encoding device to which an adaptive loop filter is applied.
  • the part 27 is omitted. Also, arrows and the like are omitted as appropriate. Therefore, in the example of FIG. 3, the reference image from the frame memory 22 is directly input to the motion prediction / compensation unit 25, and the prediction image from the motion prediction / compensation unit 25 is directly output to the calculation units 13 and 20. ing.
  • the image encoding device 61 in FIG. 3 differs from the image encoding device 1 in FIG. 1 only in that an adaptive loop filter 71 is added between the deblock filter 21 and the frame memory 22.
  • the adaptive loop filter 71 calculates an adaptive loop filter coefficient so as to minimize a residual with the original image from the screen rearrangement buffer 12 (not shown), and uses this adaptive loop filter coefficient to perform deblocking. Filter processing is performed on the decoded image from the filter 21.
  • this filter for example, a Wiener filter is used.
  • the adaptive loop filter 71 sends the calculated adaptive loop filter coefficient to the lossless encoding unit 16.
  • this adaptive loop filter coefficient is subjected to lossless encoding processing such as variable length encoding and arithmetic encoding, and inserted into the header portion of the compressed image.
  • FIG. 4 is a block diagram showing a configuration example of an image decoding apparatus corresponding to the image encoding apparatus of FIG.
  • arrows and the like are omitted as appropriate. Therefore, in the example of FIG. 4, the reference image from the frame memory 49 is directly input to the motion compensation unit 52, and the predicted image from the motion compensation unit 52 is directly output to the calculation unit 45.
  • the image decoding device 81 in FIG. 4 differs from the image decoding device 31 in FIG. 2 only in that an adaptive loop filter 91 is added between the deblock filter 46 and the frame memory 49.
  • the adaptive loop filter 91 is supplied with the adaptive loop filter coefficient decoded from the lossless decoding unit 42 and extracted from the header.
  • the adaptive loop filter 91 performs a filter process on the decoded image from the deblocking filter 46 using the supplied filter coefficient.
  • this filter for example, a Wiener filter is used.
  • the image quality of the decoded image can be improved, and further the image quality of the reference image can be improved.
  • JM ⁇ ⁇ Joint Model
  • JM JM
  • High Complexity Mode Low Complexity Mode.
  • a cost function value for each prediction mode Mode is calculated, and a prediction mode that minimizes the cost function value is selected as the optimum mode for the block or macroblock.
  • the cost function in High Complexity Mode is as shown in the following formula (1).
  • ⁇ is the entire set of candidate modes for encoding the block or macroblock
  • D is the difference energy between the decoded image and the input image when encoded in the prediction mode Mode.
  • ⁇ is a Lagrange undetermined multiplier given as a function of the quantization parameter.
  • R is a total code amount when encoding is performed in the mode Mode, including orthogonal transform coefficients.
  • the cost function in Low Complexity Mode is as shown in the following formula (2).
  • D is the difference energy between the predicted image and the input image, unlike the case of High Complexity Mode.
  • QP2Quant QP
  • HeaderBit is a code amount related to information belonging to Header, such as a motion vector and a mode, which does not include an orthogonal transform coefficient.
  • HEVC method High Efficiency Video Coding
  • a hierarchical structure is defined by macroblocks and sub-macroblocks.
  • CU Coding Unit
  • CU is also called Coding Tree Block (CTB) and is a coding unit similar to a macroblock in the AVC method.
  • CTB Coding Tree Block
  • the latter is fixed to a size of 16 ⁇ 16 pixels, whereas the size of the former is not fixed, and is specified in the image compression information in each sequence.
  • SPS Sequence Coding Unit
  • LCU Large Coding Unit
  • SCU Smallest Coding Unit
  • the LCU size is 128 and the maximum hierarchical depth is 5.
  • split_flag the value of split_flag is “1”
  • the 2N ⁇ 2N size CU is divided into N ⁇ N size CUs that are one level below.
  • the CU may be divided into one or more prediction units (Prediction Units: PUs) that mean units of intra or inter prediction processing.
  • PU can be divided
  • Prediction Units: PUs Prediction Units: PUs
  • Unit: TU transformation units
  • Unit TU
  • a macroblock in the AVC method can be considered to correspond to an LCU.
  • the size of the LCU in the highest hierarchy is H.264, for example, 128 ⁇ 128 pixels. Generally, it is set larger than the macroblock of the H.264 / AVC format.
  • each processing unit of LCU, CU, PU, and TU is also referred to as a block as appropriate.
  • Non-Square Quadtree Transform In the HEVC method, a rectangular TU called NSQT (Non-Square Quadtree Transform) as shown in FIG. 6 can be used.
  • a single TU is applied to the CU.
  • NSQT a plurality of squares or a rectangular TU
  • Non-Patent Document 3 This is described, for example, as a short distance intra prediction method in Non-Patent Document 3 in the case of intra prediction.
  • prediction units of various sizes such as 1 ⁇ 4 pixels, 2 ⁇ 8 pixels, 4 ⁇ 16 pixels, 4 ⁇ 1 pixels, 8 ⁇ 2 pixels, and 16 ⁇ 4 pixels are included in the image.
  • the adaptive offset filter (Picture Quality Adaptive Offset: PQAO) is provided between the deblock filter (DB) and the adaptive loop filter (ALF) as shown in FIG.
  • band offsets There are two types of adaptive offsets called band offsets and six types called edge offsets, and it is also possible not to apply offsets. Then, the image is divided into quad-trees, and it is possible to select which of the above-described adaptive offset types is used for encoding each region.
  • This selection information is encoded as PQAO Info.
  • the encoding unit Entropy Coding
  • a bit stream is generated, and the generated bit stream is transmitted to the decoding side.
  • a cost function value J0 of Level-0 (division depth 0) indicating a state where the region 0 is not divided is calculated.
  • cost function values J1, J2, J3, and J4 of Level-1 (division depth 0) indicating a state where the area 0 is divided into four areas 1 to 4 are calculated.
  • Partitions of Level-1 is selected by J0> (J1 + J2 + J3 + J4).
  • cost function values J5 to J20 of Level-2 (division depth 2) indicating a state where the area 0 is divided into 16 areas 5 to 20 are calculated.
  • a partition region (Partitions) of Level-1 is selected in region 1 by J1 ⁇ (J5 + J6 + J9 + J10).
  • a Level-2 partition region (Partitions) is selected by J2> (J7 + J8 + J11 + J12).
  • J3> J13 + J14 + J17 + J18
  • J4> J15 + J16 + J19 + J20
  • the division region (Partitions) of Level-1 is selected in the region 4.
  • the final quad-tree region (Partitions) indicated by A4 in the quad-tree structure is determined. Then, for each region of the quad-tree structure determined, cost function values are calculated for all of the two types of band offsets, six types of edge offsets, and no offset, and it is determined which offset is used for encoding.
  • EO (4) that is, the fourth type of edge offset is determined for the region 1 as indicated by the white arrow.
  • OFF that is, no offset is determined
  • EO (2) that is, the second type of edge offset is determined.
  • OFF that is, no offset is determined.
  • BO (1) that is, the first type of band offset
  • EO (2) that is, 2 of edge offset
  • the type has been determined.
  • BO (2) that is, the second type of band offset
  • BO (1) that is, the first type of band offset.
  • EO (1) that is, the first type of edge offset is determined.
  • the central 16 bands are divided into the first group, and the 8 bands on both sides are divided into the second group.
  • the offset of only one of the first group and the second group is encoded and sent to the decoding side.
  • the first group and the second group have pixels. For this reason, by sending only one offset, it is possible to suppress an increase in the amount of coding due to transmission of pixel values that are not included in each quad-tree region.
  • the luminance signal is limited to 16,235
  • the color difference signal is limited to 16,240.
  • the broadcast legal shown in the lower part of FIG. 9 is applied, and the offset value for each of the two bands on both sides indicated by the crosses is not transmitted.
  • the pixel value is compared with the adjacent pixel value adjacent to the pixel value, and the offset value is transmitted to the category corresponding thereto.
  • edge offset there are four one-dimensional patterns shown in FIGS. 10A to 10D and two two-dimensional patterns shown in FIG. 10E and FIG. 10F.
  • the offset is transmitted in the indicated category.
  • FIG. 10A shows that 1-D, 0-degree in which adjacent pixels are arranged one-dimensionally on the left and right sides with respect to the pixel C, that is, 0 degrees with respect to the pattern of A in FIG. Represents a pattern.
  • FIG. 10B adjacent pixels are arranged one-dimensionally above and below the pixel C, that is, 1-D, 90-degree, which forms 90 degrees with respect to the pattern of A in FIG. Represents a pattern.
  • FIG. 10 is such that adjacent pixels are arranged one-dimensionally in the upper left and lower right with respect to the pixel C, that is, 1-D, which forms 135 degrees with respect to the pattern of A in FIG. Represents a 135-degree pattern.
  • FIG. 10D adjacent pixels are arranged one-dimensionally on the upper right and lower left with respect to the pixel C, that is, 45 degrees with respect to the pattern of A in FIG. Represents the -degree pattern.
  • FIG. 10 E in FIG. 10 represents a 2-D, cross pattern in which adjacent pixels are arranged two-dimensionally in the vertical and horizontal directions with respect to the pixel C, that is, intersect with the pixel C.
  • FIG. 10F shows that 2-D, diagonal in which adjacent pixels are two-dimensionally arranged with respect to the pixel C in the upper right lower left and upper left lower right, that is, obliquely intersect the pixel C. Represents a pattern.
  • 11A shows a rule list of one-dimensional patterns (Classification rule for 1-D patterns).
  • the patterns A in FIG. 11 to D in FIG. 11 are classified into five types of categories as shown in FIG. 11A, offsets are calculated based on the categories, and sent to the decoding unit.
  • the pixel value of the pixel C When the pixel value of the pixel C is smaller than the pixel values of two adjacent pixels, it is classified into category 1. When the pixel value of the pixel C is smaller than the pixel value of one adjacent pixel and matches the pixel value of the other adjacent pixel, it is classified into category 2. When the pixel value of the pixel C is larger than the pixel value of one adjacent pixel and matches the pixel value of the other adjacent pixel, it is classified into category 3. When the pixel value of the pixel C is larger than the pixel values of two adjacent pixels, it is classified into category 4. If none of the above, it is classified into category 0.
  • B in FIG. 11 shows a two-dimensional pattern rule list (Classification rule for 2-D ⁇ 2patterns).
  • the patterns of E of FIG. 10 and F of FIG. 10 are classified into seven types of categories as shown in B of FIG. 11, and offsets are sent to the decoding unit according to the categories.
  • the pixel C When the pixel value of the pixel C is smaller than the pixel values of the four adjacent pixels, it is classified into category 1. When the pixel value of the pixel C is smaller than the pixel values of the three adjacent pixels and matches the pixel value of the fourth adjacent pixel, the pixel C is classified into category 2. When the pixel value of the pixel C is smaller than the pixel values of the three adjacent pixels and larger than the pixel value of the fourth adjacent pixel, the pixel C is classified into category 3.
  • the pixel C When the pixel value of the pixel C is larger than the pixel values of the three adjacent pixels and smaller than the pixel value of the fourth adjacent pixel, it is classified into category 4. When the pixel value of the pixel C is larger than the pixel values of the three adjacent pixels and matches the pixel value of the fourth adjacent pixel, the pixel C is classified into category 5. When the pixel value of the pixel C is larger than the pixel values of the four adjacent pixels, it is classified into category 6. If none of the above, it is classified into category 0.
  • the 1-bit offset value is sent to the decoding side with higher accuracy than the low-delay encoding condition.
  • the quad-tree structure (including information about the type of offset and no offset) described above with reference to FIG. 8 and the offset value are sent to the decoding side.
  • the category may be sent to the decoding side, or may be obtained in each device.
  • FIG. 12 illustrates a configuration of an embodiment of an image encoding device as an image processing device to which the present disclosure is applied.
  • the image encoding apparatus 101 shown in FIG. 12 encodes image data using a prediction process.
  • a method according to HEVC High Efficiency Video Coding
  • the image encoding device 101 in FIG. 12 includes a calculation unit 20, a deblock filter 21, a frame memory 22, a selection unit 23, an intra prediction unit 24, a motion prediction / compensation unit 25, a predicted image selection unit 26, and a rate control unit 27. 1 in common with the image encoding device 1 of FIG.
  • the image coding apparatus 101 in FIG. 12 is different from the image coding in FIG. 1 in that the orthogonal transform unit 14 is replaced with the orthogonal transform unit 111 and that a class classification unit 112 and an adaptive offset unit 113 are added. Different from the device 1.
  • the orthogonal transform unit 111 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference information supplied from the computation unit 13 in the same manner as the orthogonal transform unit 14 of FIG. Is supplied to the quantization unit 15. At that time, the orthogonal transform unit 111 determines the orthogonal transform size for the block (processing unit, TU in this case) by mode determination based on the cost function value. Then, the orthogonal transform unit 111 supplies the determined TU size, which is a unit of the determined orthogonal transform, to the lossless encoding unit 16 and the class classification unit 112.
  • orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform
  • the class classification unit 112 performs class classification according to the TU size from the orthogonal transformation unit 111.
  • the class classification in the class classification unit 112 is setting (classification) of the type of offset. That is, the class classification unit 112 sets whether to apply edge offset (EO) or band offset (BO) according to the TU size from the orthogonal transform unit 111.
  • the class classification unit 112 supplies information indicating the set offset type to the adaptive offset unit 113.
  • the deblock filter 21, the adaptive offset unit 113, and the adaptive loop filter 71 are provided in the motion compensation loop in that order.
  • the motion compensation loop is an arithmetic unit 13, an orthogonal transformation unit 111, a quantization unit 15, an inverse quantization unit 18, an inverse orthogonal transformation unit 19, an arithmetic unit 20, a frame memory 22, a selection unit 23, an intra prediction unit 24, or a motion.
  • This block includes a prediction / compensation unit 25 and a predicted image selection unit 26.
  • the filter processing performed by the deblock filter 21, the adaptive offset unit 113, and the adaptive loop filter 71 in the motion compensation loop is also collectively referred to as in-loop filter processing.
  • the adaptive offset unit 113 performs an offset process on the decoded image (baseband information after local decoding) from the deblocking filter 21 based on the information indicating the type of offset from the class classification unit 112.
  • the adaptive offset unit 113 determines whether or not to perform the offset (on / off of the adaptive offset filter), and when it is determined to perform the offset, obtains the category and the offset value, and calculates the arithmetic unit 20.
  • the offset process is performed on the decoded image from.
  • the adaptive offset unit 113 supplies the image after the offset process to the adaptive loop filter 71.
  • the adaptive offset unit 113 supplies the determined on / off information and information indicating the offset value to the lossless encoding unit 16.
  • the adaptive loop filter 71 calculates an adaptive loop filter coefficient so as to minimize the residual from the original image (not shown) from the screen rearrangement buffer 12, and uses this adaptive loop filter coefficient to perform an adaptive offset. Filter processing is performed on the decoded image from the unit 113. As this filter, for example, a Wiener filter is used. This improves the image quality. Although not shown, the adaptive loop filter 71 sends the calculated adaptive loop filter coefficient to the lossless encoding unit 16.
  • the lossless encoding unit 16 in FIG. 12 encodes the quantized transform coefficient as well as various types of filter coefficients, prediction mode information, quantization parameters, and the like, similar to the lossless encoding unit 16 in FIG.
  • the information is part of the header information of the encoded data.
  • the lossless encoding unit 16 uses the on / off information from the adaptive offset unit 113 and the information indicating the offset value as part of the header information of the encoded data as the adaptive offset parameter. Further, the lossless encoding unit 16 also uses information about the TU size from the orthogonal transform unit 111, information indicating an adaptive loop filter coefficient, and the like as part of the header information of the encoded data.
  • the orthogonal transform size for the luminance signal can be any of 4 ⁇ 4, 8 ⁇ 8, 16 ⁇ 16, and 32 ⁇ 32.
  • a smaller TU size tends to be selected for a block (processing unit) including an edge.
  • larger TU sizes tend to be selected. That is, whether or not the block includes an edge has a correlation with the size of the block.
  • the present technology it is determined whether to use the edge offset or the band offset in the adaptive offset filter by using the correlation with the block size described above. Therefore, the information regarding the adaptive offset filter sent to the decoding side is reduced, and the coding efficiency is improved.
  • an edge offset is applied to a block for which a smaller TU size is selected, and a band offset is applied to a block for which a larger TU size is selected.
  • the class classification unit 112 sets an edge offset for a block having a TU size of 4 ⁇ 4 or 8 ⁇ 8, and blocks other TU sizes (or more). Set the band offset.
  • an edge offset can be set to be applied to a TU having a TU size of 4 ⁇ 4 or 8 ⁇ 8.
  • a band offset it is possible to set to be applied to a TU having a TU size of 16 ⁇ 16 and not to apply an adaptive offset to a 32 ⁇ 32 TU.
  • an edge offset is applied to a TU to which NSQT described above with reference to FIG. 6 is applied. This is because a block to which NSQT is applied is considered to include an edge in the texture.
  • whether to apply an edge offset or a band offset may be set for the size of the shorter side.
  • the adaptive offset filter determines whether to use the edge offset or the band offset, and based on this, the adaptive offset filter is applied to the decoded image. I did it.
  • class classification is performed according to the TU size as a block (processing unit), but is not limited to a TU. That is, class classification (setting of filter type) may be performed according to PU size or CU size, and an adaptive offset filter may be performed for each PU or CU.
  • a block having a larger PU size or CU size is likely to be a flat region, and is therefore set to use a band offset.
  • a block having a smaller large PU size or CU size is more likely to include an edge, and thus is set to use an edge offset.
  • the CU includes a PU and the PU includes a TU.
  • the amount of calculation for determining the edge offset and the band offset by the existing method is the smallest based on the CU size, followed by the case based on the PU size and the TU size.
  • an edge offset or band offset may be set for each LCU.
  • the processing unit for example, TU, PU, or CU
  • the processing unit included in the LCU is determined depending on whether there are more units larger than a predetermined size or smaller than a predetermined size. That is, setting of the edge offset or band offset is performed according to how much area the processing unit of each size occupies in the LCU, not the number of processing units.
  • the LCU has a size of 32 ⁇ 32 and is configured to include three 16 ⁇ 16 TUs, two 8 ⁇ 8 TUs, and eight 4 ⁇ 4 TUs.
  • the class classification unit 112 includes a counter for each size, and the area is obtained from the number and size indicated by the counter.
  • the method according to the present technology described above can be applied to both the luminance signal and the color difference signal.
  • FIG. 16 is a block diagram illustrating a configuration example of the orthogonal transform unit 111 and the adaptive offset unit 113.
  • the orthogonal transform unit 111 includes a 4 ⁇ 4 orthogonal transform unit 131, an 8 ⁇ 8 orthogonal transform unit 132, a 16 ⁇ 16 orthogonal transform unit 133, a 32 ⁇ 32 orthogonal transform unit 134, a cost function calculation unit 135, And a TU size determination unit 136.
  • the adaptive offset unit 113 is configured to include an on / off determination unit 141, a category classification unit 142, and an offset processing unit 143.
  • Difference information (PU) indicating a difference value from the calculation unit 13 is supplied to the 4 ⁇ 4 orthogonal transform unit 131, the 8 ⁇ 8 orthogonal transform unit 132, the 16 ⁇ 16 orthogonal transform unit 133, and the 32 ⁇ 32 orthogonal transform unit 134. Is done.
  • the 4 ⁇ 4 orthogonal transform unit 131 performs orthogonal transform on the difference information from the calculation unit 13 with a 4 ⁇ 4 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the 8 ⁇ 8 orthogonal transform unit 132 performs orthogonal transform on the difference information from the calculation unit 13 with an 8 ⁇ 8 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the 16 ⁇ 16 orthogonal transform unit 133 performs orthogonal transform on the difference information from the calculation unit 13 with a 16 ⁇ 16 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the 32 ⁇ 32 orthogonal transform unit 134 performs orthogonal transform on the difference information from the calculation unit 13 with a 32 ⁇ 32 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the cost function calculation unit 135 calculates the cost function value using the conversion coefficient of each TU size, and supplies the conversion coefficient of each TU size and the corresponding cost function value to the TU size determination unit 136.
  • the TU size determination unit 136 determines the optimal TU size for the block based on the cost function value calculated by the cost function calculation unit 135 and supplies the orthogonal transform coefficient of the determined TU size to the quantization unit 15. Further, the TU size determination unit 136 supplies information on the determined TU size to the class classification unit 112 and the lossless encoding unit 16.
  • the class classification unit 112 Based on the TU size from the TU size determination unit 136, the class classification unit 112 sets the type of offset (edge offset or band offset) to be applied to the block by the method of the present technology described above. To do. The class classification unit 112 supplies the set offset type information to the on / off determination unit 141.
  • the on / off determination unit 141 determines the on / off of the adaptive offset processing for the block (for example, TU) using the pixel value after the deblocking filter processing from the deblocking filter 21. For example, the on / off determination unit 141 calculates the cost function value in the adaptive offset process according to the type of offset set by the class classification unit 112 and the cost function value without the adaptive offset process. Then, the on / off determination unit 141 determines on / off of the adaptive offset process based on the calculated cost function value.
  • the on / off determination unit 141 supplies the pixel value after the deblocking filter processing from the deblocking filter 21 and the determined on / off information to the category classification unit 142. Further, the on / off determination unit 141 supplies the determined on / off information to the lossless encoding unit 16.
  • the category classification unit 142 uses the pixel value after the deblocking filter processing to determine the category in the offset type set by the class classification unit 112. Classify.
  • the category classification unit 142 supplies information indicating the classified category and the pixel value after deblocking filter processing to the offset processing unit 143.
  • the category classification unit 142 stores the pixel value after the deblocking filter processing from the on / off determination unit 141 as it is in the offset processing unit 143. Supply.
  • the offset processing unit 143 When the on / off information from the on / off determination unit 141 indicates on, the offset processing unit 143 performs adaptive offset processing on the pixel value after the deblocking filter processing from the category classification unit 142. The offset processing unit 143 supplies the pixel value after adaptive offset processing to the adaptive loop filter 71.
  • the offset processing unit 143 uses the pixel value from the screen rearrangement buffer 12 and the pixel value after the deblocking filter processing from the category classification unit 142 for the set offset type and the classified category. Find the offset value.
  • the offset processing unit 143 performs offset processing on the pixel value after the deblocking filter processing from the category classification unit 142 with the set offset type, the classified category, and the obtained offset value.
  • the offset processing unit 143 also supplies the obtained offset value to the lossless encoding unit 16.
  • the offset processing unit 143 uses the pixel value after the deblocking filter processing from the category classification unit 142 as it is (without performing the offset processing). To the adaptive loop filter 71.
  • the lossless encoding unit 16 adds information on the TU size from the TU size determination unit 136 to the encoded stream.
  • the lossless encoding unit 16 uses the on / off information from the on / off determination unit 141 and the offset value information from the offset processing unit 143 (when the on / off information indicates on) as an adaptive offset parameter. Is added to the encoded stream.
  • step S101 the A / D conversion unit 11 performs A / D conversion on the input image.
  • step S102 the screen rearrangement buffer 12 stores the A / D converted image, and rearranges the picture from the display order to the encoding order.
  • the decoded image to be referred to is read from the frame memory 22 and the intra-prediction unit via the selection unit 23 24.
  • the intra prediction unit 24 performs intra prediction on the pixels of the block to be processed in all candidate intra prediction modes.
  • decoded pixels to be referred to pixels that are not filtered or offset by the deblock filter 21, the adaptive offset unit 113, and the adaptive loop filter 71 are used.
  • intra prediction is performed in all candidate intra prediction modes, and the cost function shown in the equation (1) or equation (2) is used for all candidate intra prediction modes, A cost function value is calculated. Then, based on the calculated cost function value, the optimal intra prediction mode is selected, and the predicted image generated by the intra prediction in the optimal intra prediction mode and its cost function value are supplied to the predicted image selection unit 26.
  • the processing target image supplied from the screen rearrangement buffer 12 is an inter-processed image
  • the referenced image is read from the frame memory 22 and supplied to the motion prediction / compensation unit 25 via the selection unit 23. Is done.
  • the motion prediction / compensation unit 25 performs motion prediction / compensation processing.
  • motion prediction processing is performed in all candidate inter prediction modes, and the cost function shown in Equation (1) or Equation (2) is used for all candidate inter prediction modes.
  • a cost function value is calculated. Based on the calculated cost function value, the optimal inter prediction mode is determined, and the predicted image generated in the optimal inter prediction mode and its cost function value are supplied to the predicted image selection unit 26.
  • step S ⁇ b> 105 the predicted image selection unit 26 optimizes one of the optimal intra prediction mode and the optimal inter prediction mode based on the cost function values output from the intra prediction unit 24 and the motion prediction / compensation unit 25. Determine the prediction mode. Then, the predicted image selection unit 26 selects the predicted image in the determined optimal prediction mode and supplies it to the calculation units 13 and 20. This predicted image is used for calculations in steps S106 and S112 described later.
  • the prediction image selection information is supplied to the intra prediction unit 24 or the motion prediction / compensation unit 25.
  • the intra prediction unit 24 supplies information indicating the optimal intra prediction mode (that is, intra prediction mode information) to the lossless encoding unit 16.
  • the motion prediction / compensation unit 25 When the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 25 further includes information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode as a lossless encoding unit. 16 is output.
  • Information according to the optimal inter prediction mode includes motion vector information and reference frame information.
  • step S106 the calculation unit 13 calculates a difference between the image rearranged in step S102 and the predicted image selected in step S105.
  • the predicted image is supplied from the motion prediction / compensation unit 25 in the case of inter prediction, and from the intra prediction unit 24 in the case of intra prediction, to the calculation unit 13 via the predicted image selection unit 26, respectively.
  • ⁇ Difference data has a smaller data volume than the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.
  • the orthogonal transform unit 111 determines an orthogonal transform size. That is, the 4 ⁇ 4 orthogonal transform unit 131 performs orthogonal transform on the difference information from the calculation unit 13 with a 4 ⁇ 4 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the 8 ⁇ 8 orthogonal transform unit 132 performs orthogonal transform on the difference information from the calculation unit 13 with an 8 ⁇ 8 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the 16 ⁇ 16 orthogonal transform unit 133 performs orthogonal transform on the difference information from the calculation unit 13 with a 16 ⁇ 16 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the 32 ⁇ 32 orthogonal transform unit 134 performs orthogonal transform on the difference information from the calculation unit 13 with a 32 ⁇ 32 TU size, and supplies the transform coefficient to the cost function calculation unit 135.
  • the cost function calculation unit 135 calculates the cost function value using the conversion coefficient of each TU size, and supplies the conversion coefficient of each TU size and the corresponding cost function value to the TU size determination unit 136.
  • the TU size determination unit 136 determines an optimal TU size for the block based on the cost function value calculated by the cost function calculation unit 135.
  • the TU size determination unit 136 supplies information on the determined TU size to the class classification unit 112 and the lossless encoding unit 16.
  • step S108 the TU size determination unit 136 performs orthogonal transformation.
  • the TU size determination unit 136 performs orthogonal transform with the TU size determined in step S ⁇ b> 107 and supplies the orthogonal transform coefficient to the quantization unit 15.
  • step S109 the quantization unit 15 quantizes the transform coefficient.
  • the quantization unit 15 sets the quantization parameter based on the information regarding the target value of the code amount supplied from the rate control unit 27 and performs quantization, as will be described in the process of step S119 described later.
  • step S110 the inverse quantization unit 18 inversely quantizes the transform coefficient quantized by the quantization unit 15 with characteristics corresponding to the characteristics of the quantization unit 15.
  • step S ⁇ b> 111 the inverse orthogonal transform unit 19 performs inverse orthogonal transform on the transform coefficient inversely quantized by the inverse quantization unit 18 with characteristics corresponding to the characteristics of the orthogonal transform unit 14.
  • step S112 the calculation unit 20 adds the predicted image input via the predicted image selection unit 26 to the locally decoded difference information, and the locally decoded (that is, locally decoded) image. (An image corresponding to the input to the calculation unit 13) is generated.
  • step S113 the deblocking filter 21 performs a deblocking process on the image from the calculation unit 20, and supplies the pixel value after the deblocking process to the adaptive offset unit 113. By this processing, block distortion is suppressed.
  • step S114 the class classification unit 112 and the adaptive offset unit 113 perform an adaptive offset process according to the orthogonal transform size determined in step S107. Details of the adaptive offset processing will be described later with reference to FIG.
  • the pixel value after adaptive offset processing is supplied to the adaptive loop filter 71 by the processing in step S114.
  • Information indicating on / off of the adaptive offset processing and information indicating the offset value are supplied to the lossless encoding unit 16. Ringing and the like are suppressed by this adaptive offset processing.
  • step S115 the adaptive loop filter 71 performs an adaptive loop filter on the pixel value after the adaptive offset process, and supplies the pixel value after the adaptive loop filter to the frame memory 22.
  • the adaptive loop filter 71 calculates the adaptive loop filter coefficient so as to minimize the residual with the original image (not shown) from the screen rearrangement buffer 12, and uses the adaptive loop filter coefficient to Filter processing is performed on the decoded image from the adaptive offset unit 113. Although not shown, the adaptive loop filter 71 sends the calculated adaptive loop filter coefficient to the lossless encoding unit 16.
  • step S116 the frame memory 22 stores the filtered image.
  • images that are not filtered or offset by the deblocking filter 21, the adaptive offset unit 113, and the adaptive loop filter 71 are also supplied from the arithmetic unit 20 and stored.
  • step S109 the transform coefficient quantized in step S109 described above is also supplied to the lossless encoding unit 16.
  • the lossless encoding unit 16 encodes the quantized transform coefficient output from the quantization unit 15. That is, the difference image is subjected to lossless encoding such as variable length encoding and arithmetic encoding, and is compressed.
  • the intra prediction mode information from the intra prediction unit 24 or the information corresponding to the optimal inter prediction mode from the motion prediction / compensation unit 25 input to the lossless encoding unit 16 in step S105 described above, etc. is also encoded and added to the header information.
  • the information indicating the orthogonal transform size input to the lossless encoding unit 16 in step S107 described above, the information indicating on / off input to the lossless encoding unit 16 in step S114 described above, and the offset information are also encoded. And added to the header information.
  • information indicating the inter prediction mode is encoded for each LCU.
  • Motion vector information and reference frame information are encoded for each target PU.
  • step S118 the accumulation buffer 17 accumulates the difference image as a compressed image.
  • the compressed image stored in the storage buffer 17 is appropriately read out and transmitted to the decoding side via the transmission path.
  • step S119 the rate control unit 27 controls the quantization operation rate of the quantization unit 15 based on the compressed image stored in the storage buffer 17 so that overflow or underflow does not occur.
  • step S119 ends, the encoding process ends.
  • the TU size determination unit 136 of the orthogonal transform unit 111 supplies information related to the TU size to the class classification unit 112 by the process of step S108 in FIG.
  • the class classification unit 112 determines whether the block (for example, TU) has an edge offset (EO) or a band offset (BO). Set whether to apply.
  • the class classification unit 112 supplies the set offset type information to the on / off determination unit 141.
  • step S152 the on / off determination unit 141 determines on / off of the adaptive offset processing for this TU using the pixel value after the deblocking filter processing from the deblocking filter 21.
  • the on / off determination unit 141 calculates the cost function value in the adaptive offset process according to the type of offset set by the class classification unit 112 and the cost function value without the adaptive offset process. Then, the on / off determination unit 141 determines on / off of the adaptive offset process based on the calculated cost function value. The on / off determination unit 141 supplies the pixel value after the deblocking filter processing from the deblocking filter 21 and the determined on / off information to the category classification unit 142.
  • step S153 the on / off determination unit 141 supplies the determined on / off information to the lossless encoding unit 16, and encodes the on / off information.
  • step S154 the category classification unit 142 determines whether the adaptive offset filter is on for the TU based on on / off information from the on / off determination unit 141. If it is determined in step S154 that the adaptive offset filter is on, the process proceeds to step S155.
  • step S155 the category classification unit 142 uses the pixel value after the deblocking filter processing from the on / off determination unit 141 to classify the category in the offset type set by the class classification unit 112.
  • the category classification unit 142 supplies information indicating the classified category and the pixel value after deblocking filter processing to the offset processing unit 143.
  • step S156 the offset processing unit 143 obtains an offset value and performs an offset process. That is, the offset processing unit 143 uses the pixel value from the screen rearrangement buffer 12 and the pixel value after the deblocking filter processing from the category classification unit 142 for the set offset type and the classified category. Find the offset value.
  • the pixel value after deblocking filter processing from the category classification unit 142 is subjected to offset processing with the set offset type, the classified category, and the obtained offset value.
  • the offset processing unit 143 supplies the pixel value after adaptive offset processing to the adaptive loop filter 71.
  • step S157 the offset processing unit 143 supplies the obtained offset value to the lossless encoding unit 16, and encodes information indicating the offset value.
  • step S154 if it is determined in step S154 that the adaptive offset filter is off, the adaptive offset processing ends. That is, in this case, the category classification unit 142 supplies the pixel value after the deblocking filter processing from the on / off determination unit 141 to the offset processing unit 143 as it is. Further, the offset processing unit 143 supplies the pixel value after the deblocking filter processing from the category classification unit 142 to the adaptive loop filter 71 as it is (without performing the offset processing).
  • the image encoding device 101 determines whether it is an edge offset or a band offset according to the size of a block (for example, TU). Therefore, since it is not necessary to send the information indicating whether it is a region division and edge offset or band offset (that is, the map information of the quad-tree structure described above with reference to FIG. 8) to the decoding side, encoding is performed. Efficiency can be improved.
  • FIG. 19 illustrates a configuration of an embodiment of an image decoding device as an image processing device to which the present disclosure is applied.
  • An image decoding apparatus 201 shown in FIG. 19 is a decoding apparatus corresponding to the image encoding apparatus 101 in FIG.
  • encoded data encoded by the image encoding device 101 is transmitted to an image decoding device 201 corresponding to the image encoding device 101 via a predetermined transmission path and decoded.
  • the image decoding device 201 in FIG. 19 is common to the image decoding device 31 in FIG. 2 in that the storage buffer 41, the lossless decoding unit 42, the inverse quantization unit 43, the calculation unit 45, and the deblocking filter 46 are provided. .
  • the image decoding apparatus 201 in FIG. 19 includes a screen rearrangement buffer 47, a D / A conversion unit 48, a frame memory 49, a selection unit 50, an intra prediction unit 51, a motion compensation unit 52, and an image selection unit 53. It is common with the image decoding apparatus 31 of FIG.
  • the image decoding apparatus 201 in FIG. 19 is different from the image decoding in FIG. 2 in that the inverse orthogonal transform unit 44 is replaced with the inverse orthogonal transform unit 211 and that the class classification unit 212 and the adaptive offset unit 213 are added. Different from the device 31.
  • the lossless decoding unit 42 converts the information supplied from the accumulation buffer 41 and encoded by the lossless encoding unit 16 of FIG. 12 into the code of the lossless encoding unit 16 as in the case of the lossless decoding unit 42 of FIG. Decoding is performed using a method corresponding to the conversion method.
  • motion vector information, reference frame information, prediction mode information (information indicating an intra prediction mode or an inter prediction mode), information on a TU (orthogonal transform) size, an adaptive offset parameter, and the like are also decoded. Is done.
  • the adaptive offset parameter includes information indicating on / off of the adaptive offset processing (hereinafter, also referred to as an on / off flag) encoded by the lossless encoding unit 16 in FIG. 12, information indicating an offset value, and the like. Consists of.
  • the adaptive offset parameter is supplied to the adaptive offset unit 213. Further, the lossless decoding unit 42 supplies information regarding the TU size to the inverse orthogonal transform unit 211.
  • the inverse orthogonal transform unit 211 performs inverse orthogonal transform corresponding to the TU size from the lossless decoding unit 42, and the decoded residual data corresponding to the residual data before being orthogonally transformed by the image encoding device 101 in FIG. obtain.
  • the decoded residual data obtained by the inverse orthogonal transform is supplied to the calculation unit 45.
  • the inverse orthogonal transform unit 211 supplies the class classification unit 212 with information regarding the TU size from the lossless decoding unit 42.
  • the class classification unit 212 is configured similarly to the class classification unit 112 of FIG. That is, the class classification unit 212 performs class classification according to the TU size from the inverse orthogonal transform unit 211.
  • the class classification in the class classification unit 212 is setting (classification) of the type of offset. That is, the class classification unit 212 sets whether to apply edge offset (EO) or band offset (BO) according to the TU size from the inverse orthogonal transform unit 211.
  • the class classification unit 212 supplies information indicating the set offset type to the adaptive offset unit 213.
  • the deblock filter 46, the adaptive offset unit 213, and the adaptive loop filter 91 are provided in the motion compensation loop in that order.
  • the motion compensation loop is a block composed of a calculation unit 45, a frame memory 49, a selection unit 50, a motion compensation unit 52, and an image selection unit 53.
  • the filter processing performed by the deblock filter 46, the adaptive offset unit 213, and the adaptive loop filter 91 in the motion compensation loop is also collectively referred to as in-loop filter processing.
  • the adaptive offset unit 213 is supplied with information indicating the on / off flag of the adaptive offset process and the offset value, which are the adaptive offset parameters from the lossless decoding unit 42.
  • the adaptive offset unit 213 performs offset processing on the pixel value of the decoded image from the deblocking filter 46 using the information, and supplies the pixel value after the offset processing to the adaptive loop filter 91.
  • the adaptive offset unit 213 performs category classification based on the offset type set by the class classification unit 212 using the pixel value after the deblocking filter. . Then, the adaptive offset unit 213 performs offset processing on the pixel value after the deblocking filter with the offset value from the lossless decoding unit 42 in the classified category with the set offset type.
  • the adaptive offset unit 213 supplies the post-offset pixel value to the adaptive loop filter 91.
  • the adaptive loop filter coefficient decoded by the lossless decoding unit 42 and extracted from the header is supplied.
  • the adaptive loop filter 91 performs a filtering process on the decoded image from the adaptive offset unit 213 using the supplied filter coefficient.
  • the basic operation principles related to the present technology in the class classification unit 212 and the adaptive offset unit 213 are the same as those of the class classification unit 112 and the adaptive offset unit 113 in FIG.
  • the TU size (or PU size) is determined by the mode determination, and thereby classifying the adaptive offset processing.
  • the image decoding apparatus 201 shown in FIG. 19 information on the TU size (or PU size) is added to the encoded stream and sent from the encoding side. Therefore, the image decoding apparatus 201 obtains the information by decoding the information, and classifies the adaptive offset process based on the obtained size information.
  • FIG. 20 is a block diagram illustrating a configuration example of the inverse orthogonal transform unit 211 and the adaptive offset unit 213.
  • the inverse orthogonal transform unit 211 includes a TU size buffer 231, a 4 ⁇ 4 inverse orthogonal transform unit 232, an 8 ⁇ 8 inverse orthogonal transform unit 233, a 16 ⁇ 16 inverse orthogonal transform unit 234, and a 32 ⁇ 32 inverse.
  • An orthogonal transform unit 235 is included.
  • the adaptive offset unit 213 is configured to include an on / off flag buffer 241, a category classification unit 242, and an offset processing unit 243.
  • the TU size buffer 231 includes information on the TU size among the 4 ⁇ 4 inverse orthogonal transform unit 232, the 8 ⁇ 8 inverse orthogonal transform unit 233, the 16 ⁇ 16 inverse orthogonal transform unit 234, and the 32 ⁇ 32 inverse orthogonal transform unit 235. , To the inverse orthogonal transform unit of the corresponding size. As a result, the inverse orthogonal transform unit having the corresponding size becomes enable, and the orthogonal transform coefficient from the inverse quantization unit 43 is inversely orthogonal transformed to correspond to the residual data before being orthogonally transformed by the image coding apparatus 101. Decoding residual data is obtained.
  • the orthogonal transform coefficients from the inverse quantization unit 43 are 4 ⁇ 4 inverse orthogonal transform unit 232, 8 ⁇ 8 inverse orthogonal transform unit 233, 16 ⁇ 16 inverse orthogonal transform unit 234, and 32 ⁇ 32 inverse orthogonal transform unit 235. To be supplied.
  • the 4 ⁇ 4 inverse orthogonal transform unit 232 is enabled when the TU size from the TU size buffer 231 indicates 4 ⁇ 4, and the decoding residual corresponding to the residual data before being subjected to orthogonal transform in the image encoding device 101. Get the data.
  • the 4 ⁇ 4 inverse orthogonal transform unit 232 supplies the obtained decoded residual data (difference value) to the calculation unit 45.
  • the 8 ⁇ 8 inverse orthogonal transform unit 233 is enabled when the TU size from the TU size buffer 231 indicates 8 ⁇ 8, and the decoding residual corresponding to the residual data before being subjected to orthogonal transform in the image encoding device 101. Get the data.
  • the 8 ⁇ 8 inverse orthogonal transform unit 233 supplies the obtained decoded residual data (difference value) to the calculation unit 45.
  • the 16 ⁇ 16 inverse orthogonal transform unit 234 is enabled when the TU size from the TU size buffer 231 indicates 16 ⁇ 16, and the decoding residual corresponding to the residual data before being subjected to orthogonal transform in the image encoding device 101. Get the data.
  • the 16 ⁇ 16 inverse orthogonal transform unit 234 supplies the obtained decoded residual data (difference value) to the calculation unit 45.
  • the 32 ⁇ 32 inverse orthogonal transform unit 235 is enabled when the TU size from the TU size buffer 231 indicates 32 ⁇ 32, and the decoding residual corresponding to the residual data before being orthogonally transformed in the image encoding device 101. Get the data.
  • the 32 ⁇ 32 inverse orthogonal transform unit 235 supplies the obtained decoded residual data (difference value) to the calculation unit 45.
  • the class classification unit 212 is basically configured similarly to the class classification unit 112 of FIG. That is, the class classification unit 112 determines the type of offset to be applied to the block (whether it is an edge offset or a band offset) based on the TU size from the TU size buffer 231 by the above-described method according to the present technology. Set. The class classification unit 212 supplies the set offset type information to the category classification unit 242.
  • the on / off flag from the lossless decoding unit 42 is supplied to the on / off flag buffer 241.
  • the offset value from the lossless decoding unit 42 is supplied to the offset processing unit 243.
  • the on / off flag buffer 241 temporarily stores the on / off flag from the lossless decoding unit 42 and supplies it to the category classification unit 242 at a predetermined timing.
  • the category classification unit 242 is further supplied with information indicating which of the edge offset / edge offset is applied and the pixel value after deblocking filter processing from the deblocking filter 46.
  • the category classification unit 242 classifies the category in the offset type set by the class classification unit 212 using the pixel value after the deblocking filter processing. To do.
  • the category classification unit 242 supplies information indicating the classified category and the pixel value after deblocking filter processing to the offset processing unit 243.
  • the category classification unit 242 supplies the pixel value after the deblocking filter processing from the on / off flag buffer 241 to the offset processing unit 243 as it is. To do.
  • the offset processing unit 243 When the on / off information from the on / off flag buffer 241 indicates on, the offset processing unit 243 performs adaptive offset processing on the pixel value after the deblocking filter processing from the category classification unit 242. The offset processing unit 243 supplies the pixel value after adaptive offset processing to the adaptive loop filter 91.
  • the offset processing unit 243 performs an offset process on the post-deblock filter-processed pixel value from the category classification unit 242 with the set offset type, the classified category, and the offset value from the lossless decoding unit 42. Apply.
  • the offset processing unit 243 uses the pixel value after the deblocking filter processing from the category classification unit 242 as it is (without performing offset processing). To the adaptive loop filter 91.
  • category classification is performed on each of the encoding side and the decoding side.
  • the present invention is not limited to this, and category classification is performed on the encoding side and the information may be sent to the decoding side. Good.
  • step S201 the accumulation buffer 41 accumulates the transmitted encoded data.
  • step S ⁇ b> 202 the lossless decoding unit 42 decodes the encoded data supplied from the accumulation buffer 41. That is, the I picture, P picture, and B picture encoded by the lossless encoding unit 16 of FIG. 12 are decoded.
  • motion vector information reference frame information
  • prediction mode information intra prediction mode or inter prediction mode
  • information on the TU size information on adaptive offset parameters
  • the prediction mode information is intra prediction mode information
  • the prediction mode information is supplied to the intra prediction unit 51.
  • the prediction mode information is inter prediction mode information
  • motion vector information corresponding to the prediction mode information is supplied to the motion compensation unit 52.
  • Information about the TU size is supplied to the inverse orthogonal transform unit 211.
  • the on / off flag information, which is an adaptive offset parameter, and information indicating the offset value are supplied to the adaptive offset unit 213.
  • step S203 the intra prediction unit 51 or the motion compensation unit 52 performs a prediction image generation process corresponding to the prediction mode information supplied from the lossless decoding unit 42, respectively.
  • the intra prediction unit 51 when the intra prediction mode information is supplied from the lossless decoding unit 42, the intra prediction unit 51 generates Most Probable ⁇ Mode and generates an intra prediction image of the intra prediction mode by parallel processing.
  • the motion compensation unit 52 When the inter prediction mode information is supplied from the lossless decoding unit 42, the motion compensation unit 52 performs a motion prediction / compensation process in the inter prediction mode, and generates an inter prediction image.
  • the prediction image (intra prediction image) generated by the intra prediction unit 51 or the prediction image (inter prediction image) generated by the motion compensation unit 52 is supplied to the image selection unit 53.
  • step S204 the image selection unit 53 selects a predicted image. That is, the prediction image generated by the intra prediction unit 51 or the prediction image generated by the motion compensation unit 52 is supplied. Therefore, the supplied predicted image is selected and supplied to the calculation unit 45, and is added to the output of the inverse orthogonal transform unit 44 in step S208 described later.
  • step S202 the transform coefficient decoded by the lossless decoding unit 42 is also supplied to the inverse quantization unit 43.
  • the inverse quantization unit 43 inverts the transform coefficient decoded by the lossless decoding unit 42 with the quantization parameter decoded by the lossless decoding unit 42 with the characteristic corresponding to the characteristic of the quantization unit 15 of FIG. Quantize.
  • the TU size buffer 231 of the inverse orthogonal transform unit 211 receives information regarding the TU (orthogonal transform) size supplied in step S202.
  • This information about the TU size corresponds to the TU size among the 4 ⁇ 4 inverse orthogonal transform unit 232, the 8 ⁇ 8 inverse orthogonal transform unit 233, the 16 ⁇ 16 inverse orthogonal transform unit 234, and the 32 ⁇ 32 inverse orthogonal transform unit 235.
  • the information regarding the TU size is also supplied to the class classification unit 212.
  • step S207 the inverse orthogonal transform unit corresponding to the TU size in the inverse orthogonal transform unit 211 converts the transform coefficient inversely quantized by the inverse quantization unit 43 with characteristics corresponding to the characteristics of the orthogonal transform unit 111 in FIG. Perform inverse orthogonal transform.
  • the difference information corresponding to the input of the orthogonal transform unit 111 in FIG. 12 (the output of the calculation unit 13) is decoded.
  • step S208 the calculation unit 45 adds the predicted image selected in the processing in step S204 described above and input via the image selection unit 53 to the difference information. As a result, the original image is decoded.
  • step S209 the deblock filter 46 performs deblock filter processing on the image from the calculation unit 45. Thereby, block distortion is suppressed.
  • the deblock filter 46 supplies the post-deblock filter processed pixel value to the adaptive offset unit 213.
  • step S210 the class classification unit 212 and the adaptive offset unit 213 perform adaptive offset processing according to the orthogonal transform size received in step S206. Details of the adaptive offset processing will be described later with reference to FIG. Thus, the adaptive offset process is performed to remove ringing and the like.
  • the pixel value after adaptive offset processing is supplied to the adaptive loop filter 91 by the processing in step S210.
  • the adaptive loop filter 91 performs an adaptive loop filter process on the pixel value after the adaptive offset process, and supplies the pixel value after the adaptive loop filter to the frame memory 49 or the screen rearrangement buffer 47.
  • step S212 the frame memory 49 stores the adaptively filtered image.
  • step S213 the screen rearrangement buffer 47 rearranges the images after the adaptive loop filter 91. That is, the order of frames rearranged for encoding by the screen rearrangement buffer 12 of the image encoding device 101 is rearranged to the original display order.
  • step S214 the D / A converter 48 D / A converts the image from the screen rearrangement buffer 47. This image is output to a display (not shown), and the image is displayed.
  • step S214 When the process of step S214 is completed, the decoding process is terminated.
  • the TU size buffer 231 of the inverse orthogonal transform unit 211 supplies information on the TU size to the class classification unit 212 by the process of step S206 in FIG.
  • the class classification unit 212 determines whether the edge offset (EO) or the band offset (BO) is applied to the block (eg, TU) according to the method of the present technology described above according to the supplied TU size. Set whether to apply.
  • the class classification unit 212 supplies the set offset type information to the category classification unit 242.
  • step S252 the on / off flag buffer 241 receives the on / off flag supplied from the lossless decoding unit 42 by the processing in step S202 of FIG.
  • step S253 the category classification unit 242 determines whether the adaptive offset filter is on for the TU based on the on / off flag from the on / off flag buffer 241. If it is determined in step S253 that the adaptive offset filter is on, the process proceeds to step S254.
  • step S254 the category classification unit 242 classifies the category in the offset type set by the class classification unit 212 using the pixel value after the deblocking filter processing from the deblocking filter 46.
  • the category classification unit 242 supplies information indicating the classified category and the pixel value after deblocking filter processing to the offset processing unit 243.
  • step S255 the offset processing unit 243 receives the offset value supplied from the lossless decoding unit 42 by the process in step S202 of FIG.
  • step S256 the offset processing unit 243 performs an offset process. That is, the offset processing unit 243 performs an offset process on the post-deblock filter processed pixel value from the category classification unit 242 with the set offset type, the classified category, and the received offset value.
  • the offset processing unit 243 supplies the pixel value after adaptive offset processing to the adaptive loop filter 91.
  • step S253 if it is determined in step S253 that the adaptive offset filter is off, the adaptive offset processing ends. That is, in this case, the category classification unit 242 supplies the pixel value after the deblocking filter processing from the on / off flag buffer 241 to the offset processing unit 243 as it is. Further, the offset processing unit 143 supplies the pixel value after the deblocking filter processing from the category classification unit 242 as it is (without performing the offset processing) to the adaptive loop filter 91.
  • whether it is an edge offset or a band offset is set according to the size of a block (for example, TU). Therefore, since it is not necessary to send the information indicating whether it is a region division and edge offset or band offset (that is, the map information of the quad-tree structure described above with reference to FIG. 8) to the decoding side, encoding is performed. Efficiency can be improved.
  • the present technology can be applied to a device using another coding method as long as the device performs adaptive offset processing.
  • this disclosure includes, for example, MPEG, When receiving image information (bitstream) compressed by orthogonal transform such as discrete cosine transform and motion compensation, such as 26x, via network media such as satellite broadcasting, cable television, the Internet, or mobile phones.
  • the present invention can be applied to an image encoding device and an image decoding device used in the above.
  • the present disclosure can be applied to an image encoding device and an image decoding device that are used when processing on a storage medium such as an optical disk, a magnetic disk, and a flash memory.
  • the present disclosure can also be applied to motion prediction / compensation devices included in such image encoding devices and image decoding devices.
  • FIG. 23 shows an example of a multi-view image encoding method.
  • the multi-viewpoint image includes a plurality of viewpoint images, and a predetermined one viewpoint image among the plurality of viewpoints is designated as the base view image.
  • Each viewpoint image other than the base view image is treated as a non-base view image.
  • each view (same view), information about TU size, and parameters such as adaptive offset parameters such as on / off information and offset values (hereinafter simply referred to as parameters). ) Can also be set.
  • parameters such as adaptive offset parameters such as on / off information and offset values (hereinafter simply referred to as parameters).
  • Each view can also share parameters set in other views.
  • the parameters set in the base view are used in at least one non-base view.
  • FIG. 24 is a diagram illustrating a multi-view image encoding apparatus that performs the multi-view image encoding described above. As illustrated in FIG. 24, the multi-view image encoding device 600 includes an encoding unit 601, an encoding unit 602, and a multiplexing unit 603.
  • the encoding unit 601 encodes the base view image and generates a base view image encoded stream.
  • the encoding unit 602 encodes the non-base view image and generates a non-base view image encoded stream.
  • the multiplexing unit 603 multiplexes the base view image encoded stream generated by the encoding unit 601 and the non-base view image encoded stream generated by the encoding unit 602 to generate a multi-view image encoded stream. To do.
  • the image encoding device 101 (FIG. 12) can be applied to the encoding unit 601 and the encoding unit 602 of the multi-view image encoding device 600.
  • the multi-view image encoding apparatus 600 sets and transmits the parameters set by the encoding unit 601 and the parameters set by the encoding unit 602.
  • the parameters set by the encoding unit 601 as described above may be set and transmitted so as to be shared and used by the encoding unit 601 and the encoding unit 602.
  • the parameters set by the encoding unit 602 may be set and transmitted so as to be shared by the encoding unit 601 and the encoding unit 602.
  • FIG. 25 is a diagram illustrating a multi-view image decoding apparatus that performs the above-described multi-view image decoding.
  • the multi-view image decoding device 610 includes a demultiplexing unit 611, a decoding unit 612, and a decoding unit 613.
  • the demultiplexing unit 611 demultiplexes the multi-view image encoded stream in which the base view image encoded stream and the non-base view image encoded stream are multiplexed, and the base view image encoded stream and the non-base view image The encoded stream is extracted.
  • the decoding unit 612 decodes the base view image encoded stream extracted by the demultiplexing unit 611 to obtain a base view image.
  • the decoding unit 613 decodes the non-base view image encoded stream extracted by the demultiplexing unit 611 to obtain a non-base view image.
  • the image decoding device 201 (FIG. 19) can be applied to the decoding unit 612 and the decoding unit 613 of the multi-view image decoding device 610.
  • the multi-view image decoding apparatus 610 performs processing using the parameters set by the encoding unit 601 and decoded by the decoding unit 612 and the parameters set by the encoding unit 602 and decoded by the decoding unit 613.
  • the parameters set by the encoding unit 601 (or encoding 602) as described above may be set and transmitted so as to be shared by the encoding unit 601 and the encoding unit 602.
  • processing is performed using the parameters set by the encoding unit 601 (or encoding 602) and decoded by the decoding unit 612 (or decoding unit 613).
  • FIG. 26 shows an example of the multi-view image encoding method.
  • a hierarchical image includes images of a plurality of layers (resolutions), and an image of a predetermined one layer among the plurality of resolutions is designated as a base layer image. Images in each layer other than the base layer image are treated as non-base layer images.
  • each layer in each layer (same layer), information on the TU size, and parameters such as adaptive offset parameters such as on / off information and offset values (hereinafter simply referred to as “only”). (Referred to as parameters) can also be set.
  • each layer different layers can share a buffer index set in another view.
  • the parameters set in the base layer are used in at least one non-base layer.
  • FIG. 27 is a diagram illustrating a hierarchical image encoding apparatus that performs the above-described hierarchical image encoding.
  • the hierarchical image encoding device 620 includes an encoding unit 621, an encoding unit 622, and a multiplexing unit 623.
  • the encoding unit 621 encodes the base layer image and generates a base layer image encoded stream.
  • the encoding unit 622 encodes the non-base layer image and generates a non-base layer image encoded stream.
  • the multiplexing unit 623 multiplexes the base layer image encoded stream generated by the encoding unit 621 and the non-base layer image encoded stream generated by the encoding unit 622 to generate a hierarchical image encoded stream. .
  • the image encoding device 101 (FIG. 12) can be applied to the encoding unit 621 and the encoding unit 622 of the hierarchical image encoding device 620.
  • the hierarchical image encoding device 620 sets and transmits the parameters set by the encoding unit 621 and the parameters set by the encoding unit 622.
  • the parameters set by the encoding unit 621 as described above may be set and transmitted so as to be shared by the encoding unit 621 and the encoding unit 622.
  • the parameters set by the encoding unit 622 may be set and transmitted so as to be shared by the encoding unit 621 and the encoding unit 622.
  • FIG. 28 is a diagram illustrating a hierarchical image decoding apparatus that performs the hierarchical image decoding described above.
  • the hierarchical image decoding device 630 includes a demultiplexing unit 631, a decoding unit 632, and a decoding unit 633.
  • the demultiplexing unit 631 demultiplexes the hierarchical image encoded stream in which the base layer image encoded stream and the non-base layer image encoded stream are multiplexed, and the base layer image encoded stream and the non-base layer image code Stream.
  • the decoding unit 632 decodes the base layer image encoded stream extracted by the demultiplexing unit 631 to obtain a base layer image.
  • the decoding unit 633 decodes the non-base layer image encoded stream extracted by the demultiplexing unit 631 to obtain a non-base layer image.
  • the image decoding device 201 (FIG. 19) can be applied to the decoding unit 632 and the decoding unit 633 of the hierarchical image decoding device 630.
  • the parameter set by the encoding unit 621, the parameter decoded by the decoding unit 632, and the encoding unit 622 are set, and the decoding unit 633 performs processing using the parameter.
  • the parameter set by the encoding unit 621 may be set and transmitted so as to be shared by the encoding unit 621 and the encoding unit 622.
  • processing is performed using the parameters set by the encoding unit 621 (or encoding 622) and decoded by the decoding unit 632 (or decoding unit 633).
  • the series of processes described above can be executed by hardware or can be executed by software.
  • a program constituting the software is installed in the computer.
  • the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like.
  • FIG. 29 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processes by a program.
  • a CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • an input / output interface 805 is connected to the bus 804.
  • An input unit 806, an output unit 807, a storage unit 808, a communication unit 809, and a drive 810 are connected to the input / output interface 805.
  • the input unit 811 includes a keyboard, a mouse, a microphone, and the like.
  • the output unit 812 includes a display, a speaker, and the like.
  • the storage unit 813 includes a hard disk, a nonvolatile memory, and the like.
  • the communication unit 814 includes a network interface or the like.
  • the drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • the CPU 801 loads the program stored in the storage unit 83 into the RAM 803 via the input / output interface 810 and the bus 804 and executes the program, for example. Is performed.
  • the program executed by the computer 800 can be provided by being recorded on a removable medium 821 as a package medium, for example.
  • the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program can be installed in the storage unit 813 via the input / output interface 810 by attaching the removable medium 821 to the drive 815. Further, the program can be received by the communication unit 814 via a wired or wireless transmission medium and installed in the storage unit 813. In addition, the program can be installed in the ROM 802 or the storage unit 813 in advance.
  • the program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.
  • the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but may be performed in parallel or It also includes processes that are executed individually.
  • system represents the entire apparatus composed of a plurality of devices (apparatuses).
  • the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units).
  • the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit).
  • a configuration other than that described above may be added to the configuration of each device (or each processing unit).
  • a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). . That is, the present technology is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present technology.
  • An image encoding device and an image decoding device include a transmitter or a receiver in optical broadcasting, satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to terminals by cellular communication, etc.
  • the present invention can be applied to various electronic devices such as a recording device that records an image on a medium such as a magnetic disk and a flash memory, or a playback device that reproduces an image from these storage media.
  • a recording device that records an image on a medium such as a magnetic disk and a flash memory
  • a playback device that reproduces an image from these storage media.
  • FIG. 30 illustrates an example of a schematic configuration of a television device to which the above-described embodiment is applied.
  • the television apparatus 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, an external interface 909, a control unit 910, a user interface 911, And a bus 912.
  • Tuner 902 extracts a signal of a desired channel from a broadcast signal received via antenna 901, and demodulates the extracted signal. Then, the tuner 902 outputs the encoded bit stream obtained by the demodulation to the demultiplexer 903. In other words, the tuner 902 serves as a transmission unit in the television apparatus 900 that receives an encoded stream in which an image is encoded.
  • the demultiplexer 903 separates the video stream and audio stream of the viewing target program from the encoded bit stream, and outputs each separated stream to the decoder 904. Further, the demultiplexer 903 extracts auxiliary data such as EPG (Electronic Program Guide) from the encoded bit stream, and supplies the extracted data to the control unit 910. Note that the demultiplexer 903 may perform descrambling when the encoded bit stream is scrambled.
  • EPG Electronic Program Guide
  • the decoder 904 decodes the video stream and audio stream input from the demultiplexer 903. Then, the decoder 904 outputs the video data generated by the decoding process to the video signal processing unit 905. In addition, the decoder 904 outputs audio data generated by the decoding process to the audio signal processing unit 907.
  • the video signal processing unit 905 reproduces the video data input from the decoder 904 and causes the display unit 906 to display the video.
  • the video signal processing unit 905 may cause the display unit 906 to display an application screen supplied via a network.
  • the video signal processing unit 905 may perform additional processing such as noise removal on the video data according to the setting.
  • the video signal processing unit 905 may generate a GUI (Graphical User Interface) image such as a menu, a button, or a cursor, and superimpose the generated image on the output image.
  • GUI Graphic User Interface
  • the display unit 906 is driven by a drive signal supplied from the video signal processing unit 905, and displays an image on a video screen of a display device (for example, a liquid crystal display, a plasma display, or an OELD (Organic ElectroLuminescence Display) (organic EL display)). Or an image is displayed.
  • a display device for example, a liquid crystal display, a plasma display, or an OELD (Organic ElectroLuminescence Display) (organic EL display)). Or an image is displayed.
  • the audio signal processing unit 907 performs reproduction processing such as D / A conversion and amplification on the audio data input from the decoder 904, and outputs audio from the speaker 908.
  • the audio signal processing unit 907 may perform additional processing such as noise removal on the audio data.
  • the external interface 909 is an interface for connecting the television apparatus 900 to an external device or a network.
  • a video stream or an audio stream received via the external interface 909 may be decoded by the decoder 904. That is, the external interface 909 also has a role as a transmission unit in the television apparatus 900 that receives an encoded stream in which an image is encoded.
  • the control unit 910 includes a processor such as a CPU and memories such as a RAM and a ROM.
  • the memory stores a program executed by the CPU, program data, EPG data, data acquired via a network, and the like.
  • the program stored in the memory is read and executed by the CPU when the television apparatus 900 is activated.
  • the CPU executes the program to control the operation of the television device 900 according to an operation signal input from the user interface 911, for example.
  • the user interface 911 is connected to the control unit 910.
  • the user interface 911 includes, for example, buttons and switches for the user to operate the television device 900, a remote control signal receiving unit, and the like.
  • the user interface 911 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 910.
  • the bus 912 connects the tuner 902, the demultiplexer 903, the decoder 904, the video signal processing unit 905, the audio signal processing unit 907, the external interface 909, and the control unit 910 to each other.
  • the decoder 904 has the function of the image decoding apparatus according to the above-described embodiment. Therefore, when the image is decoded by the television apparatus 900, information to be sent to the decoding side is reduced, and the encoding efficiency can be improved.
  • FIG. 31 shows an example of a schematic configuration of a mobile phone to which the above-described embodiment is applied.
  • a mobile phone 920 includes an antenna 921, a communication unit 922, an audio codec 923, a speaker 924, a microphone 925, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, a control unit 931, an operation A portion 932 and a bus 933.
  • the antenna 921 is connected to the communication unit 922.
  • the speaker 924 and the microphone 925 are connected to the audio codec 923.
  • the operation unit 932 is connected to the control unit 931.
  • the bus 933 connects the communication unit 922, the audio codec 923, the camera unit 926, the image processing unit 927, the demultiplexing unit 928, the recording / reproducing unit 929, the display unit 930, and the control unit 931 to each other.
  • the mobile phone 920 has various operation modes including a voice call mode, a data communication mode, a shooting mode, and a videophone mode, and is used for sending and receiving voice signals, sending and receiving e-mail or image data, taking images, and recording data. Perform the action.
  • the analog voice signal generated by the microphone 925 is supplied to the voice codec 923.
  • the audio codec 923 converts an analog audio signal into audio data, A / D converts the compressed audio data, and compresses it. Then, the audio codec 923 outputs the compressed audio data to the communication unit 922.
  • the communication unit 922 encodes and modulates the audio data and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal.
  • the communication unit 922 demodulates and decodes the received signal to generate audio data, and outputs the generated audio data to the audio codec 923.
  • the audio codec 923 decompresses the audio data and performs D / A conversion to generate an analog audio signal. Then, the audio codec 923 supplies the generated audio signal to the speaker 924 to output audio.
  • the control unit 931 generates character data constituting the e-mail in response to an operation by the user via the operation unit 932.
  • the control unit 931 causes the display unit 930 to display characters.
  • the control unit 931 generates e-mail data in response to a transmission instruction from the user via the operation unit 932, and outputs the generated e-mail data to the communication unit 922.
  • the communication unit 922 encodes and modulates email data and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921.
  • the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal.
  • the communication unit 922 demodulates and decodes the received signal to restore the email data, and outputs the restored email data to the control unit 931.
  • the control unit 931 displays the content of the electronic mail on the display unit 930 and stores the electronic mail data in the storage medium of the recording / reproducing unit 929.
  • the recording / reproducing unit 929 has an arbitrary readable / writable storage medium.
  • the storage medium may be a built-in storage medium such as a RAM or a flash memory, or an externally mounted type such as a hard disk, magnetic disk, magneto-optical disk, optical disk, USB (Universal Serial Bus) memory, or memory card. It may be a storage medium.
  • the camera unit 926 images a subject to generate image data, and outputs the generated image data to the image processing unit 927.
  • the image processing unit 927 encodes the image data input from the camera unit 926 and stores the encoded stream in the storage medium of the storage / playback unit 929.
  • the demultiplexing unit 928 multiplexes the video stream encoded by the image processing unit 927 and the audio stream input from the audio codec 923, and the multiplexed stream is the communication unit 922. Output to.
  • the communication unit 922 encodes and modulates the stream and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921.
  • the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal.
  • These transmission signal and reception signal may include an encoded bit stream.
  • the communication unit 922 demodulates and decodes the received signal to restore the stream, and outputs the restored stream to the demultiplexing unit 928.
  • the demultiplexing unit 928 separates the video stream and the audio stream from the input stream, and outputs the video stream to the image processing unit 927 and the audio stream to the audio codec 923.
  • the image processing unit 927 decodes the video stream and generates video data.
  • the video data is supplied to the display unit 930, and a series of images is displayed on the display unit 930.
  • the audio codec 923 decompresses the audio stream and performs D / A conversion to generate an analog audio signal. Then, the audio codec 923 supplies the generated audio signal to the speaker 924 to output audio.
  • the image processing unit 927 has the functions of the image encoding device and the image decoding device according to the above-described embodiment. Accordingly, when encoding and decoding an image with the mobile phone 920, information to be sent to the decoding side is reduced, and encoding efficiency can be improved.
  • FIG. 32 shows an example of a schematic configuration of a recording / reproducing apparatus to which the above-described embodiment is applied.
  • the recording / reproducing device 940 encodes audio data and video data of a received broadcast program and records the encoded data on a recording medium.
  • the recording / reproducing device 940 may encode audio data and video data acquired from another device and record them on a recording medium, for example.
  • the recording / reproducing device 940 reproduces data recorded on the recording medium on a monitor and a speaker, for example, in accordance with a user instruction. At this time, the recording / reproducing device 940 decodes the audio data and the video data.
  • the recording / reproducing apparatus 940 includes a tuner 941, an external interface 942, an encoder 943, an HDD (Hard Disk Drive) 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) 948, a control unit 949, and a user interface. 950.
  • Tuner 941 extracts a signal of a desired channel from a broadcast signal received via an antenna (not shown), and demodulates the extracted signal. Then, the tuner 941 outputs the encoded bit stream obtained by the demodulation to the selector 946. That is, the tuner 941 has a role as a transmission unit in the recording / reproducing apparatus 940.
  • the external interface 942 is an interface for connecting the recording / reproducing apparatus 940 to an external device or a network.
  • the external interface 942 may be, for example, an IEEE1394 interface, a network interface, a USB interface, or a flash memory interface.
  • video data and audio data received via the external interface 942 are input to the encoder 943. That is, the external interface 942 serves as a transmission unit in the recording / reproducing device 940.
  • the encoder 943 encodes video data and audio data when the video data and audio data input from the external interface 942 are not encoded. Then, the encoder 943 outputs the encoded bit stream to the selector 946.
  • the HDD 944 records an encoded bit stream in which content data such as video and audio is compressed, various programs, and other data on an internal hard disk. Further, the HDD 944 reads out these data from the hard disk when reproducing video and audio.
  • the disk drive 945 performs recording and reading of data to and from the mounted recording medium.
  • the recording medium mounted on the disk drive 945 is, for example, a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.) or a Blu-ray (registered trademark) disk. It may be.
  • the selector 946 selects an encoded bit stream input from the tuner 941 or the encoder 943 when recording video and audio, and outputs the selected encoded bit stream to the HDD 944 or the disk drive 945. In addition, the selector 946 outputs the encoded bit stream input from the HDD 944 or the disk drive 945 to the decoder 947 during video and audio reproduction.
  • the decoder 947 decodes the encoded bit stream and generates video data and audio data. Then, the decoder 947 outputs the generated video data to the OSD 948. The decoder 904 outputs the generated audio data to an external speaker.
  • OSD 948 reproduces the video data input from the decoder 947 and displays the video. Further, the OSD 948 may superimpose a GUI image such as a menu, a button, or a cursor on the video to be displayed.
  • the control unit 949 includes a processor such as a CPU and memories such as a RAM and a ROM.
  • the memory stores a program executed by the CPU, program data, and the like.
  • the program stored in the memory is read and executed by the CPU when the recording / reproducing apparatus 940 is activated, for example.
  • the CPU controls the operation of the recording / reproducing apparatus 940 in accordance with an operation signal input from the user interface 950, for example, by executing the program.
  • the user interface 950 is connected to the control unit 949.
  • the user interface 950 includes, for example, buttons and switches for the user to operate the recording / reproducing device 940, a remote control signal receiving unit, and the like.
  • the user interface 950 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 949.
  • the encoder 943 has the function of the image encoding apparatus according to the above-described embodiment.
  • the decoder 947 has the function of the image decoding apparatus according to the above-described embodiment.
  • FIG. 33 illustrates an example of a schematic configuration of an imaging apparatus to which the above-described embodiment is applied.
  • the imaging device 960 images a subject to generate an image, encodes the image data, and records it on a recording medium.
  • the imaging device 960 includes an optical block 961, an imaging unit 962, a signal processing unit 963, an image processing unit 964, a display unit 965, an external interface 966, a memory 967, a media drive 968, an OSD 969, a control unit 970, a user interface 971, and a bus. 972.
  • the optical block 961 is connected to the imaging unit 962.
  • the imaging unit 962 is connected to the signal processing unit 963.
  • the display unit 965 is connected to the image processing unit 964.
  • the user interface 971 is connected to the control unit 970.
  • the bus 972 connects the image processing unit 964, the external interface 966, the memory 967, the media drive 968, the OSD 969, and the control unit 970 to each other.
  • the optical block 961 includes a focus lens and a diaphragm mechanism.
  • the optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962.
  • the imaging unit 962 includes an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and converts an optical image formed on the imaging surface into an image signal as an electrical signal by photoelectric conversion. Then, the imaging unit 962 outputs the image signal to the signal processing unit 963.
  • CCD Charge-Coupled Device
  • CMOS Complementary Metal-Oxide Semiconductor
  • the signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the image signal input from the imaging unit 962.
  • the signal processing unit 963 outputs the image data after the camera signal processing to the image processing unit 964.
  • the image processing unit 964 encodes the image data input from the signal processing unit 963 and generates encoded data. Then, the image processing unit 964 outputs the generated encoded data to the external interface 966 or the media drive 968. The image processing unit 964 also decodes encoded data input from the external interface 966 or the media drive 968 to generate image data. Then, the image processing unit 964 outputs the generated image data to the display unit 965. In addition, the image processing unit 964 may display the image by outputting the image data input from the signal processing unit 963 to the display unit 965. Further, the image processing unit 964 may superimpose display data acquired from the OSD 969 on an image output to the display unit 965.
  • the OSD 969 generates a GUI image such as a menu, a button, or a cursor, and outputs the generated image to the image processing unit 964.
  • the external interface 966 is configured as a USB input / output terminal, for example.
  • the external interface 966 connects the imaging device 960 and a printer, for example, when printing an image.
  • a drive is connected to the external interface 966 as necessary.
  • a removable medium such as a magnetic disk or an optical disk is attached to the drive, and a program read from the removable medium can be installed in the imaging device 960.
  • the external interface 966 may be configured as a network interface connected to a network such as a LAN or the Internet. That is, the external interface 966 has a role as a transmission unit in the imaging device 960.
  • the recording medium mounted on the media drive 968 may be any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory.
  • a recording medium may be fixedly mounted on the media drive 968, and a non-portable storage unit such as an internal hard disk drive or an SSD (Solid State Drive) may be configured.
  • the control unit 970 includes a processor such as a CPU and memories such as a RAM and a ROM.
  • the memory stores a program executed by the CPU, program data, and the like.
  • the program stored in the memory is read and executed by the CPU when the imaging device 960 is activated, for example.
  • the CPU controls the operation of the imaging device 960 according to an operation signal input from the user interface 971 by executing the program.
  • the user interface 971 is connected to the control unit 970.
  • the user interface 971 includes, for example, buttons and switches for the user to operate the imaging device 960.
  • the user interface 971 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 970.
  • the image processing unit 964 has the functions of the image encoding device and the image decoding device according to the above-described embodiment. Thereby, when encoding and decoding an image in the imaging device 960, information to be sent to the decoding side is reduced, and encoding efficiency can be improved.
  • various types of information such as syntax elements such as information on TU size, adaptive offset parameters such as on / off information and offset values, etc. are multiplexed into the encoded stream, and from the encoding side to the decoding side.
  • An example of transmission has been described.
  • the method for transmitting such information is not limited to such an example.
  • these pieces of information may be transmitted or recorded as separate data associated with the encoded bitstream without being multiplexed into the encoded bitstream.
  • the term “associate” means that an image (which may be a part of an image such as a slice or a block) included in the bitstream and information corresponding to the image can be linked at the time of decoding. Means.
  • information may be transmitted on a transmission path different from that of the image (or bit stream).
  • Information may be recorded on a recording medium (or another recording area of the same recording medium) different from the image (or bit stream).
  • the information and the image (or bit stream) may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.
  • this technique can also take the following structures.
  • a decoding unit that generates an image by decoding an encoded stream encoded in units having a hierarchical structure;
  • An offset setting unit that sets an offset type of adaptive offset processing according to the size or area of the block of the image generated by the decoding unit;
  • An image processing apparatus comprising: an adaptive offset processing unit that performs the adaptive offset processing on an image generated by the decoding unit with an offset type set by the offset setting unit.
  • the offset setting unit sets a band offset for the block when the size or area of the block is large, and sets an edge offset for the block when the size or area of the block is small. Set The image processing apparatus according to (1).
  • the decoding unit decodes the encoded stream received by the receiving unit to generate the image
  • the adaptive offset processing unit targets the image generated by the decoding unit with the type of offset set by the offset setting unit.
  • the image processing apparatus according to any one of (1) to (3), wherein the adaptive offset processing is performed.
  • the offset setting unit sets a band offset when the size or area of the block is equal to or larger than the first size or the first area, and the size or area of the block is the first size or the first size.
  • the image processing apparatus according to any one of (1) to (3), wherein an edge offset is set when the area is smaller than 1.
  • the offset setting unit sets the adaptive offset processing to be off.
  • the image processing apparatus according to (5).
  • the offset setting unit sets an edge offset for the block. Any one of (1) to (6) An image processing apparatus according to 1.
  • the offset setting unit determines the type of the offset in the adaptive offset processing according to the size or area of the short side of the block.
  • the image processing apparatus according to any one of (1) to (6) to be set.
  • the block is an LCU (Largest Coding Unit), The image processing apparatus according to any one of (1) and (2), wherein the offset setting unit sets an offset type of the adaptive offset processing according to integration of areas of subblocks included in the LCU.
  • the image processing apparatus is An image is generated by decoding an encoded stream encoded in units having a hierarchical structure, Set the offset type of adaptive offset processing according to the size or area of the block of the generated image, An image processing method for performing the adaptive offset processing on a generated image with a set type of offset.
  • an offset setting unit that sets an offset type of adaptive offset processing according to the size or area of a block of an image subjected to local decoding processing when encoding an image;
  • An image processing apparatus comprising: an encoding unit that encodes the image in units having a hierarchical structure using the image on which the adaptive offset processing has been performed by the adaptive offset processing unit.
  • the offset setting unit sets a band offset for the block when the size or area of the block is large, and sets an edge offset for the block when the size or area of the block is small. Setting The image processing apparatus according to (11).
  • the image processing device according to (11) or (12), wherein the block is a TU (Transform Unit).
  • the adaptive offset processing unit determines whether to turn on or off the adaptive offset processing. When the adaptive offset processing is on, the adaptive offset processing unit sets the adaptive offset for the image with the type of offset set by the offset setting unit.
  • Process The image processing apparatus according to any one of (10) to (13), wherein the transmission unit transmits on / off information indicating on or off of the adaptive offset processing.
  • the offset setting unit sets a band offset, and the size or area of the block is the first size or the first size.
  • the image processing apparatus according to any one of (11) to (13), wherein an edge offset is set when the area is smaller than 1.
  • the offset setting unit sets the adaptive offset processing to be off.
  • the image processing apparatus according to (15).
  • the offset setting unit sets an edge offset for the block when NSQT (Non-Square Quadtree Transform) is applied to the block.
  • NSQT Non-Square Quadtree Transform
  • the offset setting unit determines the type of offset of the adaptive offset processing according to the size or area of the short side of the block.
  • the block is an LCU (Largest Coding Unit), The image processing apparatus according to (11), wherein the offset setting unit sets an offset type of the adaptive offset process according to an integration of areas of subblocks included in the LCU.
  • the image processing apparatus is Set the offset type of adaptive offset processing according to the block size or area of the locally decoded image block when encoding the image, Performing the adaptive offset process for the image with the set offset type, An image processing method for encoding the image in units having a hierarchical structure using the image on which the adaptive offset processing has been performed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention porte sur un appareil et un procédé de traitement d'image permettant de réduire les informations envoyées à un côté décodage et d'améliorer le rendement de codage. Une unité de classification établit d'appliquer un décalage de bord (EO) ou un décalage de bande (BO) en fonction de la taille d'unité de transformation (TU) provenant d'une unité de transformation orthogonale. En conséquence, une unité de décalage adaptatif détermine d'effectuer ou non un décalage (état actif/inactif d'un filtre de décalage adaptatif), et dans le cas où il est déterminé que le décalage doit être effectué, la catégorie et la valeur de décalage dans le type du décalage établi par l'unité de classification sont obtenues, et un traitement de décalage est effectué relativement à une image décodée provenant d'une unité de calcul. La présente invention peut être appliquée, par exemple, à l'appareil de traitement d'image.
PCT/JP2013/060362 2012-04-13 2013-04-04 Appareil et procédé de traitement d'image Ceased WO2013154026A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2012-092015 2012-04-13
JP2012092015 2012-04-13

Publications (1)

Publication Number Publication Date
WO2013154026A1 true WO2013154026A1 (fr) 2013-10-17

Family

ID=49327594

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/060362 Ceased WO2013154026A1 (fr) 2012-04-13 2013-04-04 Appareil et procédé de traitement d'image

Country Status (1)

Country Link
WO (1) WO2013154026A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014050732A1 (fr) * 2012-09-28 2014-04-03 ソニー株式会社 Dispositif et procédé de codage, et dispositif et procédé de décodage
WO2015083796A1 (fr) * 2013-12-04 2015-06-11 三菱電機株式会社 Dispositif de codage d'image
JP2015128239A (ja) * 2013-12-27 2015-07-09 キヤノン株式会社 画像符号化装置、撮像装置、画像符号化方法、及びプログラム
JP2015216626A (ja) * 2014-04-23 2015-12-03 ソニー株式会社 画像処理装置及び画像処理方法
WO2018168484A1 (fr) * 2017-03-15 2018-09-20 ソニー株式会社 Dispositif de codage, procédé de codage, dispositif de décodage et procédé de décodage

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013046990A1 (fr) * 2011-09-29 2013-04-04 シャープ株式会社 Appareil de décodage de décalage, appareil de codage de décalage, appareil de filtre d'image et structure de données

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013046990A1 (fr) * 2011-09-29 2013-04-04 シャープ株式会社 Appareil de décodage de décalage, appareil de codage de décalage, appareil de filtre d'image et structure de données

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHIH-MING FU ET AL.: "CE8 Subset3: Picture Quadtree Adaptive Offset", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG16 WP3 AND ISO/IEC JTC1/SC29/WG11 4TH MEETING, 20 January 2011 (2011-01-20), DAEGU, KR *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014050732A1 (fr) * 2012-09-28 2014-04-03 ソニー株式会社 Dispositif et procédé de codage, et dispositif et procédé de décodage
WO2015083796A1 (fr) * 2013-12-04 2015-06-11 三菱電機株式会社 Dispositif de codage d'image
JP5951144B2 (ja) * 2013-12-04 2016-07-13 三菱電機株式会社 画像符号化装置
JP2015128239A (ja) * 2013-12-27 2015-07-09 キヤノン株式会社 画像符号化装置、撮像装置、画像符号化方法、及びプログラム
JP2015216626A (ja) * 2014-04-23 2015-12-03 ソニー株式会社 画像処理装置及び画像処理方法
US10477207B2 (en) 2014-04-23 2019-11-12 Sony Corporation Image processing apparatus and image processing method
WO2018168484A1 (fr) * 2017-03-15 2018-09-20 ソニー株式会社 Dispositif de codage, procédé de codage, dispositif de décodage et procédé de décodage

Similar Documents

Publication Publication Date Title
JP5907367B2 (ja) 画像処理装置および方法、プログラム、並びに記録媒体
JP6419113B2 (ja) 画像処理装置および方法、並びに、プログラム
US11601685B2 (en) Image processing device and method using adaptive offset filter in units of largest coding unit
CN104662901B (zh) 图像处理装置及方法
JPWO2014002895A1 (ja) 画像処理装置および方法
US20140233660A1 (en) Image processing device and method
US20140092979A1 (en) Image processing apparatus and method
JP2013012995A (ja) 画像処理装置および方法
WO2014050731A1 (fr) Dispositif et procédé de traitement d'image
WO2013108688A1 (fr) Dispositif de traitement d'image et procédé
WO2014156708A1 (fr) Dispositif et procede de decodage d'image
US20140226715A1 (en) Image processing device and method
WO2013154026A1 (fr) Appareil et procédé de traitement d'image
WO2013051453A1 (fr) Dispositif et procédé de traitement d'image
JP5999449B2 (ja) 画像処理装置および方法、プログラム、並びに記録媒体
JP6341304B2 (ja) 画像処理装置および方法、プログラム、並びに記録媒体
WO2013065567A1 (fr) Dispositif et procédé de traitement d'image
WO2013002106A1 (fr) Dispositif et procédé de traitement d'image
JP6094838B2 (ja) 画像処理装置および方法、プログラム、並びに記録媒体
JPWO2014002900A1 (ja) 画像処理装置および画像処理方法
WO2012157488A1 (fr) Dispositif et procédé de traitement d'image
AU2015255161B2 (en) Image Processing Device and Method
WO2013002111A1 (fr) Dispositif et procédé de traitement d'image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13774982

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13774982

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP