HK1237163B - Image processing device and method - Google Patents

Description

Image processing device and method

This application is a divisional application of the PCT application with national application number 201280004667.8, which entered the Chinese national phase on July 4, 2013, and the invention name is “Image Processing Device and Method”.

Technical Field

The present disclosure relates to an image processing apparatus and method, and more particularly to an image processing apparatus and method capable of enhancing encoding efficiency while suppressing a decrease in encoding processing efficiency.

Background Art

In recent years, devices compatible with compression formats such as MPEG (Motion Experts Picture Group), in which image information is digitally processed and compressed using orthogonal transforms (such as discrete cosine transforms) and motion compensation for the purpose of achieving efficient transmission and accumulation of information by utilizing the redundancy unique to image information, have become widely used both for information dissemination by broadcasting stations and for information reception in ordinary households.

Specifically, MPEG-2 (ISO (International Organization for Standardization)/IEC (International Electrotechnical Commission) 13818-2) is defined as a general-purpose image coding format. It is a standard that covers both interlaced and progressive scan images, as well as standard-definition and high-definition images, and is currently widely used in a wide range of applications for both professional and consumer use. The MPEG-2 compression format, for example, can achieve a high compression ratio and good image quality by allocating a coding amount (bit rate) of 4 to 8 Mbps to a standard-definition interlaced image having 720×480 pixels, or by allocating a coding amount (bit rate) of 18 to 22 Mbps to a high-definition interlaced image having 1920×1088 pixels.

MPEG-2 is primarily used for high-quality video encoding suitable for broadcasting, but is not compatible with encoding formats with a lower encoding capacity (i.e., higher compression ratio) than MPEG-1. With the widespread use of mobile terminals, demand for such encoding formats is expected to increase in the future, and in response, the MPEG-4 encoding format was standardized. Regarding image encoding formats, MPEG-4 was designated as an international standard as ISO/IEC 14496-2 in December 1998.

Furthermore, in recent years, standardization of the H.26L format (ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Q6/16 VCEG (Video Coding Experts Group)), originally intended for image coding for video conferencing, has been underway. It is known that, while encoding and decoding according to H.26L involve significant computational effort, H.26L achieves higher coding efficiency compared to previous coding formats such as MPEG-2 and MPEG-4. Furthermore, as part of the MPEG-4 initiative, standardization is currently underway to achieve higher coding efficiency based on H.26L by introducing functions not supported by H.26L as part of the Joint Model for Enhanced Compression Video Coding.

As a schedule for standardization, standards called H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as AVC) were designated as international standards in March 2003.

In addition, as an extension of the above, the standardization of FRExt (Fidelity Range Extension) was completed in February 2005. FRExt standardization includes encoding tools required for commercial use (such as RGB, 4:2:2, and 4:4:4) and the 8×8 DCT (Discrete Cosine Transform) and quantization matrix defined in MPEG-2. As a result, a coding format has been established that can well express even film noise included in movies by using AVC, and this coding format is used in a wide range of applications (such as Blu-ray Discs).

However, recently there has been an increasing demand for encoding at a higher compression rate, for example, compression of an image having approximately 4000×2000 pixels (which is four times the number of pixels included in a high-definition image), or distribution of high-definition images in an environment with limited transmission capacity (such as the Internet). Therefore, in VCEG (Video Coding Experts Group) under ITU-T, ongoing research has been carried out to enhance coding efficiency.

Meanwhile, with the goal of achieving higher coding efficiency than AVC, JCTVC (Joint Collaboration Team - Video Coding), a standardization group of ITU-T and ISO/IEC, is currently standardizing a coding format called HEVC (High Efficiency Video Coding) (for example, see NPL 1).

In the HEVC encoding format, a coding unit (CU) is defined as a processing unit similar to a macroblock used in AVC. Unlike macroblocks used in AVC, the size of a CU is not fixed to 16×16 pixels, but is specified in the image compression information of each sequence.

CUs are hierarchically arranged from the largest coding unit (LCU) to the smallest coding unit (SCU). That is, it can be generally considered that LCUs correspond to macroblocks used in AVC, and CUs in layers lower than LCUs correspond to sub-macroblocks used in AVC.

At the same time, there is a coding format in which a coding mode for encoding image data and outputting the image data and a non-coding mode for outputting the image data without encoding the image data are provided, and whether to use the coding mode or the non-coding mode is selected in units of macroblocks, and the coding mode and the non-coding mode can be used in combination in a single picture (for example, see PTL 1). In addition, in the AVC coding format, the I_PCM mode for outputting image data without encoding the image data is supported as mb_type (for example, see PTL 2). This is for ensuring real-time operation of arithmetic coding processing when the quantization parameter is set to a small value (such as QP=0) and when the amount of information of the coded data is greater than the amount of information of the input image. In addition, lossless coding can be achieved by using I-PCM.

In addition, a method for increasing internal operations has been proposed (for example, see NPL 2). As a result, internal operation errors caused in processes such as orthogonal transformation and motion compensation can be reduced, and encoding efficiency can be enhanced.

In addition, a technique for setting an FIR filter in a motion compensation loop has been proposed (for example, see NPL 3). In an encoding device, by using a Wiener filter to obtain FIR filter coefficients to minimize the error with respect to an input image, degradation of a reference image can be minimized, and encoding efficiency of image compression information to be output can be enhanced.

Reference List

Patent Literature

PTL 1: Japanese Patent No. 3992303

PTL 2: Japanese Patent No. 4240283

Non-patent literature

NPL 1: "Test Model under Consideration", JCTVC-B205, Joint CollaborativeTeam on Video Coding(JCT-VC) of ITU-T SG16WP3and ISO/IEC JTC1/SC29/WG112ndMeeting:Geneva,CH,21-28July,2010

NPL 2: Takeshi Chujoh, Reiko Noda, "Internal bit depth increase exceptframe memory", VCEG-AF07, ITU-Telecommunications Standardization Section STUDYGROUP 16Question 6Video Coding Experts Group (VCEG) 32nd Meeting: San Jose, USA, 20-21April, 2007

NPL 3: Takeshi Chujoh, Goki Yasuda, Naofumi Wada, Takashi Watanabe, TomooYamakage, "Block-based Adaptive Loop Filter", VCEG-AI18, ITU-TelecommunicationsStandardization Section STUDY GROUP 16Question 6Video Coding Experts Group(VCEG)35th Meeting: Berlin, Germany, 16-18July, 2008

Summary of the Invention

Technical issues

However, in the case of a coding format that defines CUs and performs various processing operations in units of CUs, as in HEVC, considering that macroblocks used in AVC correspond to LCUs, if I_PCM can only be set in units of LCUs, then because the processing unit is a maximum of 128×128 pixels, unnecessary coding processing is added and the efficiency of the coding process may be reduced. For example, it may be difficult to ensure real-time operation of CABAC.

In addition, the encoding formats proposed in NPL 2 and NPL 3 are not included in the AVC encoding format, and compatibility with the I_PCM mode is not disclosed therein.

The present disclosure has been made in view of these circumstances, and aims to enhance encoding efficiency while suppressing a decrease in encoding process efficiency.

Solution to the problem

According to aspects of the present disclosure, an image processing apparatus is provided. The apparatus includes: a decoder for decoding, in units of coding units obtained by recursively dividing a maximum coding unit, coded data including non-coded data of the coding unit set to a non-compression mode, to generate image data; and a controller for controlling the coding units including the non-coded data to skip filter processing performed on the image data generated by the decoder after deblocking filter processing.

According to aspects of the present disclosure, an image processing method for an image processing device is provided. The method includes: decoding, in units of coding units obtained by recursively dividing a maximum coding unit, coded data including non-coded data of the coding unit set to a non-compression mode, to generate image data, wherein the non-compression mode is a coding mode in which the image data is used as the coded data; and controlling the coding unit including the non-coded data to skip filter processing performed on the image data generated by the decoding after deblocking filter processing.

According to aspects of the present disclosure, an image processing device is provided. The image processing device includes: a coding mode setter that sets, in units of coding units having a hierarchical structure, whether to select a non-compression mode as a coding mode for encoding image data, the non-compression mode being a coding mode in which the image data is output as coded data; and an encoder that encodes the image data in units of coding units according to the mode set by the coding mode setter.

The image processing device may also include: a shift processing controller that controls the coding unit set to a non-compression mode by the coding mode setter to skip the shift processing, in which the bit accuracy for encoding or decoding is increased; and a shift processor that performs the shift processing on the coding unit of the image data, which is controlled by the shift processing controller to undergo the shift processing.

The image processing device may also include: a filter processing controller that controls the coding unit set to a non-compression mode by the coding mode setter to skip filter processing, in which filtering is performed on the local decoded image; a filter coefficient calculator that calculates filter coefficients for filter processing by using image data corresponding to the coding unit controlled by the filter processing controller to undergo filter processing; and a filter processor that performs filter processing in units of blocks by using the filter coefficients calculated by the filter coefficient calculator, the block being the unit of filter processing.

The filter processor may perform the filter process only on pixels controlled by the filter process controller to be subjected to the filter process, the pixels being included in the current block that is a target to be processed.

The image processing apparatus may further include a filter identification information generator that generates filter identification information in units of blocks, the filter identification information being identification information indicating whether filter processing is to be performed.

The filter processor may perform adaptive loop filtering on the locally encoded image, the adaptive loop filtering being an adaptive filter process using a classification process.

In a case where the amount of encoded data obtained by encoding image data corresponding to the current coding unit which is the target of the encoding process is less than or equal to the amount of input data which is the data amount of the image data corresponding to the current coding unit, the encoding mode setter may set the encoding mode of the current coding unit to a non-compression mode.

The image processing apparatus may further include an input data amount calculator that calculates an amount of input data. The encoding mode setter may compare the amount of input data calculated by the input data amount calculator with the encoding amount with respect to the current encoding unit.

The image processing apparatus may further include: an identification information generator that generates identification information in units of encoding units, the identification information indicating whether the encoding mode setter sets the non-compression mode.

According to aspects of the present disclosure, an image processing method for an image processing device is provided. The method includes: using a coding mode setter to set, in units of hierarchical coding units, whether to select a non-compression mode as a coding mode for encoding image data, wherein the non-compression mode is a coding mode in which the image data is output as coded data; and using an encoder to encode the image data in units of coding units according to the set mode.

According to another aspect of the present disclosure, an image processing apparatus is provided. The apparatus includes: a coding mode determiner that determines, in units of coding units having a hierarchical structure, whether a non-compression mode is selected as a coding mode for encoding image data, the non-compression mode being a coding mode in which the image data is output as coded data; and a decoder that decodes the coded result in units of coding units according to the mode determined by the coding mode determiner.

The image processing device may further include: a shift processing controller that performs control on a coding unit determined by the coding mode determiner to select a non-compression mode to skip a shift processing in which bit accuracy for encoding or decoding is increased; and a shift processor that performs a shift processing on a coding unit of image data, the coding unit being controlled by the shift processing controller so as to undergo a shift processing.

The image processing apparatus may further include: a filter processing controller that controls a coding unit determined by the coding mode determiner to select a non-compression mode to skip filter processing in which filtering is performed on the local decoded image; and a filter processor that performs filter processing on the image data in units of blocks, the blocks being the units of filter processing. The filter processor may perform filter processing only on pixels controlled by the filter processing controller to undergo filter processing, the pixels being included in a current block being a target for processing.

The filter processor may perform adaptive loop filtering on the local decoded image, the adaptive loop filtering being an adaptive filter process using a classification process.

In a case where the filter identification information indicating whether filter processing has been performed indicates that filter processing has been performed on image data corresponding to the current block as a target to be processed, the filter processor may perform filter processing only when the filter processing controller performs control so as to perform filter processing on all pixels included in the current block.

The encoding mode determiner may determine whether the non-compression mode is selected based on identification information indicating whether the non-compression mode is selected in units of coding units.

According to another aspect of the present disclosure, an image processing method for an image processing device is provided. The image processing method includes: using a coding mode determiner to determine, in units of hierarchical coding units, whether a non-compression mode is selected as a coding mode for encoding image data, wherein the non-compression mode is a coding mode in which the image data is output as coded data; and using a decoder to decode the coded data in units of coding units according to the determined mode.

According to aspects of the present disclosure, whether to select a non-compression mode as a coding mode for encoding image data is set in units of coding units having a hierarchical structure. The non-compression mode is a coding mode in which image data is output as encoded data, and the image data is encoded in units of coding units according to the set mode.

According to another aspect of the present disclosure, whether a non-compression mode is selected as a coding mode for encoding image data is determined in units of coding units having a hierarchical structure, the non-compression mode being a coding mode in which image data is output as encoded data, and the encoded data is decoded in units of coding units according to the determined mode.

Advantageous Effects of the Invention

According to the present disclosure, it is possible to process an image, and in particular, it is possible to enhance encoding efficiency while suppressing a decrease in encoding processing efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an image encoding apparatus that outputs image compression information based on the AVC encoding format.

FIG. 2 is a block diagram illustrating an image decoding apparatus that receives image compression information based on the AVC encoding format.

FIG. 3 is a diagram showing examples of types of macroblocks.

FIG. 4 is a diagram illustrating an example configuration of a description coding unit.

FIG. 5 is a diagram describing a method for increasing the amount of bits in an internal operation.

FIG6 is a diagram describing an adaptive loop filter.

FIG7 is a block diagram showing a main example configuration of an image encoding device.

FIG8 is a block diagram showing a main example configuration of the lossless encoder, loop filter, and PCM encoder in FIG7.

FIG. 9 is a block diagram showing a main example configuration of the PCM decision unit in FIG. 8 .

FIG10 is a flowchart describing an example of the flow of encoding processing.

FIG11 is a flowchart describing an example of the flow of PCM encoding control processing.

FIG12 is a flowchart describing an example of the flow of PCM encoding processing.

FIG13 is a flowchart describing an example of the flow of reference image generation processing.

FIG. 14 is a flowchart describing an example of the flow of a loop filter process.

FIG15 is a block diagram showing a main example configuration of an image decoding device.

FIG. 16 is a block diagram showing a main example configuration of the lossless decoder, loop filter, and PCM decoder in FIG. 15 .

FIG17 is a flowchart describing an example of the flow of a decoding process.

FIG18 is a flowchart describing an example of the flow of decoding processing, continuing from FIG17 .

FIG19 is a flowchart describing an example of the flow of a loop filter process.

FIG20 is a diagram describing an example of I_PCM information.

FIG21 is a block diagram showing a main example configuration of a personal computer.

FIG22 is a block diagram showing a main example configuration of a television receiver.

FIG23 is a block diagram showing a main example configuration of a mobile phone.

FIG. 24 is a block diagram showing a main example configuration of a hard disk recorder.

FIG. 25 is a block diagram showing a main example configuration of an image pickup apparatus.

DETAILED DESCRIPTION

Hereinafter, embodiments for implementing the present technology (hereinafter referred to as embodiments) will be described. Note that the description will be made in the following order.

1. First Embodiment (Image Coding Device)

2. Second embodiment (image decoding device)

3. Third embodiment (personal computer)

4. Fourth Embodiment (Television Receiver)

5. Fifth embodiment (mobile phone)

6. Sixth embodiment (hard disk recorder)

7. Seventh Embodiment (Image Capture Device)

<1. First embodiment>

[Image encoding device compatible with the AVC encoding format]

1 shows a configuration of an image encoding device according to an embodiment, which encodes an image using H.264 and MPEG (Motion Experts Group) 4 Part 10 (AVC (Advanced Video Coding)) encoding formats.

The image encoding device 100 shown in FIG1 is a device that encodes an image using a coding format based on the AVC standard and outputs the encoded image. As shown in FIG1, the image encoding device 100 includes an A/D converter 101, a screen rearrangement buffer 102, a calculation unit 103, an orthogonal transform unit 104, a quantizer 105, a lossless encoder 106, and an accumulation buffer 107. In addition, the image encoding device 100 includes a dequantizer 108, an inverse orthogonal transform unit 109, a calculation unit 110, a deblocking filter 111, a frame memory 112, a selector 113, an intra-frame prediction unit 114, a motion prediction/compensation unit 115, a selector 116, and a rate controller 117.

The A/D converter 101 performs A/D conversion on the image data input thereto and outputs the image data to the screen rearrangement buffer 102 to store the image data therein. The screen rearrangement buffer 102 rearranges the frame images stored therein, which are arranged in display order, according to the GOP (Group of Picture) structure so that the frame images are rearranged in encoding order. The screen rearrangement buffer 102 supplies the rearranged frame images to the calculation unit 103. In addition, the screen rearrangement buffer 102 supplies the rearranged frame images to the intra-frame prediction unit 114 and the motion prediction/compensation unit 115.

The calculation unit 103 subtracts the predicted image supplied from the intra prediction unit 114 or the motion prediction/compensation unit 115 via the selector 116 from the image read out from the screen rearrangement buffer 102 , and outputs difference information thereof to the orthogonal transformation unit 104 .

For example, in the case of an image on which intra encoding is to be performed, the calculation unit 103 subtracts the predicted image supplied from the intra prediction unit 114 from the image read out from the screen rearrangement buffer 102. Also, for example, in the case of an image on which inter encoding is to be performed, the calculation unit 103 subtracts the predicted image supplied from the motion prediction/compensation unit 115 from the image read out from the screen rearrangement buffer 102.

The orthogonal transform unit 104 performs orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform on the difference information supplied from the calculation unit 103 , and supplies the transform coefficient thereof to the quantizer 105 .

The quantizer 105 quantizes the transform coefficients output from the orthogonal transform unit 104. The quantizer 105 performs quantization by setting quantization parameters based on information on a target value of an encoding amount supplied from the rate controller 117. The quantizer 105 supplies the quantized transform coefficients to the lossless encoder 106.

The lossless encoder 106 performs lossless encoding (such as variable length encoding or arithmetic encoding) on the quantized transform coefficients. The coefficient data is quantized under the control performed by the rate controller 117, and its encoding amount is equal to (or approximate to) the target value set by the rate controller 117.

The lossless encoder 106 acquires information indicating intra prediction and the like from the intra prediction unit 114, and acquires information indicating the inter prediction mode, motion vector information, and the like from the motion prediction/compensation unit 115. Note that information indicating intra prediction (intra-frame prediction) will be referred to as intra prediction mode information hereinafter. In addition, information indicating an information mode indicating inter prediction (inter-frame prediction) will be referred to as inter prediction mode information.

The lossless encoder 106 encodes the quantized transform coefficients and also makes various information (such as filter coefficients, intra-frame prediction mode information, inter-frame prediction mode information, and quantization parameters) part of the header information of the encoded data (multiplexing the various information). The lossless encoder 106 supplies the encoded data obtained by encoding to the accumulation buffer 107 to store the encoded data therein.

For example, in the lossless encoder 106, lossless encoding processing (such as variable length encoding or arithmetic coding) is performed. Examples of variable length encoding include CAVLC (Context Adaptive Variable Length Coding) defined in the H.264/AVC format. Examples of arithmetic coding include CABAC (Context Adaptive Binary Arithmetic Coding).

The accumulation buffer 107 temporarily holds the encoded data supplied from the lossless encoder 106 and outputs the encoded data to a recording device or a transmission channel at a subsequent stage (not shown) as an encoded image encoded using the H.264/AVC format, for example, at a specific timing.

In addition, the transform coefficient quantized by the quantizer 105 is also supplied to the dequantizer 108. The dequantizer 108 dequantizes the quantized transform coefficient using a method corresponding to the quantization performed by the quantizer 105. The dequantizer 108 supplies the transform coefficient thus obtained to the inverse orthogonal transform unit 109.

The inverse orthogonal transform unit 109 performs inverse orthogonal transform on the transform coefficient supplied thereto using a method corresponding to the orthogonal transform process performed by the orthogonal transform unit 104. The output (restored difference information) obtained by the inverse orthogonal transform is supplied to the computing unit 110.

The calculation unit 110 adds the predicted image supplied from the intra-frame prediction unit 114 or the motion prediction/compensation unit 115 via the selector 116 to the result of the inverse orthogonal transform supplied from the inverse orthogonal transform unit 109 (i.e., the restored differential information), and obtains a local decoded image (decoded image).

For example, in the case where the difference information corresponds to an image on which intra encoding is to be performed, the calculation unit 110 adds the predicted image supplied from the intra prediction unit 114 to the difference information. Also, for example, in the case where the difference information corresponds to an image on which inter encoding is to be performed, the calculation unit 110 adds the predicted image supplied from the motion prediction/compensation unit 115 to the difference information.

The addition result is supplied to the deblocking filter 111 or the frame memory 112 .

The deblocking filter 111 performs deblocking filter processing as appropriate, thereby removing block distortion from the decoded image. The deblocking filter 111 supplies the result of the filter processing to the frame memory 112. Note that the decoded image output from the calculation unit 110 may be supplied to the frame memory 112 without passing through the deblocking filter 111. In other words, the deblocking filter processing performed by the deblocking filter 111 may be skipped.

The frame memory 112 stores the decoded image supplied thereto, and outputs the stored decoded image as a reference image to the intra prediction unit 114 or the motion prediction/compensation unit 115 via the selector 113 at a specific timing.

For example, in the case of an image on which intra-frame encoding is to be performed, the frame memory 112 supplies the reference image to the intra-frame prediction unit 114 via the selector 113. Also, for example, in the case of an image to be inter-frame encoding, the frame memory 112 supplies the reference image to the motion prediction/compensation unit 115 via the selector 113.

In the case where the reference image supplied from the frame memory 112 is an image on which intra-frame encoding is to be performed, the selector 113 supplies the reference image to the intra-frame prediction unit 114. On the other hand, in the case where the reference image supplied from the frame memory 112 is an image on which inter-frame encoding is to be performed, the selector 113 supplies the reference image to the motion prediction/compensation unit 115.

The intra prediction unit 114 performs intra prediction (intra-screen prediction) in which a predicted image is generated using pixel values of a target image to be processed supplied from the frame memory 112 via the selector 113. The intra prediction unit 114 performs intra prediction using a plurality of prepared modes (intra prediction modes).

The H.264 image information coding format defines an intra 4×4 prediction mode, an intra 8×8 prediction mode, and an intra 16×16 prediction mode for the luminance signal. Furthermore, for color difference signals, a prediction mode independent of the prediction mode for the luminance signal can be defined for each macroblock. With the intra 4×4 prediction mode, one intra prediction mode is defined for each 4×4 luminance block. With the intra 8×8 prediction mode, one intra prediction mode is defined for each 8×8 luminance block. With the intra 16×16 prediction mode and color difference signals, one prediction mode is defined for each macroblock.

The intra prediction unit 114 generates prediction images using all candidate intra prediction modes, evaluates the cost function values of the respective prediction images using the input image supplied from the screen rearrangement buffer 102, and selects the optimal mode. When the optimal intra prediction mode is selected, the intra prediction unit 114 supplies the prediction image generated using the optimal mode to the calculation unit 103 and the calculation unit 110 via the selector 116.

In addition, as described above, the intra prediction unit 114 appropriately supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the lossless encoder 106 .

The motion prediction/compensation unit 115 performs motion prediction (inter-frame prediction) on an image to be inter-coded using the input image supplied from the screen rearrangement buffer 102 and the reference image supplied from the frame memory 112 via the selector 113, performs motion compensation processing in accordance with the detected motion vector, and generates a predicted image (inter-frame predicted image information). The motion prediction/compensation unit 115 performs such inter-frame prediction using a plurality of prepared modes (inter-frame prediction modes).

The motion prediction/compensation unit 115 generates a predicted image using all candidate inter prediction modes, evaluates the cost function values of the respective predicted images, and selects the optimal mode. The motion prediction/compensation unit 115 supplies the generated predicted image to the calculation unit 103 and the calculation unit 110 via the selector 116.

In addition, the motion prediction/compensation unit 115 supplies inter prediction mode information indicating the employed inter prediction mode and motion vector information indicating the calculated motion vector to the lossless encoder 106 .

In the case of an image on which intra-frame encoding is to be performed, the selector 116 supplies the output of the intra-frame prediction unit 114 to the calculation unit 103 and the calculation unit 110. In the case of an image on which inter-frame encoding is to be performed, the selector 116 supplies the output of the motion prediction/compensation unit 115 to the calculation unit 103 and the calculation unit 110.

The rate controller 117 controls the rate of the quantization operation performed by the quantizer 105 based on the compressed image accumulated in the accumulation buffer 107 so that overflow or underflow does not occur.

[Image decoding device compatible with the AVC encoding format]

FIG2 is a block diagram showing a main example configuration of an image decoding device that uses orthogonal transform (such as discrete cosine transform or Karhunen-Loeve) and motion compensation to achieve image compression. The image decoding device 200 shown in FIG2 is a decoding device corresponding to the image encoding device 100 shown in FIG1.

The encoded data encoded by the image encoding device 100 is supplied to the image decoding device 200 corresponding to the image encoding device 100 via an arbitrary path (for example, a transmission channel, a recording medium, etc.), and is decoded.

2 , the image decoding apparatus 200 includes an accumulation buffer 201, a lossless decoder 202, a dequantizer 203, an inverse orthogonal transform unit 204, a calculation unit 205, a deblocking filter 206, a screen rearrangement buffer 207, and a D/A converter 208. In addition, the image decoding apparatus 200 includes a frame memory 209, a selector 210, an intra-prediction unit 211, a motion prediction/compensation unit 212, and a selector 213.

The accumulation buffer 201 accumulates the encoded data transmitted thereto. The encoded data has been encoded by the image encoding device 100. The lossless decoder 202 decodes the encoded data read out from the accumulation buffer 201 at a specific timing using a format corresponding to the encoding format used by the lossless encoder 106 shown in FIG. 1 .

In addition, when the target frame is intra-coded, intra-prediction mode information is stored in the header of the coded data. The lossless decoder 202 also decodes the intra-prediction mode information and supplies it to the frame prediction unit 211. Conversely, when the target frame is inter-coded, motion vector information is stored in the header of the coded data. The lossless decoder 202 also decodes the motion vector information and supplies it to the motion prediction/compensation unit 212.

The dequantizer 203 dequantizes the coefficient data (quantized coefficients) obtained by the decoding performed by the lossless decoder 202 using a method corresponding to the quantization method used by the quantizer 105 shown in FIG 1. That is, the dequantizer 203 dequantizes the quantized coefficients using a method similar to the method used by the dequantizer 108 shown in FIG 1.

The dequantizer 203 supplies the dequantized coefficient data (i.e., the orthogonal transform coefficient) to the inverse orthogonal transform unit 204. The inverse orthogonal transform unit 204 performs an inverse orthogonal transform on the orthogonal transform coefficient using a method corresponding to the orthogonal transform method used by the orthogonal transform unit 104 shown in FIG1 (a method similar to the method used by the inverse orthogonal transform unit 109 shown in FIG1 ), and obtains decoded residual data corresponding to the residual data before the orthogonal transform is performed by the image encoding device 100. For example, a fourth-order inverse orthogonal transform is performed.

The decoded residual data obtained by the inverse orthogonal transform is supplied to the calculation unit 205. In addition, the prediction information from the intra prediction unit 211 or the motion prediction/compensation unit 212 is supplied to the calculation unit 205 via the selector 213.

The calculation unit 205 adds the decoded residual data and the predicted image, thereby obtaining decoded image data corresponding to the image data from which the predicted image is not subtracted by the calculation unit 103 of the image encoding device 100 . The calculation unit 205 supplies the decoded image data to the deblocking filter 206 .

The deblocking filter 206 removes block distortion of the decoded image supplied thereto, and supplies the decoded image to the screen rearrangement buffer 207 .

The screen rearrangement buffer 207 rearranges the images. That is, the frames rearranged in the encoding order by the screen rearrangement buffer 102 shown in FIG1 are rearranged in the original display order. The D/A converter 208 performs D/A conversion on the images supplied from the screen rearrangement buffer 207 and outputs the images to a display (not shown) for display thereon.

The output of the deblocking filter 206 is also supplied to the frame memory 209 .

The frame memory 209, the selector 210, the intra-frame prediction unit 211, the motion prediction/compensation unit 212 and the selector 213 respectively correspond to the frame memory 112, the selector 113, the intra-frame prediction unit 114, the motion prediction/compensation unit 115 and the selector 116 of the image encoding device 100.

The selector 210 reads out an image on which inter processing is to be performed and a reference image from the frame memory 209, and supplies the images to the motion prediction/compensation unit 212. In addition, the selector 210 reads out an image for intra prediction from the frame memory 209, and supplies the image to the intra prediction unit 211.

For example, information indicating the intra prediction mode obtained by decoding the header information is appropriately supplied to the intra prediction unit 211 from the lossless decoder 202. The intra prediction unit 211 generates a predicted image from the reference image obtained from the frame memory 209 based on the information, and supplies the generated predicted image to the selector 213.

The motion prediction/compensation unit 212 obtains information obtained by decoding header information (prediction mode information, motion vector information, reference frame information, flags, various parameters, etc.) from the lossless decoder 202 .

The motion prediction/compensation unit 212 generates a predicted image from the reference image obtained from the frame memory 209 based on the information supplied from the lossless decoder 202 , and supplies the generated predicted image to the selector 213 .

The selector 213 selects a predicted image generated by the motion prediction/compensation unit 212 or the intra prediction unit 211 , and supplies the image to the calculation unit 205 .

[macroblock type]

Incidentally, as disclosed in PTL 1, there is an encoding format in which an encoding mode for encoding and outputting image data and a non-encoding mode for outputting the image data without encoding the image data are provided, the encoding mode or the non-encoding mode is selected on a macroblock basis, and the encoding mode and the non-encoding mode can be used in combination in a single picture. As disclosed in PTL 2, also in the AVC encoding format, as shown in FIG3 , the I_PCM (Intra-Block Pulse Code Modulation) mode (non-compression mode) for outputting image data without encoding the image data is supported as one of the macroblock types (mb_type).

This is used to ensure real-time operation of the arithmetic coding process when the quantization parameter is set to a small value (such as QP = 0) and when the amount of information of the encoded data is greater than that of the input image. In addition, lossless encoding can be achieved by using the I-PCM mode (non-compressed mode).

[Cost function]

Meanwhile, in order to obtain higher coding efficiency in the AVC coding format, it is important to select an appropriate prediction mode.

An example of a selection method is a method incorporated in the reference software of H.264/MPEG-4 AVC (so-called JM (Joint Model)) (published at http://iphome.hhi.de/suehring/tml/index.htm).

According to JM, the following two mode determination methods can be selected, namely, high complexity mode and low complexity mode. In either mode, cost function values are calculated for each prediction mode, and the prediction mode with the smallest cost function value is selected as the optimal mode for the target block or macroblock.

The cost function for the high complexity mode is expressed by the following equation (1).

Cost(Mode∈Ω)＝D+λ*R…(1)

Here, "Ω" represents the entire set of candidate modes for encoding the target block or macroblock, and "D" represents the differential energy between the decoded image and the input image when encoding is performed using the prediction mode. "λ" represents the Lagrange undetermined multiplier given as a function of the quantization parameter. "R" represents the total amount of code when encoding is performed using the mode, which includes orthogonal transform coefficients.

That is, in order to perform encoding using the high complexity mode, it is necessary to perform a preliminary encoding process using all candidate modes at once to calculate parameters D and R, which involves a large amount of calculation.

The cost function in the low complexity mode is expressed by the following equation (2).

Cost(Mode∈Ω)＝D+QP2Quant(QP)*HeaderBit…(2)

Here, unlike the High Complexity mode, "D" represents the differential energy between the predicted image and the input image. "QP2Quant (QP)" is given as a function of the quantization parameter QP, and "HeaderBit" represents the amount of code related to information belonging to the header (such as motion vectors and modes, excluding orthogonal transform coefficients).

That is, in Low Complexity Mode, prediction processing needs to be performed on each candidate mode, but decoding of the image is not required, and therefore encoding processing does not need to be performed. Therefore, Low Complexity Mode can be implemented with a lower amount of calculation than High Complexity Mode.

[Coding unit]

Next, the coding units defined in the HEVC encoding format described in NPL 1 will be described.

A coding unit (CU), also known as a coding tree block (CTB), is a partial area of an image in each picture that plays a similar role to a macroblock in AVC and is a coding unit with a hierarchical structure. The size of a macroblock is fixed to 16×16 pixels, but the size of a CU is not fixed and is specified in the image compression information of each sequence.

Specifically, the CU with the largest size is called an LCU (largest coding unit), and the CU with the smallest size is called an SCU (smallest coding unit). For example, the sizes of these regions are specified in a sequence parameter set (SPS) included in the image compression information. Each region is square in shape, and its size is limited to a size expressed by a power of 2.

FIG4 shows an example of a coding unit defined in HEVC. In the example shown in FIG4 , the size of the LCU is 128 and the maximum layer depth is 5. When the value of split_flag is “1”, a CU having a size of 2N×2N is split into CUs each having a size of N×N in the immediately lower layer.

In addition, the CU is divided into prediction units (PUs), each of which is a region used as a processing unit for intra-frame or inter-frame prediction (a partial region of the image of each picture); or is divided into transform units (TUs), each of which is a region used as a processing unit for orthogonal transform (a partial region of the image of each picture). Currently, in HEVC, 16×16 and 32×32 orthogonal transforms can be used in addition to 4×4 and 8×8 orthogonal transforms.

[IBDI]

Meanwhile, NPL 2 proposes a method for increasing internal operations (IBDI (Internal bitdepth increase except frame memory)) as shown in FIG5 . In this method, as shown in FIG5 , the bit depth of data is increased, for example, from 8 bits to 12 bits, in quantization processing, lossless encoding processing, dequantization processing, filter processing, prediction processing, lossless decoding processing, etc. performed by encoding and decoding devices. As a result, internal operation errors in processing (such as orthogonal transform or motion compensation) can be reduced, and encoding efficiency can be enhanced.

[BALF]

Meanwhile, NPL 3 proposes a method in which an FIR filter is provided in a motion compensation loop, and loop filter processing using a filter (BALF (Block-based Adaptive Loop Filter)) is adaptively performed, as shown in FIG5. In the encoding device, the FIR filter coefficients are obtained using a Wiener filter so as to minimize the error with respect to the input image, and thereby degradation of the reference image can be minimized, and encoding efficiency of image compression information to be output can be enhanced.

[Efficiency of encoding processing]

Meanwhile, in the case of a coding format that defines a CU and performs various processing operations in units of CUs, as in HEVC, it can be considered that a macroblock in AVC corresponds to an LCU. However, since the CU has a hierarchical structure as shown in FIG4 , the size of the LCU in the top layer is generally set to be larger than the macroblock in AVC, for example, to 128×128 pixels.

Therefore, in such an encoding format, as in the case of HEVC, if the I_PCM mode is set in units of LCUs, the unit of processing becomes larger than that in AVC, for example, is set to 128×128 pixels.

As described above, the intra prediction or inter prediction mode is determined by calculating and comparing cost function values. That is, prediction and encoding are performed using all modes, each cost function value is calculated, the optimal mode is selected, and encoded data is generated using the optimal mode.

However, when the I_PCM mode is adopted, the coded data generated using the optimal mode is discarded, and the input image (non-coded data) is adopted as the coding result. Therefore, when the I_PCM mode is selected, all processing operations used to generate the coded data of the optimal mode are unnecessary. That is, if the selection control unit of the I_PCM mode becomes larger, unnecessary processing operations are further increased. That is, as described above, if the selection of whether to adopt the I_PCM mode is performed for each LCU, the efficiency of the coding process is further reduced. Therefore, for example, it may be difficult to ensure the real-time operation of CABAC.

In addition, the above-mentioned techniques (such as IBDI and BALF) are not included in the AVC encoding format. In the case of adopting the I_PCM mode, it is known how to control these processing operations.

Therefore, this embodiment can control the selection of the I_PCM mode (non-compressed mode) in more detail, and can also enhance the encoding efficiency while suppressing the reduction of the encoding processing efficiency. In addition, this embodiment can appropriately control the execution of IBDI and BALF according to the selection of the I_PCM mode, and can further suppress the reduction of the encoding processing efficiency.

[Image Coding Device]

FIG7 is a block diagram showing a main example configuration of an image encoding device.

The image encoding device 300 shown in FIG7 is basically similar to the image encoding device 100 shown in FIG1 and encodes image data. As shown in FIG7, the image encoding device 300 includes an A/D converter 301, a screen rearrangement buffer 302, an adaptive left shift unit 303, a calculation unit 304, an orthogonal transform unit 305, a quantizer 306, a lossless encoder 307, and an accumulation buffer 308. In addition, the image encoding device 300 includes a dequantizer 309, an inverse orthogonal transform unit 310, a calculation unit 311, a loop filter 312, an adaptive right shift unit 313, a frame memory 314, an adaptive left shift unit 315, a selector 316, an intra-frame prediction unit 317, a motion prediction/compensation unit 318, a selector 319, and a rate controller 320.

The image encoding apparatus 300 further includes a PCM encoder 321 .

As with the A/D converter 101, the A/D converter 301 performs A/D conversion on the image data input thereto. The A/D converter 301 supplies the converted image data (digital data) to the screen rearrangement buffer 302 to store the image data therein. As with the screen rearrangement buffer 102, the screen rearrangement buffer 320 rearranges the frame images stored therein, which are arranged in display order, according to the GOP (Group of Picture) structure, so that the frame images are rearranged in encoding order. The screen rearrangement buffer 302 supplies the rearranged frame images to the adaptive left shift unit 303.

In addition, the screen rearrangement buffer 302 also supplies the rearranged frame images to the lossless encoder 307 and the PCM encoder 321 .

The adaptive left shift unit 303 is controlled by the PCM encoder 321 and shifts the image data read out from the screen rearrangement buffer 302 in the left direction while increasing its bit depth by a specific bit amount (e.g., 4 bits). For example, the adaptive left shift unit 303 increases the bit depth of the image data read out from the screen rearrangement buffer 302 from 8 bits to 12 bits. As a result of increasing the bit depth in this manner, the accuracy of the internal operations of each processing operation (such as orthogonal transform processing, quantization processing, lossless encoding processing, prediction processing, etc.) can be increased, and errors can be suppressed.

Note that the amount of left shift (amount of bits) is not specified and may be fixed or variable. In addition, the left shift process may be skipped in accordance with the control performed by the PCM encoder 321.

The adaptive left shift unit 303 supplies the image data subjected to the left shift process to the calculation unit 304 (in the case of skipping the process, the image data output from the screen rearrangement buffer 302 is supplied to the calculation unit 304). In addition, the adaptive left shift unit 303 also supplies the image data to the intra-frame prediction unit 317 and the motion prediction/compensation unit 318.

As in the case of the calculation unit 103, the calculation unit 304 subtracts the predicted image supplied from the intra prediction unit 317 or the motion prediction/compensation unit 318 via the selector 319 from the image supplied from the adaptive left shift unit 303. The calculation unit 304 outputs its difference information to the orthogonal transform unit 305.

For example, in the case of an image on which intra encoding is to be performed, the calculation unit 304 subtracts the predicted image supplied from the intra prediction unit 317 from the image supplied from the adaptive left shift unit 303. Also, for example, in the case of an image on which inter encoding is to be performed, the calculation unit 304 subtracts the predicted image supplied from the motion prediction/compensation unit 318 from the image supplied from the adaptive left shift unit 303.

As in the case of the orthogonal transform unit 104, the orthogonal transform unit 305 performs an orthogonal transform (such as a discrete cosine transform or a Karhunen-Loeve transform) on the difference information supplied from the calculation unit 304. The method used for the orthogonal transform is not specified. The orthogonal transform unit 305 supplies its transform coefficients to the quantizer 306.

As in the case of the quantizer 105, the quantizer 306 quantizes the transform coefficients supplied from the orthogonal transform unit 305. The quantizer 306 performs quantization by setting a quantization parameter based on information about the target value of the encoding amount supplied from the rate controller 320. The method used for quantization is not specified. The quantizer 306 supplies the quantized transform coefficients to the lossless encoder 307.

As in the case of the lossless encoder 106, the lossless encoder 307 performs lossless encoding (such as variable length encoding or arithmetic encoding) on the transform coefficient quantized by the quantizer 306. The coefficient data is quantized under the control performed by the rate controller 320, and thus its encoding amount is equal to (or approximate to) the target value set by the rate controller 320.

Note that in the case where the PCM encoder 321 selects the I_PCM mode, the lossless encoder 307 regards the input image (non-encoded data) supplied from the screen rearrangement buffer 302 as the encoding result (ie, actually skips encoding).

The lossless encoder 307 also obtains information indicating the intra prediction mode and the like from the intra prediction unit 317 and information indicating the inter prediction mode, motion vector information and the like from the motion prediction/compensation unit 318. The lossless encoder 307 also obtains filter coefficients used by the loop filter 312.

As in the case of the lossless encoder 106, the lossless encoder 307 encodes various information (such as filter coefficients, information indicating the intra-frame prediction mode or the inter-frame prediction mode, and quantization parameters), and makes the various information part of the header information of the encoded data (multiplexing the various information). The lossless encoder 307 supplies the encoded data obtained by encoding (including non-encoded data in the case of I_PCM mode) to the accumulation buffer 308 to store the encoded data therein.

For example, as in the case of the lossless encoder 106, in the lossless encoder 307, lossless encoding processing (such as variable length encoding or arithmetic coding) is performed. Examples of variable length coding include CAVLC (Context Adaptive Variable Length Coding) defined in the H.264/AVC format. Examples of arithmetic coding include CABAC (Context Adaptive Binary Arithmetic Coding). Of course, the lossless encoder 307 can perform encoding using methods other than these methods.

As in the case of the accumulation buffer 107, the accumulation buffer 308 temporarily stores the encoded data (including non-encoded data in the case of the I_PCM mode) supplied from the lossless encoder 307. For example, the accumulation buffer 308 outputs the encoded data stored therein to a recording device (recording medium) or a transmission channel at a subsequent stage (not shown) at a specific timing.

In addition, the transform coefficient quantized by the quantizer 306 is also supplied to the dequantizer 309. As in the case of the dequantizer 108, the dequantizer 309 dequantizes the quantized transform coefficient using a method corresponding to the quantization performed by the quantizer 306. The method used for dequantization is not limited as long as the method corresponds to the quantization process performed by the quantizer 306. The dequantizer 309 supplies the transform coefficient obtained thereby to the inverse orthogonal transform unit 310.

As in the case of the inverse orthogonal transform 109, the inverse orthogonal transform unit 310 performs an inverse orthogonal transform on the transform coefficient supplied from the dequantizer 309 using a method corresponding to the orthogonal transform process performed by the orthogonal transform unit 305. The method used for the inverse orthogonal transform is not limited as long as the method corresponds to the orthogonal transform process performed by the orthogonal transform unit 305. The output (restored difference information) obtained by the inverse orthogonal transform is supplied to the calculation unit 311.

As in the case of the computing unit 110, the computing unit 311 adds the predicted image supplied from the intra-frame prediction unit 317 or the motion prediction/compensation unit 318 via the selector 319 to the result of the inverse orthogonal transform supplied from the inverse orthogonal transform unit 310 (i.e., the restored differential information), and obtains a locally decoded image (decoded image).

For example, when the difference information corresponds to an image on which intra-coding is to be performed, the calculation unit 311 adds the predicted image supplied from the intra-prediction unit 317 to the difference information. Also, for example, when the difference information corresponds to an image on which inter-coding is to be performed, the calculation unit 311 adds the predicted image supplied from the motion prediction/compensation unit 318 to the difference information.

The result of the addition (decoded image) is supplied to the loop filter 302 or the adaptive right shift unit 313 .

The loop filter 312 includes a deblocking filter, an adaptive loop filter, and the like, and appropriately performs filter processing on the decoded image supplied from the calculation unit 311. For example, the loop filter 312 performs deblocking filter processing similar to that performed by the deblocking filter 111 on the decoded image, thereby removing block distortion from the decoded image. Furthermore, for example, the loop filter 312 is controlled by the PCM encoder 321 and performs loop filter processing on the result of the deblocking filter processing (the decoded image from which block distortion has been removed) using a Wiener filter, thereby improving image quality. Note that the adaptive loop filter processing can be skipped according to the control performed by the PCM encoder 321.

Alternatively, the loop filter 312 may perform arbitrary filter processing on the decoded image. In addition, if necessary, the loop filter 312 may supply the filter coefficients used for the filter processing to the lossless encoder 307 so that the filter coefficients are encoded.

The loop filter 312 supplies the result of the filter processing (the decoded image on which the filter processing has been performed) to the adaptive right shift unit 313. Note that, as described above, the decoded image output from the calculation unit 311 may be supplied to the adaptive right shift unit 313 without passing through the loop filter 312. That is, the filter processing by the loop filter 312 may be skipped.

The adaptive right shift unit 313 is controlled by the PCM encoder 321, and shifts the image data supplied from the calculation unit 311 or the loop filter 312 in the right direction, and reduces its bit depth by a specific bit amount (for example, 4 bits). That is, the adaptive right shift unit 313 shifts the image data to the right by the bit amount by which the image data was left-shifted by the adaptive left shift unit 303, so as to change the bit depth of the image data to the state before the image data was left-shifted (the state when the image data was read out from the screen rearrangement buffer 302).

For example, the adaptive right shift unit 313 reduces the bit depth of image data supplied from the calculation unit 311 or the loop filter 312 from 12 bits to 8 bits. As a result of reducing the bit depth in this manner, the amount of image data stored in the frame memory can be reduced.

Note that the amount of right shift (amount of bits) is not specified as long as it matches the amount of left shift in the adaptive left shift unit 303. That is, the amount may be fixed or variable. In addition, the right shift process may be skipped according to the control performed by the PCM encoder 321.

The adaptive right shift unit 313 supplies the image data on which the right shift process is performed to the frame memory 314 (in the case of skipping the process, the image data output from the calculation unit 311 or the loop filter 312 is supplied to the frame memory 314 ).

As in the case of the frame memory 112 , the frame memory 314 stores the decoded image supplied thereto, and outputs the stored decoded image as a reference image to the adaptive left shift unit 315 at a certain timing.

The adaptive left shift unit 315 is a processing unit similar to the adaptive left shift unit 303, which is controlled by the PCM encoder 321 to appropriately shift the image data (reference image) read out from the frame memory 314 in the left direction and increase its bit depth by a specific bit amount (for example, 4 bits).

For example, in the case where the mode is not the I_PCM mode, the data of the input image is shifted to the left by the adaptive left shift unit 303. Therefore, the adaptive left shift unit 315 shifts the data of the reference image read out from the frame memory 314 to the left in accordance with the control performed by the PCM encoder 321, and increases the bit depth by the same amount of bits as in the case of the adaptive left shift unit 303 (for example, changes the bit depth from 8 bits to 12 bits).

The adaptive left shift unit 315 then supplies the image data subjected to the left shift process to the selector 316. As a result of increasing the bit depth in this manner, the bit depth of the reference image can be made the same as that of the input image, and the reference image can be added to the input image. In addition, the accuracy of internal operations (such as prediction processing) can be increased, and errors can be suppressed.

On the other hand, for example, in the case of I_PCM mode, the adaptive left shift unit 303 does not shift the data of the input image to the left. Therefore, the adaptive left shift unit 315 supplies the reference image read out from the frame memory 314 to the selector 316 without increasing the bit depth in accordance with the control performed by the PCM encoder 321.

As in the case of the selector 113, in the case of intra prediction, the selector 316 supplies the reference image supplied from the adaptive left shift unit 315 to the intra prediction unit 317. Also, as in the case of the selector 113, in the case of inter prediction, the selector 316 supplies the reference image supplied from the adaptive left shift unit 315 to the motion prediction/compensation unit 318.

The intra-frame prediction unit 317 performs intra-frame prediction (intra-frame picture prediction) in which a predicted image is generated using the reference image supplied from the adaptive left shift unit 315 via the selector 316. The intra-frame prediction unit 317 performs intra-frame prediction using a plurality of prepared modes (intra-frame prediction modes). The intra-frame prediction unit 317 can also perform intra-frame prediction using any mode other than the mode defined in the AVC encoding format.

The intra prediction unit 317 generates a predicted image using all candidate intra prediction modes, evaluates the cost function value of each predicted image using the input image supplied from the adaptive left shift unit 303, and selects the optimal mode. After selecting the optimal intra prediction mode, the intra prediction unit 317 supplies the predicted image generated using the optimal mode to the calculation unit 304 and the calculation unit 311 via the selector 319.

In addition, as described above, the intra prediction unit 317 appropriately supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the lossless encoder 307 so as to cause the information to be encoded.

The motion prediction/compensation unit 318 performs motion prediction (inter-frame prediction) on the image to be inter-coded using the input image supplied from the adaptive left shift unit 303 and the reference image supplied from the adaptive left shift unit 315 via the selector 316, performs motion compensation processing according to the detected motion vector, and generates a predicted image (inter-frame predicted image information). The motion prediction/compensation unit 318 performs such inter-frame prediction using a plurality of prepared modes (inter-frame prediction modes). The motion prediction/compensation unit 318 can also perform inter-frame prediction using any mode other than the mode defined in the AVC encoding format.

The motion prediction/compensation unit 318 generates a predicted image using all candidate inter prediction modes, evaluates the cost function values of the respective predicted images, and selects the optimal mode. After selecting the optimal inter prediction mode, the motion prediction/compensation unit 318 supplies the predicted image generated using the optimal mode to the calculation unit 304 and the calculation unit 311 via the selector 319.

In addition, the motion prediction/compensation unit 318 supplies inter prediction mode information indicating the employed inter prediction mode and motion vector information indicating the calculated motion vector to the lossless encoder 307 so as to cause the information to be encoded.

As in the case of the selector 116, in the case of an image on which intra-frame encoding is to be performed, the selector 319 supplies the output of the intra-frame prediction unit 317 to the calculation unit 304 and the calculation unit 311. In the case of an image on which inter-frame encoding is to be performed, the selector 319 supplies the output of the motion prediction/compensation unit 318 to the calculation unit 304 and the calculation unit 311.

The rate controller 320 controls the rate of the quantization operation performed by the quantizer 306 based on the code amount of the encoded data accumulated in the accumulation buffer 308 so that overflow or underflow does not occur.

In addition, the rate controller 320 supplies the code amount of the encoded data accumulated in the accumulation buffer 308 (the generated code amount) to the PCM encoder 321 .

The PCM encoder 321 compares the code size supplied from the rate controller 320 with the data size of the input image supplied from the screen rearrangement buffer 302 and selects whether to adopt the I_PCM mode. At this time, the PCM encoder 321 performs the selection in units of CUs (units smaller than LCUs). In other words, the PCM encoder 321 controls whether to adopt the I_PCM mode in more detail.

According to the result of the selection, the PCM encoder 321 controls the operations of the lossless encoder 307 , the adaptive left shift unit 303 , the adaptive right shift unit 313 , the adaptive left shift unit 315 , and the loop filter 312 .

[Lossless encoder, PCM encoder, and loop filter]

FIG8 is a block diagram showing a main example configuration of the lossless encoder 307, the PCM encoder 321, and the loop filter 312 shown in FIG7.

As shown in FIG. 8 , the lossless encoder 307 includes a NAL (Network Abstraction Layer) encoder 331 and a CU encoder 332 .

The NAL encoder 331 encodes NALs such as SPS (Sequence Parameter Set), PPS (Picture Parameter Set), etc. based on user instructions, specifications, etc. input via a user interface (not shown). The NAL encoder 331 supplies the encoded NAL (NAL data) to the accumulation buffer 308 so that the NAL data is added to the CU data, which is the encoded VCL (Video Coding Layer) supplied to the accumulation buffer 308 from the CU encoder 332.

The CU encoder 332 is controlled by the PCM encoder 321 (based on the On/Off control signal supplied from the PCM encoder 321) and encodes the VCL. For example, when the PCM encoder 321 does not select the I_PCM mode (when the PCM encoder 321 supplies a control signal indicating "On"), the CU encoder 332 encodes the quantized orthogonal transform coefficients of each CU. The CU encoder 332 supplies the encoded data (CU data) of each CU to the accumulation buffer 308.

In addition, for example, when the PCM encoder 321 selects the I_PCM mode (when the PCM encoder 321 supplies a control signal representing "Off"), the CU encoder 332 supplies the input pixel value supplied from the screen rearrangement buffer 302 to the accumulation buffer 308 as the encoding result (CU data).

In addition, the CU encoder 332 also encodes a flag (I_PCM_flag) indicating whether the encoding mode is the I_PCM mode and supplied from the PCM encoder 321, and supplies the encoded flag as CU data to the accumulation buffer 308. Furthermore, the CU encoder 332 encodes information about filter processing (such as an adaptive filter flag and filter coefficients) supplied from the loop filter 312, and supplies the encoded information as CU data to the accumulation buffer 308.

The method for encoding (eg, CABAC, CAVLC, etc.) used by the CU encoder 332 is not specified. The NAL data and CU data supplied to the accumulation buffer 308 are combined together and accumulated therein.

Note that the PCM encoder 321 actually controls whether to select the I_PCM mode by using the code amount of the encoded data generated by encoding the quantized orthogonal transform coefficients using the CU encoder 332 .

Therefore, for example, when the I_PCM mode is not selected, the encoded data supplied to the accumulation buffer 308 is used as the encoding result of the quantized orthogonal transform coefficient of the target CU. Therefore, the CU encoder 332 only needs to encode additional information such as I_PCM_flag.

On the other hand, for example, when the I_PCM mode is selected, the CU encoder 332 supplies the input pixel values of the target CU supplied from the screen rearrangement buffer 302 as the encoding result (CU data) to the accumulation buffer 308. Therefore, in this case, the supplied encoded data of the target CU (encoded data generated by encoding the quantized orthogonal transform coefficients) is discarded. In other words, all processing operations related to the generation of the encoded data are redundant.

As shown in FIG. 8 , the PCM encoder 321 includes an I_PCM_flag generator 341 and a PCM decision unit 342 .

The I_PCM_flag generator 341 generates an I_PCM_flag according to the decision made by the PCM decision unit 342 and determines its value. The I_PCM_flag generator 341 supplies the generated I_PCM_flag to the CU encoder 332 of the lossless encoder 307. For example, if the PCM decision unit 342 selects the I_PCM mode, the I_PCM_flag generator 341 sets the value of the I_PCM_flag to a value indicating that the I_PCM mode is selected (e.g., "1") and supplies the I_PCM_flag to the CU encoder 332. Alternatively, if the PCM decision unit 342 does not select the I_PCM mode, the I_PCM_flag generator 341 sets the value of the I_PCM_flag to a value indicating that the I_PCM mode is not selected (e.g., "0") and supplies the I_PCM_flag to the CU encoder 332.

The PCM decision unit 342 determines whether the encoding mode is the I_PCM mode. The PCM decision unit 342 obtains the data amount of the input pixel value supplied from the screen rearrangement buffer 302, compares this data amount with the generated encoding amount supplied from the rate controller 320, and determines whether to select the I_PCM mode based on the comparison result. The PCM decision unit 342 supplies an on/off control signal representing the selection result to the CU encoder 332 and the I_PCM_flag generator 341, thereby controlling the operation according to the selection result.

For example, if the amount of input pixel value data is greater than the generated code amount, the PCM decision unit 342 does not select the I_PCM mode. In this case, the PCM decision unit 342 supplies a control signal representing "On" to the CU encoder 322, and causes the CU encoder 332 to encode the quantized orthogonal coefficients. In addition, the PCM decision unit 342 supplies a control signal representing "On" to the I_PCM_flag generator 341, and causes the I_PCM_flag generator 341 to generate an I_PCM_flag having a value (e.g., "0") indicating that the I_PCM mode is not selected.

On the other hand, for example, if the amount of input pixel value data is less than or equal to the generated code amount, the PCM decision unit 342 selects the I_PCM mode. In this case, the PCM decision unit 342 supplies a control signal representing "Off" to the CU encoder 332, and causes the CU encoder 332 to output the input pixel value as the encoding result (CU data). In addition, the PCM decision unit 342 supplies a control signal representing "Off" to the I_PCM_flag generator 341, and causes the I_PCM_flag generator 341 to generate an I_PCM_flag having a value (e.g., "1") indicating that the I_PCM mode is selected.

The PCM decision unit 342 can determine whether to select the I_PCM mode in units of CUs and LCUs of all sizes (in any layer) set in the sequence parameter set. Therefore, for example, a process of suppressing the generation of many bits with low QP by using the I_PCM mode can be performed in units of smaller CUs. As a result, the amount of coding (including the amount of non-coded data in the I_PCM mode) can be controlled in more detail, and redundant processing occurring in the I_PCM mode can be reduced.

Furthermore, the PCM decision unit 342 supplies an on/off control signal representing the selection result to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, thereby controlling the IBDI according to the selection result. Specifically, if the target CU mode is I_PCM mode, the PCM decision unit 342 controls the adaptive shifting device so that the bit precision is not increased or decreased.

For example, when the I_PCM mode is not selected, the PCM decision unit 342 supplies a control signal representing "On" to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, and performs left shift processing and right shift processing to increase the bit accuracy in the internal processing.

On the contrary, for example, when the I_PCM mode is selected, the PCM decision unit 342 supplies a control signal representing "Off" to the adaptive left shift unit 303, the adaptive right shift unit 313 and the adaptive left shift unit 315, and skips the left shift processing and the right shift processing so as not to increase the bit accuracy in the internal processing.

In I_PCM mode, the input image pixel values are transferred to the image compression information, and therefore no operation error occurs. Increasing the bit operation precision is therefore redundant processing. The PCM decision unit 342 can eliminate such redundant processing by performing the processing in the above manner.

Furthermore, the PCM decision unit 342 supplies an on/off control signal representing the selection result to the loop filter 312, thereby controlling the adaptive loop filtering (BALF) process according to the selection result. Specifically, if the target CU mode is I_PCM mode, the PCM decision unit 342 controls the loop filter 312 so that it does not perform the adaptive loop filtering process.

For example, if the I_PCM mode is not selected, the PCM decision unit 342 supplies a control signal representing "On" to the loop filter 312 to execute the adaptive loop filter process. Conversely, if the I_PCM mode is selected, the PCM decision unit 342 supplies a control signal representing "Off" to the loop filter 312 to skip the adaptive loop filter process.

In the I_PCM mode, the input image pixel values are transferred to the image compression information and thus are not degraded. The adaptive loop filter processing performed thereon is redundant. The PCM decision unit 342 can eliminate such redundant processing by performing the processing in the above manner.

As shown in FIG. 8 , the loop filter 312 includes a deblocking filter 351 , a pixel classification unit 352 , a filter coefficient calculator 353 , and a filtering unit 354 .

As in the case of the deblocking filter 111 , the deblocking filter 351 performs deblocking filter processing on the decoded image (pixel values before deblocking filter) supplied from the calculation unit 311 , thereby removing block distortion.

Even if the target CU is processed in I_PCM mode, it is not always possible to process the CUs adjacent to the target CU in I_PCM mode. Therefore, even if the target CU is processed in I_PCM mode, blocking artifacts may occur. Therefore, deblocking filtering is performed regardless of whether the target CU is in I_PCM mode.

The deblocking filter 351 supplies the result of the filter process (pixel value after the deblocking filter) to the pixel classification unit 352 .

The pixel classification unit 352 classifies each result of the filter processing (pixel value after the deblocking filter) into pixel values on which adaptive loop filter processing is to be performed or pixel values on which adaptive loop filter processing is not to be performed, according to the value of the On/Off control signal supplied from the PCM decision unit 342.

For example, when the PCM decision unit 342 supplies a control signal representing "On", the pixel classification unit 352 classifies the pixel values of the corresponding CU after the deblocking filter as pixel values on which the adaptive loop filter process is to be performed. Conversely, for example, when the PCM decision unit 342 supplies a control signal representing "Off", the pixel classification unit 352 classifies the pixel values of the corresponding CU after the deblocking filter as pixel values on which the adaptive loop filter process is not to be performed.

The pixel classification unit 352 supplies the pixel values of the respective pixels after classification (pixel values after the deblocking filter) to the filter coefficient calculator 353 .

The filter coefficient calculator 353 uses a Wiener filter to calculate the filter coefficients (FIR filter coefficients) of the adaptive loop filter for the pixel values to be processed by the adaptive loop filter among the pixel values after the deblocking filter supplied thereto, so as to minimize the error with respect to the input image. In other words, the filter coefficient calculator 353 calculates the filter coefficients by excluding the pixels to be processed using the I_PCM mode.

The filter coefficient calculator 353 supplies the pixel value after the deblocking filter and the calculated filter coefficient to the filtering unit 354 .

The filtering unit 354 performs adaptive loop filter processing on the pixel values classified as pixel values to be subjected to adaptive loop filter processing using the filter coefficients supplied thereto. The filtering unit 354 supplies the pixel values after the adaptive filter, the pixel values on which the filter processing is performed, and the pixel values classified as pixel values not to be subjected to the adaptive loop filter processing to the adaptive right shift unit 313.

In addition, the filter unit 354 generates an adaptive filter flag (on/off_flag) as filter flag information indicating whether filter processing is performed for each block in specific blocks set independently of the CU. The method for setting the value of the adaptive filter flag is not specified.

For example, in the case where adaptive loop filter processing is performed on some or all pixels in the target block to be processed (current block), the adaptive filter flag may be set to a value (e.g., "1") indicating that filter processing is performed. Alternatively, for example, in the case where adaptive loop filter processing is not performed on all pixels in the block, the adaptive filter flag may be set to a value (e.g., "0") indicating that filter processing is not performed. The value of the adaptive loop filter flag may be set based on another criterion.

The filtering unit 354 supplies the generated adaptive filter flag to the CU encoder 332 of the lossless encoder 307, so that the adaptive filter flag is encoded and provided to the decoding side. Note that when the value of the adaptive filter flag is a value indicating that filter processing is not performed (for example, "0"), the provision of the adaptive filter flag to the decoding side can be skipped (the adaptive filter flag is not provided to the decoding side).

For example, when the value of the adaptive filter flag is a value indicating that filter processing is not performed (for example, "0") and the encoding format used by the lossless encoder 307 (CU encoder 332) is VLC, the filtering unit 354 skips supplying the adaptive filter flag (does not supply the adaptive filter flag to the decoding side). In addition, for example, when the value of the adaptive filter flag is a value indicating that filter processing is not performed (for example, "0") and the encoding format used by the lossless encoder 307 (CU encoder 332) is CABAC, the filtering unit 354 supplies the adaptive filter flag to the CU encoder 332 of the lossless encoder 307 (supplies the adaptive filter flag to the decoding side).

This is because, in the case of VLC, if the amount of information input is small, higher encoding efficiency can be achieved, but in the case of CABAC, if the same information is continuously input, the probability when performing arithmetic coding deviates, and higher encoding efficiency can be achieved.

Furthermore, the filtering unit 354 supplies the filter coefficient used for the adaptive loop filter process to the CU encoder 332 of the lossless encoder 307 , so that the filter coefficient is encoded and supplied to the decoding side.

[PCM decision unit]

FIG. 9 is a block diagram showing a main example configuration of the PCM decision unit 342 shown in FIG. 8 .

As shown in FIG. 9 , the PCM decision unit 342 includes an input data amount calculator 361 , a PCM determination unit 362 , an encoding controller 363 , an adaptive shift controller 364 , and a filter controller 365 .

The input data amount calculator 361 calculates the input data amount, which is the data amount of the input pixel values supplied from the screen rearrangement buffer 302 , for the target CU, and supplies the calculated input data amount to the PCM determination unit 362 .

The PCM determination unit 362 obtains the generated code amount (generated bits) supplied from the code rate controller 320, compares the generated code amount with the input data amount supplied from the input data amount calculator 361, and determines whether to select the I_PCM mode for the CU based on the comparison result. That is, the PCM determination unit 362 determines whether to select the I_PCM mode for each CU in any layer. The PCM determination unit 362 supplies the determination result to the encoding controller 363, the adaptive shift controller 364, and the filter controller 365.

Based on the determination result (identification information indicating whether the I_PCM mode is selected) supplied from the PCM determination unit 362 , the encoding controller 363 supplies an On/Off control signal to the CU encoder 332 and the I_PCM_flag generator 341 .

Therefore, the encoding controller 361 can control the encoding mode in units of CUs in any layer. Therefore, the encoding controller 363 can control the encoding amount in more detail (including the amount of non-encoded data in I_PCM mode) and can also reduce redundant processing when I_PCM mode is selected.

Based on the determination result (information indicating whether the I_PCM mode is selected) supplied from the PCM determination unit 362 , the adaptive shift controller 364 supplies On/Off control signals to the adaptive left shift unit 303 , the adaptive right shift unit 313 , and the adaptive left shift unit 315 .

Therefore, the adaptive shift controller 364 can perform control so as not to increase the bit depth in internal operations when the I_PCM mode is selected. Therefore, the adaptive shift controller 364 can reduce redundant processing.

Based on the determination result (information indicating whether the I_PCM mode is selected) supplied from the PCM determination unit 362 , the filter controller 365 supplies an On/Off control signal to the pixel classification unit 352 .

Therefore, the filter controller 365 can perform control so that the adaptive loop filter process is not performed when the I_PCM mode is selected.Therefore, the filter controller 365 can reduce redundant processing.

As described above, the image encoding device 300 can reduce redundant processing and suppress a decrease in encoding process efficiency. In addition, the image encoding device 300 can select the I_PCM mode (non-compression mode) in more detail (in units of smaller data units) and enhance encoding efficiency. Therefore, the image encoding device 300 can enhance encoding efficiency while suppressing a decrease in encoding process efficiency.

[Encoding process flow]

Next, a description will be given of the flow of each processing operation performed by the above-described image encoding device 300. First, an example of the flow of encoding processing will be described with reference to the flowchart in FIG.

In step S301, the A/D converter 301 performs A/D conversion on an input image. In step S302, the screen rearrangement buffer 302 stores the A/D-converted image therein and rearranges pictures from display order to encoding order.

In step S303, the adaptive left shift unit 303 adaptively performs left shifting on the input image based on the control performed by the PCM encoder 321. In step S304, the adaptive left shift unit 315 adaptively performs left shifting on the reference image.

In step S305, the intra prediction unit 317 performs intra prediction processing in the intra prediction mode by using the reference image shifted to the left in step S304. In step S306, the motion prediction/compensation unit 318 performs inter motion prediction processing by using the reference image shifted to the left in step S304, wherein motion prediction or motion compensation is performed in the inter prediction mode.

Note that, actually in the intra prediction process or the inter motion prediction process, a process of shifting the bit depth of the reference image to the left may be performed when the reference image is read out from the frame memory 314 .

In step S307, the selector 319 determines the optimal mode based on the respective cost function values output from the intra prediction unit 317 and the motion prediction/compensation processing unit 318. That is, the selector 319 selects either the predicted image generated by the intra prediction unit 317 or the predicted image generated by the motion prediction/compensation unit 318.

In addition, selection information indicating which prediction image was selected is supplied to the one corresponding to the selected prediction image, between the intra prediction unit 317 and the motion prediction/compensation unit 318. When the prediction image in the optimal intra prediction mode is selected, the intra prediction unit 317 supplies intra prediction mode information indicating the optimal intra prediction mode, etc., to the lossless encoder 307. When the prediction image in the optimal inter prediction mode is selected, the motion prediction/compensation unit 318 supplies information indicating the optimal inter prediction mode and, if necessary, information based on the optimal inter prediction mode, to the lossless encoder 307. Examples of information based on the optimal inter prediction mode include motion vector information, flag information, and reference frame information.

In step S308, the calculation unit 304 calculates the difference between the image whose bit depth is shifted to the left by the process in step S303 and the predicted image selected by the process in step S307. When inter prediction is to be performed, the predicted image is supplied to the calculation unit 304 from the motion prediction/compensation unit 318 via the selector 319, and when intra prediction is to be performed, the predicted image is supplied to the calculation unit 304 from the intra prediction unit 317 via the selector 319.

The amount of difference data is smaller than that of the original image data, so the amount of data can be reduced compared to the case of encoding the image itself.

In step S309, the orthogonal transform unit 305 performs an orthogonal transform on the difference information generated by the process in step S308. Specifically, an orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform is performed, and a transform coefficient is output.

In step S310 , the quantizer 306 quantizes the orthogonal transform coefficient obtained by the process in step S309 .

In step S311, the lossless encoder 307 encodes the transform coefficient quantized by the process in step S310. That is, lossless encoding (such as variable length encoding or arithmetic encoding) is performed on the difference image.

The lossless encoder 307 encodes the quantization parameter calculated in step S310 and adds the quantization parameter to the encoded data. Furthermore, the lossless encoder 307 encodes information about the prediction image mode selected by the process in step S307 and adds this information to the encoded data obtained by encoding the difference image. Specifically, the lossless encoder 307 also encodes the optimal intra-frame prediction mode information supplied from the intra-frame prediction unit 317 or the information base based on the optimal inter-frame prediction mode supplied from the motion prediction/compensation unit 318 and adds this information to the encoded data.

Furthermore, the lossless encoder 307 encodes the filter coefficient and flag information obtained from the loop filter 312, and adds them to the encoded data. Furthermore, the lossless encoder 307 encodes NAL data.

In step S312, the accumulation buffer 308 accumulates the encoded data output from the lossless encoder 307. The encoded data accumulated in the accumulation buffer 308 is read out as appropriate and transmitted to the decoding side via a transmission channel or a recording medium.

In step S313, the rate controller 320 calculates the code amount (generated code amount) of the encoded data accumulated in the accumulation buffer 308 by the process in step S312, and controls the rate of quantization performed by the quantizer 306 based on the code amount so that overflow or underflow does not occur. In addition, the rate controller 320 supplies the generated code amount to the PCM encoder 321.

In step S314, the PCM encoder 321 performs PCM encoding control processing by using the generated code amount calculated in step S313. In step S315, the lossless encoder 307 performs PCM encoding processing in accordance with the control performed by the PCM encoder 321.

In step S316 , the dequantizer 309 to the frame memory 314 perform a reference image generation process in which the difference information quantized by the process in step S310 is locally decoded to generate a reference image.

After the process in step S316 is completed, the encoding process is completed. For example, the encoding process is repeatedly performed on each CU.

[PCM encoding control processing]

Next, an example of the flow of the PCM encoding control process executed in step S314 of FIG. 10 will be described with reference to the flowchart in FIG. 11 .

After the PCM encoding control process starts, in step S331 , the PCM determination unit 362 of the PCM decision unit 342 obtains the generated code amount of the encoded data of the quantized orthogonal transform coefficient of the target CU from the rate control unit 320 .

In step S332 , the input data amount calculator 361 calculates the input data amount of the input pixel values of the target CU.

In step S333 , the PCM determination unit 362 compares the code amount obtained in step S331 with the input data amount calculated in step S332 , and determines whether to perform encoding using the I_PCM mode.

In step S334 , the I_PCM_flag generator 341 generates I_PCM_flag based on the On/Off control signal representing the determination result generated in step S333 and supplied from the encoding controller 363 .

In step S335 , the encoding controller 363 supplies an On/Off control signal representing the determination result generated in step S333 to the CU encoder 332 , thereby controlling encoding of the CU data.

In step S336 , the adaptive shift controller 364 supplies an On/Off control signal representing the determination result generated in step S333 to the adaptive left shift unit 303 , the adaptive right shift unit 313 , and the adaptive left shift unit 315 , thereby controlling the adaptive shift process.

In step S337 , the encoding controller 363 supplies an On/Off control signal representing the determination result generated in step S333 to the pixel classification unit 352 of the loop filter 312 , thereby controlling the adaptive loop filter process.

After the processing in step S337 ends, the PCM decision unit 342 ends the PCM encoding control processing, the processing returns to step S314 in Figure 10, and the processing is performed from step S315.

[PCM encoding process flow]

Next, an example of the flow of the PCM encoding process performed in step S315 of FIG. 10 will be described with reference to the flowchart in FIG. 12 .

After the PCM encoding process begins, the CU encoder 332 determines in step S351 whether to perform encoding using the I_PCM mode. If control is exercised in the aforementioned PCM encoding control process to perform encoding using the I_PCM mode, the CU encoder 332 proceeds to step S352. In step S352, the CU encoder 332 selects the input pixel values of the target CU as the encoding result. The CU encoder 332 discards the CU data of the target CU in the accumulation buffer 308 and accumulates the input pixel values in the accumulation buffer 308.

After step S352 ends, the CU encoder 332 causes the process to proceed to step S353. On the other hand, if it is determined in step S351 that encoding is not performed using the I_PCM mode, the CU encoder 332 causes the process to proceed to step S353.

In step S353 , the CU encoder 332 encodes the I_PCM_flag generated in the above-described PCM encoding control process, and accumulates the I_PCM_flag in the accumulation buffer 308 .

After step S353 ends, the CU encoder 332 ends the PCM encoding process, the process returns to step S315 in FIG. 10 , and the process is performed from step S316 .

[Flow of reference image generation process]

Next, an example of the flow of the reference image generation process executed in step S316 of FIG. 10 will be described with reference to the flowchart in FIG. 13 .

After the reference image generation process starts, the adaptive left shift unit 315 determines whether the I_PCM mode is selected based on the control performed by the adaptive shift controller 364 in step S371. If the I_PCM mode is selected, the bit depth is not increased in the internal operation, and the process is performed again starting from the prediction process. That is, if it is determined that the I_PCM mode is selected, the adaptive left shift unit 315 causes the process to continue to step S372 without performing the left shift process on the reference image.

In step S372, the intra prediction unit 317 performs intra prediction processing using the reference image whose bit depth is not shifted to the left. In step S373, the motion prediction/compensation unit 318 performs inter motion prediction processing using the reference image whose bit depth is not shifted to the left.

In step S374, the selector 319 determines the optimal mode based on the respective cost function values output from the intra-frame prediction unit 317 and the motion prediction/compensation unit 318. That is, the selector 319 selects either the predicted image generated by the intra-frame prediction unit 317 or the predicted image generated by the motion prediction/compensation unit 318.

In addition, selection information indicating which prediction image was selected is supplied to the one corresponding to the selected prediction image, between the intra prediction unit 317 and the motion prediction/compensation unit 318. When the prediction image in the optimal intra prediction mode is selected, the intra prediction unit 317 supplies intra prediction mode information indicating the optimal intra prediction mode, etc., to the lossless encoder 307. When the prediction image in the optimal inter prediction mode is selected, the motion prediction/compensation unit 318 supplies information indicating the optimal inter prediction mode and, if necessary, information based on the optimal inter prediction mode, to the lossless encoder 307. Examples of information based on the optimal inter prediction mode include motion vector information, flag information, and reference frame information.

When I_PCM mode is selected, the bit depth in internal operations is not increased. That is, the left shifting process performed on the reference image by the adaptive left shifting unit 303 is skipped. In step S375, the calculation unit 304 calculates the difference between the input image whose bit depth has not been shifted to the left and the predicted image selected by the process in step S374. When inter-frame prediction is performed, the predicted image is supplied to the calculation unit 304 from the motion prediction/compensation unit 318 via the selector 319, and when intra-frame prediction is performed, the predicted image is supplied to the calculation unit 304 from the intra-frame prediction unit 317 via the selector 319.

In step S376, the orthogonal transform unit 305 performs an orthogonal transform on the difference information generated by the process in step S375. Specifically, an orthogonal transform (such as a discrete cosine transform or a Karhunen-Loeve transform) is performed and the transform coefficients are output. In step S377, the quantizer 306 quantizes the orthogonal transform coefficients obtained by the process in step S376.

After quantizing the orthogonal transform coefficients, the quantizer 306 causes the process to proceed to step S378 , where a reference image is generated using the orthogonal transform coefficients quantized in step S377 (data on which the left shift process is not performed).

On the other hand, if it is determined in step S371 that the I_PCM mode is not selected, the adaptive left shift unit 315 skips the processing in steps S372 to S377 and causes the processing to proceed to step S378. That is, if the I_PCM mode is not selected, the reference image is generated using the orthogonal transform coefficients quantized in step S310 in FIG. 10 (data subjected to the left shift processing).

In step S378, the dequantizer 309 inversely quantizes the quantized orthogonal transform coefficient (also referred to as the quantized coefficient) in accordance with the characteristics corresponding to the characteristics of the quantizer 306. In step S379, the inverse orthogonal transform unit 310 performs inverse orthogonal transform on the orthogonal transform coefficient obtained by the process in step S378 in accordance with the characteristics corresponding to the characteristics of the orthogonal transform unit 305.

In step S380, the calculation unit 311 adds the predicted image to the locally decoded difference information, thereby generating a locally decoded image (an image corresponding to the input of the calculation unit 304). For example, if the I_PCM mode is selected, the calculation unit 311 adds the predicted image that has not been left-shifted to the difference information that has not been left-shifted to generate a decoded image that has not been left-shifted. Alternatively, if the I_PCM mode is not selected, the calculation unit 311 adds the predicted image that has been left-shifted to the difference information that has been left-shifted to generate a decoded image that has been left-shifted.

In step S381, the loop filter 312 performs loop filtering processing on the local decoded image obtained by the processing in step S380 based on the control performed by the filter controller 365, and appropriately performs loop filter processing (including deblocking filter processing, adaptive loop filter processing, etc.).

In step S382, the adaptive right shift unit 313 determines whether the I_PCM mode is selected based on the control performed by the adaptive shift controller 364. If it is determined that the I_PCM mode is not selected, the adaptive right shift unit 313 causes the process to proceed to step S383.

If the I_PCM mode is not selected, the decoded image undergoes an increase in bit depth in the internal operation. Therefore, in step S383, the adaptive right shift unit 313 performs a right shift on the bit depth of the filter processing result (decoded image) obtained by the loop filter processing in step S381. After the processing in step S383 is completed, the adaptive right shift unit 313 causes the processing to continue to step S384.

If it is determined in step S382 that the I_PCM mode is selected, the adaptive right shift unit 313 causes the process to proceed to step S384 without performing the right shift process.

In step S384, the frame memory 314 stores the decoded image. After the process in step S384 ends, the frame memory 314 ends the reference image generation process, the process returns to step S316 in FIG10, and the encoding process ends.

[Flow of loop filter processing]

Next, an example of the flow of the loop filter process executed in step S381 in FIG. 13 will be described with reference to the flowchart in FIG. 14 .

After the loop filter process starts, in step S401 , the deblocking filter 351 of the loop filter 312 performs a deblocking filter process on the decoded image (pixel values before deblocking filter) supplied from the calculation unit 311 .

In step S402 , the pixel classification unit 352 classifies each pixel of the decoded image based on whether the mode is the I_PCM mode, under the control performed by the filter controller 365 of the PCM decision unit 342 .

In step S403, the filter coefficient calculator 353 calculates filter coefficients for pixels classified as being subjected to filter processing (target pixels to be processed). In step S404, the filtering unit 354 performs adaptive filter processing on the target pixels to be processed using the filter coefficients calculated in step S403.

In step S405 , the filtering unit 354 sets the adaptive filter flag for the target block to be processed, and supplies the adaptive filter flag and the filter coefficient to the CU encoder 332 , so that the adaptive filter flag and the filter coefficient are encoded.

After the processing in step S405 ends, the loop filter 321 ends the loop filter processing, the processing returns to step S381 in FIG. 13 , and the processing is performed from step S382 .

By performing the respective processing operations in the manner described above, the image encoding device 300 can enhance encoding efficiency while suppressing a decrease in encoding processing efficiency.

[Second embodiment]

[Image decoding device]

Fig. 15 is a block diagram showing a main example configuration of an image decoding device. An image decoding device 500 shown in Fig. 15 is a device basically similar to the image decoding device 200 shown in Fig. 2 , and decodes encoded data generated by encoding image data.

The image decoding device 500 shown in Fig. 15 is a decoding device corresponding to the image encoding device 300 shown in Fig. 7. The encoded data encoded by the image encoding device 300 is supplied to the image decoding device 500 via an arbitrary path (for example, a transmission channel, a recording medium, etc.) and decoded.

15 , the image decoding apparatus 500 includes an accumulation buffer 501, a lossless decoder 502, a dequantizer 503, an inverse orthogonal transform unit 504, a calculation unit 505, a loop filter 506, an adaptive right shift unit 507, a screen rearrangement buffer 508, and a D/A converter 509. In addition, the image decoding apparatus 500 includes a frame memory 510, an adaptive left shift unit 511, a selector 512, an intra-prediction unit 513, a motion prediction/compensation unit 514, and a selector 515.

The image decoding apparatus 500 further includes a PCM decoder 516 .

The accumulation buffer 501 accumulates the encoded data transmitted thereto as in the case of the accumulation buffer 201 . The encoded data is encoded by the image encoding device 300 .

The lossless decoder 502 reads out the encoded data from the accumulation buffer 501 at a specific timing, and decodes the encoded data using a format corresponding to the encoding format used by the lossless encoder 307 shown in FIG 15. At this time, the lossless decoder 502 supplies the I_PCM_flag included in the encoded data to the PCM decoder 516, and causes the PCM decoder 516 to determine whether the mode is the I_PCM mode (non-compression mode).

In the case where the mode is I_PCM mode, the CU data obtained from the accumulation buffer 501 is non-coded data. Therefore, the lossless decoder 502 supplies the CU data to the dequantizer 503 in accordance with the control performed by the PCM decoder 516.

When the mode is not I_PCM mode, the CU data obtained from the accumulation buffer 501 is encoded data. Therefore, the lossless decoder 502 decodes the CU data in accordance with the control performed by the PCM decoder 516 and supplies the decoding result to the dequantizer 503.

Note that, for example, when the target CU is intra-coded, the intra-prediction mode information is stored in the header portion of the coded data. The lossless decoder 502 also decodes the intra-prediction mode information and supplies this information to the intra-prediction unit 513. On the other hand, when the target CU is inter-coded, the motion vector information and the inter-prediction mode information are stored in the header portion of the coded data. The lossless decoder 502 also decodes the motion vector information and the inter-prediction mode information and supplies this information to the motion prediction/compensation unit 514.

As in the case of the dequantizer 203, when the mode is not the I_PCM mode, the dequantizer 503 dequantizes the coefficient data (quantized coefficients) supplied from the lossless decoder 502 using a method corresponding to the quantization method used by the quantizer 306 shown in FIG7. That is, the dequantizer 503 dequantizes the quantized coefficients using a method similar to the method used by the dequantizer 309 shown in FIG7. The dequantizer 503 supplies the dequantized coefficient data (that is, the orthogonal transform coefficients) to the inverse orthogonal transform unit 504.

In addition, in the case where the mode is the I_PCM mode, the dequantizer 503 supplies the CU data (unencoded image data) supplied from the lossless decoder 502 to the inverse orthogonal transform unit 504 .

As in the case of the inverse orthogonal transform unit 204, when the mode is not I_PCM mode, the inverse orthogonal transform unit 504 performs an inverse orthogonal transform on the orthogonal transform coefficients using a method corresponding to the orthogonal transform method used by the orthogonal transform unit 305 shown in FIG7 (a method similar to the method used by the inverse orthogonal transform unit 310 shown in FIG7 ). Using the inverse orthogonal transform process, the inverse orthogonal transform unit 504 obtains decoded residual data corresponding to the residual data before the orthogonal transform performed in the image encoding device 300. For example, a fourth-order inverse orthogonal transform is performed. The inverse orthogonal transform unit 504 supplies the decoded residual data obtained by the inverse orthogonal transform to the calculation unit 505.

In addition, when the mode is the I_PCM mode, the inverse orthogonal transform unit 504 supplies the CU data (unencoded image data) supplied from the dequantizer 503 to the calculation unit 505 .

In addition, the predicted image is supplied from the intra prediction unit 513 or the motion prediction/compensation unit 514 to the calculation unit 505 via the selector 515 .

As in the case of the calculation unit 205, when the mode is not the I_PCM mode, the calculation unit 505 adds the decoded residual data and the predicted image, and thereby obtains decoded image data corresponding to the image data before the predicted image is subtracted by the calculation unit 304 of the image encoding device 300. The calculation unit 505 supplies the decoded image data to the loop filter 506.

When the mode is I_PCM mode, the calculation unit 505 supplies the CU data (uncoded image data) supplied from the inverse orthogonal transform unit 504 to the loop filter 506. In this case, the CU data is not residual information and therefore does not need to be added to the predicted image.

The loop filter 506 appropriately performs loop filter processing (including deblocking filter processing, adaptive filter processing, and the like) on the decoded image supplied from the calculation unit 505 under control performed by the PCM decoder 516 .

More specifically, the loop filter 506 performs a deblocking filter process similar to the deblocking filter process performed by the deblocking filter 206 on the decoded image, thereby removing block distortion from the decoded image. In addition, the loop filter 506 performs a loop filter process on the result of the deblocking filter process (decoded image from which block distortion has been removed) using a Wiener filter in accordance with control performed by the PCM decoder 516, thereby improving image quality.

Alternatively, the loop filter 506 may perform arbitrary filter processing on the decoded image. Alternatively, the loop filter 506 may perform filter processing by using filter coefficients supplied from the image encoding device 300 shown in FIG. 7 .

The loop filter 506 supplies the result of the filter process (the decoded image on which the filter process has been performed) to the adaptive right shift unit 507 .

The adaptive right shift unit 507 is a processing unit similar to the adaptive right shift unit 313 ( FIG. 7 ), and is controlled by the PCM decoder 516. When the mode is not the I_PCM mode, the adaptive right shift unit 507 shifts the decoded image data supplied from the loop filter 506 in the right direction to reduce the bit depth by a specific bit amount (e.g., 4 bits), for example, from 12 bits to 8 bits. That is, the adaptive right shift unit 507 shifts the decoded image data to the right by the bit amount by which the image data was left-shifted in the image encoding device 300, so as to change the bit depth of the image data to a state in which the left shift is not performed (the state when the image data is read out from the screen rearrangement buffer 302 ( FIG. 7 )).

The adaptive right shift unit 507 supplies the image data on which the right shift process has been performed to the screen rearrangement buffer 508 and the frame memory 510 .

Note that the amount of right shift (amount of bits) is not specified, as long as it is the same as the amount of left shift in the image encoding device 300 (adaptive left shift unit 303). That is, the amount can be fixed or variable. For example, the amount of shift can be predetermined (shared in advance between the image encoding device 300 and the image decoding device 500), or the image decoding device 500 can be allowed to calculate the amount of shift in the image encoding device 300, or information indicating the amount of shift can be provided from the image encoding device 300.

Alternatively, the right shift processing may be skipped according to the control performed by the PCM decoder 516. For example, in the case of the I_PCM mode, the image data does not undergo the left shift processing in the image encoding device 300. Therefore, in this case, the adaptive right shift unit 507 is controlled by the PCM decoder 516, and supplies the decoded image data supplied from the loop filter 506 to the screen rearrangement buffer 508 and the frame memory 510 without shifting the decoded image data in the right direction.

The screen rearrangement buffer 508 rearranges the images as in the case of the screen rearrangement buffer 207. That is, the frames rearranged in the encoding order by the screen rearrangement buffer 302 shown in FIG7 are rearranged in the original display order. The screen rearrangement buffer 508 supplies the decoded image data of each frame to the D/A converter 509 in the original display order.

As in the case of the D/A converter 208 , the D/A converter 509 D/A-converts the frame image supplied from the screen rearrangement buffer 508 , and outputs the frame image to a display (not shown) so that the frame image is displayed.

The frame memory 510 stores the decoded image supplied from the adaptive right shift unit 507 , and supplies the stored decoded image as a reference image to the adaptive left shift unit 511 at a specific timing.

The adaptive left shift unit 511 is a processing unit similar to the adaptive left shift unit 315, which is controlled by the PCM decoder 516 to appropriately shift the image data (reference image) read out from the frame memory 510 in the left direction and increase its bit depth by a specific number of bits (for example, 4 bits).

For example, in the case where the mode is not the I_PCM mode, the decoded image data supplied from the inverse orthogonal transform unit 504 to the calculation unit 505 is image data (for example, the bit depth is 12 bits) that has been shifted to the left in the image encoding device 300. Therefore, the adaptive left shift unit 511 increases the bit depth of the reference image read out from the frame memory 510 by a specific number of bits (for example, increases the bit depth from 8 bits to 12 bits) in accordance with the control performed by the PCM decoder 516.

The adaptive left shift unit 511 then supplies the image data subjected to the left shift process to the selector 512. As a result of increasing the bit depth in this manner, the bit depth of the reference image can be made the same as the bit depth of the decoded image, so that the reference image can be added to the decoded image. In addition, the accuracy of internal operations (such as prediction processing) can be increased, and errors can be suppressed.

On the other hand, for example, in the case where the mode is the I_PCM mode, the decoded image data supplied from the inverse orthogonal transform unit 504 to the calculation unit 505 is image data (for example, the bit depth is 8 bits) on which the left shift process is not performed in the image encoding device 300. Therefore, the adaptive left shift unit 511 supplies the reference image read out from the frame memory 510 to the selector 512 without increasing the bit depth in accordance with the control performed by the PCM decoder 516.

In the case of intra prediction, the selector 512 supplies the reference image supplied from the adaptive left shift unit 511 to the intra prediction unit 513. On the other hand, in the case of inter prediction, the selector 512 supplies the reference image supplied from the adaptive left shift unit 511 to the motion prediction/compensation unit 154.

Information indicating the intra prediction mode obtained by decoding the header information, etc., is appropriately supplied from the lossless decoder 502 to the intra prediction unit 513. The intra prediction unit 513 performs intra prediction using the reference image obtained from the frame memory 510 and the intra prediction mode used by the intra prediction unit 317, and generates a predicted image. That is, similar to the intra prediction unit 317, the intra prediction unit 513 can also perform intra prediction using any mode other than the mode defined in the AVC encoding format.

The intra prediction unit 513 supplies the generated predicted image to the selector 515 .

Information obtained by decoding the header information (prediction mode information, motion vector information, reference frame information, flags, various parameters, and the like) is supplied from the lossless decoder 502 to the motion prediction/compensation unit 514 .

The motion prediction/compensation unit 514 performs inter-frame prediction using the reference image obtained from the frame memory 510, using the inter-frame prediction mode used by the motion prediction/compensation unit 318, thereby generating a predicted image. That is, similar to the motion prediction/compensation unit 318, the motion prediction/compensation unit 514 can also perform intra-frame prediction using an arbitrary mode other than the mode defined in the AVC encoding format.

The motion prediction/compensation unit 514 supplies the generated prediction image to the selector 515 .

As in the case of the selector 213 , the selector 515 selects the predicted image generated by the motion prediction/compensation unit 514 or the intra prediction unit 513 , and supplies the predicted image to the calculation unit 505 .

The PCM decoder 516 controls the lossless decoder 502 , the loop filter 506 , the adaptive right shift unit 507 , and the adaptive left shift unit 511 based on the I_PCM_flag supplied from the lossless decoder 502 .

[Lossless decoder, PCM decoder and loop filter]

FIG16 is a block diagram showing a main example configuration of the lossless decoder 502, the PCM decoder 516, and the loop filter 506 shown in FIG15.

16 , the lossless decoder 502 includes a NAL decoder 531 and a CU decoder 532 . The NAL decoder 531 decodes the encoded NAL data supplied from the accumulation buffer 501 , and supplies the decoded NAL data to the CU decoder 532 .

The CU decoder 532 decodes the encoded CU data supplied from the accumulation buffer 501 based on the NAL data supplied from the NAL decoder 531 .

In addition, when I_PCM_flag is obtained by decoding CU data, the CU decoder 532 supplies I_PCM_flag to the PCM decoder 516. The CU decoder 532 decodes the encoded CU data in accordance with the control performed by the PCM decoder 516 based on I_PCM_flag (based on the On/Off control signal supplied from the PCM encoder 321).

For example, when I_PCM_flag has a value indicating that the mode is not I_PCM mode and a control signal representing "On" is obtained from the PCM decoder 516, the CU decoder 532 decodes the encoded CU data supplied from the accumulation buffer 501 based on the NAL data supplied from the NAL decoder 531, and supplies the quantized orthogonal transform coefficients to the dequantizer 503.

On the other hand, when I_PCM_flag has a value indicating that the mode is I_PCM mode and a control signal representing "Off" is obtained from the PCM decoder 516, the CU decoder 532 supplies the CU data (unencoded image data) supplied from the accumulation buffer 501 to the dequantizer 503.

Note that the CU decoder 532 supplies information obtained by decoding the CU data, such as the filter coefficient and the adaptive filter flag, to the loop filter 506 .

As shown in FIG. 8 , the PCM decoder 516 includes an I_PCM_flag buffer 541 and a PCM controller 542 .

The I_PCM_flag buffer 541 stores I_PCM_flag supplied from the CU decoder 532 of the lossless decoder 502 , and supplies I_PCM_flag to the PCM controller 542 at a specific timing.

The PCM controller 542 determines whether the I_PCM mode is selected in encoding in the image encoding device 300 based on the value of I_PCM_flag obtained from the I_PCM_flag buffer 541. Based on the determination result, the PCM controller 542 supplies an On/Off control signal to the CU decoder 532 and controls its operation.

For example, when it is determined that the I_PCM mode is not selected, the PCM controller 542 supplies a control signal representing "On" to the CU decoder 532, so that the CU decoder 532 decodes the encoded CU data, and the CU decoder 532 supplies the quantized orthogonal transform coefficients obtained by decoding to the dequantizer 503.

On the other hand, for example, in the case where it is determined that the I_PCM mode is selected, the PCM controller 542 supplies a control signal representing "Off" to the CU decoder 532, and causes the CU decoder 532 to supply CU data as non-encoded data to the dequantizer 503 as an output pixel value.

With the above control, the CU decoder 532 can appropriately decode the encoded data supplied from the image encoding device 300. That is, the CU decoder 532 can decode the encoded data more appropriately and encode the encoded data even when the mode is controlled to be the I_PCM mode in units of CUs smaller than the LCU.

In addition, the PCM controller 542 supplies On/Off control signals to the adaptive right shift unit 507 and the adaptive left shift unit 511 based on the determination result about the I_PCM mode, and controls the operations thereof.

For example, when it is determined that the I_PCM mode is not selected, the PCM controller 542 supplies a control signal representing "On" to the adaptive right shift unit 507 and the adaptive left shift unit 511, and performs left shift processing and right shift processing to increase the bit accuracy in internal processing.

On the other hand, when it is determined that the I_PCM mode is selected, the PCM control 542 supplies a control signal representing "Off" to the adaptive right shift unit 507 and the adaptive left shift unit 511, and skips the left shift processing and the right shift processing so as not to increase the bit accuracy in the internal processing.

By adopting the above control, the PCM controller 542 can appropriately eliminate redundant processing even when controlling whether the mode is the I_PCM mode in units of CUs smaller than LCUs.

Furthermore, the PCM controller 542 supplies an on/off control signal to the loop filter 506 based on the result of the determination regarding the I_PCM mode, and controls its operation. For example, if it is determined that the I_PCM mode is not selected, the PCM controller 542 supplies a control signal representing "on" to the loop filter 506. On the other hand, if it is determined that the I_PCM mode is selected, the PCM controller 542 supplies a control signal representing "off" to the loop filter 506.

With the above control, the PCM controller 542 can more appropriately eliminate redundant processing, and can appropriately eliminate redundant processing even when controlling whether the mode is the I_PCM mode in units of CUs smaller than LCUs.

As shown in FIG. 8 , the loop filter 506 includes a deblocking filter 551 , a pixel sorting unit 552 , and a filtering unit 553 .

As in the case of the deblocking filter 206 , the deblocking filter 551 performs deblocking filter processing on the decoded image (pixel values before deblocking filter) supplied from the calculation unit 505 , thereby removing block distortion.

As with the image encoding device 300 , deblocking filter processing is performed regardless of whether the mode for the target CU is the I_PCM mode. The deblocking filter 551 supplies the result of the filter processing (pixel value after deblocking filtering) to the pixel sorting unit 552 .

In accordance with the value of the On/Off control signal supplied from the PCM controller 542, the pixel classification unit 552 classifies the individual results of the filter processing (pixel values after the deblocking filter) into pixel values on which adaptive loop filter processing is to be performed and pixel values on which adaptive loop filter processing is not to be performed.

For example, when the PCM controller 542 supplies a control signal representing "On", the pixel classification unit 552 classifies the pixel values of the CU after the deblocking filter as pixel values on which the adaptive loop filter process is to be performed. Conversely, for example, when the PCM controller 542 supplies a control signal representing "Off", the pixel classification unit 552 classifies the pixel values of the CU after the deblocking filter as pixel values on which the adaptive loop filter process is not to be performed.

The pixel classification unit 552 supplies the pixel value after classification of each pixel (pixel value after the deblocking filter) to the filtering unit 553 .

The filtering unit 553 performs the adaptive loop filter process on the pixel values that have been classified as pixel values on which the adaptive loop filter process is to be performed, based on the adaptive loop filter flag supplied from the CU decoder 532 .

Adaptive loop filter processing is performed in units of specific blocks set independently of the CU. When the value of the adaptive loop filter flag of the target block is a value indicating that the filtering process is not performed on the encoding side (for example, "0"), the filtering unit 553 skips the adaptive loop filter processing for the target block and supplies the pixel value after the deblocking filter supplied thereto to the adaptive right shift unit 507 as the pixel value after the adaptive filter.

Incidentally, the method for setting the adaptive loop filter flag depends on the specifications of the image encoding device 300, etc. Therefore, depending on the method for setting the adaptive loop filter flag by the image encoding device 300, even if the value of the adaptive loop filter flag of the target block is a value indicating that filter processing is performed on the encoding side (for example, "1"), there is a possibility that pixels in the I_PCM mode are included in the target block.

Therefore, in the case where there are pixel values in the target block that are classified as pixel values on which adaptive loop filter processing is not to be performed, the filtering unit 553 skips the adaptive loop filter processing for the target block, and even if the value of the adaptive loop filter flag of the target block is a value indicating that filter processing is performed on the encoding side (for example, "1"), the adaptive loop filter processing for the target block is skipped, and the pixel value after the deblocking filter supplied thereto is supplied to the adaptive right shift unit 507 as the pixel value after the deblocking filter.

That is, the filtering unit 553 performs adaptive loop filter processing on the target block only when the value of the adaptive loop filter flag of the target block is a value indicating that filter processing is performed on the encoding side (for example, "1") and all pixels in the target block are classified as pixels on which adaptive loop filter processing is to be performed.

Note that in the case of performing the adaptive loop filter process, the filtering unit 553 performs the adaptive loop filter process using the filter coefficients supplied from the CU decoder 32 (the filter coefficients used in the adaptive loop filter process on the encoding side).

The filtering unit 553 supplies the pixel value on which the adaptive loop filter process is performed to the adaptive right shift unit 507 as the pixel value after the adaptive filter.

With the above control, the PCM controller 542 can appropriately eliminate redundant processing even when controlling whether the mode is the I_PCM mode (non-compression mode) in units of CUs smaller than the LCU.

As described above, with the processing operations performed by the respective processing units, the image decoding device 500 is able to achieve enhanced encoding efficiency while suppressing a decrease in encoding processing efficiency.

[Decoding Process Flow]

Next, description will be made of the flow of respective processing operations performed by the above-described image decoding apparatus 500. First, an example of the flow of a decoding process will be described with reference to the flowcharts in FIG17 and FIG18.

After the decoding process begins, in step S501, the accumulation buffer 501 accumulates the image (encoded data) transmitted thereto. In step S502, the CU decoder 532 obtains the adaptive filter flag from the encoded data accumulated in step S501. In step S503, the CU decoder 532 obtains the I_PCM_flag from the encoded data accumulated in step S501. The I_PCM_flag buffer 541 stores the I_PCM_flag.

In step S504 , the PCM controller 542 determines whether the encoding mode of the encoded data (CU data) accumulated in step S501 is the I_PCM mode (ie, non-encoded data) based on the value of I_PCM_flag stored in the I_PCM_flag buffer 541 .

If it is determined that the encoding mode is not I_PCM mode, the PCM controller 542 causes the process to proceed to step S505. In step S505, the lossless decoder 502 performs lossless decoding processing, decodes the encoded data (CU data) accumulated in step S501, and obtains quantized orthogonal transform coefficients, filter coefficients, etc.

In step S506, the dequantizer 503 dequantizes the quantized orthogonal transform coefficients obtained by the process in step S505. In step S507, the inverse orthogonal transform unit 504 performs inverse orthogonal transform on the dequantized orthogonal transform coefficients by the process in step S506, and generates decoded image data.

In step S508 , the adaptive left shift unit 511 obtains a reference image corresponding to the target CU from the frame memory 510 , and performs a left shift process on the reference image in accordance with the control performed by the PCM decoder 516 .

In step S509 , the intra prediction unit 513 or the motion prediction/compensation unit 514 performs a prediction process using the reference image on which the left shift process has been performed in step S508 , thereby generating a predicted image.

In step S510 , the calculation unit 505 adds the predicted image generated by the intra prediction unit 513 or the motion prediction/compensation unit 514 in step S509 to the difference information obtained by the process in step S507 .

In step S511 , the loop filter 506 performs loop filter processing on the addition result obtained in step S510 .

In step S512 , the adaptive right shift unit 507 performs right shift processing on the result of the loop filter processing obtained in step S511 in accordance with the control performed by the PCM decoder 516 .

In step S513 , the frame memory 510 stores the decoded image data as a reference image.

In step S514, the screen rearrangement buffer 508 rearranges the frames of the decoded image data. That is, the frames of the decoded image data rearranged in encoding order by the screen rearrangement buffer 302 (FIG. 7) of the image encoding device 300 are rearranged in the original display order.

In step S515, the D/A converter 509 performs D/A conversion on the decoded image data whose frames are rearranged in step S514. The decoded image data is output to a display (not shown), and an image thereof is displayed.

In addition, if it is determined in step S504 that the encoding mode is the I_PCM mode, the PCM controller 542 causes the processing to proceed to step S521 in Figure 18. In step S521, the lossless decoder 502 performs lossless decoding processing and regards the encoded data (non-compressed data) accumulated in step S501 as the encoding result (output pixel value).

In step S522, the loop filter 506 performs loop filter processing on the output pixel value obtained in step S521. After the processing in step S522 ends, the loop filter 506 returns the processing to step S513 in FIG17 so that the subsequent steps are executed.

[Flow of loop filtering processing]

Next, an example of the flow of the loop filter process performed in step S511 in FIG. 17 and step S522 in FIG. 18 will be described with reference to the flowchart in FIG. 19 .

After the loop filter process starts, in step S541 , the deblocking filter 551 performs a deblocking filter process on the pixel values before deblocking filtering obtained in step S510 or step S521 .

In step S542, the filtering unit 553 obtains the filter coefficient. In step S543, the pixel classification unit 552 determines whether to perform adaptive loop filtering based on the value of the adaptive loop filter flag. If it is determined that the adaptive loop filter process is to be performed, the pixel classification unit 552 causes the process to proceed to step S544.

In step S544 , the pixel classification unit 552 classifies the pixel values after the deblocking filter according to whether the mode is the I_PCM mode in accordance with the control performed by the PCM controller 542 .

In step S545, the filtering unit 545 performs adaptive loop filter processing on the pixel value after the deblocking filter that is classified as being subjected to adaptive loop filter processing by using the filter coefficient obtained in step S542. After the processing in step S545 ends, the filtering unit 545 ends the loop filter processing, the processing returns to step S511 in Figure 17 or step S522 in Figure 18, and the processing continues to step S512 in Figure 17 or step S513 in Figure 17.

If it is determined in step S543 in Figure 19 that adaptive loop filter processing is not to be performed, the pixel classification unit 52 ends the loop filter processing, the processing returns to step S511 in Figure 17 or step S512 in Figure 18, and the processing continues to step S512 in Figure 17 or step S513 in Figure 17.

As a result of performing the respective processing operations in the manner described above, the image decoding device 500 is able to achieve enhanced encoding efficiency while suppressing a decrease in encoding processing efficiency.

[Location of I_PCM information]

As described above, the selection control of the I_PCM mode can be performed in units of CUs, so that encoding using the I_PCM mode can be performed only on some CUs in the LCU, as shown in part A of Figure 20. Part A of Figure 20 shows the structure of the CUs in one LCU. The LCU shown in part A of Figure 20 is composed of seven CUs (i.e., CU0 to CU6). Each mark indicates the processing order in the LCU. In the example shown in part A of Figure 20, the shaded CUs (CU1, CU3, and CU6) are encoded using the I_PCM mode.

In this case, as shown in part B of FIG. 20 , I_PCM information, which is information related to I_PCM inside the LCU, may be added to the top of the data of the LCU of the encoded stream.

For example, the I_PCM information includes flag information indicating whether the LCU includes a CU to be encoded using the I_PCM mode. In the case of the example shown in FIG20, CU1, CU3, and CU6 are CUs to be encoded using the I_PCM mode, and therefore the flag information is set to a value (e.g., "1") indicating that the LCU includes a CU to be encoded using the I_PCM mode.

In this manner, the image encoding apparatus 300 adds information related to I_PCM inside the LCU to the top of the LCU, and thereby the image decoding apparatus 500 can easily determine whether the LCU includes a CU to be encoded using the I_PCM mode before decoding the LCU.

In addition, for example, the I_PCM information includes information indicating the CUs included in the LCU to be encoded using the I_PCM mode. In the case of the example shown in FIG20 , CU1, CU3, and CU6 are CUs to be encoded using the I_PCM mode, and therefore the I_PCM information includes information indicating these CUs (CU1, CU3, and CU6).

In this way, the image encoding device 300 adds information related to the I_PCM inside the LCU to the top of the LCU, and thus the image decoding device 500 can easily determine which CU in the LCU is encoded using the I_PCM mode (non-compression mode) before decoding the LCU.

Note that in the description given above, information other than image data, such as I_PCM_flag, adaptive filter flag, filter coefficients, and I_PCM information, is provided from the image encoding device 300 to the image decoding device 500 as needed, but such information may be added to any position of the encoded data. For example, the information may be added to the top of the LCU or CU, or may be added to the slice header, or may be stored in a sequence parameter set (SPS), a picture parameter set (PPS), or the like. Alternatively, for example, the information may be stored in a parameter set (e.g., a header of a sequence or picture) such as SEI (Supplemental Enhancement Information).

Furthermore, information may be transmitted to the decoding side separately from the encoded data. In that case, it is necessary to clarify (allow the decoding side to determine) the correspondence between the information and the encoded data, but the method for this is not specified. For example, table information indicating the correspondence may be separately created, or link information indicating the other side may be embedded in the data of each side.

Alternatively, the image encoding device 300 and the image decoding device 500 may share information in advance. In that case, the transmission of the information can be omitted.

<3. Third embodiment>

[Personal Computer]

The sequence of the above-mentioned processing operations can be executed by hardware or can be executed by software. In this case, for example, the hardware or software can be constructed as a personal computer shown in FIG21.

21 , a CPU (Central Processor Unit) 601 of a personal computer 600 executes various processing operations in accordance with a program stored in a ROM (Read Only Memory) 602 or in accordance with a program loaded from a storage unit 613 into a RAM (Random Access Memory) 603. In addition, data required for the CPU 601 to execute various processing operations is appropriately stored in the RAM 603.

The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. In addition, an input/output interface 610 is connected to the bus 604.

Connected to the input/output interface 610 are: an input unit 611 including a keyboard, a mouse, etc.; an output unit 612 including a display (such as a CRT (cathode ray tube) or an LCD (liquid crystal display)) and a speaker; a storage unit 613 including a hard disk; and a communication unit 614 including a modem. The communication unit 614 performs communication processing via a network (including the Internet).

In addition, if necessary, a drive 615 is connected to the input/output interface 610, a removable medium 621 (such as a disk, an optical disk, a magneto-optical disk, or a semiconductor memory) is appropriately loaded into the input/output interface 610, and if necessary, a computer program read from the input/output interface 610 is installed in the storage unit 613.

In the case of causing software to execute the above-described series of processing operations, the program constituting the software is installed via a network or a recording medium.

For example, as shown in FIG21, the recording medium is composed of a removable medium 621, which is provided separately from the main body of the device to distribute the program to the user, and contains the program recorded thereon, and is composed of a magnetic disk (including a floppy disk), an optical disk (including a CD-ROM (Compact Disc Read Only Memory) and a DVD (Digital Versatile Disc)), a magneto-optical disk (including an MD (Mini Disc)), or a semiconductor memory. Alternatively, the recording medium is composed of a ROM 602 containing the program recorded thereon or a hard disk included in a storage unit 613, and the recording medium is provided to the user in advance in a state of being incorporated into the main body of the device.

The program executed by the computer may be a program that performs processing operations in time series in the order described in this specification, or may be a program that performs processing operations in parallel or at necessary timing (for example, when a processing operation is called).

In addition, in this specification, steps describing the program recorded on the recording medium may be processing operations performed in time series in the order described, or may be processing operations performed in parallel or independently.

In addition, in this specification, a "system" is an entire device composed of a plurality of devices.

In addition, the configuration described above as a single device (or processing unit) can be divided into multiple devices (or processing units). Conversely, the configuration described above as multiple devices (or processing units) can be combined into a single device (or processing unit). In addition, configurations other than the configurations described above can of course be added to the configurations of the various devices (processing units). In addition, as long as the operation and configuration of the overall system are substantially the same, a part of the configuration of a certain device (or processing unit) can be included in the configuration of another device (or processing unit). That is, the embodiments of the present technology are not limited to the embodiments described above, and various modifications can be made without departing from the gist of the present technology.

For example, each of the lossless encoder 307, the loop filter 312, and the PCM encoder 321 shown in FIG8 may be configured as an independent device. In addition, each of the NAL encoder 331, the CU encoder 332, the I_PCM_flag generator 341, the PCM decision unit 342, the deblocking filter 351, the pixel classification unit 352, the filter coefficient calculator 353, and the filtering unit 354 shown in FIG8 may be configured as an independent device.

Furthermore, each of the input data amount calculator 361 , the PCM determination unit 362 , the encoding controller 363 , the adaptive shift controller 364 , and the filter controller shown in FIG. 9 may be configured as an independent device.

Alternatively, these processing units may be arbitrarily combined together to constitute an independent device. Of course, these processing units may be combined with any processing unit shown in Figures 7 to 9, or may be combined with processing units not shown.

This is the same as in the image decoding apparatus 500. For example, each of the lossless decoder 502, the loop filter 506, and the PCM decoder 516 shown in FIG15 can be configured as an independent device. In addition, each of the NAL decoder 531, the CU decoder 532, the I_PCM_flag buffer 541, the PCM controller 542, the deblocking filter 551, the pixel classification unit 552, and the filtering unit 553 shown in FIG16 can be configured as an independent device.

In addition, these processing units can be arbitrarily combined together to form an independent device. Of course, these processing units can be combined with any processing unit shown in Figures 15 and 16, or can be combined with processing units not shown.

In addition, for example, the above-described image encoding device and image decoding device can be applied to any electronic device. Hereinafter, an example thereof will be described.

<4. Fourth embodiment>

[TV receiver]

FIG. 22 is a block diagram showing a main example configuration of a television receiver including the image decoding device 500 .

A television receiver 1000 shown in FIG. 22 includes a terrestrial broadcast tuner 1013 , a video decoder 1015 , a video signal processing circuit 1018 , an image generating circuit 1019 , a panel driving circuit 1020 , and a display panel 1021 .

The terrestrial broadcast tuner 1013 receives a broadcast wave signal of a terrestrial analog broadcast via an antenna, decodes the signal, obtains a video signal, and supplies the video signal to the video decoder 1015. The video decoder 1015 performs decoding processing on the video signal supplied from the terrestrial broadcast tuner 1013, and supplies the digital component signal obtained thereby to the video signal processing circuit 1018.

The video signal processing circuit 1018 performs certain processing (such as noise reduction) on the video data supplied from the video decoder 1015 , and supplies the obtained video data to the graphics generation circuit 1019 .

For example, the graphics generation circuit 1019 generates video data of a program to be displayed on the display panel 1021, or generates image data by executing processing based on an application supplied via a network, and supplies the generated video data or image data to the panel drive circuit 1020. In addition, the graphics generation circuit 1019 appropriately performs processing for generating video data (graphics) for displaying a screen used for selecting an item or the like by a user, and supplies video data obtained by superimposing the generated video data on the video data of the program to the panel drive circuit 1020.

The panel driving circuit 1020 drives the display panel 1021 based on the data supplied from the graphic generating circuit 1019 , and causes the display panel 1021 to display the video of a program or the above-described various screens.

The display panel 1021 is configured by an LCD (Liquid Crystal Display) or the like, and displays a video of a program or the like in accordance with control performed by the panel drive circuit 1020 .

In addition, the television receiver 1000 includes an audio A/D (Analog/Digital) converter circuit 1014 , an audio signal processing circuit 1022 , an echo cancellation/audio synthesis circuit 1023 , an audio amplifier circuit 1024 , and a speaker 1025 .

The terrestrial tuner 1013 demodulates the received broadcast wave signal and thereby obtains an audio signal and a video signal. The terrestrial tuner 1013 supplies the obtained audio signal to the audio A/D converter circuit 1014.

The audio A/D converter circuit performs A/D conversion processing on the audio signal supplied from the terrestrial tuner 1013 , and supplies the digital audio signal obtained thereby to the audio signal processing circuit 1022 .

The audio signal processing circuit 1022 performs certain processing (such as noise reduction) on the audio data supplied from the audio A/D converter circuit 1014 , and supplies the audio data obtained thereby to the echo cancellation/audio synthesis circuit 1023 .

The echo cancellation/audio synthesis circuit 1023 supplies the audio data supplied from the audio signal processing circuit 1022 to the audio amplifier circuit 1024 .

The audio amplifier circuit 1024 performs D/A conversion processing and amplification processing on the audio data supplied from the echo cancellation/audio synthesis circuit 1023 , adjusts the audio data so that it has a certain volume, and causes audio to be output from the speaker 1023 .

Furthermore, the television receiver 1000 includes a digital tuner 1016 and an MPEG decoder 1017 .

The digital tuner 1016 receives the broadcast wave signal of digital broadcasting (terrestrial digital broadcasting, BS (broadcast satellite)/CS (communication satellite) digital broadcasting) via an antenna, demodulates the signal, obtains MPEG-TS (Moving Picture Experts Group-Transport Stream), and supplies it to the MPEG decoder 1017.

The MPEG decoder 1017 descrambles the MPEG-TS supplied from the digital tuner 1016 and extracts a stream containing data of a program to be reproduced (to be viewed or listened to). The MPEG decoder 1017 decodes the audio packets constituting the extracted stream and supplies the audio data obtained thereby to the audio signal processing circuit 1022. It also decodes the video data constituting the stream and supplies the video data obtained thereby to the video signal processing circuit 1018. Furthermore, the MPEG decoder 1017 supplies EPG (Electronic Program Guide) data extracted from the MPEG-TS to the CPU 1032 via a path not shown.

The television receiver 1000 includes the above-described image decoding device 500 serving as the MPEG decoder 1017, which decodes the video packets in this manner. Note that the MPEG-TS transmitted from a broadcast station or the like is encoded by the image encoding device 300.

As in the case of the image decoding device 500, the MPEG decoder 1017 appropriately decodes the coded data whose I_PCM mode selection is controlled in units of CUs smaller than LCUs. Therefore, the MPEG decoder 1017 can reduce redundant processing for encoding and reduce redundant information included in the coded data. Therefore, the MPEG decoder 1017 can achieve enhanced encoding efficiency while suppressing a decrease in encoding process efficiency.

As in the case of the video data supplied from the video decoder 1015, the video data supplied from the MPEG decoder 1017 undergoes specific processing in the video signal processing circuit 1018, the video data generated in the graphics generation circuit 1019, etc. are appropriately superimposed thereon, and the video data is supplied to the display panel 1021 via the panel driving circuit 1020, and its image is displayed.

As with the audio data supplied from the audio A/D converter circuit 1014, the audio data supplied from the MPEG decoder 1017 undergoes specific processing in the audio signal processing circuit 1022, is supplied to the audio amplifier circuit 1024 via the echo cancellation/audio synthesis circuit 1023, and undergoes D/A conversion processing and amplifier processing. As a result, audio adjusted to a specific volume is output from the speaker 1025.

In addition, the television receiver 1000 includes a microphone 1026 and an A/D converter circuit 1027 .

The A/D converter circuit 1027 receives a signal of a user's voice captured by the microphone 1026 for sound conversion provided in the television receiver 1000 , performs A/D conversion on the received audio signal, and supplies the obtained digital audio data to the echo cancellation/audio synthesis circuit 1023 .

When the audio data of the user (user A) of the television receiver 1000 is supplied from the A/D converter circuit 1027, the echo cancellation/audio synthesis circuit 1023 performs echo cancellation on the audio data of user A, and causes the audio data obtained by synthesizing with other audio to be output from the speaker 1025 via the audio amplifier circuit 1024.

Furthermore, the television receiver 1000 includes an audio decoding encoder 1028 , an internal bus 1029 , an SDRAM (Synchronous Dynamic Random Access Memory) 1030 , a flash memory 1031 , a CPU 1032 , a USB (Universal Serial Bus) I/F 1033 , and a network I/F 1034 .

The A/D converter circuit 1027 receives a signal of a user's voice captured by the microphone 1026 for sound conversion provided in the television receiver 1000 , performs A/D conversion processing on the received audio signal, and supplies the obtained digital audio data to the audio codec 1028 .

The audio codec 1028 converts the audio signal supplied from the A/D converter circuit 1027 into data in a specific format for transmitting it via a network, and supplies the audio data to the network I/F 1034 via the internal bus 1029 .

The network I/F 1034 is connected to the network via a cable attached to the network terminal 1035. For example, the network I/F 1034 transmits audio data supplied from the audio codec 1028 to another device connected to the network. In addition, for example, the network I/F 1034 receives audio data transmitted from another device connected via the network via the network terminal 1035, and supplies the audio data to the audio codec 1028 via the internal bus 1029.

The audio codec 1028 converts the audio data supplied from the network I/F 1034 into data in a specific format, and supplies the data to the echo cancellation/audio synthesis circuit 1023 .

The echo cancellation/audio synthesis circuit 1023 performs echo cancellation on the audio data supplied from the audio codec 1028 , and causes audio data obtained by synthesis with other audio data to be output from the speaker 1025 via the audio amplifier circuit 1024 .

The SDRAM 1030 stores various data required for the CPU 1032 to execute processing.

The flash memory 1031 stores programs executed by the CPU 1032. The programs stored in the flash memory 1031 are read out by the CPU 1032 at a specific timing (for example, when the television receiver 1000 is activated). The flash memory 1031 also stores EPG data obtained via digital broadcasting and data obtained from a specific server via a network.

For example, the flash memory 1031 stores MPEG-TS including content data obtained from a specific server via a network under the control performed by the CPU 1032. For example, under the control performed by the CPU 1032, the flash memory 1031 supplies the MPEG-TS to the MPEG decoder 1017 via the internal bus 1029.

The MPEG decoder 1017 processes the MPEG-TS as in the case of being supplied from the digital tuner 1016. In this way, the television receiver 1000 can receive content data such as video and audio via a network, decode the data using the MPEG decoder 1017, and display the video or output the audio.

In addition, the television receiver 1000 includes a light receiver 1037 that receives an infrared signal transmitted from the remote controller 1051 .

The light receiver 1037 receives infrared rays from the remote controller 1051 and outputs a control code representing details of a user operation obtained by demodulation to the CPU 1032 .

The CPU 1032 executes the program stored in the flash memory 1031, and controls the overall operation of the television receiver 1000 in accordance with a control code or the like supplied from the light receiver 1037. The CPU 1032 is connected to the respective units of the television receiver 1000 via an unillustrated path.

The USB I/F 1033 transmits/receives data to/from an external device of the television receiver 1000, the device being connected via a USB cable attached to the USB terminal 1036. The network I/F 1034 is connected to a network via a cable attached to the network terminal 1035, and transmits/receives data other than audio data to/from various devices connected to the network.

The television receiver 1000 includes an image decoding device 500 serving as an MPEG decoder 1017, thereby enabling enhanced encoding efficiency of content data while suppressing reduction in encoding processing efficiency when generating content data, the content data being obtained via a broadcast wave signal received via an antenna or a network.

<5. Fifth embodiment>

[Mobile Phone]

FIG. 23 is a block diagram showing a main example configuration of a mobile phone including the image encoding device 300 and the image decoding device 500 .

The mobile phone 1100 shown in FIG23 includes a main controller 1150 configured to generally control the respective units, a power supply circuit unit 1151, an operation input controller 1152, an image encoder 1153, a camera I/F unit 1154, an LCD controller 1155, an image decoder 1156, a multiplexer/demultiplexer unit 1157, a recording/reproducing unit 1162, a modulation/demodulation circuit unit 1158, and an audio codec 1159. These units are connected to one another via a bus 1160.

In addition, the mobile phone 1100 includes operation keys 1119 , a CCD (Charge Coupled Device) camera 1116 , a liquid crystal display 1118 , a storage unit 1123 , a transmission/reception circuit unit 1163 , an antenna 1114 , a microphone 1121 , and a speaker 1117 .

When a call is ended or a power key is turned on by a user operation, the power supply circuit unit 1151 supplies power from the battery pack to each unit, thereby putting the mobile phone 1100 into an operable state.

Based on the control performed by the main controller 1150 including a CPU, ROM, RAM, etc., the mobile phone 1100 performs various operations (such as transmission/reception of audio signals, transmission/reception of email or image data, image capture, or data recording) in various modes (such as audio call mode or data communication mode).

For example, in audio call mode, the mobile phone 1100 converts the audio signal collected by the microphone 1121 into digital audio data using the audio codec 1159, performs spectrum spread processing thereon using the modulation/demodulation circuit unit 1158, and performs digital-to-analog conversion processing and frequency conversion processing using the transmission/reception circuit unit 1163. The mobile phone 1100 transmits the signal to be transmitted obtained through the conversion process to a base station (not shown) via the antenna 1114. The signal to be transmitted (audio signal) transmitted to the base station is supplied to the mobile phone at the other end of the call via the public telephone line network.

In addition, for example, in the audio call mode, the mobile phone 1100 amplifies the reception signal received by the antenna 1114 using the transmission/reception circuit unit 1163, further performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum inverse spread processing using the modulation/demodulation circuit unit 1158, and converts the signal into an analog audio signal using the audio codec 1159. The mobile phone 1100 outputs the analog audio signal obtained by the conversion from the speaker 1117.

Furthermore, for example, when transmitting an e-mail in the data communication mode, the mobile phone 1100 receives text data of the e-mail input by operating the operation key 1119 in the operation input controller 1152. The mobile phone 1100 processes the text data using the main controller 1150 and causes the text data to be displayed as an image on the liquid crystal display 1118 via the LCD controller 1155.

In addition, the mobile phone 1100 generates email data in the main controller 1150 based on the text data or user instructions received by the operation input controller 1152. The mobile phone 1100 performs spectrum spread processing on the email data using the modulation/demodulation circuit unit 1158, and performs digital-to-analog conversion processing and frequency conversion processing using the transmission/reception circuit unit 1163. The mobile phone 1100 transmits the signal to be transmitted obtained through the conversion processing to a base station (not shown) via the antenna 1114. The signal to be transmitted (email) transmitted to the base station is supplied to a specific destination via a network or a mail server, etc.

Furthermore, for example, when receiving an email in digital communication mode, mobile phone 1100 receives a signal transmitted from a base station via antenna 1114 using transmission/reception circuit unit 1163, amplifies the signal, and further performs frequency conversion processing and analog-to-digital conversion processing. Mobile phone 1100 then performs spectrum despreading processing on the received signal using modulation/demodulation circuit unit 1158 to restore the original email data. Mobile phone 1100 displays the restored email data on liquid crystal display 1118 via LCD controller 1155.

In addition, the mobile phone 1100 can also cause received email data to be recorded (stored) in the storage unit 1123 via the recording/reproducing unit 1162 .

The storage unit 1123 is any rewritable storage medium. For example, the storage unit 1123 can be a semiconductor memory (such as RAM or built-in flash memory), a hard disk, or a removable medium (such as a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card). Of course, other types of media can be used.

Furthermore, for example, when transmitting image data in data communication mode, the mobile phone 1100 generates image data by capturing an image using the CCD camera 1116. The CCD camera 1116 includes an optical device (such as a lens and a diaphragm) and a CCD serving as a photoelectric conversion element, captures an image of a subject, converts the intensity of received light into an electrical signal, and generates image data of the subject's image. The CCD camera 1116 encodes the image data using the image encoder 1153 via the camera I/F unit 1154, thereby converting the image data into encoded image data.

Mobile phone 1100 includes the aforementioned image encoding device 300, which functions as an image encoder 1153 that performs the aforementioned processing. As with image encoding device 300, image encoder 1153 controls the selection of the I_PCM mode in units of CUs (units of a CU) that are smaller than LCUs. In other words, image encoder 1153 can further reduce redundant processing for encoding and further reduce redundant information included in the encoded data. Consequently, image encoder 1153 can enhance encoding efficiency while suppressing a decrease in encoding process efficiency.

In addition, at the same time, during image capture by the CCD camera 1116 , the mobile phone 1100 performs analog-to-digital conversion on the audio collected by the microphone 1121 in the audio codec 1159 , and also encodes it.

The mobile phone 1100 multiplexes the encoded image data supplied from the image encoder 1153 and the digital audio data supplied from the audio codec 1159 using a specific method in the multiplexer/demultiplexer unit 1157. The mobile phone 1100 performs spectrum spread processing on the resulting multiplexed data using the modulation/demodulation circuit unit 1158, and performs digital-to-analog conversion processing and frequency conversion processing using the transmission/reception circuit unit 1163. The mobile phone 1100 transmits the signal to be transmitted obtained through the conversion processing to a base station (not shown) via the antenna 1114. The signal to be transmitted (image data) transmitted to the base station is supplied to the other end of the communication via a network or the like.

Note that, without transmitting image data, the mobile phone 1100 can cause image data generated by the CCD camera 1116 to be displayed on the liquid crystal display 1118 via the LCD controller 1155 rather than via the image encoder 1153 .

In addition, for example, when receiving data of a moving image file linked to a simple web page or the like in data communication mode, mobile phone 1100 uses transmission/reception circuit unit 1163 to receive the signal transmitted from the base station via antenna 1114, amplifies the signal, and further performs frequency conversion processing and analog-to-digital conversion processing on it. Mobile phone 1100 uses modulation/demodulation circuit unit 1158 to perform spectrum despreading processing on the received signal to restore the original multiplexed data. Mobile phone 1100 uses multiplexer/demultiplexer unit 1157 to demultiplex the multiplexed data into encoded image data and audio data.

The mobile phone 1100 decodes the encoded image data using the image decoder 1156 to generate reproduced moving image data, and causes the data to be displayed on the liquid crystal display 1118 via the LCD controller 1155. Thus, for example, moving image data included in a moving image file linked to a simple web page is displayed on the liquid crystal display 1118.

Mobile phone 1100 includes the aforementioned image decoding device 500, which functions as an image decoder 1156 for performing such processing. Specifically, as with image decoding device 500, image decoder 1156 appropriately decodes coded data whose I_PCM mode selection is controlled in units of CUs (units smaller than LCUs). Consequently, image decoder 1156 can reduce redundant processing for encoding and reduce redundant information included in the coded data. Consequently, image decoder 1156 can enhance coding efficiency while suppressing reductions in coding process efficiency.

At this time, the mobile phone 1100 converts the digital audio data into an analog audio signal using the audio codec 1159, and causes the signal to be output from the speaker 1117. Thus, for example, audio data included in a moving image file linked to a simple web page is reproduced.

Note that the mobile phone 1100 is also capable of causing received data linked to a simple web page or the like to be recorded (stored) in the storage unit 1123 via the recording/reproducing unit 1162 as in the case of electronic mail.

In addition, the mobile phone 1100 can analyze the two-dimensional code obtained by image capture by the CCD camera 1116 using the main controller 1150 and obtain information recorded in the two-dimensional code.

Furthermore, the mobile phone 1100 can communicate with external devices through infrared rays using the infrared communication unit 1181 .

By including the image encoding device 300 serving as the image encoder 1153 , the mobile phone 1100 can enhance encoding efficiency while suppressing a decrease in encoding process efficiency, for example, when encoding and transmitting image data generated by the CCD camera 1116 .

In addition, by including the image decoding device 500 serving as the image decoder 1156, the mobile phone 1100 can achieve enhanced data encoding efficiency while suppressing a decrease in encoding processing efficiency when generating data (encoded data) linked to a moving image file such as a simple web page.

Note that although the mobile phone 1100 is described above as including the CCD camera 1116, an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) (CMOS image sensor) may be used instead of the CCD camera 1116. In this case, as in the case of using the CCD camera 1116, the mobile phone 1100 can capture an image of a subject and generate image data of the image of the subject.

In addition, although the mobile phone 1100 is described above, as in the case of the mobile phone 1100, the image encoding device 300 and the image decoding device 500 can be applied to any device (such as a PDA (personal digital assistant), a smart phone, a UMPC (ultra mobile personal computer), a netbook, or a notebook personal computer) having an image capture function and a communication function similar to those of the mobile phone 1100.

<6. Sixth embodiment>

[Hard Disk Recorder]

FIG. 24 is a block diagram showing a main example configuration of a hard disk recorder including the image encoding device 300 and the image decoding device 500 .

The hard disk recorder (HDD recorder) 1200 shown in Figure 24 is a device that stores audio data and video data of a broadcast program included in a broadcast wave signal (television signal) transmitted by a satellite, an antenna on the ground, etc. and received by a tuner in a built-in hard disk therein, and provides the stored data to the user at a timing corresponding to an instruction provided by the user.

For example, the hard disk recorder 1200 can extract audio data and video data from a broadcast wave signal, appropriately decode it, and store it in a hard disk built therein. In addition, for example, the hard disk recorder 1200 can obtain audio data and video data from another device via a network, appropriately decode it, and store it in a hard disk built therein.

Furthermore, for example, the hard disk recorder 1200 can decode audio data and video data recorded on a hard disk built therein, supply the data to the monitor 1260, display the image on the screen of the monitor 1260, and output the audio from the speaker of the monitor 1260. Furthermore, for example, the hard disk recorder 1200 can decode audio data and video data extracted from a broadcast wave signal obtained via a tuner or obtained from another device via a network, supply the data to the monitor 1260, display the image on the screen of the monitor 1260, and output the audio from the speaker of the monitor 1260.

Of course, additional operations may be performed.

24 , the hard disk recorder 1200 includes a receiving unit 1221, a demodulating unit 1222, a demultiplexer 1223, an audio decoder 1224, a video decoder 1225, and a recorder controller 1226. The hard disk recorder 1200 also includes an EPG data memory 1227, a program memory 1228, a work memory 1229, a display converter 1230, an OSD (On Screen Display) controller 1231, a display controller 1232, a recording/reproducing unit 1223, a D/A converter 1234, and a communication unit 1235.

In addition, the display converter 1230 includes a video encoder 1241. The recording/reproducing unit 1233 includes an encoder 1251 and a decoder 1252.

The receiving unit 1221 receives an infrared signal from a remote controller (not shown), converts the signal into an electric signal, and outputs the electric signal to the recorder controller 1226. For example, the recorder controller 1226 is composed of a microphone or the like, and performs various processing operations in accordance with a program stored in the program memory 1228. At this time, the recorder controller 1226 uses the work memory 1229 as necessary.

The communication unit 1235 is connected to the network and performs communication processing with another device via the network. For example, the communication unit 1235 is controlled by the recorder controller 1226, communicates with a tuner (not shown), and outputs a channel selection control signal mainly to the tuner.

The demodulation unit 1222 demodulates the signal supplied from the tuner and outputs the signal to the demultiplexer 1223. The demultiplexer 1223 demultiplexes the data supplied from the demodulation unit 1222 into audio data, video data, and EPG data, and outputs them to the audio decoder 1224, the video decoder 1225, and the recording controller 1226, respectively.

The audio decoder 1224 decodes the audio data input thereto, and outputs the audio data to the recording/reproducing unit 1223. The video decoder 1225 decodes the video data input thereto, and outputs the video data to the display converter 1230. The recorder controller 1226 supplies the EPG data input thereto to the EPG data memory 1227 so as to store it therein.

The display converter 1230 encodes the video data supplied from the video decoder 1225 or the recorder controller 1226 into, for example, video data in the NTSC (National Television Standards Committee) format using the video encoder 124, and outputs the video data to the recording/reproducing unit 1223. In addition, the display converter 1230 converts the screen size of the video data supplied from the video decoder 1225 or the recorder controller 1226 into a size corresponding to the size of the monitor 1260, converts the video data into video data in the NTSC format using the video encoder 1241, converts the video data into an analog signal, and outputs the analog signal to the display controller 1232.

Under the control performed by the recorder controller 1226, the display controller 1232 superimposes the OSD signal output from the OSD (screen display) controller 1231 on the video signal input from the display converter 1230, outputs it to the display of the monitor 1260, and causes it to be displayed on the display of the monitor 1260.

In addition, the audio data output from the audio decoder 1224 and converted into an analog signal by the D/A converter 1234 is supplied to the monitor 1260. The monitor 1260 outputs this audio signal from a speaker built therein.

The recording/reproducing unit 1233 includes a storage medium for storing video data, audio data, and the like recorded thereon.

For example, the recording/reproducing unit 1233 encodes the audio data supplied from the audio decoder 1224 using the encoder 1251. Furthermore, the recording/reproducing unit 1233 encodes the video data supplied from the video encoder 1241 of the display converter 1230 using the encoder 1251. The recording/reproducing unit 1233 combines the coded audio data and the coded video data using a multiplexer. The recording/reproducing unit 1233 performs channel coding on the combined data to amplify it, and writes the data to the hard disk via a recording head.

The recording/reproducing unit 1233 reproduces the data recorded on the hard disk via a reproduction head, amplifies the data, and demultiplexes the data into audio data and video data using a demultiplexer. The recording/reproducing unit 1233 decodes the audio data and video data using a decoder 1252. The recording/reproducing unit 1233 performs D/A conversion on the decoded audio data and outputs the audio data to the speaker of the monitor 1260. In addition, the recording/reproducing unit 1233 performs D/A conversion on the decoded video data and outputs the video data to the display of the monitor 1260.

The recorder controller 1226 reads out the latest EPG data from the EPG data memory 1227 based on a user instruction represented by an infrared signal supplied from the remote controller and received via the receiving unit 1221, and supplies the EPG data to the OSD controller 1231. The OSD controller 1231 generates image data corresponding to the input EPG data and outputs the image data to the display controller 1232. The display controller 1232 outputs the video data input from the OSD controller 1231 to the display of the monitor 1260, and causes the video data to be displayed on the display of the monitor 1260. Thus, an EPG (Electronic Program Guide) is displayed on the display of the monitor 1260.

In addition, the hard disk recorder 1200 can obtain various data (such as video data, audio data, or EPG data) supplied from another device via a network (such as the Internet).

The communication unit 1235 is controlled by the recorder controller 1226, obtains encoded data such as video data, audio data, and EPG data transmitted from another device via the network, and supplies the encoded data to the recorder controller 1226. For example, the recorder controller 1226 supplies the obtained encoded data of the video data and audio data to the recording/reproducing unit 1233 and causes the hard disk to store the encoded data. At this time, the recorder controller 1226 and the recording/reproducing unit 1233 may perform processing (such as re-encoding) as needed.

In addition, the recorder controller 1226 decodes the obtained encoded data of the video data and audio data, and supplies the obtained video data to the display converter 1230. Similar to the video data supplied from the video decoder 1225, the display converter 1230 processes the video data supplied from the recorder controller 1226, supplies the video data to the monitor 1260 via the display controller 1232, and causes its image to be displayed on the monitor 1260.

In addition, in accordance with the display of the image, the recorder controller 1226 can supply the decoded audio data to the monitor 1260 via the D/A converter 1234 and cause audio to be output from the speaker.

Furthermore, the recorder controller 1226 decodes the encoded data of the obtained EPG data, and supplies the decoded EPG data to the EPG data memory 1227 .

The hard disk recorder 1200 described above includes the image decoding device 500 serving as a decoder included in the video decoder 1225, the decoder 1252, and the recorder controller 1226. That is, as in the case of the image decoding device 500, the video decoder 1225, the decoder 1252, and the decoder included in the recorder controller 1226 appropriately decode the coded data whose I_PCM mode selection is controlled in units of CUs smaller than LCUs. Therefore, the video decoder 1225, the decoder 1252, and the decoder included in the recorder controller 1226 can achieve a reduction in redundant processing for encoding and a reduction in redundant information included in the coded data. Therefore, the video decoder 1225, the decoder 1252, and the decoder included in the recorder controller 1226 can achieve enhanced coding efficiency while suppressing a decrease in coding process efficiency.

Therefore, when generating video data (encoded data) received by the tuner or communication unit 1235 or video data (encoded data) reproduced by the recording/reproducing unit 1233, the hard disk recorder 1200 is able to achieve enhanced data encoding efficiency while suppressing a reduction in encoding efficiency.

In addition, the hard disk recorder 1200 includes the image encoding device 300 serving as an encoder 1251. Therefore, as in the case of the image encoding device 300, the encoder 1251 controls the selection of the I_PCM mode in units of CUs (units of a CU) that are smaller than LCUs. That is, the encoder 1251 can further reduce redundant processing for encoding and also reduce redundant information included in the encoded data. Therefore, the encoder 1251 can enhance encoding efficiency while suppressing a decrease in encoding process efficiency.

Therefore, when generating encoded data to be recorded on a hard disk, for example, the hard disk recorder 1200 can enhance encoding efficiency while suppressing a decrease in encoding processing efficiency.

Note that although the above description describes the hard disk recorder 1200 that records video data and audio data on a hard disk, any type of recording medium can be used. For example, as in the case of the hard disk recorder 1200 described above, the image encoding device 300 and the image decoding device 500 can be applied to a recorder that uses a recording medium other than a hard disk (e.g., a flash memory, an optical disk, or a video tape).

<7. Seventh embodiment>

[Camera]

FIG. 25 is a block diagram showing a main example configuration of a camera including the image encoding device 300 and the image decoding device 500 .

The camera 1300 shown in FIG. 25 captures an image of a subject, causes an LCD 1316 to display the image of the subject, and records it on a recording medium 1333 as image data.

The lens block 1311 allows light (ie, an image of a subject) to enter the CCD/CMOS 1312 . The CCD/CMOS 1312 is an image sensor including a CCD or a CMOS, converts the intensity of received light into an electric signal, and supplies the electric signal to the camera signal processor 1313 .

The camera signal processor 1313 converts the electrical signal supplied from the CCD/CMOS 1312 into color difference signals Y, Cr, and Cb, and supplies the color difference signals to the image signal processor 1314. Under the control of the controller 1321, the image signal processor 1314 performs specific image processing on the image signal supplied from the camera signal processor 1313, and encodes the image signal using the encoder 1341. The image signal processor 1314 supplies the encoded data generated by encoding the image signal to the decoder 1315. Furthermore, the image signal processor 1314 obtains data to be displayed generated by the on-screen display (OSD) 1320, and supplies the data to the decoder 1315.

In the foregoing processing, the camera signal processor 1313 appropriately uses a DRAM (Dynamic Random Access Memory) connected via the bus 1317 and causes the DRAM 1318 to store image data, encoded data obtained by encoding the image data, if necessary.

The decoder 1315 decodes the encoded data supplied from the image signal processor 1314, and supplies image data (decoded image data) obtained thereby to the LCD 1316. In addition, the decoder 1315 supplies the data to be displayed supplied from the image signal processor 1314 to the LCD 1316. The LCD 1316 appropriately combines the image of the decoded image data supplied from the decoder 1315 and the image of the data to be displayed, and displays the synthesized image.

Under control performed by the controller 1321 , the screen display 1320 outputs data to be displayed, such as a menu screen and icons composed of symbols, characters, or pictures, to the image signal processor 1314 via the bus 1317 .

The controller 1321 performs various processing operations based on a signal representing details of an instruction provided by a user using the operation unit 1322, and controls the image signal processor 1314, the DRAM 1318, the external interface 1319, the screen display 1320, the media drive 1323, and the like via the bus 1317. Programs, data, and the like required for the controller 1321 to perform various processing operations are stored in the flash ROM 1324.

For example, the controller 1321 can encode the image data stored in the DRAM 1318 on behalf of the image signal processor 1314 or the decoder 1315, and can decode the encoded data stored in the DRAM 1318. In this case, the controller 1321 can perform encoding/decoding processing using a format similar to that of the image signal processor 1314 or the decoder 1315, or can perform encoding/decoding processing using a format incompatible with the image signal processor 1314 or the decoder 1315.

In addition, for example, when an instruction to start printing an image is provided by the operation unit 1322, the controller 1321 reads out image data from the DRAM 1318 and supplies it to the printer 1334 connected to the external interface 1319 via the bus 1317 to print it.

Furthermore, for example, when an instruction to record an image is provided by the operation unit 1322 , the controller 1321 reads out the encoded data from the DRAM 1318 and supplies it to the recording medium 1333 loaded to the media drive 1323 via the bus 1317 to store it.

For example, the recording medium 1333 is any readable and writable removable medium (such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory). Of course, the recording medium 1333 can be any type of removable medium and can be a magnetic tape device, a disk, or a memory card. Of course, the recording medium 1333 can be a contactless IC card, etc.

In addition, the media drive 1323 and the recording medium 1333 may be integrated together, and may be configured of a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive), for example.

For example, the external interface 1319 is composed of a USB input/output terminal or the like, and in the case of printing an image, is connected to a printer 1334. In addition, if necessary, a drive 1331 is connected to the external interface 1319, a removable medium 1332 (such as a magnetic disk, an optical disk, or a magneto-optical disk) is appropriately loaded therein, and if necessary, a computer program read therefrom is installed in the flash ROM 1324.

Furthermore, the external interface 1319 includes a network interface for connecting to a specific network (such as a LAN or the Internet). For example, in accordance with an instruction provided by the operation unit 1322, the controller 1321 can read out coded data from the DRAM 1318 and supply it from the external interface 1319 to another device connected via the network. Furthermore, the controller 1321 can obtain coded data or image data supplied from another device via the network via the external interface 1319 and cause the DRAM 1318 to store the data or supply it to the image signal processor 1314.

The camera 1300 described above includes the image decoding device 500 serving as the decoder 1315. That is, as in the case of the image decoding device 500, the decoder 1315 appropriately decodes the coded data whose I_PCM mode selection is controlled in units of CUs smaller than LCUs. Therefore, the decoder 1315 can reduce redundant processing for encoding and reduce redundant information included in the coded data. Therefore, the decoder 1315 can achieve enhanced coding efficiency while suppressing a decrease in coding process efficiency.

Therefore, for example, when generating image data generated by the CCD/CMOS 1312, encoded data of video data read out from the DRAM 1318 or the recording medium 1333, or encoded data of video data obtained via a network, the camera device 1300 is able to achieve enhanced data encoding efficiency while suppressing a decrease in encoding processing efficiency.

In addition, the camera 1300 includes the image encoding device 300 serving as an encoder 1341. As with the image encoding device 300, the encoder 1341 controls the selection of the I_PCM mode in units of CUs (units of a CU) that are smaller than LCUs. That is, the encoder 1341 can further reduce redundant processing for encoding and further reduce redundant information included in the encoded data. Therefore, the encoder 1341 can enhance encoding efficiency while suppressing a decrease in encoding process efficiency.

Therefore, for example, when generating encoded data to be recorded on the DRAM 1318 or the recording medium 1333 or encoded data to be provided to another device, the camera 1300 can enhance encoding efficiency while suppressing a decrease in encoding processing efficiency.

In addition, the decoding method of the image decoding device 500 can be applied to the decoding process performed by the controller 1321. Likewise, the encoding method of the image encoding device 300 can be applied to the encoding process performed by the controller 1321.

In addition, the image data captured by the camera 1300 may be a moving image or a still image.

Of course, the image encoding apparatus 300 and the image decoding apparatus 500 can be applied to apparatuses or systems other than the above-described apparatuses.

The present technology can be applied to the following image encoding devices and image decoding devices: for example, as in MPEG, H.26x, etc., the image encoding devices and image decoding devices are used to receive image information (bit stream) compressed by orthogonal transform (such as discrete cosine transform) and motion compensation via a network medium (such as satellite broadcasting, cable TV, the Internet or mobile phone), or are used to process image information on a storage medium (such as an optical disc or magnetic disk, or a flash memory).

Additionally, the present technology may also provide the following configurations.

(1) An image processing device comprising:

an encoding mode setter that sets, in units of encoding units having a hierarchical structure, whether a non-compression mode is to be selected as an encoding mode for encoding image data, the non-compression mode being an encoding mode in which the image data is output as encoded data; and

An encoder encodes the image data in units of the encoding units according to the mode set by the encoding mode setter.

(2) The image processing device according to (1), further comprising:

a shift processing controller that performs control to skip a shift processing in which bit accuracy for encoding or decoding is increased, on a coding unit for which the non-compression mode is set by the coding mode setter; and

A shift processor performs the shift process on a coding unit of the image data, the coding unit being controlled by the shift process controller so as to be subjected to the shift process.

(3) The image processing apparatus according to (1) or (2), further comprising:

a filter processing controller that performs control to skip a filter processing in which filtering is performed on a local decoded image, for a coding unit for which the non-compression mode is set by the coding mode setter;

a filter coefficient calculator that calculates a filter coefficient for the filter process by using image data corresponding to a coding unit controlled by the filter process controller so as to be subjected to the filter process; and

A filter processor performs the filter processing in units of blocks, which are units of the filter processing, by using the filter coefficients calculated by the filter coefficient calculator.

(4) The image processing apparatus according to (3), wherein the filter processor performs the filter processing only on pixels controlled by the filter processing controller to undergo the filter processing, the pixels being included in a current block that is a target to be processed.

(5) The image processing apparatus according to (3) or (4), further comprising:

A filter identification information generator generates filter identification information in units of the block, the filter identification information being flag information indicating whether the filter process is to be performed.

(6) The image processing apparatus according to any one of (3) to (5), wherein the filter processor performs adaptive loop filtering on the local decoded image, the adaptive loop filtering being an adaptive filter process using a classification process.

(7). An image processing device according to any one of (1) to (6), wherein, when the amount of encoded data obtained by encoding the image data corresponding to the current encoding unit as the target of encoding processing is less than or equal to the input data amount as the data amount of the image data corresponding to the current encoding unit, the encoding mode setter sets the encoding mode of the current encoding unit to the non-compression mode.

(8) The image processing device according to (7), further comprising:

An input data amount calculator for calculating the input data amount.

The encoding mode setter compares the input data amount calculated by the input data amount calculator with the encoding amount with respect to the current encoding unit.

(9) The image processing apparatus according to any one of (1) to (8), further comprising:

An identification information generator generates identification information in units of the coding unit, wherein the identification information indicates whether the coding mode setter sets the non-compression mode.

(10) An image processing method for an image processing device, comprising:

using a coding mode setter to set, in units of coding units having a hierarchical structure, whether a non-compression mode is to be selected as a coding mode for encoding image data, the non-compression mode being a coding mode in which the image data is output as coded data; and

The image data is encoded using the encoding unit according to the set mode.

(11) An image processing device comprising:

an encoding mode determiner that determines, in units of encoding units having a hierarchical structure, whether a non-compression mode is selected as an encoding mode for encoding image data, the non-compression mode being an encoding mode in which the image data is output as encoded data; and

A decoder decodes the encoded data in units of the encoding units according to the mode determined by the encoding mode determiner.

(12) The image processing apparatus according to (11), further comprising:

a shift processing controller that performs control to skip a shift processing in which bit accuracy for encoding or decoding is increased, on a coding unit determined by the coding mode determiner to have selected the non-compression mode; and

A shift processor performs the shift process on a coding unit of the image data, the coding unit being controlled by the shift process controller so as to be subjected to the shift process.

(13) The image processing apparatus according to (11) or (12), further comprising:

a filter processing controller that performs control to skip a filter process in which filtering is performed on a local decoded image, for a coding unit determined by the encoding mode determiner to have selected the non-compression mode; and

a filter processor that performs the filter processing on the image data in units of blocks, the blocks being the units of the filter processing,

Here, the filter processor performs the filter process only on pixels controlled by the filter process controller to be subjected to the filter process, the pixels being included in a current block that is a target to be processed.

(14) The image processing apparatus according to (13), wherein the filter processor performs adaptive loop filtering on the local decoded image, the adaptive loop filtering being an adaptive filter process using a classification process.

(15). An image processing device according to (13) or (14), wherein, in a case where the filter identification information indicating whether the filter processing has been performed indicates that the filter processing has been performed on the image data corresponding to the current block as a target to be processed, the filter processor performs the filter processing only when control is performed by the filter processing controller so as to perform the filter processing on all pixels included in the current block.

(16) The image processing device according to any one of (11) to (15), wherein the encoding mode determiner determines whether the non-compression mode is selected based on identification information indicating whether the non-compression mode is selected in units of encoding units.

(17) An image processing method for an image processing device, comprising:

determining, using a coding mode determiner, whether a non-compression mode is selected as a coding mode for encoding image data in units of coding units having a hierarchical structure, the non-compression mode being a coding mode in which the image data is output as coded data; and

The coded data is decoded using the coding unit according to the determined mode using a decoder.

300 Image Coding Device

303 Adaptive left shift unit

307 Lossless Encoder

312 Loop Filter

313 Adaptive right shift unit

315 Adaptive left shift unit

320 bitrate controller

321 PCM encoder

331 NAL encoder

332 CU encoder

341 I_PCM_flag generator

342 PCM decision unit

351 Deblocking Filter

352 pixel classification units

353 Filter Coefficient Calculator

354 filter unit

361 Input Data Calculator

362 PCM determination unit

363 Encoding Controller

364 Adaptive Shift Controller

365 filter controller

500 Image Decoding Device

502 lossless decoder

507 Adaptive right shift unit

511 Adaptive left shift unit

516 PCM decoder

531 NAL decoder

532 CU decoder

541 I_PCM_flag buffer

542 PCM controller

551 Deblocking Filter

552 pixel classification units

553 filter unit

Claims (28)

1. An image processing apparatus, comprising: A processing device and a memory, the memory having a program stored thereon, the program including instructions that, when executed by the processing device, cause the processing device to: Using the encoded data obtained by recursively segmenting the largest encoded unit as the unit, the encoded data of the encoded unit set to uncompressed mode is decoded to generate image data, wherein the uncompressed mode is an encoding mode that uses image data as encoded data; and Control is applied to the encoding units set in the uncompressed mode to skip the filter processing performed on the image data after the deblocking filter processing. 2. The image processing apparatus according to claim 1, further comprising instructions that cause the processing apparatus to: The deblocking filter process is performed on the image data, and The filter processing is performed on the image data that has undergone the deblocking filter processing. 3. The image processing apparatus according to claim 2, wherein, The filter processing described is an adaptive loop filter processing that uses a Wiener filter. 4. The image processing apparatus according to claim 2, wherein the instruction causes the processing apparatus to: The filter processing is performed on the image data using filter identification information that indicates whether the filter processing should be performed. 5. The image processing apparatus according to claim 4, wherein the instruction causes the processing apparatus to: The identification information is obtained from the encoded data. The filter processing is performed on the image data using the identification information obtained by the decoder. 6. The image processing apparatus according to claim 2, wherein, The deblocking filter process and the filter process are performed on the image data using loop filter identification information that identifies whether the deblocking filter process and the filter process are performed as loop filter processes. 7. The image processing apparatus according to claim 6, wherein the instruction causes the processing apparatus to: The identification information is obtained from the encoded data. The deblocking filter process is performed on the image data using the loop filter identification information obtained by the decoder, and The filter processing is performed on the image data that has undergone deblocking filter processing. 8. The image processing apparatus according to claim 1, further comprising instructions that cause the processing apparatus to: Using the encoding unit as the unit, determine whether the uncompressed mode is selected for each unit. Specifically, control is applied to the encoding unit that has selected the uncompressed mode to skip the filter processing. 9. The image processing apparatus according to claim 1, wherein the instruction causes the processing apparatus to: Whether the uncompressed mode has been selected is determined based on the encoding mode identifier information indicating whether the uncompressed mode has been selected in units of the encoding unit. 10. The image processing apparatus according to claim 2, wherein the instructions cause the processing apparatus to: The filter processing is performed on the image data by using identifier information indicating that the uncompressed mode of the encoding unit exists in a block consisting of multiple encoding units. 11. The image processing apparatus according to claim 10, wherein, The block composed of the multiple coding units is the largest coding unit (LCU) with the largest size. 12. The image processing apparatus according to claim 11, wherein, The encoding unit recursively divides the LCU according to a quadtree structure. 13. The image processing apparatus according to claim 12, wherein, The LCU is the top-level encoding unit in the quadtree structure. 14. The image processing apparatus according to claim 13, wherein, The LCU is a fixed-size block based on a sequence. The encoding unit is a variable-size block. 15. An image processing method for an image processing device, comprising: Using the encoded data obtained by recursively segmenting the largest encoded unit as the unit, the encoded data of the encoded unit set to uncompressed mode is decoded to generate image data, wherein the uncompressed mode is an encoding mode that uses image data as encoded data; and Control is applied to the encoding units set in the uncompressed mode to skip the filter processing performed on the image data after the deblocking filter processing. 16. The image processing method according to claim 15, further comprising: Perform the deblocking filter processing on the image data; and The filter processing is performed on the image data that has undergone the deblocking filter processing. 17. The image processing method according to claim 16, wherein, The filter processing described is an adaptive loop filter processing that uses a Wiener filter. 18. The image processing method according to claim 16, comprising: The filter processing is performed on the image data using filter identification information that indicates whether filter processing should be performed. 19. The image processing method according to claim 18, comprising: The identification information is obtained from the encoded data. The obtained identification information is used to perform the filter processing on the image data. 20. The image processing method according to claim 16, wherein, The deblocking filter process and the filter process are performed on the image data using loop filter identification information that identifies whether the deblocking filter process and the filter process are performed as loop filter processes. 21. The image processing method according to claim 20, comprising: The identification information is obtained from the encoded data. The image data is processed using the acquired loop filter identification information; and The filter processing is performed on the image data that has undergone deblocking filter processing. 22. The image processing method according to claim 15, further comprising: Using the encoding unit as the unit, determine whether the uncompressed mode is selected for each unit, and Control is applied to the encoding unit that has selected the uncompressed mode to skip the filter processing. 23. The image processing method according to claim 15, comprising: The selection of the uncompressed mode is determined based on the encoding mode identifier information indicating whether the uncompressed mode has been selected in units of the encoding unit. 24. The image processing method according to claim 16, comprising: The filter processing is performed on the image data by using identifier information indicating the presence of the uncompressed encoding unit selected in a block consisting of multiple encoding units. 25. The image processing method according to claim 24, wherein, The block composed of the multiple coding units is the largest coding unit (LCU) with the largest size. 26. The image processing method according to claim 25, wherein, The encoding unit recursively divides the LCU according to a quadtree structure. 27. The image processing method according to claim 26, wherein, The LCU is the top-level encoding unit in the quadtree structure. 28. The image processing method according to claim 27, wherein, The LCU is a fixed-size block based on a sequence. The encoding unit is a variable-size block.