KR20120083200A - Depth map coding/decoding apparatus and method - Google Patents

Abstract

We propose a depth image encoding / decoding apparatus and method. According to an aspect, an apparatus for encoding a depth image includes an intra prediction mode that predicts the direction of a current macro block using encoded macro blocks, and intra-codes the intra direction using the predicted direction as one of intra candidate encoding modes. Priorities of the encoding modes included in the candidate encoding modes and the inter candidate encoding modes are encoded according to the characteristics of the depth image.

Description

Depth image coding / decoding apparatus and method {Depth Map Coding / Decoding Apparatus and Method}

TECHNICAL FIELD The present invention relates to an apparatus and a method for encoding and decoding a depth image.

A stereoscopic image refers to a 3D image that simultaneously provides shape information about depth and space. In this case, unlike stereo, which provides images of different viewpoints to the left and right eyes, in order to provide the same image as viewed from different directions every time the observer views a different image, images captured from various viewpoints are required. There are free view TVs, 3D TVs, and other application fields using images captured from various viewpoints. Free-view TV can freely change the point of view of objects by acquiring and analyzing multi-view images of the same scene, and in the case of 3-D TVs, it is possible to recognize realistic 3D depth by changing the images reflected by both eyes of a person. To help. However, since images taken at various points of view have a large amount of data, it is difficult to secure network infrastructure and terrestrial bandwidth in order to compress and transmit them.

Instead of compressing and transmitting all the videos from multiple viewpoints, a depth image can be created and compressed together with the images of some viewpoints among the images of multiple viewpoints to reduce the amount of data generated during compression. Since the depth image is an image representing a distance of an object from the viewer in the color image with a value of 0 to 255, the feature is similar to the color image. However, the depth image has a flat characteristic unlike the color image. Therefore, there is a need for a more effective coding method for depth images.

In one aspect, an intra mode prediction unit for predicting the direction of the current macro block using the encoded neighboring macro blocks, an intra encoder for intra coding the current macro block for each of the intra candidate encoding modes, and the current macro An inter encoder which inter-codes a block for each inter candidate encoding modes, and rate-distortion optimization among the encoding modes in consideration of the priority of each of the intra candidate encoding modes and the encoding modes included in the inter candidate encoding modes. There is provided a depth image encoding apparatus including an encoding mode determiner configured to determine an encoding mode having a lowest RDO (cost-distortion optimization) cost as an encoding mode of the current macroblock, wherein the intra candidate encoding modes are included in the prediction candidates. Intra coding using one direction Intra prediction mode is included.

In this case, the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode, and the intra skip mode encodes the predicted directional information and the residual signal. The intra direct mode is a mode in which the predicted directional information is not encoded and transmitted to the decoding apparatus, and the intra direction mode may be a mode in which no residual signal is transmitted to the decoding apparatus. have.

In this case, the intra skip mode may be the highest priority or the next highest priority among the priorities of the respective encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes.

In this case, the predicted directionality is a vertical direction extending vertically from an upper boundary pixel of the current macroblock to generate a prediction block, and horizontally extending from a left boundary pixel of the current macroblock to generate a prediction block. A DC direction for generating a prediction block based on a horizontal direction and an average of the left boundary pixel and the upper boundary pixel of the current macro block, and the prediction is performed by considering the upper boundary pixel and the left boundary pixel of the current macro block together. It may be one of plane directions for generating a block.

In this case, the types of intra candidate encoding modes may be classified according to a difference between at least one of the size of the macro block, whether the directional information is encoded, and whether there is a residual signal for the brightness component of the macro block.

In this case, the inter candidate encoding modes refer to one motion vector per macroblock, and the type of the inter candidate encoding modes includes at least one of a size of a macroblock, whether a motion vector is encoded, and whether a residual signal is encoded. It can be classified according to the difference.

In this case, the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be set to have a higher priority than the inter candidate encoding modes.

In this case, the priority between the intra candidate encoding modes is set to a higher priority as the size of the macroblock is larger, and a mode that does not encode the directional information is set to have a higher priority than a mode that encodes the directional information, and a residual The mode in which no signal is encoded may be set to have a higher priority than the mode in which the residual signal is encoded, and the mode having a directionality may be set to have a higher priority than a mode having no directionality.

In this case, the priority between the inter candidate encoding modes is set to a higher priority as the size of the macroblock is larger, a mode not encoding a motion vector is set to a higher priority than a coding mode, and the residual signal is encoded. A mode that does not have a higher priority may be set than a mode that encodes a residual signal.

In this case, the information about the priority of each of the intra candidate encoding modes and the encoding modes included in the inter candidate encoding modes may be preset or defined when encoding a depth image and transmitted to the decoding apparatus.

In this case, the encoding mode determiner may change the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to a frequency at which encoding modes are finally determined.

In this case, the current macro block may be one of a macro block of a depth image, a macro block of a black and white image, a macro block of a color image having a flat characteristic, or a macro block of an X-ray image.

The deblocking filter unit may further include a deblocking filter unit to remove a blocking phenomenon by filtering the reconstructed block by using the prediction block, and the deblocking filter unit may be intra-coded from at least one of both blocks on the basis of a block boundary. On the basis of the boundary of the block, both blocks may be filtered relatively more strongly than the case where the inter coding is performed.

The encoded macroblock is encoded by using an encoding mode reconstruction unit configured to identify an encoding mode of the received encoded macroblock, and neighboring macroblocks decoded when the encoding mode of the encoded macroblock is an intra prediction mode. There is provided a depth image decoding apparatus including an intra mode reconstruction unit for predicting a directionality of a and an intra decoder for generating a prediction block using the predicted directionality.

In this case, the encoding mode reconstruction unit identifies the encoding mode of the encoded macroblock by using priority information of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes, and the intra prediction mode is determined as the above. It may be included in intra candidate encoding modes.

In this case, the encoding mode reconstruction unit may change the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to the frequency of the identified encoding modes.

In this case, the information about the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be preset or defined when decoding a depth image and may be received by the encoding apparatus.

In this case, the predicted directionality extends vertically from an upper boundary pixel of the coded macroblock to generate a prediction block, and extends horizontally from a left boundary pixel of the coded macroblock to expand the prediction block. A horizontal direction to generate and a DC direction to generate a prediction block with an average of the upper boundary pixel and the left boundary pixel of the coded macro block, the upper boundary pixel and the left boundary pixel of the coded macro block Considered together, it may be one of the plane directions for generating the prediction block.

In this case, the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode, and the intra skip mode encodes the predicted directional information and the residual signal. The intra direct mode is a mode that is not provided from an apparatus, the intra direct mode is a mode that receives a residual signal from the encoding apparatus, and the predicted directional information is not provided from the encoding apparatus, and the intra direction mode transmits a residual signal to the decoding apparatus. The mode may not be.

The deblocking filter unit may further include a deblocking filter unit configured to remove a blocking phenomenon by filtering the reconstructed depth image by using the prediction block, wherein the deblocking filter unit is intra-coded from at least one of both blocks based on a block boundary. Based on the boundary of the block, both blocks may be filtered relatively more strongly than the case where both blocks are inter coded.

In an aspect, the method may further include estimating a directionality of a current macroblock using encoded neighboring macroblocks, intra-coding the current macroblock according to intra candidate encoding modes, and wherein the current macroblock is a P-picture. Or, if the macro block corresponds to a B-picture, inter encoding the current macro block according to inter candidate encoding modes and prioritizing each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes. In consideration of this, a depth image encoding method including determining an encoding mode having a lowest rate-distortion optimization (RDO) cost among the encoding modes as an encoding mode of the current macroblock is provided. The intra candidate encoding mode uses the predicted directionality. An intra prediction mode for intra coding is included.

In this case, the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode, and the intra skip mode encodes the predicted directional information and the residual signal. The intra direct mode is a mode in which the predicted directional information is not encoded and transmitted to the decoding apparatus, and the intra direction mode may be a mode in which no residual signal is transmitted to the decoding apparatus. have.

In this case, the intra skip mode may be the highest priority or the next highest priority among the priorities of the respective encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes.

In this case, the predicted directionality is a vertical direction extending vertically from an upper boundary pixel of the current macroblock to generate a prediction block, and horizontally extending from a left boundary pixel of the current macroblock to generate a prediction block. A DC direction for generating a prediction block based on a horizontal direction and an average of the left boundary pixel and the upper boundary pixel of the current macro block, and the prediction is performed by considering the upper boundary pixel and the left boundary pixel of the current macro block together. It may be one of plane directions for generating a block.

The method of claim 1, further comprising: checking an encoding mode of an encoded macroblock using priority information of respective encoding modes included in intra candidate encoding modes and inter candidate encoding modes, and encoding mode of the encoded macroblock. If is an intra prediction mode, there is provided a depth image decoding method comprising predicting the directionality of the encoded macroblock using decoded neighboring macroblocks and generating a prediction block using the predicted directionality.

In this case, the information about the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be preset or defined when decoding a depth image and may be received by the encoding apparatus.

In this case, the predicted directionality extends vertically from an upper boundary pixel of the coded macroblock to generate a prediction block, and extends horizontally from a left boundary pixel of the coded macroblock to expand the prediction block. A horizontal direction to generate and a DC direction to generate a prediction block with an average of the upper boundary pixel and the left boundary pixel of the coded macro block, the upper boundary pixel and the left boundary pixel of the coded macro block Considered together, it may be one of plane directions for generating a prediction block.

In this case, the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode, and the intra skip mode encodes the predicted directional information and the residual signal. The intra direct mode is a mode that is not provided from an apparatus, the intra direct mode is a mode that receives a residual signal from the encoding apparatus, and the predicted directional information is not provided from the encoding apparatus, and the intra direction mode transmits a residual signal to the decoding apparatus. The mode may not be.

According to embodiments of the present invention, the directionality of the current macroblock is predicted using the encoded macroblocks in the neighboring region, intra-coded using the predicted directionality, and included in the intra candidate encoding modes and the inter candidate encoding modes. By encoding the priorities of the encoding modes aligned with the characteristics of the depth image, the encoding efficiency of the depth image may be improved.

1 is a diagram illustrating an encoding structure of a multiview image;
2 is a diagram illustrating a configuration of a depth image encoding apparatus encoding a depth image;
3 is a diagram illustrating a configuration of a depth image decoding apparatus for decoding a depth image;
4 is a diagram illustrating a region referenced when predicting directionality for an intra prediction mode;
5 is a flowchart illustrating a process of determining an encoding mode in a depth image encoding apparatus;
6 is a flowchart illustrating a process of generating a prediction block for decoding in a depth image decoding apparatus;
7 is a flowchart illustrating a process of setting a filtering strength for deblocking in a depth image encoding apparatus or a depth image decoding apparatus.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to an apparatus and a method for encoding and decoding a depth image. First, a detailed description will be made of an encoding structure and an encoding mode of a depth image to be encoded.

1 is a diagram illustrating an encoding structure of a multiview image.

Referring to FIG. 1, a depth image may be configured of an encoding structure of a multiview image. In general, a coding structure of a multiview image includes an anchor frame that is encoded using only reference between adjacent viewpoints, and a non-anchor frame that uses both a reference in the time direction and a view in the view direction. The anchor frame is needed for random access and is encoded using only inter-view prediction, and thus shows the largest difference from the conventional single view compression. The encoding structure of the multi-view image of FIG. 1 represents a multiview video coding (MVC) structure in which three view images are encoded as a group of picture (GOP) '8' when three view images are input (left view, center view, right view).

When compressing multi-view images, the concept of hierarchical B pictures is applied to the temporal axis and the view axis, which can reduce a lot of redundancy between images. have. The MVC structure of FIG. 1 currently compresses a left image and then compresses a right image and a center image in order to compress three viewpoint images. At this time, the left image is encoded while removing a temporal redundancy by finding a similar region from previous images by performing motion estimation (Motion Estimation) as in the case of compressing a conventional 2D image. However, when the right image is compressed, the left image that has already been compressed can be used as a reference image, so that not only temporal redundancy through motion search but also view redundancy is removed by disparity estimation. In the case of the center image, both left and right images can be used as reference images, and thus, redundancy between viewpoints can be eliminated by searching for mutation in both directions.

In FIG. 1, a view that is encoded without prediction from another view, such as a left image, is predicted in one direction from an image of another view, such as an I-View (Intra View), a P-View (Predictive View), or a Center image. As described above, a viewpoint that is encoded by bidirectionally predicting images from left and right viewpoints is generally defined in MVC as a B-View (Interpolative View).

Each picture is divided into an I-picture, a P-picture, and an B-picture according to a coding type. In this case, the I-picture encodes the image itself without inter-prediction, and the P-picture encodes the difference from the predicted value by predicting the inter-image using the reference image only in the forward direction, and the B-picture refers to the reference image in both forward and reverse directions. The inter prediction is encoded using

Conventional H.264 / AVC uses various macroblock modes when encoding each macroblock. Among them, the optimal macroblock mode is selected and encoded, and information about the mode type is encoded and sent for each macroblock. At this time, in order to minimize the bits used for the mode information of the macroblock, macroblocks are encoded by giving priority to modes that are frequently generated and have fewer bits to be encoded. <Table 1>, <Table 2> and <Table 3> are tables for various macroblock types used in frame encoding of I pictures, P pictures, and B pictures in H.264 / AVC, respectively.

[Table 1]

[Table 2]

[Table 3]

However, the macroblock modes shown in <Table 1>, <Table 2> and <Table 3> are designed in terms of color images and are not effective for depth images. Among the macroblock modes, the SKIP mode (macroblock mode information and residual components not encoded), which can reduce bits most effectively, has a lower probability of generating a depth image than a color image.

This conventional SKIP mode (hereinafter referred to as inter SKIP mode) makes predictions based on median motion vector values from the motion vectors of adjacent macroblocks. Because of this, inter SKIP mode is not effective.

In addition, the depth image is currently commonly provided in the form of 4: 2: 0 yuv. In this case, the values of the color difference signals are all set to 128 so that residuals of the color difference components do not occur. Therefore, the macroblock mode having the residual of the color difference in the conventional H.264 / AVC mode is not effective.

Also, in the H.264 / AVC mode, the coding method based on the motion vector is not effective except for the region where some textures exist in the depth image. In addition, even in the case of Bi-prediction (bidirectional prediction) used for the B picture frame of the conventional H.264 / AVC, the depth image does not perform well. Therefore, the macroblock modes designed from the perspective of the existing color image are inefficient for the depth image. The unnecessarily large number of macroblock modes increases the bits used for encoding macroblock information, which is inefficient in terms of encoding. In addition, the prediction method for each mode and the priority between macroblocks are also inefficient in terms of color image.

In order to overcome this problem, an intra prediction method, which is an intra-based prediction method, is proposed in an embodiment of the present invention, and the intra prediction method is commonly applied to frames of an I picture, a P picture, and a B picture. The intra prediction method may include an intra SKIP mode and an intra direct mode. The description of the intra SKIP mode and the intra direct mode will be described later in the description of FIG. 2.

The macroblock mode may be deleted when the depth image is encoded, and the priority of the macroblock may be changed to be suitable for the depth image.

In order to change the priority of the macroblock mode in depth image encoding, some considerations are considered.

First, the depth image is commonly provided in the form of 4: 2: 0 yuv. In this case, the color difference signal is all set to 128 so that no residual of the color difference component occurs. Therefore, mode types 5 ~ 12 and 17 ~ 24 are meaningless when the cbp of the color difference component is not 0 among Intra 16x16 modes. It is therefore desirable to eliminate these modes. In the form of I_16x16_a_b_c, a represents cbp information of the chrominance component of b and the brightness component of c of directionality of Intra 16x16.

Second, in the case of depth images, complex regions are often encoded by Intra4x4 rather than by Inter 16x8, 8x16, and 8x8. In addition, the depth image has a higher frequency of intra generation in the B screen or the P screen than the color image. In summary, an embodiment of a priority for various macroblock modes may be about SKIP> Direct> Inter16x16> Intra NxN> Inter8x16> Inter16x16> InterP8x8> Intra16x16. In this case, the SKIP and Direct modes are examples of the case where the proposed intra mode is replaced.

A newly defined macroblock mode for an I picture, a P picture, and a B picture frame and an embodiment thereof may be configured as shown in Tables 4, 5, and 6 below.

[Table 4]

[Table 5]

TABLE 6

The above-described intra prediction mode and the reordering of the priorities are optimized according to the characteristics of the depth image. However, intra SKIP mode or intra direct mode of intra prediction mode may be inefficient in encoding non-anchor frames. This is because the existing SKIP method, inter SKIP mode, shows stable performance in non-anchor. Therefore, for the effective application of SKIP and direct modes to anchors and non-anchors

First, we will check the efficiency of intra SKIP mode and inter SKIP mode in anchor and non-anchor. In general, intra SKIP is effective in anchors and inter SKIP is more effective in non-anchors. Therefore, if you use them all together, you can use SKIP for more efficient prediction methods and other methods for direct mode.

In one embodiment, in the coding of the anchor frame, the intra SKIP mode may be positioned at the highest priority and the inter SKIP mode may be positioned at the priority position next to the next higher priority. In contrast, in non-anchor, the inter SKIP mode can be placed at the highest priority, and the intra SKIP mode can be placed at the next higher priority.

In addition, you can rearrange priorities in various combinations. In addition, a flag bit may be defined after the SKIP mode or other mode information to distinguish between the inter skip mode and the intra skip mode.

2 is a diagram illustrating a configuration of a depth image encoding apparatus for encoding a depth image.

Referring to FIG. 2, the depth image encoding apparatus includes an intra mode predictor 210, an intra encoder 212, a motion predictor 214, an inter encoder 216, an encoding mode determiner 218, and a prediction block. The generator 220, the difference unit 222, the transform and quantization unit 224, the entropy encoder 226, the inverse transform and inverse quantizer 228, the combiner 230, and the deblocking filter unit 232. Include.

The intra mode prediction unit 210 predicts the direction of the current macro block by using the encoded macro blocks of the surrounding. In this case, the predicted directionality extends vertically from the upper boundary pixel of the current macro block to generate a prediction block, and horizontal to extend the horizontal direction from the left boundary pixel of the current macro block to generate a prediction block. One of a DC direction for generating a prediction block with a direction and an average of the upper boundary pixel and the left boundary pixel of the current macro block, and one of the plane directions for generating the prediction block considering the upper boundary pixel and the left boundary pixel of the current macro block together. Can be.

The intra encoder 212 intra encodes the current macroblock according to intra candidate encoding modes.

The intra candidate coding mode includes an intra prediction mode for intra coding using the predicted directionality. The intra prediction mode may include at least one of an intra skip mode, an intra direct mode, and an intra direction mode. The intra skip mode is a mode in which the predicted directional information and the residual signal are not transmitted to the decoding apparatus, and the intra direct mode is a mode in which the predicted directional information is not transmitted to the decoding apparatus, and the intra direction mode decodes the residual signal. This mode does not pass to the device.

In addition, the intra skip mode is the highest priority or the next highest priority among the priority of each of the intra candidate encoding modes and the encoding modes included in the inter candidate encoding modes.

The types of intra candidate encoding modes may be classified according to a difference between at least one of the size of the macro block, whether the directional information is encoded, and whether there is a residual signal for the brightness component of the macro block.

The motion predictor 214 estimates a motion vector using the current macroblock to be encoded and the previously input macroblock.

The inter encoder 216 inter-encodes the current macro block according to inter candidate encoding modes based on the motion vector estimated by the motion predictor 214.

In this case, the inter candidate encoding modes are encoded by referring to one motion vector per macroblock. The types of inter candidate encoding modes may be classified in consideration of the type of the size of the macro block and whether the residual signal is encoded. At this time, the macroblock means the largest block to be encoded and may be larger than 16x16.

The encoding mode determiner 218 considers the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes, and thus, has a rate-distortion optimization (RDO) cost among the encoding modes. The smallest encoding mode is determined as the encoding mode of the current macroblock.

The priority of each of the intra candidate encoding modes and the encoding modes included in the inter candidate encoding modes is set such that the intra candidate encoding modes have a higher priority than the inter candidate encoding modes.

The priority between the intra candidate encoding modes is set to a higher priority as the macroblock size increases, and a mode that does not encode directional information is set to have a higher priority than a mode that encodes directional information. A mode without encoding is set to have a higher priority than a mode for encoding the residual signal, and a mode with a direction is set to have a higher priority than a mode without a direction.

The priority between the inter candidate encoding modes is set to a higher priority as the macroblock size increases, a mode not encoding a motion vector is set higher than a mode encoding, and a mode not encoding a residual signal. Priority is set higher than the mode for encoding the residual signal.

Meanwhile, information about the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be preset or defined when encoding a depth image, and may be transmitted to the decoding apparatus.

The encoding mode determiner 218 may change the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to the frequency at which the encoding modes are finally determined.

The prediction block generator 220 generates a prediction block by using the encoding mode determined by the encoding mode determiner 218.

The difference unit 222 subtracts the prediction block from the current macro block to generate a residual signal in units of macro blocks.

The transform and quantization unit 224 converts the difference signal generated by the difference unit 222 using a transform scheme, and quantizes the transformed differential signal to generate a quantized transform coefficient. An example of a transformation method is a Discrete cosine transform (DCT).

The entropy encoder 226 generates a bitstream by entropy encoding encoding information such as a quantized transform coefficient and a motion vector. The generated bitstream is a compressed depth image.

The inverse transform and inverse quantizer 228 inverse quantizes the difference signal quantized by the transform and quantizer 224 for prediction of a frame to be encoded next, inverse discrete cosine transform, and reconstructs the difference signal before encoding. .

The combiner 230 reconstructs the current macroblock before encoding by adding the reconstructed difference signal and the predictive block generated by the predictive block generator 220.

The deblocking filter unit 232 removes the blocking phenomenon by filtering the reconstructed block or the reconstructed depth image. The deblocking filter unit 232 filters relatively more strongly when both of the blocks are intra-encoded based on the boundary of the block. In this case, the intra encoding includes an intra prediction mode, and the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode. The deblocking filter 232 may set the strength of the filtering as shown in FIG. 7 afterwards.

3 is a diagram illustrating the configuration of a depth image decoding apparatus for decoding a depth image.

Referring to FIG. 3, an apparatus for decoding a depth image may include an entropy decoder 310, an encoding mode restoring unit 312, a motion restoring unit 314, an inter decoding unit 316, an intra mode restoring unit 318, and an intra decoding unit. The unit 320, an inverse transform and inverse quantization unit 322, a coupling unit 324, and a deblocking filter unit 326.

The entropy decoder 310 may entropy decode the input compressed bitstream to extract encoding information such as a quantized coefficient value and a motion vector of the difference image.

The encoding mode reconstruction unit 312 checks the encoding mode of the encoded macroblock by using priority information of respective encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes.

The encoding mode reconstruction unit 312 may change the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to the identified encoding modes.

In addition, the information about the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be preset or defined when decoding the depth image, and may be received by the encoding apparatus.

The motion reconstruction unit 314 checks the motion vector from the encoding information in the entropy decoding unit 310.

The inter decoder 316 inter decodes the motion vector identified by the motion reconstructor 314 to generate a prediction block.

The intra mode reconstructor 318 predicts the direction of the encoded macro block using the decoded neighboring macro blocks when the encoding mode of the encoded macro block is the intra prediction mode. In this case, the predicted directionality is a vertical direction extending vertically from the upper boundary pixel of the encoded macro block to generate a prediction block, and a horizontal direction extending horizontally from the left boundary pixel of the encoded macro block to generate a prediction block. In the (Horizontal) direction and the DC direction for generating the prediction block by the average of the upper boundary pixel and the left boundary pixel of the encoded macro block, the prediction block is considered in consideration of the upper boundary pixel and the left boundary pixel of the encoded macro block. One of the plane directions to create.

If the encoding mode of the encoded macroblock is one of intra candidate encoding modes, the intra mode reconstructor 318 checks the directional information included in the received bit stream.

The intra decoder 320 generates a prediction block using the directional information predicted by the intra mode reconstructor 318 or the confirmed directional information.

The inverse transform and inverse quantizer 322 inversely quantizes the extracted quantization coefficient value in macroblock units to generate coefficient values of the differential signal corresponding to the inverse transform. The inverse transform and inverse quantization unit 322 may obtain a differential signal by performing discrete cosine transform (IDCT) on the coefficient values of the generated difference image. As a result, the compressed bitstream is decoded and temporarily restored to the depth image.

The combiner 324 reconstructs the depth image by adding the difference signal and the prediction block. The differential signal is input to the combiner 324 from the inverse transform and inverse quantizer 322, and the prediction block is input to the combiner 324 from the inter decoder 316 or the intra decoder 320.

The deblocking filter 326 filters the depth image reconstructed by the combiner 324 to remove the blocking phenomenon. The deblocking filter unit 326 filters relatively more strongly when both of the blocks are intra-encoded based on the boundary of the block, if both blocks are inter-encoded based on the boundary of the block. In this case, the intra encoding includes an intra prediction mode, and the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode. The deblocking filter 326 may set the strength of the filtering as shown in FIG. 7 afterwards.

The intra mode prediction unit 210 of the encoding apparatus and the intra mode reconstruction unit 318 of the decoding apparatus may predict the directional prediction by the method shown in FIG. 4 below.

4 is a diagram illustrating a region referred to when predicting directionality for the intra prediction mode.

Referring to FIG. 4, the intra mode prediction unit 210 and the intra mode reconstruction unit 318 may include the boundary value of the upper block 420 and the left block (420) adjacent to the current macroblock 410 as shown in Equation 1 below. The complexity of the boundary value of 430 is compared to take the method of following the more complex boundary direction.

[Equation 1]

Here, A_SSE is the complexity of the boundary value of the upper block, and L_SSE is the complexity of the boundary value of the left block.

Equation 1 is an embodiment for predicting the direction of the current macroblock. In addition to this method, the predicted direction may be predicted by using a depth image of an adjacent encoded view or using color information. . In addition, the directional prediction method can be predicted by analyzing mode information, motion information, or edge components of adjacent blocks, not boundary pixels of adjacent blocks. Also, the equation for obtaining directional information can use various differences such as simple sum or variance, not the sum of squares of differnece.

According to an embodiment, the proposed encoding apparatus and the decoding apparatus are optimized for encoding and decoding a depth image. However, the encoding apparatus and the decoding apparatus are not limited to the depth image, but may be applied not only to the depth image but also to a medical image such as a black and white image and an x-ray image and a color image having flat characteristics.

Hereinafter, a method for encoding and decoding a depth image using an intra prediction mode in the depth image encoding / decoding apparatus according to the present invention configured as described above will be described with reference to the accompanying drawings.

5 is a flowchart illustrating a process of determining an encoding mode in a depth image encoding apparatus.

Referring to FIG. 5, in operation 510, the encoding apparatus receives a macroblock of a depth image to be encoded.

In operation 512, the encoding apparatus predicts the directionality of the current macroblock using neighboring macroblocks encoded for the intra prediction mode, and determines the directionality from the RDO perspective for the remaining intra candidate encoding modes except the intra prediction mode. .

In operation 514, the encoding apparatus intra-encodes the current macroblock according to intra candidate encoding modes.

In this case, the intra candidate encoding mode may include an intra prediction mode, and the intra prediction mode may include at least one of an intra skip mode, an intra direct mode, and an intra direction mode. The intra skip mode is a mode in which the predicted directional information and the residual signal are not encoded and transmitted to the decoding apparatus, and the intra direct mode is a mode in which the predicted directional information is not encoded and transmitted to the decoding apparatus. In this mode, the residual signal is not transmitted to the decoding apparatus.

In addition, the intra skip mode is the highest priority or the next highest priority among the priority of each of the intra candidate encoding modes and the encoding modes included in the inter candidate encoding modes. The types of intra candidate encoding modes may be classified according to a difference between at least one of the size of the macro block, whether the directional information is encoded, and whether there is a residual signal for the brightness component of the macro block.

In operation 516, the encoding apparatus checks whether the current macroblock is a macroblock corresponding to a P-picture or a B-picture.

If the current macroblock is not a macroblock corresponding to the P-picture or the B-picture, the encoding apparatus proceeds to step 520.

In operation 516, if the current macroblock is a macroblock corresponding to the P-picture or the B-picture, the encoding apparatus inter-encodes the current macroblock according to the inter candidate encoding modes in operation 518.

In operation 520, the encoding apparatus has the highest rate-distortion optimization (RDO) cost among the encoding modes in consideration of the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes. The small encoding mode is determined as the encoding mode of the current macroblock.

6 is a flowchart illustrating a process of generating a prediction block for decoding in a depth image decoding apparatus.

Referring to FIG. 6, when the decoding apparatus receives an encoded macroblock in step 610, the decoding apparatus verifies the encoding mode by checking encoding information included in the encoded macroblock in step 612.

In operation 614, the decoding apparatus determines whether an encoding mode of the encoded macroblock is an intra mode.

If the encoding mode is not the intra mode in operation 614, the decoding apparatus generates a prediction block according to the inter mode in operation 616.

In operation 614, if the encoding mode is the intra mode, the decoding apparatus determines whether the encoding mode is the intra prediction mode in operation 618.

In operation 618, if the encoding mode is the intra prediction mode, the decoding apparatus predicts the direction of the encoded macroblock using the neighboring macroblocks decoded in operation 620. In operation 622, the decoding apparatus generates a prediction block using the predicted direction.

In operation 618, if the encoding mode is not the intra prediction mode, that is, when the encoding mode of the encoded macroblock is one of the intra candidate encoding modes, the decoding apparatus checks directional information included in the encoding information in operation 624. The decoding apparatus generates a prediction block by using the directional information identified in step 626.

7 is a flowchart illustrating a process of setting a filtering strength for deblocking in a depth image encoding apparatus or a depth image decoding apparatus.

Referring to FIG. 7, in operation 710, the deblocking filtering unit of the depth image encoding apparatus or the depth image decoding apparatus checks whether at least one of both blocks is intra coded based on a boundary. In this case, the intra encoding includes an intra prediction mode, and the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an intra direction mode.

If at least one of both blocks is intra-encoded based on the boundary as a result of operation 710, the deblocking filtering unit determines whether the boundary between the two blocks is an edge of the macro block in step 720.

In operation 722, if the boundary is the edge of the macro block, the deblocking filtering unit sets the boundary strength (BS) to 4. At this time, the BS can be set to a value from 0 to 4, where 0 means no filtering and 4 indicates the strongest filtering strength.

If the boundary of step 720 is not the edge of the macro block, the deblocking filtering unit sets the BS to 3 in step 724.

If both blocks are inter-coded based on the boundary as a result of checking in step 710, the deblocking filtering unit determines whether there is residual data in any one of both blocks based on the boundary in step 730.

If the block having the residual data exists in step 730, the deblocking filtering unit sets the BS to 2 in step 732.

If the block having the residual data does not exist as a result of checking in step 730, the deblocking filtering unit checks in step 740 whether the motion difference between the two blocks is greater than or equal to the reference value or both blocks refer to different frames.

As a result of checking in step 740, when the blocks have a motion difference greater than a reference value or both blocks refer to different frames, the deblocking filtering unit sets the BS to 1 in step 742.

As a result of checking in step 740, when both blocks have a motion difference smaller than the reference value and both blocks refer to the same frame, the deblocking filtering unit sets the BS to 0 in step 744.

Referring to FIG. 7, in the case of intra coding, the filtering strength setting may be determined as one of steps 722 or 724, and in the case of inter encoding, the filtering strength setting may be determined as one of steps 732, 742, or 744. Comparing the determined value of the filtering strength, it can be seen that the case of intra coding is relatively stronger than the case of inter coding. Therefore, the intra prediction mode including at least one of the proposed intra skip mode, the intra direct mode, and the intra direction mode belongs to intra coding, and thus is filtered relatively stronger than inter coding. .

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

Claims (28)

An intra mode predictor for predicting the direction of the current macro block using the encoded neighboring macro blocks;
An intra encoder for intra coding the current macro block according to intra candidate encoding modes;
An inter encoder for inter encoding the current macroblock according to inter candidate encoding modes; And
In consideration of the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes, an encoding mode having a smallest rate-distortion optimization (RDO) cost among the encoding modes is described. An encoding mode determiner configured to determine an encoding mode of a current macroblock,
In the intra candidate encoding modes,
An intra prediction mode for intra coding using the predicted directionality is included.
Depth video encoding device.
The method of claim 1,
The intra prediction mode,
At least one of an intra skip mode, an intra direct mode, and an intra direction mode,
The intra skip mode is a mode in which the predicted directional information and the residual signal are not transmitted to a decoding apparatus.
The intra direct mode is a mode in which the predicted directional information is not transmitted to the decoding apparatus.
The intra direction mode is a mode in which the residual signal is not transmitted to the decoding apparatus.
Depth video encoding device.
The method of claim 2,
The intra skip mode,
The highest priority or the next highest priority among the priorities of the encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes.
Depth video encoding device.
The method of claim 1,
The predicted directionality is,
A vertical direction extending vertically from an upper boundary pixel of the current macroblock to generate a prediction block, a horizontal direction extending horizontally from a left boundary pixel of the current macroblock to generate a prediction block, and the current One of a DC direction for generating a prediction block with an average of the upper boundary pixel and the left boundary pixel of the macro block, and one of the plane directions for generating the prediction block by considering the upper boundary pixel and the left boundary pixel of the current macro block together. sign
Depth video encoding device.
The method of claim 1,
Types of the intra candidate encoding modes are
According to the difference between at least one of the type of the size of the macroblock, whether the directional information is encoded, and the residual signal for the brightness component of the macroblock
Depth video encoding device.
The method of claim 1,
The inter candidate coding modes refer to one motion vector per macro block,
The types of the inter candidate encoding modes are classified according to at least one of a kind of a size of a macroblock, whether a motion vector is encoded, and whether a residual signal is encoded.
Depth video encoding device.
The method of claim 1,
Priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes is:
Wherein the intra candidate encoding modes are set to have a higher priority than the inter candidate encoding modes.
Depth video encoding device.
The method of claim 1,
Priority between the intra candidate encoding modes,
The larger the macro block size is, the higher the priority is set.
The mode which does not encode directional information has a higher priority than the mode which encodes directional information,
The mode in which the residual signal is not encoded has a higher priority than the mode in which the residual signal is encoded.
Directional modes have a higher priority than non-directional modes.
Depth video encoding device.
The method of claim 1,
Priority between the inter candidate encoding modes,
The larger the macro block size is, the higher the priority is set.
Modes that do not encode motion vectors have a higher priority than modes that encode.
The mode in which the residual signal is not encoded has a higher priority than the mode in which the residual signal is encoded.
Depth video encoding device.
The method of claim 1,
The information on the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may include:
Pre-defined or defined when encoding the depth image is transmitted to the decoding device
Depth video encoding device.
The method of claim 1,
The encoding mode determiner,
Changing the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to a frequency at which encoding modes are finally determined;
Depth video encoding device.
The method of claim 1,
The current macro block is,
One of the macroblock of the depth image, the macroblock of the black and white image, the macroblock of the color image with flat characteristics, or the macroblock of the X-ray image,
Depth video encoding device.
The method of claim 1,
The apparatus may further include a deblocking filter configured to remove a blocking phenomenon by filtering the reconstructed block by using the prediction block.
The deblocking filter unit,
If any one of both blocks is intra coded based on a block boundary, the filtering is performed more strongly than when both blocks are inter coded based on a block boundary.
Depth video encoding device.
An encoding mode restoration unit which checks an encoding mode of the received encoded macroblock;
An intra mode reconstruction unit predicting the directionality of the encoded macro block using the decoded neighboring macro blocks when the encoding mode of the encoded macro block is an intra prediction mode; And
And an intra decoder configured to generate a predicted block using the predicted directionality.
Depth video decoding device.
15. The method of claim 14,
The encoding mode recovery unit,
Identifying the encoding mode of the encoded macroblock by using priority information of respective encoding modes included in intra candidate encoding modes and inter candidate encoding modes,
The intra prediction mode is included in the intra candidate coding modes.
Depth video decoding device.
16. The method of claim 15,
The encoding mode recovery unit,
Changing the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to a frequency of identified encoding modes,
Depth video decoding device.
16. The method of claim 15,
The information on the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may include:
Predetermined or defined when decoding the depth image and received by the encoding apparatus
Depth video decoding device.
15. The method of claim 14,
The predicted directionality is,
A vertical direction extending vertically from an upper boundary pixel of the coded macroblock to generate a prediction block, a horizontal direction extending horizontally from a left boundary pixel of the coded macroblock to generate a prediction block, and A DC direction for generating a prediction block by an average of the upper boundary pixel and the left boundary pixel of the coded macro block, and generating the prediction block by considering the upper boundary pixel and the left boundary pixel of the coded macro block together One of the plane directions
Depth video decoding device.
15. The method of claim 14,
The intra prediction mode,
At least one of an intra skip mode, an intra direct mode, and an intra direction mode,
The intra skip mode is a mode in which the predicted directional information and the residual signal are not provided from an encoding device.
The intra direct mode is a mode in which a residual signal is received from the encoding apparatus and the predicted directional information is not provided from the encoding apparatus.
The intra direction mode is a mode in which the residual signal is not transmitted to the decoding apparatus.
Depth video decoding device.
15. The method of claim 14,
The apparatus may further include a deblocking filter unit configured to remove a blocking phenomenon by filtering the reconstructed depth image by using the prediction block.
The deblocking filter unit,
If any one of the two blocks is intra coded based on the boundary of the block, the filtering is performed more strongly than the case where both blocks are inter coded based on the boundary of the block.
Depth video decoding device.
Predicting the direction of the current macro block using the encoded neighboring macro blocks;
Intra coding the current macro block according to intra candidate encoding modes;
If the current macro block is a macro block corresponding to a P-picture or a B-picture, inter encoding the current macro block according to inter candidate encoding modes; And
In consideration of the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes, an encoding mode having a smallest rate-distortion optimization (RDO) cost among the encoding modes is described. Determining the encoding mode of the current macroblock;
The intra candidate encoding mode is
An intra prediction mode for intra coding using the predicted directionality;
Depth image coding method.
The method of claim 21,
The intra prediction mode,
At least one of an intra skip mode, an intra direct mode, and an intra direction mode,
The intra skip mode is a mode in which the predicted directional information and the residual signal are not encoded and transmitted to the decoding apparatus.
The intra direct mode is a mode in which the predicted directional information is not encoded and transmitted to the decoding apparatus.
The intra direction mode is a mode in which the residual signal is not transmitted to the decoding apparatus.
Depth image coding method.
The method of claim 22,
The intra skip mode,
The highest priority or the next highest priority among the priorities of the encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes.
Depth image coding method.
The method of claim 21,
The predicted directionality is,
A vertical direction extending vertically from an upper boundary pixel of the current macroblock to generate a prediction block, a horizontal direction extending horizontally from a left boundary pixel of the current macroblock to generate a prediction block, and the current One of a DC direction for generating a prediction block with an average of the upper boundary pixel and the left boundary pixel of the macro block, and one of the plane directions for generating the prediction block by considering the upper boundary pixel and the left boundary pixel of the current macro block together. sign
Depth image coding method.
Identifying an encoding mode of an encoded macroblock using priority information of respective encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes;
Estimating the directionality of the encoded macroblock using the decoded neighboring macroblocks if the encoding mode of the encoded macroblock is an intra prediction mode:
Generating a prediction block using the predicted directionality;
Depth image decoding method.
26. The method of claim 25,
The information on the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may include:
Predetermined or defined when decoding the depth image and received by the encoding apparatus
Depth image decoding method.
26. The method of claim 25,
The predicted directionality is,
A vertical direction extending vertically from an upper boundary pixel of the coded macroblock to generate a prediction block, a horizontal direction extending horizontally from a left boundary pixel of the coded macroblock to generate a prediction block, and A prediction block is generated by considering a DC direction for generating a prediction block as an average of the upper boundary pixel and the left boundary pixel of the coded macro block, and the upper boundary pixel and the left boundary pixel of the coded macro block. One of the plane directions
Depth image decoding method.
26. The method of claim 25,
The intra prediction mode,
At least one of an intra skip mode, an intra direct mode, and an intra direction mode,
The intra skip mode is a mode in which the predicted directional information and the residual signal are not provided from an encoding device.
The intra direct mode is a mode in which a residual signal is received from the encoding apparatus and the predicted directional information is not provided from the encoding apparatus.
The intra direction mode is a mode in which the residual signal is not transmitted to the decoding apparatus.
Depth image decoding method.

Images