JP2014112748A - Image coding device and image decoding device - Google Patents

Abstract

An encoding device and a decoding device for encoding / decoding a distance image are provided.
An encoding apparatus includes: a dividing unit that divides an input distance image into rectangular blocks of a predetermined size; and a group of pixels that form an encoded block around an encoding target block divided by the dividing unit. The copy approximation means for approximating the block to be encoded by copying based on a predetermined copy format, and the block to be encoded is approximated by using the predetermined drawing format for the block to be encoded divided by the dividing means. A drawing format approximating means for storing depth value information of the used drawing format, a copy approximating means, a selecting means for selecting one of the drawing format approximating means, and a copy format selected for the encoding target block or Code word generation means for transmitting a code word generated based on the format identification information of the drawing format and the accumulated depth value information is provided. The decoding device performs decoding based on the encoding method used in the encoding device.
[Selection] Figure 1

Description

  The present invention relates to an image encoding device and an image decoding device.

  Accurate and efficient recording of the three-dimensional shape of the subject is an important theme, and various methods have been proposed. As one of the methods, a texture image that is a general two-dimensional image that represents the subject space with the color of each subject and the background, and an image that represents the subject space with the distance from the viewpoint to each subject and the background. There is a method of recording in association with two types of image data (hereinafter referred to as “distance image”). A distance image is an image that expresses a distance value (depth value) from a viewpoint to a corresponding point in a subject space for each pixel. This distance image can be acquired, for example, by a distance measuring device such as a depth camera installed in the vicinity of the camera that records the texture image. Alternatively, a distance image can be acquired by analyzing a plurality of texture images obtained by photographing with a multi-viewpoint camera, and many analysis methods have been proposed.

  In addition, distance values are expressed in 256 levels (8-bit luminance values) in the Moving Picture Experts Group (MPEG), which is a working group of the International Organization for Standardization / ISO / IEC, as a standard for distance images. The standard MPEG-C part3 is defined, and the standard distance image is an 8-bit grayscale image. In addition, since it is defined that a higher luminance value is assigned as the distance from the viewpoint is shorter, in a standard distance image, a subject located in front is expressed as white and a subject located in the back is expressed in black. As a feature of the distance image, it can be said that a single pixel value tends to appear in a wider area than the texture image. For example, even if a person wearing a fancy pattern is drawn on the texture image, the distance value of the clothes portion is almost constant in the distance image.

  If a texture image and a distance image representing the same subject space are obtained, the distance from the viewpoint of each pixel constituting the subject image drawn in the texture image is known from the distance image, so that the subject has the maximum depth. It can be restored as a three-dimensional shape expressed in 256 stages. Furthermore, by projecting the 3D shape onto the 2D plane geometrically, the original texture image is converted into a texture image in the subject space when the subject is photographed from another angle within a certain range from the original angle. It is possible to convert. That is, since a 3D shape can be restored when viewed from an arbitrary angle within a certain range by a set of texture images and distance images, a free viewpoint image of 3D shapes can be obtained by using multiple sets of texture images and distance images. Can be expressed with a small amount of data.

  By the way, the video compression standard H.264. As in the case of H.264, a technique for compressing and encoding video by efficiently eliminating temporal or spatial redundancy in the video is known (for example, Non-Patent Document 1). When each video of a texture video (video having a texture image as each frame) and a distance video (video having a distance image as each frame) is encoded by an encoding device using this technology, the redundancy that each video has Can be eliminated, and the data amount of each video transmitted to the decoding device can be further reduced.

  This H. In the H.264 standard, two conversion methods, an integer precision DCT conversion and a Hadamard conversion, are adopted as image conversion methods. Both of these are orthogonal transform methods. The integer precision DCT transform is an approximation of the real number precision DCT transform (ordinary DCT), and has a feature that the amount of calculation is smaller than that. The Hadamard transform has a smaller amount of computation than the integer precision DCT, and is used for transforming a block (DC block) generated by collecting only DC components. This orthogonal transform is used to calculate the correlation within the block. In the H.264 standard, it is used for a maximum of 16 × 16 pixel blocks. That is, the correlation between pixels in a 16 × 16 pixel block is used for information compression. These methods can compress information very efficiently within an appropriate bit rate range when compressing natural images. However, when the bit rate is extremely low, the image is blurred as a whole and block noise appears. have.

  H. The H.264 standard also employs a method called intra prediction encoding in order to further compress information. In this method, the pixel value of the encoding target block is predicted using an encoded pixel adjacent to the encoding target block. In the encoding target block, information compression is performed by orthogonally transforming the difference from the predicted value. The orthogonal transformation described above only uses the correlation within the block of 16 × 16 pixels at the maximum, but by using this method of predictive coding within the screen, the compression using the correlation with the adjacent pixels is performed. Can do.

“ITU-T Recommendation H.264”, International Telecommunication Union-Telecommunication Standardization Sector, March 2009

  However, H.C. When the compression coding technique standardized in the H.264 standard is applied to distance images, blur and block noise appear as described above in an environment where the bit rate is extremely low. This is because if the bit rate of encoding distance video is lowered, the number of bits allocated to transform coefficients obtained by orthogonal transform such as integer precision DCT transform and Hadamard transform is reduced, thereby increasing quantization distortion. This is because all the pixels in the block have only a DC component value.

  The distance video is usually used to generate a video of a viewpoint different from the viewpoint from which the texture video was shot after decoding, and blur and block noise are major factors that degrade the quality of the synthesized video. This is because the position / continuity of the contour portion of the subject is very important for the quality of the composite image in the distance video. When the contour of the subject of the texture image is continuous but the contour of the corresponding distance image is discontinuous, the contour of the subject of the synthesized texture image is also discontinuous. That is, H.I. The H.264 standard is a very efficient method for encoding video composed of natural images using an objective measure such as PSNR (Peak Signal-to-Noise Ratio) as an index. It cannot be said that it is an efficient method for special images that are used only to synthesize viewpoint images. Even when the PSNR is the same, the quality of the synthesized video is generally higher especially when the contour portion of the subject matches the texture video corresponding thereto.

  The present invention has been made in view of such circumstances, and an image encoding device capable of reducing the amount of encoded data of a distance image as compared with the conventional one and the encoding supplied from the image encoding device. An object of the present invention is to provide an image decoding apparatus that decodes a distance image from data.

  The present invention is an image encoding apparatus that encodes a distance image, a dividing unit that divides an inputted distance image into rectangular blocks of a predetermined size, and a surrounding area of an encoding target block divided by the dividing unit. A copy approximating unit for approximating the encoding target block by copying a pixel group constituting the encoded block based on a predetermined copying format, and a coding target block divided by the dividing unit in a predetermined drawing format Is used to approximate the encoding target block and to select one of drawing format approximating means for storing depth value information of the used drawing format, copy approximating means, and drawing format approximating means. Based on selection means, format identification information of the copy format or drawing format selected for the block to be encoded, and information on the accumulated depth value Characterized in that a codeword generating means for transmitting the generated codeword.

  The present invention is characterized by further comprising depth quantization means for quantizing the depth value of the block divided by the dividing means.

  The present invention is characterized in that the drawing format includes two depth values and defines only the boundary of the depth values.

  The present invention is characterized in that the selection means selects one of the copy formats, one of the drawing formats, or one combination of one copy format and one drawing format.

  The present invention is characterized in that one of the two depth values is determined from a pixel group constituting an encoded block around the encoding target block.

  The present invention is characterized in that a depth value determined from a pixel group constituting an encoded block around the encoding target block is determined in advance from a pixel position defined for each drawing format.

  The present invention determines in advance for each drawing format whether the depth value determined from the pixel group constituting the encoded block around the block to be encoded is applied to one of the two regions included in the drawing format. It is characterized by prescribing.

  According to the present invention, the depth value accumulated using the drawing format is set to a depth value that minimizes distortion with the input block when approximated using all depth values included in the encoding target block. It is characterized by.

  In the present invention, the combination method of one copy format and one draw format is to create an approximate block based on each copy format, and from the two regions included in each draw format, the region that is adopted from the surrounding pixel group It is characterized in that it is obtained by overwriting only the region opposite to the approximate block.

  The present invention is characterized in that the selection means selects, for all pixels of the encoding target block, one that minimizes distortion with the input block.

  In the present invention, the selection means weights the distortion of the input block with respect to all the pixels of the encoding target block as it approaches the end of the block, and selects the one that minimizes the weighted distortion. And

  In the present invention, the selection means weights all the pixels of the encoding target block with respect to the distortion of the input block only for the bottom row and the rightmost column of the block, and selects the weighted distortion that is minimized. It is characterized by that.

  According to the present invention, the selection means has one copy format, two copy formats, one drawing format, or one combination of one copy format and one drawing format. One of them is selected.

  In the present invention, the two selections from among the copying formats involve the order of copying, and after copying in the first copying format, the second copying format is used among the pixel groups used for copying. Only the pixel group in contact with a depth value different from the depth value held by each is used for the second copy format and overwritten.

  The present invention is characterized in that the depth value quantizing means is associated with a quantization parameter used in encoding a texture image paired with the distance image.

  The present invention is an image decoding apparatus for decoding an encoded distance image encoded by the image encoding apparatus according to any one of claims 1 to 15, wherein a codeword of the received encoded distance image is obtained. Analyzing means for analyzing, holding means for holding a depth value group obtained by analysis by the analyzing means, identification information in a format obtained by analyzing by the analyzing means, and predetermined depth based on the depth group Decoding means for restoring the distance image for each block using a predetermined drawing format up to a copy format is provided.

  The present invention causes a computer to function as the image encoding device according to any one of claims 1 to 15.

  According to the present invention, a computer is caused to function as the image decoding apparatus according to claim 16.

  The present invention is encoded image data of a distance image, and for each block of the image, the block is approximated or prepared in advance by copying an encoded pixel group around the block according to a preset copy format. The block is approximated by using the drawing format, and when one format is selected from the copy format and the drawing format, and the drawing format is selected, the depth value used for the selected format is stored, and the number of the selected format and The encoding is based on the accumulated depth value information.

  According to the present invention, an encoding device capable of reducing the code amount of encoded data of a distance image and a decoding device that decodes a distance image from encoded data supplied from the encoding device are realized. The effect that it can be obtained.

It is a block diagram which shows the structure of one Embodiment of this invention. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which extracted and showed one of the arrow groups shown in FIG. It is explanatory drawing which shows the state which performed the copy of a pixel. It is explanatory drawing which expressed the distance image typically. It is explanatory drawing which divided | segmented the distance image shown in FIG. 12 into the block. It is explanatory drawing which shows an example of a drawing format. It is explanatory drawing which shows an example of a drawing format. It is explanatory drawing which shows an example of the codeword which the codeword generation part 16 produces | generates with respect to one image. It is explanatory drawing which shows the structure (code word production | generation rule) of the code word shown in FIG. It is explanatory drawing which shows the block after an encoding. It is a block diagram which shows the modification of the apparatus structure shown in FIG.

  Hereinafter, an image encoding device and an image decoding device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes an image encoding device that inputs a distance image, performs an encoding process on the input distance image, and transmits the image via a transmission path. Reference numeral 2 denotes an image decoding apparatus that receives a distance image that has been subjected to encoding processing via a transmission path, decodes the distance image that has been subjected to encoding processing, and outputs a distance image. The image encoding device 1 includes a dividing unit 11, a process determining unit 12, a copy format determining unit 13, a drawing format determining unit 14, a depth value accumulating unit 15, and a code word generating unit 16. The image decoding apparatus 2 includes a codeword analysis unit 21, a depth value holding unit 22, a copy format development unit 23, and a drawing format development unit 24.

  First, the processing operation of the image encoding device 1 shown in FIG. 1 will be described. When the distance image is input, the dividing unit 11 divides the input distance image into a plurality of blocks. For example, block division is performed with 16 × 16 pixels as one block. Then, the dividing unit 11 outputs the blocks as an encoding target block to the processing determination unit in the raster scan order from the upper left block. The processing determination unit 12 determines which copy format is optimal for this encoding target block, which drawing format is optimal, and whether it is better to use the two in combination.

  Here, the copy format will be described. 2 to 9 are examples of types of copy formats. 2 to 9, a 16 × 16 pixel block located at the lower right is an encoding target block, and other blocks are encoded adjacent blocks. 2 to 9, each grid in each block represents a pixel, and a line with an arrow represents a copy destination of the pixel. For example, in FIG. 2, the encoding target block is created by copying the pixel in the bottom row of the encoded block adjacent thereto. Specifically, in the encoding target block, all the pixel groups located in the nth column from the left copy the nth pixel from the left in the bottom row of the adjacent block above. The same applies to the other drawings. The meaning of the arrow will be further described. For example, FIG. 10 shows one extracted from the group of arrows in FIG. In this case, as shown in FIG. 11, the pixel shown in black is copied as the ninth pixel from the left of the bottom row of the adjacent block.

  Neighboring pixels are H.264. As in the H.264 standard, it is effective to encode a distance image by copying it and using it as it is instead of using it as a predicted value. This is due to the following reason. That is, since the distance image represents the distance to the subject, the range of the same depth value is increased to some extent. Then, except for the contour portion of the subject, the value rarely changes abruptly in units of pixels. Therefore, the probability that the adjacent blocks have the same depth value is very high. In addition, by copying adjacent pixels in this way, when the contour is continuous from the adjacent block to the encoding target block, the continuity of the contour is maintained, and formats in various directions are prepared. By doing so, it is possible to deal with contours extending in various directions.

  FIG. 12 is a diagram schematically representing a distance image, and FIG. 13 is a diagram obtained by dividing FIG. 12 into blocks. In FIG. 13, one block represents a 16 × 16 pixel block. For example, a boundary line horizontally extending across the blocks B1 to B6 is the contour of the subject. When the blocks B2 to B4 are the encoding target blocks, it is obvious that the encoding target block can be approximated very well if the copy format shown in FIG. 3 is selected and applied.

  Furthermore, as in the case of the block B6 shown in FIG. 13, two types of copy formats may be selected along with the order in the case where the contour is connected from both the upper side and the left side. First of all, if a rule is applied in which copying is performed in the copying format shown in FIG. 3 and then copying is performed in the copying format shown in FIG. 2, and columns other than those already copied at the same depth value are overwritten. It is clear that the encoding target block can be approximated very well.

  Next, the drawing format determination unit 14 will be described. FIG. 14 shows an example of a drawing format. Each square represents a block of 16 × 16 pixels, and a line drawn therein represents a boundary of depth values. The drawing format P1 is a block composed of a single depth value. The drawing format P2 is a drawing format in which blocks are vertically divided at a ratio of 1: 3 in the horizontal direction. In the example shown in FIG. 14, the model assumes that the number of depth values included in one block is two. As a result, when three or more different depth values are included in one block, the depth is reduced to two depth values, but the number of formats is limited, so that the compression efficiency can be increased. Become. The same applies to other formats. The type of this format is not limited to that shown in FIG. 14, and for example, there may be a drawing format as shown in FIG.

  The drawing format defines only the boundary of the depth value, and for the depth value, it is determined which adjacent pixel that has been encoded is used for each drawing format. For example, with respect to the drawing format P2 shown in FIG. 14, the value of the pixel located on the uppermost side of the pixel column adjacent to the left side is set as the depth value of the left part, and Use the value as the depth value for the right part. Then, for each drawing format, a distortion (sum of squares of differences in depth values) with the input image is calculated.

  However, with this alone, when a depth value that is not included in the encoded adjacent block is included in the encoding target block, it cannot be accurately approximated. If there are other depth values that were not used in the previous calculation, the distortion is calculated in the same manner using each of the depth values. As described above, when a depth value not included in the adjacent block is used, the depth value is output to the depth value accumulation unit 15. As described above, the drawing format determination unit 14 selects the format that best approximates the boundary line of the depth value for the encoding target block.

  The process determination unit 12 determines an optimum usage method from the drawing format determination unit 14 and the copy format determination unit 12 described above. Specifically, the encoding target block is approximated by (1) a copy format determination unit using only a copy format determination unit, and (2) is approximated by a drawing format using only a drawing format determination unit. (3) Select one of approximation using both the copy format and the drawing format.

  Here, (3) a method of approximation using both the drawing format and the copy format will be described. In this case, for each of the copy formats shown in FIGS. 2 to 9, the drawing format shown in FIG. 14 is combined in a round robin manner, and the two regions included in the drawing format are not adopted from the surrounding pixel group. An encoding target block is created by overwriting only the area. Then, for each, the distortion with respect to the input image of all pixels in the encoding target block is calculated. The combination with the least distortion is the optimum combination.

  Next, which of the approximation methods (1), (2), and (3) is selected will be described. First, using the distortion for all the pixels of the encoding target block as a common reference, the distortion is calculated for each of the approximation methods (1), (2), and (3), and the one with the least distortion is selected. Such a method can be considered. However, since the method of the present invention has a feature that an encoded block is propagated to a subsequent block, the right end column or the boundary of the block, particularly the block to be encoded after that, It is important to reduce the distortion of the pixel located in the bottom row. Therefore, in determining which copy format is optimal, the distortion (sum of squares of the difference) between the input distance image in the pixel group included in the rightmost row of the encoding target block and the pixel group included in the lowermost row is determined. On the other hand, the weight is compared with the other parts and calculated for each format, and the format with the smallest distortion is set as the optimum format. Alternatively, a function that is weighted closer to the block boundary may be used. By doing so, it is possible to reduce the deviation of the outline at the boundary with the adjacent block on the right side or the lower side of the encoding target block, which is effective in maintaining the continuity of the outline.

  By the way, for the block located in the uppermost row in the image, there is no adjacent block on the upper side, so there are two types of formats of FIGS. 3 and 9 and three types of formats of FIGS. Calculation is performed for a total of five types. 5 to 7, the pixel group of the upper adjacent block is also referred to, but when the encoding target block is located in the uppermost row in the image, instead of the pixel group of the upper adjacent block, It is assumed that the uppermost pixel in the pixel group of the left adjacent block is copied. Similarly, for the block located in the left end column in the image, there is no adjacent block on the left side. Therefore, in addition to the three types of FIGS. 2, 4, and 8, the three types of FIGS. Calculations are made for a total of six types. As before, for the pixels that cannot be referred to, the pixel at the left end of the upper adjacent pixel group is used. In addition, for the block located in the right end column in the image, the right end pixel of the upper adjacent pixel group is used in the formats of FIGS. Here, since there is no encoded adjacent block for the upper left block, the copy format is not adopted.

  Next, the process determination unit 12 shows information indicating which of the above approximation methods (1), (2), and (3) is encoded, and the selected drawing format when encoded by the drawing format. When the depth value is output to the information and depth value storage unit 15, the output information is output to the codeword generation unit 16, and the information indicating the selected copy format is output to the codeword generation unit 16.

  The depth value storage unit 15 holds the input distance depth value until all the blocks included in one image are encoded by the processing determination unit 12, and one image is completely encoded. Then, the accumulated depth value group is output to the codeword generation unit 16. The codeword generation unit 16 assigns a codeword composed of binary values “0” or “1” to the input depth value information. FIG. 16 is an example of a code word generated by the code word generation unit 16 for one image. In FIG. 16, X1 to X5 each represent a code word consisting of binary values. Here, it is assumed that each of X1 to X5 has a fixed length. It is assumed that the fixed length of each bit is transmitted to the decoding side in advance or before encoding or the like and is known on the decoding side. FIG. 17 is a diagram showing the configuration of the code word shown in FIG. X1 represents the number of transmitted depth values for this encoding target image. For example, when an image having 1024 × 768 pixels is divided into 16 × 16 pixel blocks, since the total number of blocks is 3072, the depth value is 3072 at the maximum, which can be represented by 12 bits.

  X2 is the number of depth values arranged in order by the number represented by X1. For example, when the distance depth value is represented by a value of 0 to 255, each depth value can be represented by 8 bits. Next, the code word consisting of two of X3 and X4 is repeated by the number of blocks in the encoding target image. X3 is information indicating which of the approximation methods (1), (2), and (3) is used, and “0” when only the copy format is used (the above method (1)). When only the drawing format is used (the above-described method (2)) is “10”, and when both the copy format and the drawing format are used (the above-described method (3)) is “11”.

  X4 is identification information for identifying a copy format or a drawing format. Here, there are 8 types of copying formats shown in FIGS. 2 to 9 and 13 types of drawing formats shown in FIG. 14. When the approximation method (1) or approximation method (3) 3 is selected in X3, the copying format is selected. In addition, when the approximation method 2 is selected in X3, it represents the identification information of the drawing format. The codeword length is 3 bits for copy format representation and 4 bits for drawing format representation. X5 exists only when the approximation method (3) is selected in X3, and represents the identification information of the drawing format with the codeword length set to 4 bits.

  With reference to FIG. 13, an operation in which the process determination unit 12 encodes the input distance image for each block by the series of processing operations described above will be described. First, when the block B7 is input to the process determining unit 12 from the dividing unit 11, the process determining unit 12 determines the rendering format because there is no pixel of the encoded adjacent block to be copied. The block B7 is output to the unit 14 to obtain the optimum drawing format P1 (see FIG. 14). At this time, since there is no depth value to be referenced, the process determination unit 12 causes the drawing format determination unit 14 to output a single depth value (for example, value 60) constituting this block to the depth value storage unit 15. The depth value accumulation unit 15 accumulates this value 60 inside. In addition, the process determination unit 12 outputs identification information indicating that the drawing format P1 has been selected and information indicating that the value has been stored in the depth value storage unit 15 to the codeword creation unit 16. When the codeword creation unit 16 generates a codeword according to the codeword generation rule shown in FIG. 17, a codeword with X3 “10” and X4 “0000” is generated.

  Next, when the block on the right is input to the processing determination unit 12, assuming that the codeword shown in FIG. 17 is assigned, using the copy format can reduce the number of assigned bits. Therefore, the copy format shown in FIG. 3 is selected. Here, it is assumed that the copy format identification information shown in FIGS. 2 to 9 is assigned to 1 to 8 (“000” to “111”), respectively. In this case, as shown in FIG. 17, the first X3 is 0, and X4 is 001. Further, since the same processing is performed for the block on the right side, X3 is 0, and X4 is 001. This is repeated up to the block B1 (see FIG. 13). However, for the leftmost blocks of the second and third lines, the copy format is the format shown in FIG. Of the blocks included in the second and third rows, the block other than the leftmost block has the same distortion as the copy format shown in FIG. 2 and that shown in FIG. You may choose.

  Next, the block B1 is input to the process determination unit 12. The processing determination unit 12 causes the copy format determination unit 13 to calculate distortion for each format. This distortion may be distortion of all pixels in the block, or may be weighted distortion as described above. For this block, the distortion is constant in any format. Next, the process determination unit 12 causes the drawing format determination unit 14 to calculate distortion for each format. At this time, the drawing format P13 shown in FIG. 14 has the least distortion. The codeword is 10 for X3 and 1100 for X4 (the identification information of the drawing format P13 is 13). Then, the depth value (for example, value 90) included in the lower right corner of the block B1 is output to the depth value accumulation unit 15. The block after encoding at this time is a block B11 shown in FIG.

  Next, the block B2 to the block B4 are sequentially input to the processing determination unit 12 in the same manner as the processing operation described above. Each code word at that time is similarly selected from the block B2 to the block B4, and the copy format determination unit 13 selects the copy format shown in FIG. 7, and X3 is 0 and X4 is 001. The blocks after encoding at this time are like blocks B21 to B41 shown in FIG.

  Next, for the block B5, the copy format determination unit 13 selects the format shown in FIG. The codeword is 0 for X3 and 101 for X4. The block after encoding at this time is a block B51 shown in FIG. For an input block such as block B6, as described above, two types of copying formats are used together. Although the code word in such a case is not defined in X4 of the code word generation rule shown in FIG. 17, for example, after 1001, code words may be defined for each combination of two copy formats. Further, in the case of this block, even when the copy format (FIG. 7) and the drawing format (drawing format P3 in FIG. 14) are combined, the distortion is approximately the same. Either may be selected as long as the distortion is the same.

  In this way, after the encoding process is performed and the process is completed for one image, the depth value group accumulated in the depth value accumulation unit 15 and the total number thereof are output to the codeword generation unit 16, and X1 and X2 Are generated, and codewords X1 to X5 are transmitted as encoded distance images via a transmission path.

  Next, the processing operation of the image decoding device 2 shown in FIG. 1 will be described. The codeword analysis unit 21 receives the encoded distance image transmitted via the transmission path, divides the received encoded distance image into codewords X1 to X5, and transmits X1 and X2 to the depth value holding unit 22. And output X3 to X5 to the copy format developing unit 23. The depth value holding unit 22 sequentially outputs the depth values to the drawing format developing unit 24 as necessary. The copy format developing unit 23 performs drawing on the block encoded in the copy format, and outputs the result to the drawing format developing unit 24. The drawing format development unit 24 performs drawing on the block encoded in the drawing format. By such processing, the input distance image encoded on the encoding side is decoded.

  Description will be made along the specific example described above with reference to FIG. First, X1 and X2 are output to the depth value holding unit 22. At this time, the head of X2 is a depth value 60 when the block B7 is encoded in the drawing format. Similarly, the second is the depth value 90 when the block B1 is encoded in the drawing format.

  Next, the code word 100000 for the block B 7 is input to the drawing format development unit 24. The drawing format developing unit 24 analyzes that X3 is 10 and X4 is 0000, selects the drawing format P1, and acquires the first depth value 60 from the depth value holding unit 22. Then, the first block is drawn in the drawing format P1 using the depth value 60. The X3 and X4 are output to the copy format developing unit 23. The copy format developing unit 23 does not perform any processing since X3 is 10, and ends the decoding of this block.

  Next, the code word 0001 for the block on the right side of the block B 7 is input to the drawing format development unit 24. The drawing format developing unit 24 analyzes that X3 is 0 and X4 is 001, and outputs X3 and X4 to the copy format developing unit without performing any processing. The copy format developing unit uses the copy format shown in FIG. 7 to copy the pixel group included in the right end column of the block B7 in the horizontal direction. By sequentially performing such processing and decoding, the encoded distance image is decoded and the distance image is restored.

  Next, a modified example of the image encoding device 1 and the image decoding device 2 shown in FIG. 1 will be described with reference to FIG. FIG. 19 is a block diagram showing a modified configuration of the image encoding device 1 and the image decoding device 2 shown in FIG. The image coding apparatus 1 shown in FIG. 19 is different from the image coding apparatus 1 shown in FIG. 1 in that a depth quantization unit 17 and an entropy coding unit 18 are newly provided. Further, the image decoding device 2 shown in FIG. 19 is different from the image decoding device 2 shown in FIG. 1 in that an entropy decoding unit 25 is newly provided.

The depth quantization unit 17 quantizes the depth value of the block output from the dividing unit 11. The quantization step may be defined in advance. The quantization parameter qP used in the H.264 standard or the like may be used and associated with the value. Or you may make it match | combine with the value of qP used at the time of the encoding of the texture image corresponding to this distance image. For the association with the quantization parameter qP, for example, the quantization step at the maximum value 51 of the quantization parameter qP is determined (for example, 16 = 2 to the fourth power), and the quantization step for each qP is determined based on the quantization step. It may be. In this case, the quantization step s is
s = 2 ^{4 + floor (51-qP / 6)}
It can be expressed as. Here, floor (x) is a function representing the maximum integer not exceeding x.

  Thus, by appropriately quantizing the depth value in advance, the distance image can be simplified, and the accuracy of the subsequent encoding using the drawing format and the copy format can be improved. Further, by performing quantization in advance in this way, the number of bits allocated to X2 shown in FIG. 17 can be limited to a number of bits sufficient to express the quantization step s. It becomes compression of.

  The entropy encoder 18 further compresses the information by entropy encoding the encoded distance image generated by the codeword generator 16. The entropy decoding unit 25 decodes the encoded distance image that has been entropy encoded. As this method, arithmetic coding, lexicographic coding, adaptive arithmetic coding for adaptively updating each occurrence probability table and codebook, adaptive lexicographic coding, and the like can be applied. This method is a method for compressing and encoding a single distance image. Regarding the removal of redundancy between a plurality of images corresponding to different times, H. The H.264 standard can also be applied. That is, H.I. Only the I frame in the H.264 standard adopts the method of the present invention, and the B frame and P frame are H.264. The H.264 standard may be used.

  In the above description, the block size is described as 16 × 16 pixels. However, the block size is not limited to this and may be 8 × 8 pixels or 4 × 4 pixels. Further, H.C. Like the macro block of the H.264 standard, the size may be variable in units of blocks. In these cases, the copy format can be used without change, and the drawing format may be reduced as it is. Furthermore, the same method can be used for rectangular blocks such as 16 × 8 pixels and 8 × 16 pixels. In this case, as for the copy format, only the pixels corresponding to the rectangle may be used, and as the drawing format, a linear scale from square to those rectangles may be used. Determination of which of these various sizes and shapes to use is made, for example, from combinations of block shape, size, copy format or drawing format with the smallest distortion for each block of 16 × 16 pixels. This may be done by selecting the one that minimizes the distortion. In this case, an arrow extending from the dividing unit 11 to the process determining unit illustrated in FIG. 1 means a flow of a plurality of data output from the divided data to the process determining unit 12 after being divided into various shapes.

  As described above, an encoding device capable of reducing the code amount of encoded data of a distance image as compared with the prior art and a decoding device that decodes a distance image from encoded data supplied from the encoding device are realized. be able to.

  A program for realizing the functions of the processing units in FIGS. 1 and 19 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute an image. You may perform an encoding process and an image decoding process. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The present invention can be applied to applications where it is indispensable to encode / decode range images.

  DESCRIPTION OF SYMBOLS 1 ... Image coding apparatus, 11 ... Division | segmentation part, 12 ... Processing determination part, 13 ... Copy format determination part, 14 ... Drawing format determination part, 15 ... Depth value storage part , 16 ... codeword generation unit, 17 ... depth value quantization unit, 18 ... entropy coding unit, 2 ... image decoding device, 21 ... codeword analysis unit, 22 ... Depth value holding unit, 23 ... Copy format developing unit, 24 ... Drawing format developing unit, 25 ... Entropy decoding unit

Claims (19)

An image encoding device for encoding a range image,
Dividing means for dividing the input distance image into rectangular blocks of a predetermined size;
Copy approximating means for approximating the encoding target block by copying a group of pixels constituting the encoded block around the encoding target block divided by the dividing means based on a predetermined copy format;
A drawing format approximating unit for approximating the encoding target block by using a predetermined drawing format for the encoding target block divided by the dividing unit, and storing depth value information of the used drawing format;
A selection means for selecting one of the copy approximation means and the drawing format approximation means;
Codeword generation means for transmitting codewords generated based on the format identification information of the copy format or drawing format selected for the block to be encoded and the accumulated depth value information, Image encoding device.   The image coding apparatus according to claim 1, further comprising a depth quantization unit that quantizes a depth value of the block divided by the division unit.   The image encoding apparatus according to claim 1, wherein the drawing format includes two depth values and defines only a boundary between the depth values.   The selection means selects one from among copy formats, one from among drawing formats, or one combination of one copying format and one drawing format. The image encoding device according to any one of the above.   The image coding apparatus according to claim 3, wherein one of the two depth values is determined from a pixel group constituting a coded block around a coding target block.   6. The image encoding apparatus according to claim 5, wherein a depth value determined from a pixel group constituting an encoded block around the encoding target block is determined in advance from a pixel position defined for each drawing format. .   Predetermining in advance for each drawing format whether to apply a depth value determined from a group of pixels constituting an encoded block around the encoding target block to one of two regions included in the drawing format. The image coding apparatus according to claim 5, wherein   The depth value accumulated using the drawing format is set to a depth value that minimizes distortion with the input block when approximated using all depth values included in the encoding target block. The image encoding device according to claim 5.   The method of combining one copy format and one drawing format creates an approximate block based on each copy format, and is the opposite of the region adopted from the surrounding pixel group among the two regions included in each drawing format. The image coding apparatus according to claim 4, wherein only the area is obtained by overwriting the approximate block.   The image encoding device according to any one of claims 1 to 9, wherein the selection unit selects a pixel that minimizes distortion with respect to an input block with respect to all pixels of an encoding target block.   The selection means weights the distortion of the input block with respect to all the pixels of the encoding target block as it approaches the end of the block, and selects the one that minimizes the weighted distortion. The image encoding device according to any one of 1 to 9.   The selection means weights only the bottom row and right end column of the block with respect to the distortion of the input block with respect to all the pixels of the encoding target block, and selects the weighted distortion that is minimized. An image encoding device according to any one of claims 1 to 9.   The selection means selects one of the copy formats, two of the copy formats, one of the drawing formats, or one combination of one copying format and one drawing format. The image encoding device according to claim 1, wherein the image encoding device is an image encoding device.   The two selections from among the copying formats involve the order of copying, and after copying in the first copying format first, each of the pixel groups used for copying holds the second copying format. 14. The image encoding apparatus according to claim 13, wherein only the pixel group that is in contact with a depth value different from the depth value is used for the second copy format and overwritten.   3. The image coding apparatus according to claim 2, wherein the depth value quantization means is associated with a quantization parameter used when coding a texture image paired with the distance image. An image decoding device that decodes an encoded distance image encoded by the image encoding device according to any one of claims 1 to 15,
Analyzing means for analyzing the codeword of the received encoded distance image;
Holding means for holding a depth value group obtained by analysis by the analyzing means;
And a decoding unit that restores the distance image for each block by using a predetermined copy format or a predetermined drawing format based on the identification information obtained by the analysis unit and the depth group. An image decoding apparatus characterized by the above.   An image encoding program for causing a computer to function as the image encoding apparatus according to any one of claims 1 to 15.   An image decoding program that causes a computer to function as the image decoding apparatus according to claim 16.   Encoded data of distance image, and for each block of the image, the block is encoded by approximating the encoded pixel group around the block according to a preset copy format, or a drawing format prepared in advance is used. The block is approximated by selecting one format from the copy format and the drawing format, and when the drawing format is selected, the depth value used for it is stored, and the number of the selected format and the accumulated depth value are stored. Coded data characterized by being encoded based on the information.