WO2010082463A1 - Image encoding apparatus and image encoding method, image decoding apparatus and image decoding method - Google Patents

Abstract

Disclosed are encoding technology and decoding technology in which high image quality and encoding efficiency are achieved by division of characteristic portions of the image, such as edges, into suitable blocks. An image encoding apparatus (100) includes: a first block division section (101) that divides an input image into first blocks, a second block division section (111) that further divides the first blocks into a plurality of second blocks, and predicted image generating sections (112, 113) that generate predicted images by intra-screen prediction or inter-screen prediction of first blocks or second blocks. The second block division section (111) determines the block shape of the second blocks on the basis of edge information of adjacent encoded blocks in the same screen as the first blocks.

Description

Image encoding device, image encoding method, image decoding device, and image decoding method

The present invention relates to an encoding technique and a decoding technique with high image quality and good encoding efficiency.

Coding methods such as MPEG (Moving Picture Experts Group) system have been established as a technique for coding and recording or transmitting moving picture data such as TV signals with high efficiency. MPEG-1 standard, MPEG-2 standard, MPEG-4 It is an international standard encoding method as a standard. As a method for further improving the compression ratio, H. The H.264 / AVC (Advanced Video Video Coding) standard is defined. In the MPEG encoding method, one screen (one frame) is divided into units of macroblocks (16 pixels × 16 lines), and predictive encoding using correlation within the screen or between the screens is performed for each block. H. In the H.264 / AVC format, a macroblock is further divided into 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4 small blocks. Compared with MPEG-2, the prediction accuracy is improved by increasing the number of division types for blocks when encoding, and a high compression rate is realized.

Regarding block division technology, for example, Patent Document 1 discloses that an input image is divided into first blocks of M × N size, and further, the first block is divided into second blocks of m × n size, The division shape of the first block is determined based on the feature information extracted from the image of the block.

JP 2008-17305 A

If there is a characteristic part such as an edge in the image to be encoded, the code amount of the block including it will increase. According to the technique described in Patent Document 1, for example, the presence / absence of an edge is detected as feature information of the second block, and if there is an edge, it is divided into the shape of the second block, and if there is no edge, the division is performed. The shape of the first block is not changed. That is, the amount of codes can be reduced by treating an area where no edge exists as a large-sized block. However, in the technique described in Patent Document 1, since the block division method is limited to a rectangular shape, for example, an image including an edge in a diagonal direction cannot always be predicted with high accuracy. There is a problem that the image quality is lowered or the code amount is increased. This is also a problem common to the conventional MPEG systems.

The present invention has been made in view of the above problems, and an object of the present invention is to divide a block suitably for an image feature portion such as an edge, thereby enabling high-quality and efficient encoding technology and decoding. It is to provide the technology.

The present invention relates to an image encoding apparatus that encodes a difference image between a prediction image generated using a plurality of encoding modes and an input image divided into first blocks of a predetermined size. One block division unit, a second block division unit that further divides the first block into a plurality of second blocks, and the plurality of encodings for the first block or the second block One encoding mode is selected from among the modes, a prediction image generation unit that generates a prediction image by intra prediction or inter prediction, a subtraction unit that calculates a difference image between the prediction image and the input image, An encoded stream generation unit configured to generate an encoded stream by performing a frequency conversion process, a quantization process, and a variable-length encoding process on the difference image, and the second block dividing unit includes the first block dividing unit Based on the rock and the edge information of encoded adjacent blocks in the same screen to determine the block shape of the second block.

The present invention relates to an image coding method for coding a difference image between a prediction image generated using a plurality of coding modes and a first block having a predetermined size. One block dividing step, a second block dividing step for further dividing the first block into a plurality of second blocks, and the plurality of encodings for the first block or the second block One encoding mode is selected from among the modes, a prediction image generation step of generating a prediction image by intra prediction or inter prediction, a subtraction step of calculating a difference image of the prediction image and the input image, An encoded stream generation step of generating an encoded stream by performing a frequency conversion process, a quantization process, and a variable-length encoding process on the difference image, Lock dividing step determines the block shape of the second blocks based on the first block and the edge information of encoded adjacent blocks in the same screen.

The present invention provides an image decoding apparatus that generates a decoded image by decoding an encoded stream, and generates a differential image by performing variable length decoding processing, inverse quantization processing, and inverse frequency transform processing on the encoded stream. A difference image decoding unit, a prediction image generation unit that divides the block into blocks to be decoded according to the encoding mode information included in the encoded stream, and generates a prediction image by intra prediction or inter prediction, and the prediction image And an addition unit that generates the decoded image by adding the difference image, and the prediction image generation unit performs the decoding based on edge information of a decoded block adjacent to the decoding target block in the same screen. The block shape of the target block is determined.

The present invention provides an image decoding method for generating a decoded image by decoding an encoded stream, and generating a differential image by performing variable length decoding processing, inverse quantization processing, and inverse frequency transform processing on the encoded stream. A differential image decoding step, a prediction image generation step of dividing the block into blocks to be decoded according to the encoding mode information included in the encoded stream, and generating a prediction image by intra prediction or inter prediction, and the prediction image And an addition step of adding the difference image to generate a decoded image, wherein the predicted image generation step is based on edge information of a decoded block adjacent to the decoding target block in the same screen. The block shape of the target block is determined.

According to the present invention, it is possible to provide an image encoding device, an image decoding device, and a method thereof with high image quality and good encoding efficiency.

1 is a configuration block diagram showing an embodiment of an image encoding device of the present invention (Embodiment 1). The figure explaining the operation example of the edge information acquisition part 110. FIG. An example of filter coefficients used to calculate edge strength. Pixel value to be referenced for calculating edge strength. The figure which shows the mode of the block division by the 2nd block division part 111. FIG. The figure which shows the operation example of prediction between screens. The figure explaining the effect of the edge division | segmentation mode in inter prediction encoding. The figure explaining the effect of the edge division | segmentation mode in inter prediction encoding. The figure which shows the operation | movement of the conventional prediction in a screen. The figure which shows the operation | movement of the conventional prediction in a screen. The figure which shows the operation | movement of the prediction in a screen by edge division | segmentation mode. The figure explaining the effect of edge division mode in intra prediction encoding. The figure explaining the effect of edge division mode in intra prediction encoding. FIG. 10 is a diagram for explaining another example of operation of the edge information acquisition unit 110 (second embodiment). The block diagram (Example 3) which shows one Example of the image decoding apparatus by this invention. 10 is a flowchart illustrating a decoding processing procedure according to the third embodiment. The flowchart which shows the procedure of the prediction process in a screen in FIG. The flowchart which shows the procedure of the prediction process between screens in FIG.

Hereinafter, embodiments of the present invention will be described separately for an image encoding device and an image decoding device.

FIG. 1 is a configuration block diagram showing an embodiment (Embodiment 1) of an image encoding apparatus according to the present invention.
In the image encoding device 100, the first block dividing unit 101 divides the input image 150 into first input image blocks 151 each having 16 × 16 pixels. The subtractor 102 performs a subtraction process between the first input image block 151 and a predicted image block 155 output from the encoding mode selection unit 114 described later, and generates a difference image signal 152. The frequency conversion unit 103 performs frequency conversion such as discrete cosine transform (DCT conversion) on the differential image signal 152 for each image block, and outputs frequency conversion data. The quantization unit 104 quantizes the data and outputs quantized data 153. The variable-length encoding unit 105 multiplexes the encoded data obtained by variable-length encoding the quantized data 153 with the information 157 such as the encoding mode information and the motion vector output from the encoding mode selection unit 114 to generate an encoded stream. 154 is generated and output.

The inverse quantization unit 106 inversely quantizes the quantized data 153 output from the quantization unit 104, and the inverse frequency transform unit 107 performs inverse frequency transform on the quantized data 153 to output difference block data. The adder 108 adds the difference block data and the predicted image block 155 output from the encoding mode selection unit 114 to generate a decoded image 156. The frame memory 109 stores the decoded image 156.

The edge information acquisition unit 110 acquires edge information for the image read from the frame memory 109. The second block division unit 111 further finely divides the input image block 151 output from the first block division unit 101 based on the acquired edge information, and outputs a second input image block 160. Detailed operations of the edge information acquisition unit 110 and the second block division unit 111 will be described later.

The intra-screen prediction unit 112 generates an intra-screen prediction image from the reference image 159 read from the frame memory 109. The inter-screen prediction unit 113 generates an inter-screen prediction image from the reference image 159 read from the frame memory 109.

The encoding mode selection unit 114 selects either the prediction image generated by the intra-screen prediction unit 112 or the prediction image generated by the inter-screen prediction unit 113, and the selected prediction image 155 is subtracted by the subtractor 102 and the adder 108. Output to. Also, the coding mode selected by the coding mode selection unit 114 and block division information 157 are output to the variable length coding unit 105.

In this embodiment, by providing the edge information acquisition unit 110 and the second block dividing unit 111, the input image block 151 is divided into finer second input image blocks 160, and intra prediction and inter prediction are performed. The method is characterized in that an optimal predicted image 155 is created.

Hereinafter, operations of the edge information acquisition unit 110 and the second block division unit 111 will be described in detail.
FIG. 2 is a diagram for explaining an operation example of the edge information acquisition unit 110. In FIG. 2, a screen 210 is an encoding target screen, and is divided into first blocks having a predetermined size (16 × 16 pixels) by the first block dividing unit 101. An arrow 210a in the screen indicates the order of blocks to be encoded, and shows an enlargement of an encoding target block 201 to be encoded next and adjacent blocks 202 to 205 referred to for obtaining edge information. The adjacent blocks 202 to 205 taken up here are blocks that are adjacent in the same screen as the encoding target block 201 and have already been encoded according to the encoding order 210a, and are decoded images read from the frame memory 109. .

The edge information acquisition unit 110 selects an edge information calculation target pixel P_edge_cal (i) (i = 0 to 47) close to the encoding target block 201 in the adjacent blocks 202 to 205 in FIG. For these pixels, the horizontal edge strength f _h and the vertical edge strength f _v are acquired, and the edge strength f is calculated by [Equation 1]. A method of calculating the horizontal edge strength f _h and the vertical edge strength f _v will be described later.

Next, the pixel P_edge_cal (i0) when the edge intensity calculated by [Equation 1] is maximum is obtained from the pixels P_edge_cal (i) (i = 0 to 47). Then, the edge angle g at the pixel P_edge_cal (i0) is calculated by [Formula 2].

From the edge angle g calculated by [Formula 2], an edge straight line at the pixel P_edge_cal (i0) is calculated by [Formula 3]. Note that x _i and y _i are the horizontal and vertical coordinates of the pixel P_edge_cal (i0) when the upper left corner of the screen is the origin.

Next, the validity / invalidity of the obtained edge straight line is determined. When the edge straight line calculated by [Formula 3] intersects the encoding target block 201 as shown by the straight line 220 in FIG. 2, the coordinates (x _i , y _i ) of the pixel i at this time and the edge angle g (i And the edge information is output to the second block dividing unit 111.

On the other hand, if the edge straight line calculated by [Equation 3] does not intersect the encoding target block 201 as shown by the straight line 230 in FIG. 2, the coordinates and edge angle of the pixel i at this time are invalidated. Then, from among the pixels P_edge_cal (i) (i = 0 to 47), the pixel having the maximum edge strength f is calculated again, and the edge angle is calculated by [Expression 2]. Then, if the edge straight line obtained again intersects with the encoding target block 201, it is validated, and the edge information at this time is output to the second block dividing unit 111. In this way, the above processing is repeated until edge information where the obtained edge straight line intersects the encoding target block 201 is found.

Here, a specific method for calculating the edge strengths f _h and f _v in the above [Formula 1] will be described.
FIG. 3 is an example of filter coefficients used for calculating the edge strength. (A) is a Sobel filter, and (b) is a pre-wit filter. Vertical filter is a factor to determine the difference in the horizontal direction of the pixel values in order to calculate the vertical edge strength f _v. The horizontal filter is a coefficient for obtaining a difference between pixel values in the vertical direction in order to calculate the horizontal edge strength f _h .

FIG. 4 shows pixel values that are referred to for calculating the edge strength. A is a pixel value, and i and j represent horizontal and vertical coordinates with the upper left corner of the screen as the origin. Pixel A (i, j) is the central pixel, and eight adjacent pixels are used.
For example, the vertical edge strength f _v and the horizontal edge strength f _h in the pixel A (i, j) when the Sobel filter of (a) is used are calculated using [Formula 4] and [Formula 5].

The same calculation is performed when the pre-witt filter of (b) is used as the filter coefficient.

Hereinafter, the operation of the present embodiment will be described separately for the case of inter-screen prediction encoding and intra-screen prediction encoding.
<Inter prediction encoding>
The operation of the second block division unit 111 and the inter-screen prediction unit 113 when performing inter-screen prediction encoding will be described. First, the operation of the second block dividing unit 111 will be described.

FIG. 5 is a diagram showing a mode of block division by the second block division unit 111. H. When performing inter-screen prediction in H.264 / AVC, as shown in FIGS. 5 (1) to (7), there are four types of modes: 16 × 16 pixel mode, 16 × 8 mode, 8 × 16 mode, and 8 × 8 mode. The total of 4 blocks when divided and divided into 8 × 8 can be divided into 4 types of 8 × 8 mode, 8 × 4 mode, 4 × 8 mode, and 4 × 4 mode independently, for a total of 7 blocks There are street split modes. Among these, in the 16 × 16 mode (1), the first input image block 151 in FIG. 1 is output to the inter-screen prediction unit 113 in the same size (16 × 16). In other modes, for example, the (2) 16 × 8 mode, the second block dividing unit 111 divides the first input image block 151 into two blocks of 16 × 8 size and inter-screen prediction unit 113. Output to. Hereinafter, the block after being divided by the second block dividing unit 111 is referred to as a sub-block.

In this embodiment, the second block dividing unit 111 adds a new mode for dividing the first input image block 151 based on the edge information acquired by the edge information acquiring unit 110 in addition to the above seven block dividing modes. Provide. Hereinafter, this mode is referred to as “edge division mode (edge_base division)”. For example, when the edge information is indicated by the straight line 220 in FIG. 2, the edge division mode shown in FIG. 5 (8) is provided. That is, in the new edge division mode, the encoding target block is divided into two sub-blocks 0 and 1 along the division line 220a corresponding to the edge information. Since the dividing line 220a is determined according to the edge information, an arbitrary inclination can be taken at an arbitrary position in the block. That is, the shapes of the two sub blocks include not only a rectangle but also a trapezoid and a triangle, and allow the shapes to be different from each other. Further, all the pixels in the encoding target block 201 belong to only one of the sub-block 0 and the sub-block 1, and there are no pixels belonging to both sub-blocks or none of the sub-blocks. And

Next, the operation of the inter-screen prediction unit 113 will be described.
Inter-screen predictive encoding is a technique of referring to a reference image block in a reference screen that has already been encoded and encoding a difference from the encoding target image. H. In H.264 / AVC, three types of slices, ie, an I slice, a P slice, and a B slice, are used as a unit for switching the coding mode, and inter-frame predictive coding can be performed in the P slice and the B slice. is there. In the P slice, motion compensation inter-screen prediction encoding is performed with reference to one reference image block from a past screen on the encoded time axis. In addition, in the B slice, in addition to the left, similarly, the motion-compensated inter-prediction code is referred by referring to two reference image blocks in any combination from the past screen or the future screen on the time axis that has already been encoded. To do.

FIGS. 6A and 6B are diagrams illustrating an example of the operation of inter-screen prediction. FIG. 6A illustrates a case where the block division mode is the 16 × 8 mode, and FIG.
FIG. 6A shows the case of the block division mode = 16 × 8 mode, where the screen 600 is a coding target screen, the screen 610 is a past reference screen on the time axis with respect to the coding target screen 600, A screen 620 shows a future reference screen on the time axis with respect to the encoding target screen 600. The encoding target block 601 in the encoding target screen 600 is composed of two sub-blocks 602 and 603 having a size of 16 × 8 (rectangular).

When the encoding target screen 600 is a P slice, an area (16 × 8 size) similar to the sub-blocks 602 and 603 is searched from the past reference screen 610 on the time axis to obtain a predicted image. For example, a square error sum or a difference error sum is used to calculate the similarity between images. Here, blocks 612 and 613 are selected as the predicted images of the sub-blocks 602 and 603. On the other hand, when the encoding target screen 600 is a B slice, in addition to the blocks 612 and 613, the blocks 622 and 623 are selected from the future reference screen 620 on the time axis as predicted images of the sub-blocks 602 and 603. It is out. Then, it is possible to refer to the two blocks of the past and the future and use the average image as the predicted image.

FIG. 6B shows the case of the edge division mode added according to this embodiment. The screen 650 shows the encoding target screen, the screen 660 shows the past reference screen on the time axis with respect to the encoding target screen 650, and the screen 670 shows the future reference screen on the time axis with respect to the encoding target screen 650. Show. The encoding target block 651 in the encoding target screen 650 is composed of, for example, two trapezoidal sub-blocks 652 and 653 by edge division.

When the encoding target screen 650 is a P slice, the sub-blocks 652 and 653 select blocks 662 and 663 as reference images from the past reference screens 660 on the time axis. These blocks 662 and 663 are regions having the same shape as the respective divided shapes (here, trapezoids) of the sub-blocks 652 and 653. On the other hand, when the encoding target screen 650 is a B slice, the sub-blocks 652 and 653 select blocks 672 and 673 as reference images from the future reference screen 670 on the time axis in addition to the blocks 662 and 663. . These blocks 672 and 673 are also regions having the same shape as the divided shapes of the sub-blocks 652 and 653, respectively. Then, it is possible to refer to two blocks of the past and the future, and to set an image as an average value as a reference image.
As described above, inter-screen prediction in the edge division mode added in the present embodiment can be performed in the same manner as in the conventional rectangular block division mode, although the shape of the sub-block is different.

The inter-screen prediction unit 113 selects a block division mode with the highest coding efficiency from among the inter-screen predictions in each block division mode according to FIGS. 5 (1) to (8). As an evaluation method when selecting the block division mode, for example, an RD-Optimization method for selecting an optimum mode from the relationship between image quality distortion due to encoding and code amount is used. The details of RD-Optimization are described in Reference Document 1 below.
[Reference 1] G. Sullivan and T. Wiegand: “Rate-Distortion Optimization for Video Compression”, IEEE Signal Processing Magazine, vol.15 no.6, pp.74-90, 1998.

FIG. 7 and FIG. 8 are diagrams for specifically explaining the effect of the edge division mode in inter-screen predictive coding.
A screen 700 shown in FIG. 7B is an encoding target screen, and is a P slice here. A screen 710 illustrated in FIG. 7A is a past reference screen on the time axis with respect to the encoding target screen 700. In the encoding target screen 700, a block 701 indicates an encoding target block, and blocks 702 to 705 indicate encoded blocks adjacent to the encoding target block 701. FIG. 8 is an enlarged view of blocks 701 to 705 in FIG. 7B. In these screens, an image including a stationary building 751 and a moving automobile 752 (752 ′) is taken as an example.

7B is an image in which a part of both the background building 751 and the moving automobile 752 is mixed, and the boundary (edge) between the building and the automobile is in an oblique direction. Consider the correlation between block 701 and reference image 710 in (a). Since the building portion in the block 701 is a still image, it matches best with the block 711 in the same spatial position in the building 751 of the reference image 710. On the other hand, the car portion is a moving image, and best matches the block 712 at the corresponding position in the car 752 ′ of the reference image 710 (that is, the image 712 shifted by the amount of movement of the car on the two screens). Here, since the boundary between the building and the car is oblique, Regardless of which block division method (FIG. 5 (1) to (7)) at the time of inter-frame prediction in H.264 / AVC is selected, buildings and automobiles are mixed in the sub-block, and the encoding target image and the prediction image The difference, i.e., the prediction error, could not be made sufficiently small.

In contrast, when the edge division mode according to the present embodiment is used, the boundary (edge straight line) 800 between the background building and the foreground car from the adjacent block (block 702 in this case) encoded by the method shown in FIG. To extract. The encoding target block 701 is divided into two trapezoidal sub-blocks 801 and 802 along the edge straight line 800. That is, by dividing the building so that it is included in the sub-block 801 and the automobile in the sub-block 802, there is no sub-block in which both the building and the automobile are mixed. Then, the sub-block 801 generates a prediction image with reference to the block 711 on the reference screen 710. Sub-block 802 refers to block 712 to generate a predicted image. According to the present embodiment, there is no sub-block in which both a building and a car are mixed. The prediction error can be reduced more than the inter-screen prediction in H.264 / AVC.

<Intra-screen predictive coding>
Next, operations of the second block division unit 111 and the intra prediction unit 112 when performing intra prediction encoding will be described.

FIG. 2 shows a conventional intra-screen prediction operation in H.264 / AVC. Here, a case where the block size when performing intra prediction encoding is 4 × 4 pixel units is shown. H. In intra-frame prediction in H.264 / AVC, prediction is performed using a total of 13 decoded pixels in four encoded blocks adjacent to the left, upper left, upper, and upper right of the encoding target block. For prediction, as shown in FIGS. 9 (1), (2), (4) to (9), a mode for generating a predicted image by extending the 13 decoded pixels in the direction of the arrow, and FIG. As shown in 3), there is a mode for generating a predicted image from the average value of adjacent decoded pixels. Any one of these nine types of prediction modes (mode 0 to mode 8) is selected. In addition to the above, the block size for performing intra-screen predictive encoding includes cases of 8 × 8 pixel units and 16 × 16 pixel units. There are nine prediction modes in the case of 8 × 8 pixel units, as in FIG. 9, and this corresponds to an operation in which the block to be encoded in FIG. 9 is expanded from 4 × 4 to 8 × 8.

On the other hand, FIG. 10 shows an operation in the case of performing intra prediction encoding in units of 16 × 16 pixels. In the case of 16 × 16 pixel units, similarly to the 4 × 4 pixel unit, the decoded pixels adjacent to the block to be encoded are extended in the direction of the arrow to generate a predicted image. One of the modes 0 to 3) is selected.

Conventional H.264 As an operation corresponding to H.264 / AVC, the second block division unit 111 divides the first input image block 151 into three sizes (16 × 16 pixel size, 8 × 8 pixel size, 4 × 4) during intra prediction. The pixel size is divided into any sub-block. Further, the in-screen prediction unit 112 performs a process of selecting any one of the nine modes shown in FIG. 9 or the four types of modes shown in FIG.

In the present embodiment, the second block division unit 111 calculates the edge information acquisition unit 110 in addition to the above three block division modes (16 × 16 pixel size, 8 × 8 pixel size, 4 × 4 pixel size). A mode for dividing the first input image block 151 based on the edge information (edge division mode) is newly provided. The edge division mode is the same as in the case of the inter-picture prediction encoding, and the description thereof is omitted here.

Next, the operation of intra prediction encoding for subblocks divided in the edge division mode will be described.
FIG. 11 shows the operation of intra prediction in the edge division mode.
In FIG. 11A, based on the edge information calculated by the edge information acquisition unit 110, the second block division unit 111 converts the first input image block 151 (encoding target block) into sub-blocks 1101 and 1102. An example of division is shown below. Then, the intra prediction unit 112 performs intra prediction for each of the divided sub blocks. As the prediction mode, any one of the prediction mode 0 and the prediction mode 1 shown in FIG. 10 is applied.

11B and 11C show the intra-screen prediction operation for the sub-block 1101. FIG. Since the sub-block 1101 is adjacent to the upper and left encoded blocks, the prediction from the encoded upper block (prediction mode 0) and the prediction from the encoded left block (prediction mode 1) ) (B) is a case where the prediction mode 0 is applied to the sub-block 1101. This is similar to FIG. 10 (1), but in FIG. 10, prediction is applied to the entire encoding target block, whereas in FIG. 11 (b), prediction is applied only to the sub-block 1101. Different points apply.

FIG. 11C shows the case where the prediction mode 1 is applied to the sub-block 1101. This case is also similar to FIG. 10 (2), except that the prediction is applied only to the sub-block 1101 in FIG. 11 (c). Here, the pixel group 1104 in FIG. 11C is not in contact with the sub-block 1101, and when a decoded image corresponding to the position of the pixel group 1104 is used as it is as a predicted image, a prediction error is expected to increase. The In order to avoid this, it is effective to copy the value of the pixel 1103 that is in contact with both the pixel group 1104 and the sub-block 1101 to the pixel group 1104 and then use it as a predicted image.

FIG. 11D shows the case where the prediction mode 1 is applied to the sub-block 1102. Since the sub-block 1102 is not adjacent to the encoded upper block and is adjacent to only the left block, the prediction mode applicable in this example is only the prediction mode 1.

FIGS. 12 and 13 are diagrams for specifically explaining the effect of the edge division mode in the intra prediction encoding.
In FIG. 12, a screen 1200 indicates an encoding target screen, a block 1201 indicates an encoding target block, and blocks 1202 to 1205 indicate encoded blocks adjacent to the encoding target block 1201.

FIG. 13 is an enlarged view of blocks 1201 to 1205. The encoding target block 1201 is a block in which a part of the building 1251 and the automobile 1252 coexists, and the boundary between the building and the automobile is in an oblique direction. In block division mode and prediction mode (FIGS. 9 (1) to (9) and FIGS. 10 (1) to (4)) for intra-screen prediction in H.264 / AVC, the prediction error could not be reduced sufficiently. . On the other hand, when the edge division mode in this embodiment is used, the edge straight line 1300 between the building and the car is calculated in the encoded adjacent block 1202 by the method shown in FIG. Then, the encoding target block 1201 can be divided into a trapezoidal sub-block 1301 and a sub-block 1302 by the edge straight line 1300. For the sub-block 1301, in the prediction mode shown in FIG. 11B or FIG. 11C, for example, a prediction image is generated with reference to the adjacent block 1203. For the sub-block 1302, a prediction image is generated with reference to the adjacent block 1202 in the prediction mode shown in FIG. Thus, the conventional H.264. The prediction error can be reduced as compared with the intra-screen prediction in H.264 / AVC.
As a result, the intra-screen prediction image generated by the intra-screen prediction unit 112 and the inter-screen prediction image generated by the inter-screen prediction unit 113 are obtained, and both are input to the encoding mode selection unit 114.

The encoding mode selection unit 114 selects an optimal prediction image from the input intra prediction image and inter prediction image. As a selection method, for example, the RD-Optimization method described in Reference Document 1 is used. When the encoding mode selection unit 114 selects an intra prediction image, the block division method (16 × 16 pixel block, 8 × 8 pixel block, 4 × 4 pixel block, edge division) is used as the encoding mode information 157. Mode) information is output to the variable length coding unit 105. At this time, when the block division method is any one of the 16 × 16 pixel block, the 8 × 8 pixel block, and the 4 × 4 pixel block, the prediction mode information illustrated in FIG. 9 or 10 is included. On the other hand, when the inter-picture prediction image is selected by the encoding mode selection unit 114, the block division method (FIG. 5 (1) to (8)) and the motion vector information are used as the variable length code as the encoding mode information 157. To the conversion unit 105.

Thus, the sub-block division method in this embodiment is the conventional H.264 standard. Compared with the block division method in H.264 / AVC, the types of division shapes (trapezoids and triangles) and the degree of freedom of size increase, so that optimum division can be performed according to the edge of the image to be encoded, and prediction errors are reduced. There is an effect to. Further, in this embodiment, the number of modes increases with the addition of the block division method is only one, and the influence on the increase in code amount can be minimized. The shape of the edge division mode added in the present embodiment is not a fixed pattern, but derived from the feature of the edge of the adjacent block. This is to determine the block division method by applying the strength of the correlation in the spatial direction of the image, that is, the property that the feature of the image is similar to the spatial direction, and it is possible to reduce the prediction error.

As described above, in the image coding apparatus according to the first embodiment, in the coding apparatus that divides an input image into first blocks and further divides the input image into a plurality of second blocks, the same as the first block. The block shape of the second block is determined based on the edge information of the encoded block adjacent in the screen, thereby reducing the prediction error and realizing image coding with high image quality and good coding efficiency. It becomes possible to do. In the above embodiment, the block is divided based on the edge information. However, the same effect can be obtained by extracting an image feature portion having a large code amount and dividing the block based on this.

Embodiment 2 describes a case where edge information is acquired by a method different from that in Embodiment 1 when block division is performed based on edge information. Note that the image coding apparatus according to the second embodiment has the same configuration as that of the first embodiment (FIG. 1), and thus the description thereof is omitted. In the first embodiment, the edge information acquisition unit 110 in FIG. 1 performs a process of calculating edge information using a decoded image block adjacent in the same screen as the block to be encoded as shown in FIG. . On the other hand, in the second embodiment, the edge information acquisition unit 110 calculates the edge information of the block at the same spatial position in the encoded screen adjacent in the time direction to the encoding target block.

FIG. 14 is a diagram for explaining an operation example of the edge information acquisition unit 110. In FIG. 14, a screen 1400 is an encoding target screen, and a screen 1410 is an encoded screen that precedes the encoding target screen 1400 in time, and is a decoded image screen obtained by decoding this. In the encoding target screen 1400, a block 1401 indicates an encoding target block. A block 1411 indicates a block in the decoded image screen 1410 that has the same spatial position in the screen as the block 1401.

The edge information acquisition unit 110 acquires the horizontal edge strength f _h and the vertical edge strength f _v for the pixel P_edge_cal2 (i) (i = 0 to 224) in the block 1411 of FIG. ] To calculate the edge strength f. Next, a pixel P_edge_cal2 (i1) having the maximum edge strength f calculated by [Equation 1] is extracted from the pixels P_edge_cal2 (i) (i = 0 to 224). The edge angle g at the pixel P_edge_cal2 (i1) is calculated. Next, an edge straight line at the pixel P_edge_cal2 (i1) is calculated from [Equation 3] in Embodiment 1 from the edge angle g calculated by [Equation 2]. The edge straight line information calculated by [Formula 3] is applied to the encoding target block 1401 and divided into sub-blocks.

The sub-block division method in the present embodiment is the same as that in the first embodiment. Compared with the block division method in H.264 / AVC, the type of division shape (trapezoid or triangle) and the degree of freedom of size increase, so that optimum division is possible according to the edge (feature) of the image to be encoded, and prediction error There is an effect to reduce. Further, in this embodiment, the number of modes increases with the addition of the block division method is only one, and the influence on the increase in code amount can be minimized. The shape of the edge division mode added in this embodiment is not a fixed pattern, but is derived from image features of temporally adjacent screens. In this method, the block division method is determined by applying the strength of the correlation in the time direction of the image, that is, the property that the feature of the image is similar to the time direction, and the prediction error can be reduced.

As described above, in the image coding apparatus according to the second embodiment, in the coding apparatus that divides an input image into first blocks and further divides the input image into a plurality of second blocks and performs coding. On the other hand, the block shape of the second block is determined based on the edge information of the block at the same spatial position in the encoded screen adjacent in the time direction, thereby reducing the prediction pixels and encoding efficiency with high image quality. It is possible to realize good image coding. In the above embodiment, the block is divided based on the edge information. However, the same effect can be obtained by extracting an image feature portion having a large code amount and dividing the block based on this.

FIG. 15 is a configuration block diagram showing an embodiment (third embodiment) of an image decoding apparatus according to the present invention.
In the image decoding device 1500, the variable length decoding unit 1501 performs variable length decoding on the input encoded stream 1550, quantized data 1551 that is a frequency conversion component of the prediction difference, and the block size and motion vector. The encoding mode information 1555 necessary for prediction processing such as the above is acquired. The inverse quantization unit 1502 inversely quantizes the quantized data 1551 and outputs inversely quantized data. The inverse frequency transform unit 1503 performs inverse frequency transform on the inversely quantized data and outputs difference block data 1552. The adder 1504 adds difference block data 1552 and a predicted image block 1556 described later, and outputs a decoded image 1553. The decoded image 1553 is output from the image decoding device 1500 and stored in the frame memory 1505.

The edge information acquisition unit 1506 outputs edge information 1557 for the image read from the frame memory 1505. The intra prediction unit 1507 generates an intra prediction image from the reference image 1554 read from the frame memory 1505 when the encoding mode information 1555 acquired from the variable length decoding unit 1501 is the intra prediction encoding mode. The predicted image block 1556 is output to the adder 1504. The inter-screen prediction unit 1508 generates an inter-screen prediction image from the reference image 1554 read from the frame memory 1505 when the encoding mode information 1555 acquired from the variable length decoding unit 1501 is the inter-screen prediction encoding mode. The predicted image block 1556 is output to the adder 1504.

Hereinafter, operations of the edge information acquisition unit 1506, the intra-screen prediction unit 1507, and the inter-screen prediction unit 1508 will be described in detail.
The edge information acquisition unit 1506 is the same as the operation in the first or second embodiment. In the first embodiment, the screen 210 in FIG. 2 is the encoding target screen and the block 201 is the encoding target block. However, in the decoding operation in this embodiment, the screen 210 is the decoding target screen and the block 201 is the decoding target. Replace with block. Note that blocks 202 to 205 in FIG. 2 represent decoded images in the operation of the first embodiment, and can be applied to the present embodiment as they are. As a result, the edge straight line 220 for the decoding target block 201 is acquired, and the edge information 1557 is output. In the second embodiment, the screen 1400 in FIG. 14 is the encoding target screen and the block 1401 is the encoding target block. However, in the decoding operation according to the present embodiment, the screen 1400 is the decoding target screen and the block 1401 is the decoding target block. To read as

Next, the intra prediction unit 1507 generates a prediction image block 1556 when the encoding mode information 1555 acquired from the variable length decoding unit 1501 is the intra prediction encoding mode. The intra-screen prediction unit 1507 acquires block division mode and prediction mode information from the coding mode information 1555. The block division mode is any one of 16 × 16 pixel size, 8 × 8 pixel size, 4 × 4 pixel size, and edge division mode. When the block division mode is 16 × 16 pixel size, the prediction mode is acquired from the encoding mode information 1555 next. The prediction mode in this case is one of prediction modes 0 to 3 shown in FIG. Even when the block division mode is 8 × 8 pixel size or 4 × 4 pixel size, the prediction mode is acquired from the coding mode information 1555. The prediction mode in this case is one of prediction modes 0 to 8 shown in FIG.

When the block division mode is the edge division mode, the shape of each sub-block of the decoding target block is obtained based on the edge information 1557 acquired by the edge information acquisition unit 1506. Next, the prediction mode of each sub block is acquired from the coding mode information 1555. The prediction mode is any one of prediction mode 0 to prediction mode 1 shown in FIG. The intra-screen prediction unit 1507 generates an intra-screen prediction image corresponding to the prediction mode based on the reference image 1554 read from the frame memory 1505, and outputs it as a prediction image block 1556.

On the other hand, the inter-screen prediction unit 1508 generates a prediction image block 1556 when the encoding mode information 1555 acquired from the variable length decoding unit 1501 is the inter-screen prediction encoding mode. The inter-screen prediction unit 1508 acquires block division mode and motion vector information from the coding mode information 1555. The block division mode is one of FIGS. 5 (1) to (8). When the block division mode is (1) to (7) in FIG. 5, next, the motion vector information of each sub-block is acquired from the encoding mode information 1555.

When the block division mode is the edge division mode of FIG. 5 (8), the shape of each sub-block of the decoding target block is obtained based on the edge information 1557 acquired by the edge information acquisition unit 1506. Next, motion vector information of each sub block is acquired from the encoding mode information 1555. The inter-screen prediction unit 1508 generates an inter-screen prediction image corresponding to the prediction mode based on the reference image 1554 read from the frame memory 1505 and outputs it as a prediction image block 1556.

FIG. 16 is a flowchart showing the decoding processing procedure for one frame in the present embodiment. First, the following processing is performed on all blocks in one frame (S1601). That is, the prediction difference is decoded by performing variable length decoding processing (S1602), inverse quantization processing (S1603), and inverse frequency transform processing (S1604) on the input stream. Subsequently, it is determined in which method (intra-screen, between screens) the target block is predictively encoded (S1605). In accordance with the determination result, an intra-screen prediction process (S1606) or an inter-screen prediction process (S1607) is executed to generate a predicted image. A decoded image generation process (S1608) is performed from the generated prediction image and the prediction difference generated in the inverse frequency conversion process (S1604). The above processing is performed for all the blocks in the frame (S1609), and decoding for one frame of the image is completed.

FIG. 17 is a flowchart showing in more detail the procedure of the intra-screen prediction process (S1606) shown in FIG. First, it is determined whether the block division mode is the edge division mode (S1701). In the edge division mode (when S1701 is YES), the block division process (S1702) based on the edge information is performed, and the screen is obtained from the coding mode information decoded in the variable length decoding process (S1602) of FIG. The intra prediction mode is acquired (S1703). On the other hand, when the block division mode is not the edge division mode (when S1701 is NO), the block division mode is acquired from the coding mode information decoded in the variable length decoding process (S1602) of FIG. 16 (S1704). Further, the in-screen prediction mode is acquired (S1705). Next, an intra-screen prediction image generation process (S1706) is performed from the block division mode and the intra-screen prediction mode acquired by any of the above methods, and the intra-screen prediction image generation process ends.

FIG. 18 is a flowchart showing in more detail the procedure of the inter-screen prediction process (S1607) shown in FIG. First, it is determined whether the block division mode is the edge division mode (S1801). In the edge division mode (when S1801 is YES), block division processing (S1802) based on edge information is performed, and motion is performed from the coding mode information decoded in the variable length decoding processing (S1602) of FIG. A vector is acquired (S1803). On the other hand, when the block division mode is not the edge division mode (when S1801 is NO), the block division mode is acquired from the coding mode information decoded in the variable length decoding process (S1602) of FIG. 16 (S1804). Further, a motion vector is acquired (S1805). Next, the inter-screen prediction image generation process (S1806) is performed from the block division mode and the motion vector information acquired by any of the above methods, and the inter-screen prediction image generation process ends.

As described above, when the block division mode of the decoding target block is the edge division mode shown in FIG. 5 (8), the decoding target block is converted into a trapezoidal or triangular shape based on the edge information 1557 output from the edge information acquisition unit 1506. The prediction image corresponding to each sub-block is generated by dividing into sub-blocks including the shape. In this case, since the edge information acquired at the time of decoding is the same as the edge information acquired at the time of encoding, a sub-block having the same shape as that at the time of encoding can be generated again. Therefore, the image decoding apparatus can perform decoding only by knowing that the block division method of the decoding target block is the edge division mode.

As described above, the image decoding apparatus according to the present embodiment is the same as that for encoding an input stream that is encoded by dividing an input image into first blocks and then dividing the input image into a plurality of second blocks. In addition, the division shape of the second block is determined based on the edge information of the decoded block adjacent in the same screen as the first block, and decoding is performed. Alternatively, as in the encoding, the division shape of the second block is determined based on the edge information of the block at the same spatial position in the encoded screen adjacent to the first block in the time direction and decoded. Is. Therefore, it is possible to perform a decoding process faithfully corresponding to the encoding process in the image encoding apparatus, and to obtain a high-quality decoded image. In the above embodiment, the block is divided based on the edge information. However, the same effect can be obtained by extracting an image feature portion having a large code amount and dividing the block based on this.

DESCRIPTION OF SYMBOLS 100 ... Image coding apparatus 101 ... 1st block division part 102 ... Subtractor 103 ... Frequency conversion part 104 ... Quantization part 105 ... Variable length coding part 106 ... Inverse quantization part 107 ... Inverse frequency conversion part 108 ... Addition 109: Frame memory 110 ... Edge information acquisition unit 111 ... Second block division unit 112 ... Intra-screen prediction unit 113 ... Inter-screen prediction unit 114 ... Coding mode selection unit 1500 ... Image decoding device 1501 ... Variable length decoding Unit 1502 ... Inverse quantization unit 1503 ... Inverse frequency conversion unit 1504 ... Adder 1505 ... Frame memory 1506 ... Edge information acquisition unit 1507 ... In-screen prediction unit 1508 ... Inter-screen prediction unit.

Claims (16)

In an image encoding device that encodes a difference image with a predicted image generated using a plurality of encoding modes for an input image,
A first block dividing unit that divides an input image into first blocks of a predetermined size;
A second block dividing unit that further divides the first block into a plurality of second blocks;
A prediction image generation unit that selects one encoding mode from among the plurality of encoding modes for the first block or the second block and generates a prediction image by intra prediction or inter prediction; ,
A subtraction unit that calculates a difference image between the predicted image and the input image;
An encoded stream generation unit that generates an encoded stream by performing frequency conversion processing, quantization processing, and variable-length encoding processing on the difference image;
The second block dividing unit determines a block shape of the second block based on edge information of an encoded block adjacent to the first block in the same screen. apparatus. In an image encoding device that encodes a difference image with a predicted image generated using a plurality of encoding modes for an input image,
A first block dividing unit that divides an input image into first blocks of a predetermined size;
A second block dividing unit that further divides the first block into a plurality of second blocks;
A prediction image generation unit that selects one encoding mode from among the plurality of encoding modes for the first block or the second block and generates a prediction image by intra prediction or inter prediction; ,
A subtraction unit that calculates a difference image between the predicted image and the input image;
An encoded stream generation unit that generates an encoded stream by performing frequency conversion processing, quantization processing, and variable-length encoding processing on the difference image;
The second block dividing unit determines a block shape of the second block based on edge information of a block at the same spatial position in an encoded screen adjacent to the first block in the time direction. An image encoding device characterized by the above. The image encoding device according to claim 1 or 2,
An image comprising an edge information acquisition unit for obtaining edge information in a horizontal direction and a vertical direction for an edge information acquisition target pixel as the edge information, and acquiring information on a pixel position and an edge angle at which the edge intensity is maximum. Encoding device. The image encoding device according to claim 1 or 2,
The image coding apparatus, wherein the coded stream includes coding mode information indicating that the second block division method is based on the edge information. In an image encoding method for encoding a difference image with a predicted image generated using a plurality of encoding modes for an input image,
A first block dividing step of dividing the input image into first blocks of a predetermined size;
A second block dividing step of further dividing the first block into a plurality of second blocks based on edge information of an encoded block adjacent to the first block;
A prediction image generation step of selecting one encoding mode from among the plurality of encoding modes for the first block or the second block and generating a prediction image by intra prediction or inter prediction; ,
A subtraction step of calculating a difference image between the predicted image and the input image;
An encoded stream generating step for generating an encoded stream by performing a frequency conversion process, a quantization process, and a variable length encoding process on the difference image;
The second block dividing step determines the block shape of the second block based on edge information of an encoded block adjacent to the first block in the same screen. Method. In an image encoding method for encoding a difference image with a predicted image generated using a plurality of encoding modes for an input image,
A first block dividing step of dividing the input image into first blocks of a predetermined size;
A second block dividing step of further dividing the first block into a plurality of second blocks;
A prediction image generation step of selecting one encoding mode from among the plurality of encoding modes for the first block or the second block and generating a prediction image by intra prediction or inter prediction; ,
A subtraction step of calculating a difference image between the predicted image and the input image;
An encoded stream generating step for generating an encoded stream by performing a frequency conversion process, a quantization process, and a variable length encoding process on the difference image;
The second block dividing step determines a block shape of the second block based on edge information of a block at the same spatial position in an encoded screen adjacent to the first block in the time direction. An image encoding method characterized by the above. The image encoding method according to claim 5 or 6,
An image comprising an edge information acquisition step of obtaining edge information in a horizontal direction and a vertical direction for the edge information acquisition target pixel as the edge information, and acquiring information on a pixel position and an edge angle at which the edge intensity is maximum. Encoding method. The image encoding method according to claim 5 or 6,
The image coding method, wherein the coded stream includes coding mode information indicating that the second block division method is based on the edge information. In an image decoding apparatus that decodes an encoded stream to generate a decoded image,
A difference image decoding unit that generates a difference image by performing variable length decoding processing, inverse quantization processing, and inverse frequency transform processing on the encoded stream;
A prediction image generation unit that divides into blocks to be decoded according to the encoding mode information included in the encoded stream, and generates a prediction image by intra prediction or inter prediction;
An adder that adds the predicted image and the difference image to generate a decoded image;
The prediction image generation unit determines the block shape of the decoding target block based on edge information of a decoded block adjacent in the same screen as the decoding target block. In an image decoding apparatus that decodes an encoded stream to generate a decoded image,
A difference image decoding unit that generates a difference image by performing variable length decoding processing, inverse quantization processing, and inverse frequency transform processing on the encoded stream;
A prediction image generation unit that divides into blocks to be decoded according to the encoding mode information included in the encoded stream, and generates a prediction image by intra prediction or inter prediction;
An adder that calculates the decoded image by adding the predicted image and the difference image;
The predicted image generation unit determines a block shape of the decoding target block based on edge information of a block at the same spatial position in a decoded screen adjacent to the decoding target block in the time direction. An image decoding apparatus. The image decoding device according to claim 9 or 10,
An image comprising an edge information acquisition unit for obtaining edge information in a horizontal direction and a vertical direction for an edge information acquisition target pixel as the edge information, and acquiring information on a pixel position and an edge angle at which the edge intensity is maximum. Decryption device. The image decoding device according to claim 9 or 10,
The predictive image generation unit uses coding mode information included in the coded stream to indicate that a block division method at the time of coding is based on edge information. . In an image decoding method for generating a decoded image by decoding an encoded stream,
A differential image decoding step of performing a variable length decoding process, an inverse quantization process, and an inverse frequency transform process on the encoded stream to generate a differential image;
A prediction image generation step of dividing into blocks to be decoded according to an encoding mode included in the encoded stream, and generating a prediction image by intra prediction or inter prediction;
An addition step of adding the predicted image and the difference image to generate a decoded image;
The predictive image generating step determines the block shape of the decoding target block based on edge information of a decoded block adjacent in the same screen as the decoding target block. In an image decoding method for generating a decoded image by decoding an encoded stream,
A differential image decoding step of performing a variable length decoding process, an inverse quantization process, and an inverse frequency transform process on the encoded stream to generate a differential image;
A prediction image generation step of dividing into blocks to be decoded according to an encoding mode included in the encoded stream, and generating a prediction image by intra prediction or inter prediction;
An addition step of calculating the decoded image by adding the predicted image and the difference image,
The predicted image generation step determines the block shape of the decoding target block based on edge information of the block at the same spatial position in a decoded screen adjacent in the time direction to the decoding target block. An image decoding method. The image decoding method according to claim 13 or 14,
An image comprising an edge information acquisition step of obtaining edge information in a horizontal direction and a vertical direction for the edge information acquisition target pixel as the edge information, and acquiring information on a pixel position and an edge angle at which the edge intensity is maximum. Decryption method. The image decoding method according to claim 13 or 14,
The predictive image generation step uses encoding mode information indicating that a block division method at the time of encoding included in the encoded stream is based on edge information. .

Images