JP2007174569A - Encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an encoding technology capable of solving a problem of an increase in a coding amount caused by information for particularizing scan sequence added to coded data in scanning DCT blocks to apply variable length encoding to an image. <P>SOLUTION: An image reduction section 12 reduces an input image to give the result to a basic layer processing block 120. A prediction section 24b applies in-frame prediction to an image frame and gives an in-frame prediction error image to a DCT section 26b. The DCT section 26b applies discrete cosine transform (DCT) to the in-frame prediction error image and gives an obtained DCT coefficient to a quantization section 28b. The quantization section 28b quantizes the DCT coefficient and gives the result to a variable length coding section 30b. The variable length coding section 30b uniquely determines a scan sequence being one of ordinary zigzag scan, horizontal priority scan, and vertical priority sequence on the basis of an aspect scaling ratio of the image to scan the DCT coefficient in the scan sequence determined in the DCT block thereby applying the variable length encoding to the image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an encoding method for encoding a moving image.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. Also, in one stream, images with different image quality (for example, high and low image quality), different resolution (for example, high and low resolution), and different frame rates (for example, high and low frame rates) depending on the code amount H. can be compressed and decompressed. There is a next-generation image compression technique such as SVC (Scalable Video Coding), which is being standardized as an extension of H.264 / AVC.

  SVC, the next-generation image compression technology, encodes moving images with various scalability such as spatial scalability, temporal scalability, and SNR scalability so that moving images can be played at multiple different resolutions, frame rates, and image quality. Turn into. Coding can be performed by arbitrarily combining these scalability, and the scalability function of SVC is very flexible.

Patent Document 1 discloses a video encoding and decoding apparatus for an arbitrarily shaped image in which a reduction in encoding efficiency is suppressed in a scalable encoding method in which resolution and image quality can be varied in multiple layers.
Japanese Patent Laid-Open No. 9-182084

  In image compression coding, an image is converted into a spatial frequency domain, and after quantization, variable length coding is performed. Here, in the block of the spatial frequency transform coefficient, generally, the transform coefficient is variable-length encoded by scanning from the low frequency component to the high frequency component in order. When changing the scan order of transform coefficients within a block for each block, it is necessary to include information for specifying the scan order for each block in the image code data, which increases the amount of code and increases the image coding efficiency. There is a problem that decreases.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image encoding technique with high encoding efficiency when the scan order in encoding is variable.

  In order to solve the above-described problem, an encoding method according to an aspect of the present invention includes a spatial frequency coefficient block obtained by performing spatial frequency conversion on an image, the degree of horizontal reduction of the image, and the vertical direction of the image. The encoded data of the image is generated by performing variable length encoding of the spatial frequency coefficient by scanning in a scan order determined from a vertical / horizontal scaling ratio which is a ratio of the degree of direction reduction.

  Here, the “image” may be a still image or a picture constituting a moving image. A “picture” is an encoding unit including a frame, a field, a VOP (Video Object Plane), and the like.

  Examples of the spatial frequency conversion of an image include a discrete cosine transform and a discrete wavelet transform.

  According to this aspect, since the scan order can be specified from the vertical / horizontal scaling ratio, it is possible to objectively provide a reference for determining the scan order when the selection of the scan order is arbitrary in the variable length coding of the image. it can.

  The encoded data may be generated without including the information specifying the scan order in the encoded data. According to this, since it is not necessary to encode the image by providing the selection bit of the scan order, the code amount of the image can be reduced.

  When the vertical / horizontal scaling ratio indicates that the degree of reduction in the vertical direction is greater than the degree of reduction in the horizontal direction, scanning may be performed in a scan order that prioritizes the vertical direction. When the vertical / horizontal scaling ratio indicates that the degree of reduction in the horizontal direction is greater than the degree of reduction in the vertical direction, scanning may be performed in a scan order that prioritizes the horizontal direction. When the image is greatly reduced in either the vertical direction or the horizontal direction, the high-frequency component becomes significant in the vertical direction or the horizontal direction. If scanning is performed with priority on a direction with a large reduction degree, significant spatial frequency coefficients can be continuously variable-length encoded in a direction in which more high-frequency components appear, so that encoding efficiency can be increased.

  When zigzag scanning the block of the spatial frequency coefficient from a low frequency component to a high frequency component, the number of pixels scanned continuously in the vertical direction and the number of pixels scanned continuously in the horizontal direction It may be determined from the vertical / horizontal scaling ratio.

  “The number of pixels scanned continuously in the vertical direction” indicates how many pixels are scanned continuously in the vertical direction when performing zigzag scanning. The “number” indicates how many pixels are continuously scanned in the horizontal direction when performing zigzag scanning. By determining the number of pixels to be continuously scanned in the vertical direction and the horizontal direction, it is possible to determine how much priority is given to the zigzag scan in the vertical direction or the horizontal direction.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, when changing the scanning order in the case of encoding, the encoding efficiency of an image can be improved.

  FIG. 1 is a configuration diagram of an encoding apparatus 100 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding apparatus 100 according to the present embodiment conforms to SVC (Scalable Video Coding), which is a next-generation image compression technology, to spatial (spatial) scalability, temporal scalability, and SNR (signal to noise) for moving images. ratio) Perform “scalable coding” in which coding is performed with at least one of scalability.

  For the coding of moving images, the standards (MPEG-1, MPEG-2 and MPEG-2) of the MPEG (Moving Picture Experts Group) standardized by ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission) MPEG-4), an H.264 standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (the official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264, respectively) are used.

  In the embodiment, a frame is used as an example of a moving image encoding unit, but the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The encoding apparatus 100 receives an input of a moving image in units of frames, performs scalable encoding of the moving image, and outputs an encoded stream of the moving image. The input moving image frame is stored in the frame memory and read / written by each processing unit related to encoding.

  The encoding device 100 includes an enhancement layer processing block 110 and a base layer processing block 120 for encoding a moving image with spatial scalability, and the base layer processing block 120 compresses and encodes the moving image at a low resolution. The enhancement layer processing block 110 compresses and encodes the moving image with high resolution. Thereby, encoded data of moving images having different spatial resolutions is generated for each layer.

  Also, the encoding apparatus 100 uses an MCTF (Motion Compensated Temporal Filtering) technique in order to encode a moving image with temporal scalability. The MCTF technique combines subband division in the time axis direction with motion compensation, and performs hierarchical motion compensation. As a result, encoded data of moving images having different frame rates for each layer is generated.

  Also, the encoding apparatus 100 compresses and encodes a moving image by changing the quantization step and the number of lower bits to be discarded by the quantization in order to encode the moving image with SNR scalability. Thereby, encoded data of moving images having different image quality for each layer is generated.

  Note that spatial scalability, temporal scalability, and SNR scalability may be arbitrarily combined.

  The image input to the encoding device 100 is supplied to the MCTF unit 20a of the enhancement layer processing block 110. The image reduction unit 12 reduces the image input to the encoding apparatus 100, supplies the reduced image data to the MCTF unit 20b of the base layer processing block 120, and sets the image reduction rate to the variable length of the base layer processing block 120. This is given to the encoding unit 30b.

  The base layer processing block 120 compresses and encodes the image data converted to a low resolution by the image reduction unit 12 and outputs the compressed data to the multiplexing unit 18.

  When performing temporal scalable coding, the MCTF unit 20b operates in the base layer processing block 120, and coding is performed with different frame rates for each layer. In addition, when performing spatial scalability encoding, the enhancement layer processing block 110 operates in addition to the base layer processing block 120, and encoding is performed with different spatial resolutions for each layer. Also, when performing SNR scalable coding, coding with different image quality for each layer is performed by changing the quantization step or the number of lower bits to be discarded by quantization.

  Each configuration of the base layer processing block 120 will be described. The MCTF unit 20b performs motion compensation time filtering according to the MCTF technique. The MCTF unit 20b obtains a motion vector from the moving image frame, and performs temporal filtering using the motion vector. Temporal filtering is performed using a Haar wavelet transform, and as a result, the temporal filtering is decomposed into a plurality of layers having different frame rates including a high frequency frame and a low frequency frame in each layer. The decomposed high-frequency frame and low-frequency frame are stored in the memory for each layer, and the motion vector is also stored in the memory for each layer.

  When the processing in the MCTF unit 20b is completed, the high frequency frames of all layers and the low frequency frames of the final layer are sent to the prediction unit 24b, and the motion vectors of all layers are sent to the motion encoding unit 22b. .

  The prediction unit 24b performs intra-frame prediction of an image frame, and provides an intra-frame prediction error image to the DCT unit 26b. The DCT unit 26b performs discrete cosine transform (DCT) on the intra-frame prediction error image supplied from the prediction unit 24b, and gives the obtained DCT coefficient to the quantization unit 28b. The quantization unit 28b quantizes the DCT coefficient and provides it to the variable length coding unit 30b.

  The variable length coding unit 30b receives the quantized DCT coefficient from the quantization unit 28b, performs variable length coding on the DCT coefficient, and supplies the DCT coefficient to the multiplexing unit 18.

  When performing SNR scalable coding, the quantizing unit 28b changes the number of lower bit planes to be discarded from among a plurality of bit planes, or changes the quantization step to generate encoded data with different image quality for each layer. Generate.

  The variable length encoding unit 30b obtains the vertical / horizontal scaling ratio of the image from the reduction ratio given by the image reducing unit 12, determines the scan order according to the vertical / horizontal scaling ratio, and the DCT in the DCT block in the determined scan order. Coefficients are scanned and variable length encoded.

  The motion encoding unit 22 b encodes the motion vector information given from the MCTF unit 20 b and provides the same to the multiplexing unit 18.

  In order to perform spatial scalable coding, the motion coding unit 22b and the prediction unit 24b of the base layer processing block 120 respectively convert the motion vector of each frame and the intraframe prediction error image in the base layer into the motion code of the enhancement layer processing block 110. To the conversion unit 22a and the interpolation processing unit 32.

  Next, each configuration of the enhancement layer processing block 110 will be described. The image data encoded by the enhancement layer processing block 110 is a high-resolution image that has not been reduced by the image reduction unit 12.

  The MCTF unit 20a of the enhancement layer processing block 110 performs the same motion compensation time filtering as the MCTF unit 20b of the base layer processing block 120 on the high-resolution image data, and motion vector information is sent to the motion encoding unit 22a. This is given to the prediction unit 24a.

  The motion encoding unit 22a of the enhancement layer processing block 110 receives information on the motion vector of the base layer image from the motion encoding unit 22b of the base layer processing block 120. The motion encoding unit 22a of the enhancement layer processing block 110 performs differential encoding between the motion vector information of the enhancement layer image and the motion vector information of the base layer image, and the motion vector that is differentially encoded between the layers. Information is given to the multiplexing unit 18.

  When motion vector information is differentially encoded between the base layer and the enhancement layer, the motion vector in the base layer is expanded to match the resolution of the enhancement layer. For example, when the height and width of the base layer image are each ½ of the height and width of the enhancement layer image, the motion vectors obtained for the base layer image are respectively expressed in the height direction and the width direction. Double it. The motion encoding unit 22a of the enhancement layer processing block 110 encodes the difference between the motion vector of the base layer and the motion vector of the enhancement layer that have been expanded according to the resolution of the enhancement layer in this way. . By differentially encoding motion vector information between layers in this manner, the amount of motion vector information can be reduced compared to encoding motion vector information of an enhancement layer image as it is.

  The interpolation processing unit 32 receives a prediction error image of the base layer image from the prediction unit 24b of the base layer processing block 120, and performs a process of interpolating pixels to match the resolution of the enhancement layer. The interpolation processing unit 32 gives the prediction error image of the base layer subjected to the interpolation processing to the prediction unit 24a of the enhancement layer processing block 110.

  The prediction unit 24a of the enhancement layer processing block 110 performs intraframe prediction encoding on the image frame provided from the MCTF unit 20a. Further, the prediction unit 24 a of the enhancement layer processing block 110 performs differential encoding between the prediction error image of the enhancement layer and the prediction error image of the base layer that is interpolated to match the resolution of the enhancement layer. By performing differential encoding of prediction error images between layers, the amount of codes can be reduced.

  The processing by the DCT unit 26a, the quantization unit 28a, and the variable length coding unit 30a of the enhancement layer processing block 110 is performed by the DCT unit 26b, the quantization unit 28b, and the variable length coding unit 30b of the base layer processing block 120. The prediction error image is compression-encoded in the enhancement layer and passed to the multiplexing unit 18.

  The multiplexing unit 18 is a hierarchized encoded stream in which the encoded data in the base layer given from the base layer processing block 120 and the coded data in the enhancement layer given from the enhancement layer processing block 110 are combined into one. Is generated and output. The encoded data of each layer includes image data and motion vector information.

  In the above description, the base layer processing block 120 and the enhancement layer processing block 110 are separately provided, and the configuration for encoding the base layer low resolution image and the enhancement layer high resolution image has been described. Components common to the enhancement layer processing block 110 may be shared between the base layer and the enhancement layer. For example, only the configuration of the base layer processing block 120 is provided, the base layer is encoded in the base layer processing block 120, and the prediction error image and motion vector information in the base layer are held in the memory. Next, using the base layer encoding result stored in the memory, the enhancement layer encoding process is executed in the base layer processing block 120. Thus, if the configuration of the encoding process in the base layer is diverted to the enhancement layer, the circuit scale of the encoding device 100 can be reduced.

  In the above description, the case where there are two layers of the spatial scalability, that is, the base layer and the enhancement layer has been described, but three or more layers of spatial scalability may be provided. In that case, the base layer processing block 120 is provided for the lowest layer, and the configuration of the extended layer processing block 110 is provided for each of the other layers. The prediction error image and motion vector information are sent from the lower layer to the upper layer, and differential encoding is performed in each layer. Alternatively, only the base layer processing block 120 may be provided, and the base layer processing block 120 may be repeatedly used for each layer so that each layer is sequentially encoded.

  When the spatial scalability hierarchy is 3 or more, the image of layer 0, that is, the image of the minimum size is encoded by the base layer processing block 120, and the image of layer 1 or more is encoded by the enhancement layer processing block 110. . For this reason, in the enhancement layer processing block 110, not only the original image of the highest layer that has not been reduced, but also the image that has been reduced by the image reduction unit 12 is input. Therefore, the variable length coding unit 30a of the enhancement layer processing block 110 receives the reduction ratio information from the image reduction unit 12 for each layer other than the highest layer, and the variable length coding unit 30b of the base layer processing block 120. In the same manner as described above, the spatial frequency coefficients are scanned and variable length coding is performed in the scan order corresponding to the vertical / horizontal scaling ratio. Hereinafter, based on the fact that the variable length coding unit 30a of the enhancement layer processing block 110 may operate in the same manner as the variable length coding unit 30b of the base layer processing block 120, the variable length coding of the enhancement layer processing block 110 may be performed. The generalization unit 30a and the variable length coding unit 30b of the base layer processing block 120 are collectively referred to simply as the variable length coding unit 30, and operations unique to the variable length coding unit 30 of the present embodiment will be described.

  FIGS. 2A and 2B are diagrams for explaining changes in the DCT coefficient when an image is reduced. FIG. 2A shows a case where the vertical stripe pattern image 300 is DCT transformed as an example. A DCT block 310 is obtained by DCT transforming the block 301 of 8 pixels in the vertical stripe pattern image 300 in the vertical and horizontal directions.

  The coefficient at the upper left corner of the DCT block 310 is a DC component, in which high frequency components are arranged from low frequency components in the horizontal and vertical directions, and the coefficient at the lower right corner is a high frequency component in both the horizontal and vertical directions. Coefficients that are hatched are non-zero values, and coefficients that are not hatched are zero values. Since the original image has a vertical stripe pattern, when the block of the image is DCT transformed, a high frequency component appears in the horizontal direction, but only a low frequency component appears in the vertical direction. Therefore, in the DCT block 310 of the vertically striped pattern image 300, non-zero values are arranged in a column of the lowest frequency in the vertical direction from a low frequency to a high frequency. In this example, as indicated by diagonal lines, three non-zero values are arranged in the column of the lowest frequency in the vertical direction.

  FIG. 2B shows an image 302 obtained by reducing the vertical stripe pattern image 300 of FIG. When a block 303 of 8 pixels in length and width is DCT-transformed in the image 302 reduced in the horizontal direction, a DCT block 312 is obtained. Since the original image is reduced in half in the horizontal direction, the number of vertical stripes is doubled in the block 303 of the reduced image 302, and the high frequency in the horizontal direction becomes remarkable. Therefore, in the DCT block 312 of the image 302 reduced in the horizontal direction, more non-zero values are arranged in the lowest frequency column in the vertical direction than in the DCT block 310 of the original vertical stripe pattern image 300. In this example, as indicated by diagonal lines, six non-zero values are arranged in the lowest frequency column in the vertical direction.

  In FIGS. 2A and 2B, the case where the vertical stripe pattern image 300 is reduced in the horizontal direction has been described as an example. However, even in a general image, when reduced in the horizontal direction, the horizontal direction is smaller than the original image. There is a tendency for high-frequency components to appear prominently. Further, when the image is reduced in the vertical direction, the high-frequency component tends to appear significantly in the vertical direction as compared with the original image.

  The DCT coefficients are usually variable-length encoded by zigzag scanning while alternately moving in the DCT block by the same number of pixels in the vertical and horizontal directions. This scan order is called “normal zigzag scan”.

  If a high-frequency component is significant in the horizontal direction in the DCT block, increasing the number of pixels advanced in the horizontal direction can increase the run length and increase the coding efficiency. In the zigzag scan, a scan order in which the number of pixels advanced in the horizontal direction is larger than the number of pixels advanced in the vertical direction is referred to as “horizontal priority scan”.

  Also, if the high frequency component is significant in the vertical direction in the DCT block, increasing the number of pixels that advance scanning in the vertical direction can similarly increase the run length and increase the encoding efficiency. In the zigzag scan, a scan order in which the number of pixels advanced in the vertical direction is larger than the number of pixels advanced in the horizontal direction is called “vertical priority scan”.

  When the DCT coefficient is variable-length encoded by selecting one of the normal zigzag scan, horizontal priority scan, and vertical priority scan for each frame of the moving image or each DCT block in the frame, each frame or each DCT block Therefore, it is necessary to include the information specifying the scan order in the encoded data of the image, the code amount increases by the amount of information specifying the scan order, and the encoding efficiency of the moving image decreases.

  Therefore, in the present embodiment, the variable length encoding unit 30 uniquely determines the scan order of normal zigzag scan, horizontal priority scan, and vertical priority scan according to the vertical / horizontal scaling ratio of the image. The DCT coefficients in the DCT block are variable length encoded in the scanned order.

  The variable length encoding unit 30 does not include information specifying the scan order in the encoded data of the image. This is because the scan order is uniquely determined by the vertical / horizontal scaling ratio of the image, so that when decoding the image, even if the encoded data of the image does not include information specifying the scan order, the vertical / horizontal direction of the image to be decoded This is because the scan order can be uniquely specified based on the scaling ratio, and the DCT coefficients after variable decoding can be correctly arranged to reproduce the image.

  3A to 3C are diagrams for explaining the scan order of the DCT block. FIG. 3A illustrates a normal zigzag scan. In the DCT block 320, starting from the pixel in the upper left corner, the pixel is scanned zigzag from low frequency to high frequency while alternately moving one pixel in the horizontal direction and one pixel in the vertical direction, and the DCT coefficient on the scan line is calculated. Variable length coding.

  FIG. 3B illustrates the horizontal priority scan order. In the DCT block 320, starting from the pixel in the upper left corner, in this example, the pixel is scanned zigzag from low frequency to high frequency while alternately moving three pixels in the horizontal direction and one pixel in the vertical direction. The DCT coefficients are variable-length encoded. As a result, the zigzag scan line is inclined in the horizontal direction, and the number of pixels arranged in the horizontal direction is traced more than the pixels arranged in the vertical direction, and the significant DCT coefficient that appears conspicuously in the horizontal direction is prioritized and variable. Long coding can be performed.

  FIG. 3C illustrates the vertical priority scan order. In the DCT block 320, starting from the pixel in the upper left corner, in this example, the pixel is scanned zigzag from low frequency to high frequency while alternately moving three pixels in the vertical direction and one pixel in the horizontal direction. The DCT coefficients are variable-length encoded. As a result, the scan line of zigzag is inclined in the vertical direction, and the number of pixels arranged in the vertical direction is traced more than the pixels arranged in the horizontal direction, and a significant DCT coefficient that appears noticeably continuously in the vertical direction is prioritized and variable. Long coding can be performed.

  In the normal zigzag scan, only one pixel crosses the scan line indicated by the arrow. Therefore, it is sufficient to sequentially encode the DCT coefficients corresponding to the pixels on the scan line. Then, the scan line may pass through the same pixel twice. In that case, the DCT coefficient corresponding to the pixel is encoded when it first passes, and the pixel may be ignored when it passes the second time.

  FIGS. 4A and 4B are diagrams illustrating the relationship between the vertical / horizontal scaling ratio of an image and the parameters that determine the scan order of DCT blocks.

  As shown in FIG. 4A, it is assumed that the image 330 of horizontal W pixels and vertical H pixels is reduced to an image 332 of horizontal W ′ pixels and vertical H ′ pixels. Here, the reduced image is hatched.

The reduction ratio R _{X in the} horizontal direction (lateral direction) is R _X = W ′ / W, and the reduction ratio R _{Y in the} vertical direction (longitudinal direction) is R _Y = H ′ / H. Variable-length coding unit 30, a preferential direction of scanning, determining _(this is called "aspect Scaling Ratio") ratio R _{X /} R Y horizontal reduction ratio R _X and vertical reduction ratio R _Y by. The variable length encoding unit 30 selects the horizontal priority scan when the vertical / horizontal scaling ratio R _X / R _Y is smaller than 1, that is, when the degree of reduction in the horizontal direction is larger than the degree of reduction in the vertical direction. Larger aspect scaling ratio R _{X /} R _Y is 1, that is, when the degree of reduction in the longitudinal direction than the degree of lateral contraction is large, selects the vertical preferential scan.

With reference to FIG. 4B, parameters in the horizontal priority scan and the vertical priority scan will be described. In the DCT block 320, the number of pixels N _X that advances scanning in the horizontal direction and the number of pixels N _Y that advances scanning in the vertical direction are shown. Horizontal priority scan is characterized by the number of pixels N _X advancing the scan in the horizontal direction is larger than the number of pixels N _Y advancing the scan in the vertical direction. On the other hand, the vertical priority scan is characterized in that the number of pixels N _Y that advances in the vertical direction is larger than the number of pixels N _X that advances in the horizontal direction.

Number of pixels N _Y advancing the scan pixel number N _X and the vertical direction to advance the scanning in the horizontal direction, as shown in the figure, the end of the DCT block 320, i.e., determined by the turning points of the zigzag scan. Thus, by determining the number of pixels N _X and N _Y to be continuously scanned at the zigzag scan folding point in the vertical direction and the horizontal direction, the number of pixels N can be obtained even in the diagonal direction other than the zigzag scan folding point. Scanning is performed while continuously following in the vertical direction and the horizontal direction with the number of pixels substantially corresponding to _{X 1} and N _Y.

Variable-length coding unit 30 not only determines the priority direction of scan based on the aspect scaling ratio, in the case of selecting the horizontal preferential scan or vertical priority scan pixel number N _X perpendicular advancing the scan in the horizontal direction The number of pixels _NY for which scanning is advanced in the direction is also determined based on the vertical / horizontal scaling ratio.

FIGS. 5A to 5D illustrate the relationship among the vertical / horizontal scaling ratio R _X / R _Y of the image, the number of pixels N _X that advances the scan in the horizontal direction, and the number of pixels N _Y that advances the scan in the vertical direction. FIG. FIG. 5A shows an example in which the image 330 is halved in the horizontal direction. The reduction ratio of the reduced image 334 in the horizontal direction is R _X = 1/2, the reduction ratio in the vertical direction is R _Y = 1, and the vertical / horizontal scaling ratio is R _X / R _Y = 1/2. At this time, as an example, as shown in FIG. 5B, in the DCT block 320 of the reduced image 334, the number of pixels N _X to be scanned in the horizontal direction is set to 2, and the number of pixels N _Y to be scanned in the vertical direction is Set to 1.

FIG. 5C is an example in which the image 330 is halved in the vertical direction. The reduction ratio of the reduced image 336 in the horizontal direction is R _X = 1, the reduction ratio in the vertical direction is R _Y = 1/2, and the horizontal / vertical scaling ratio is R _X / R _Y = 2. At this time, as an example, as shown in FIG. 5D, in the DCT block 320 of the reduced image 336, the number of pixels N _X that progresses in the horizontal direction is set to 1, and the number of pixels N _Y that advance the scan in the vertical direction are Set to 2.

In FIG. 6, the priority direction of scanning is associated with the number of pixels N _X that advances in the horizontal direction and the number of pixels N _Y that advances in the vertical direction with respect to the vertical / horizontal scaling ratio R _X / R _Y of the image. It is a figure explaining a table. This correspondence table is stored in the memory of the encoding apparatus 100, read by the variable length encoding unit 30, and used for encoding.

In this example, the vertical / horizontal scaling ratio R _X / R _Y = 1/4 is associated with the number of pixels N _X = 4 for scanning in the horizontal direction and the number of pixels N _Y = 1 for scanning in the vertical direction. This is a horizontal priority scan. N _X = 2 and N _Y = 1 are associated with R _X / R _Y = 1/2, which is also a horizontal priority scan.

R _X / R _Y = 1 is associated with N _X = 1 and N _Y = 1, which is usually a zigzag scan. The vertical / horizontal scaling ratio is 1 when the horizontal reduction ratio and the vertical reduction ratio are equal, or when the image is not reduced.

N _X = 1 and N _Y = 2 are associated with R _X / R _Y = 2, which is a vertical priority scan. N _X = 1 and N _Y = 4 are associated with R _X / R _Y = 4, which is also a vertical priority scan.

The variable length encoding unit 30b calculates the vertical / horizontal scaling ratio R _X / R _Y based on the horizontal reduction ratio R _X and the vertical reduction ratio R _Y of the image given from the image reduction unit 12, With reference to the correspondence table of FIG. 6, the number of pixels N _X for selecting the normal zigzag scan, the horizontal priority scan, or the vertical priority scan based on the vertical / horizontal scaling ratio R _X / R _Y and proceeding the scan in the horizontal direction Then, the number of pixels N _Y to be scanned in the vertical direction is uniquely determined. The image decoding apparatus also has the table of FIG. 6, and the variable length decoding unit uniquely performs normal zigzag scanning, horizontal priority scanning, vertical priority scanning based on the vertical / horizontal scaling ratio R _X / R _Y of the decoding target image. Is selected, and the number of pixels N _X for scanning in the horizontal direction and the number of pixels N _Y for scanning in the vertical direction are uniquely determined.

  The above-described method of encoding by selecting the scan order based on the vertical / horizontal scaling ratio of the image is particularly effective in the spatial scalable encoding of SVC. In spatial scalable coding, an image is reduced at a predetermined reduction rate in each layer, and image data is encoded at a different spatial resolution for each layer.

  FIG. 7 shows the hierarchy of spatial scalability. Here, an example of encoding with spatial scalability of three layers will be described. The layer 2 image 340 has horizontal W pixels and vertical H pixels, the layer 1 image 342 is obtained by halving the layer 2 image 340 in the horizontal direction, and the layer 0 image 344 is the layer 1 image. The image 342 is further halved in the vertical direction. Layer 0 is a base layer, and layers 1 and 2 are enhancement layers.

  In spatial scalable encoding, after encoding an image for each layer, differential encoding is performed between the layer 0 image 344 and the layer 1 image 342, and between the layer 1 image 342 and the layer 2 image 340. Then, differential encoding is performed.

  Here, the layer 1 image 342 is reduced only in the horizontal direction, and a high-frequency component appears remarkably in the horizontal direction. Therefore, in encoding a layer 1 image, it is preferable in terms of encoding efficiency that variable length encoding of DCT coefficients is performed by horizontal priority scanning.

  Spatial scalable encoding may be performed while simultaneously reducing the horizontal and vertical directions in each layer, but as in this example, only the horizontal direction is reduced in one layer, and the vertical in the next layer. Hierarchical coding can also be performed by reducing only the direction. The selection of the scan order based on the vertical / horizontal scaling ratio of this embodiment is particularly effective when scalable encoding is performed by reducing either the horizontal direction or the vertical direction in each layer.

  As described above, according to the encoding apparatus 100 of the present embodiment, since the scan order is linked to the vertical / horizontal scaling ratio of the image, it is not necessary to include information for specifying the scan order in the encoded data of the image. The encoding efficiency of the image can be increased.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above embodiment, the DCT transformation is taken as an example of the spatial frequency transformation of the image, and it has been described that the scan order in the DCT block is determined based on the vertical / horizontal scaling ratio of the image. Other spatial frequency transforms such as discrete wavelet transform (DWT) may be used. In the case of DCT transformation, the image is divided into blocks of the same size. In the case of DWT transformation, since the low frequency components of the image are recursively divided and spatial frequency transformation is performed, DWT coefficients of different sizes in each layer are used. A block is obtained. In the case of DWT conversion, the scan order in the DWT coefficient block of each layer is determined based on the vertical / horizontal scaling ratio of the image.

  In the above embodiment, a method for determining a scan order in variable-length coding of spatial frequency coefficients when each frame is subjected to spatial frequency conversion has been described using a moving image as an example. The present invention is not limited to moving images and can be similarly applied to a case where a still image is subjected to spatial frequency conversion. For example, in DCT conversion of JPEG (Joint Photographics Experts Group) or DWT conversion of JPEG2000, the scanning efficiency of still images can be improved by selecting the scan order of the block of conversion coefficients according to the aspect ratio of the image. .

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a figure explaining the change of the DCT coefficient at the time of reducing an image. It is a figure explaining the scanning order of a DCT block. It is a figure explaining the relationship between the vertical / horizontal scaling ratio of an image, and the parameter which determines the scanning order of a DCT block. It is a figure explaining the relationship among the vertical / horizontal scaling ratio of an image, the number of pixels which advance a scan in a horizontal direction, and the number of pixels which advance a scan in a vertical direction. It is a figure explaining the table which matched the priority direction of a scan, the number of pixels which advance a scan in the horizontal direction, and the number of pixels which advance a scan in the vertical direction with respect to the vertical / horizontal scaling ratio of the image. It is a figure explaining the hierarchy of space scalability.

Explanation of symbols

  12 image reduction unit, 18 multiplexing unit, 20a, 20b MCTF unit, 22a, 22b motion coding unit, 24a, 24b prediction unit, 26a, 26b DCT unit, 28a, 28b quantization unit, 30a, 30b variable length coding , 32 interpolation processing unit, 100 encoding device, 110 enhancement layer processing block, 120 base layer processing block.

Claims (6)

  Scans the spatial frequency coefficient block obtained by performing spatial frequency conversion of the image in a scanning order determined by the aspect ratio of the horizontal reduction of the image and the vertical reduction of the image. Then, the encoded data of the image is generated by variable-length encoding the spatial frequency coefficient.   The encoding method according to claim 1, wherein the encoded data is generated without including information specifying the scan order in the encoded data.   3. The scanning according to claim 1, wherein when the vertical / horizontal scaling ratio indicates that the degree of reduction in the vertical direction is larger than the degree of reduction in the horizontal direction, scanning is performed in a scanning order in which the vertical direction is prioritized. Encoding method.   4. The scanning according to claim 1, wherein when the vertical / horizontal scaling ratio indicates that the degree of reduction in the horizontal direction is larger than the degree of reduction in the vertical direction, scanning is performed in a scan order in which the horizontal direction is prioritized. An encoding method according to claim 1.   When zigzag scanning the block of the spatial frequency coefficient from a low frequency component to a high frequency component, the number of pixels scanned continuously in the vertical direction and the number of pixels scanned continuously in the horizontal direction 5. The encoding method according to claim 1, wherein the encoding method is determined from a vertical / horizontal scaling ratio.   The encoding according to any one of claims 1 to 5, wherein a table in which the vertical / horizontal scaling ratio is associated with the scan order is provided separately from the encoded data and used for encoding. Method.

Images