JP2001285875A - Image information conversion apparatus and method - Google Patents

Abstract

(57) [Problem] To reduce the amount of arithmetic processing and the amount of video memory when converting MPEG2 image compression information into MPEG4 image compression information. SOLUTION: An MPEG2 image information decoding unit 18 for decoding an interlaced image using only 4 × 8 components among 8 × 8 DCT coefficients of macroblocks constituting MPEG2 image compression information by interlaced scanning, and MPEG2 A scan conversion unit 19 that selects one of the first field and the second field of the interlaced image decoded by the image information decoding unit 18 to generate an image of a sequential operation, and a scan conversion unit 20 that generates the image. Down-sampler 20 that horizontally down-samples an image that has been sampled, and down-sampler 21 that vertically down-samples an image down-sampled by the down sampler 20.
And encoding means 22 for encoding the image downsampled by the downsampler 21 into MPEG4 image compression information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information conversion apparatus and method for converting image information, and more particularly, to image information (bit stream) such as MPEG compressed by orthogonal transform such as discrete cosine transform and motion compensation. The present invention relates to an image information conversion apparatus and method used when receiving an image through a network medium such as satellite broadcasting, cable TV, or the Internet, or when processing the same on a storage medium such as an optical disk or a magnetic disk.

[0002]

2. Description of the Related Art In recent years, discrete cosine transform (discrete cosine transform (discrete cosine transform) has been used in recent years for the purpose of handling image information as digital, transmitting and storing information with high efficiency, and utilizing redundancy inherent in image information.
Image information compression schemes such as MPEG that compress by orthogonal transformation such as tecosine transformation (DCT) and motion compensation are provided. Devices conforming to such an image information compression method are becoming widespread in both information distribution at broadcast stations and the like and information reception in ordinary households.

In particular, MPEG2 (ISO / IEC 13
818-2) is defined as a general-purpose image coding method, and is a standard covering both interlaced scan images and progressive scan images, as well as standard resolution images and high-definition images, and has a wide range of applications for professional use and consumer use. It is expected to be used in the future.

[0004] By using the MPEG2 compression method,
A high compression rate and good image quality can be realized. For this purpose, for example, a standard resolution interlaced scanning image having 720 × 480 pixels is 4 to 8 Mbps and 1920 × 480.
For a high-resolution interlaced scan image having 1088 pixels, it is necessary to allocate a code amount (bit rate) of 18 to 22 Mbps.

[0005] In digital broadcasting which is expected to be widely used in the future, image information is transmitted by such a compression method. Standards include an image having a standard resolution and an image having a high resolution. It is desirable that the device has a function capable of decoding both of them.

[0006] By the way, MPEG2 is mainly intended for high-quality coding suitable for broadcasting, but MPEG1
It does not correspond to an encoding method with a lower code amount (bit rate), that is, a higher compression ratio. With the spread of mobile terminals, it is expected that the need for such an encoding system will increase in the future, and in response to this, the MPEG4 encoding system has been standardized. Regarding the image coding system, the standard was approved as an international standard in December 1998 as ISO / IEC 14496-2.

[0007] Also, MPEG4 image compression information (bit rate) having a lower code amount (bit rate), which is more suitable for processing the MPEG2 image compression information (bit stream) once encoded for digital broadcasting on a portable terminal or the like. Bit stream).

[0008] As an image information conversion apparatus (transcoder) for achieving the above object, "Field-to-Frame Transco
ding with Spatial and Tempora1 Downsampling "(Susi
e J. Wee, John G. Apostolopoulos, and Nick Feamste
r, ICIP '99), an apparatus as shown in FIG. 11 has been proposed. In other words, this device includes a picture type discriminating unit 1
1. MPEG2 image information decoding unit (I / P picture) 1
2. Thinning unit 13, MPEG4 image information encoding unit (I /
(P-VOP) 14, a motion vector synthesis unit 15, and a motion vector detection unit 16.

The picture type discriminating section 11 receives MPEG2 image compression information (bit stream) for interlaced scanning. The picture type determination unit 11 determines whether the data of each frame is related to an I / P picture,
It is determined whether it is related to the picture, only the former,
Subsequent MPEG2 image information decoding unit (I / P picture)
12 is output.

The processing in the MPEG2 image information decoding section (I / P picture) 12 is the same as that in a normal MPEG2 image information decoding apparatus.
The function of the PEG2 image information decoding unit (I / P picture) 12 only needs to be able to decode only the I / P picture.

The pixel value output from the MPEG2 image information decoding section (I / P picture) 12 is input to a thinning section 13 where a 1/2 thinning process is performed in the horizontal direction and a vertical direction is performed. In this method, only one of the data of the first field and the second field is left, and the other is discarded, thereby generating a progressively scanned image having a size of 1/4 of the input image information.

The progressive scan image generated by the thinning section 13 is an MPEG4 image information encoding section (I / P-VOP).
14 and output as MPEG4 image compression information (bit stream). At this time, the motion vector information in the input MPEG2 image compression information (bit stream) is mapped to the motion vector for the decimated image information in the motion vector synthesis unit 15, and the motion vector A high-precision motion vector is detected based on the motion vector value synthesized by the unit 15.

[0013]

The image information conversion apparatus shown in FIG. 11 is designed so that the input MPEG2 image compression information (bit stream) conforms to the NTSC standard (720 × 480 pixels, interlaced scanning). It outputs MPEG4 image compression information (bit stream) having a size of SIF (352 × 240 pixels, progressive scanning) which is an approximately 1/2 × 1/2 image frame. However, in a portable information terminal which is one of the target applications of MPEG4, the resolution of the monitor unit is SI
It is possible that an F-size image cannot be displayed. Also, under the condition of the code amount (bit rate) determined by the capacity of the storage medium or the bandwidth of the transmission path, there may be a problem that good image quality cannot be obtained with the SIF size. In such a case, the QSIF (176 × 11) which is an approximately 1/4 × 1/4 picture frame of the input MPEG2 image compression information (bit stream) is used.
(2 pixels, progressive scanning). Further, information on the Takagi component of the image, which is discarded at a later stage, is also stored in the MPEG2 image information decoding unit (I / P picture) 1.
In No. 3, since the processing is performed, it can be said that the calculation amount and the memory capacity required for decoding are redundant.

The present invention has been proposed in view of the above-mentioned circumstances, and has been proposed in which MPEG-2 image compression information of interlaced scanning, which is an input, is converted into a QSIF, which is an approximately 1/4 × 1/4 image frame.
It is an object of the present invention to provide an image information conversion apparatus and method for converting image information into an image information, which reduces the amount of calculation and the area memory capacity required for decoding.

[0015]

In order to solve the above-mentioned problems, an image information conversion apparatus according to the present invention provides a discrete cosine transform of an image in units of a macroblock consisting of pixels of eight components in both the horizontal and vertical directions. In the image information conversion device for converting the resolution of the compressed image information, among the discrete cosine transform coefficient coefficients of eight components in both the horizontal and vertical directions of the macroblocks constituting the input image compression information obtained by encoding the image by the interlaced scanning, Decoding means for decoding an interlaced image using only four low-frequency components in the horizontal direction and eight components in the vertical direction; and a first field and a second field constituting the interlaced image decoded by the decoding means. Scanning conversion means for selecting any one of the fields to generate a progressively scanned image; and Horizontal downsampling means for downsampling in the horizontal direction, vertical downsampling means for vertically downsampling the image downsampled by the horizontal downsampling means, and downsampling by the vertical downsampling means Encoding means for encoding the compressed image into output image compression information having a lower resolution than the input image compression information.

An image information conversion method according to the present invention is a method for converting the resolution of image compression information obtained by performing discrete cosine conversion on an image in units of macroblocks composed of eight-component pixels in both the horizontal and vertical directions. Of the eight components of the discrete cosine transform coefficient coefficients in the horizontal and vertical directions of the macroblocks constituting the input image compression information obtained by encoding the image obtained by the interlaced scanning, the low frequency band 4 in the horizontal direction is used.
A decoding step of decoding the interlaced image using only the component and the eight components in the vertical direction, and one of the first field and the second field constituting the interlaced image decoded in the decoding step. A scan conversion step of selecting and sequentially generating an image of a progressive scan, a horizontal downsampling step of performing horizontal downsampling on the image generated by the scan conversion step, and a downsampling step performed by the horizontal downsampling step. A vertical downsampling step of performing downsampling on the image in the vertical direction, and encoding the image downsampled in the vertical downsampling step into output image compression information having a lower resolution than the input image compression information. And a process.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

First, an image information conversion apparatus according to an embodiment of the present invention will be described with reference to FIG.

This image information conversion apparatus receives an input MP
A picture type discriminating unit 17 for discriminating and processing the type of a picture constituting the image compression information (bit stream) of EG2, an MPEG decoding the picture sent from the picture type discriminating unit 17 and detecting a motion vector
And a two-image information decoding unit 18.

Image compression information (bit stream) corresponding to interlaced scanning is input to the picture type determination unit 17. This MPEG2 image compression information (bit stream) is composed of an intra-coded image (I picture) coded in a frame and a forward prediction code predicted and coded by referring to another image in the forward direction between frames. Encoded image (P picture) and a bidirectional predictive encoded image (B) that is predictively encoded with reference to another image in the forward and backward directions between frames.
Picture).

The picture type discriminating unit 17 receives the I-picture data (bit stream) from the input MPEG-2 image compression information.
The B picture is discarded while leaving only the picture and the P picture.

The MPEG2 image information decoding unit 18 outputs an I-picture or P-picture
Information about a picture (hereinafter, simply referred to as an I / P picture) is transferred in the horizontal direction to the lower 4
A 4 × 8 down decoder that performs partial decoding of a macroblock using only the next information and all of the 8th-order DCT coefficients in the vertical direction (hereinafter referred to as 4 × 8, and the same applies to the other). is there.

That is, the MPEG2 image information decoding unit 1
Reference numeral 8 denotes an interlaced scan image having a 1/2 × 1/1 picture frame of the image compression information (bit stream) of the input I / P picture.

Further, the image information conversion device includes a scan conversion unit 19 for converting the interlaced scan image output from the image information decoding unit 18 into a progressively scanned image, and a horizontal direction of the image from the scan conversion unit 19. Downsampler (horizontal direction) 20 for performing downsampling, downsampler (vertical direction) 21 for performing vertical downsampling of an image from downsampler (horizontal direction) 20, and image from downsampler (vertical direction) 21 To the motion vector detector 2
MPEG4, which encodes MPEG4 image compression information (bit stream) using the motion vector sent from
And an image information encoding unit 22.

The scan conversion section 19 includes only one of the first field data and the second field data among the pixel values output from the MPEG2 image information decoding section (I / P picture 4 × 8 down decoder) 18. , And discarding the remainder to output progressively scanned image information having an image frame of 1/2 × 1/2 of the input image compression information (bit stream).

The downsampler (horizontal direction) 20 performs サ ン プ リ ン グ times downsampling in the horizontal direction by using a low-pass filter of several taps, and outputs 圧 縮 × 1 of image compression information (bit stream) to be input. The image information of progressive scanning having an image frame of / 2 is output.

The downsampler (vertical direction) 21 performs １ ／ times downsampling in the vertical direction by using a low-pass filter with several taps, and outputs 圧 縮 × 1 of the image compression information (bit stream) to be input. The image information of progressive scanning having an image frame of / 4 is output.

An MPEG4 image information encoding unit (I / P-V
OP) 22 performs MPEG4 encoding on the image data output from the scan conversion unit 17 and outputs it as image compression information (bit stream). This image compression information (bit stream) is progressive scanning image data having a ×× １ ／ image frame of the input image compression information (bit stream).

This image compression information (bit stream)
Is a video object plane (video object plane; VOP) constituting a video object (VO) of an MPEG4 bit stream, and includes an I-VOP, which is an intra-coded VOP, and a forward prediction code. It is composed of a P-VOP which is a generalized VOP.

Further, the image information conversion device is an MPEG2
A motion vector synthesizing circuit 23 for synthesizing the motion vectors detected by the image information decoding unit 17, and detecting a motion vector based on an output from the motion vector synthesizing unit 23 and an image from the downsampler (vertical direction) 21. And a motion vector detection unit 24.

The motion vector synthesizing unit 23 calculates a motion vector value in the MPEG2 image compression information (bit stream) detected by the MPEG2 image information decoding unit 18 based on the motion vector value.
The image after the scan conversion is subjected to mapping using the motion vector value for the data.

The motion vector detection section 24 performs highly accurate motion vector detection based on the motion vector value output from the motion vector synthesis section 23.

Next, the operation of the image information conversion apparatus according to the present embodiment will be described.

The MPEG2 image compression information (bit stream) of the interlaced scanning, which is an input, is first input to a picture type discriminating section 17, where the information relating to the I / P picture is output and output to an MPEG2 image information decoding section (I / P). P
Picture 4 × 8 down decoder) 18,
Information about the B picture is discarded. The conversion of the frame rate is performed in this manner. MPEG2 image information decoding unit (I / P picture 4 × 8 down decoder) 18
Has the same configuration as that of the apparatus shown in FIG. 2 described later. However, since information relating to a B picture has already been discarded in the picture type discriminating unit 17, its function is to perform decoding of only an I / P picture. good. By performing a decoding process using only low-frequency quaternary information in the horizontal direction, this M
The capacity of the video memory required by the PEG2 image information decoding unit (I / P picture 4 × 8 down decoder) 18 is the same as that of the MPEG2 image information decoding unit (I / P
(P picture) 12 and the amount of processing required for inverse discrete cosine transform may be １ ／.

From the MPEG2 image information decoding section (I / P picture 4 × 8 down decoder) 18, interlaced scanning having a 1/2 × 1/1 picture frame of the input image compression information (bit stream) is performed. An image is output. this is,
As shown in FIG. 3, the scan converter 19 leaves only one of the first field and the second field and discards the other, thereby reducing the size of the input image compression information (bit stream) to 1/2 ×. The pixel information is converted into progressively scanned pixel information having a 1/2 picture frame and output. In FIG. 3, among the pixels a2 in the first field and the pixels a2 in the second field shown in FIG.
Is discarded, the pixel b of FIG. 3B is obtained.

Next, this progressively scanned image having an image frame of ×× の of the input image compression information (bit stream) is input to a down sampler (horizontal direction) 20.
As shown in FIG. 4, downsampling of 1/2 times in the horizontal direction is performed using a low-pass filter of several taps,
1/4 of the input image compression information (bit stream)
It is converted into progressively scanned image information having an image frame of × 1/2. That is, the pixel a in FIG.
The pixel b in FIG. 4B is obtained by performing the / 2 down-sampling.

Further, the progressively scanned image information having an image frame of １ ／ × の of the input image compression information (bit stream) is input to a down sampler (vertical direction) 21.
5, as shown in FIG. 5, by using a low-pass filter with several taps, the pixel a shown in FIG. , And is converted into progressively scanned image information having an image frame of ４ × の of the input image compression information (bit stream). The order of the scan conversion unit 19, the down sampler (horizontal direction) 20, and the down sampler (vertical direction) 21 does not necessarily have to be the same, and the order can be changed. The pixel data of this progressive scanning is stored in an MPEG4 image information encoding unit (I /
An encoding process is performed in (P-VOP) 22.

By the way, for example, if the image compression information (bit stream) to be input conforms to the NTSC standard and the size of the image is 720 × 480 pixels,
The 枠 × １ ／ image frame is 180 × 120. However, in order for the MPEG4 image information encoding unit (I / P-VOP) 22 to perform processing in units of macroblocks, the image frame needs to be a multiple of 16 in both the horizontal and vertical directions for the luminance component. is there. The image frame of the color difference component needs to be a multiple of 8 in the horizontal and vertical directions in the case of the 420 format, and 8 in the horizontal direction in the case of the 422 format.
Must be a multiple of 16 in the vertical direction,
In the case of the 444 format, it is necessary to be a multiple of 16 in both the horizontal and vertical directions, similarly to the luminance component. Adjustment of the image frame for this is also performed by the down sampler (horizontal direction) 20.
And a down sampler (vertical direction) 21. That is, in the down sampler (horizontal direction) 20, the horizontal sampler
At the same time as double downsampling is performed, for example, the right four columns of the image frame are discarded, and 176 pixels are set horizontally. Also,
In the down sampler (vertical direction) 21, 1 /
At the same time as performing double downsampling, for example,
The row is discarded to make 112 pixels in the vertical direction. The adjustment of the image frame in the vertical direction may be performed by the scan conversion unit 19.

The MPEG2 image information decoding unit (I /
The motion vector information in the input MPEG2 image compression information (bit stream) detected by the P picture 4 × 8 down decoder 18 is input to the motion vector synthesis unit 23, where the sequentially converted image after scan conversion is input. To the motion vector value at. The motion vector detecting section 24 performs high-precision motion detection based on the motion vector values in the sequentially converted scanned image output from the motion vector synthesizing section 23.

Next, the 4 × 8 decoder will be described with reference to FIG.

The 4 × 8 down decoder includes a code buffer 1 for temporarily storing input image compression information, a compression information analysis unit 2 for analyzing the image compression information sent from the code buffer 1, A variable length decoding unit 3 for performing variable length decoding on the data output from the analysis unit 2 and an inverse quantization unit 4 for inversely quantizing the data output from the variable length encoding unit 3 are provided.

The 4 × 8 down decoder is a reduced inverse discrete cosine transform unit (4 × 4) that performs inverse discrete cosine transform only on the low frequency 4 × 8 component of the 8 × 8 component output from the inverse quantization unit 4. ) 5).

Further, the 4 × 8 down decoder uses a motion compensation unit (field prediction) 7 for performing motion compensation for field prediction using an image sent from the video memory 9 and an image sent from the video memory 9. Compensator (frame prediction) 8, which performs motion compensation for frame prediction, and outputs thereof, a reduced inverse discrete cosine transform (4 × 8) 5, a motion compensator (field separation) 7, or a motion compensator (frame prediction) A) an adder 6 for adding the outputs of 8), a video memory 9 for storing the output from the adder 6, and an image frame correction unit 1 for applying an image frame correction to the image stored in the video memory 9 and outputting the image.
0.

The operation principle of the code buffer 1, the compressed information analysis unit 2, the variable length decoding unit 3, and the inverse quantization unit 4 is the same as that of a normal image decoding device. However, in the variable length decoding unit 3, as shown in FIG. 6, only the necessary coefficients are decoded in the subsequent reduced inverse discrete cosine transform unit (4 × 8) 5, and the processing is continued until an EOB is detected. It is also conceivable not to do this. The decoded coefficients are 4 × 8 pixels surrounded by a broken line in FIG. 6A shows a zigzag scan, and FIG. 6B shows an alternate scan. The numbers in the figure indicate the order in which the DCT coefficients are scanned.

The alternate scan is suitable for interlaced scanning. The inverse discrete cosine transform coefficient (4 × 8) 5 performs inverse discrete cosine transform on the inversely quantized discrete cosine transform coefficient.

When the macroblock is an intra macroblock, the data after the inverse discrete cosine transform output from the reduced inverse discrete cosine transform unit (4 × 8) 5 is stored in the video memory 9 as it is.

When the macroblock is an inter macroblock, the motion compensation unit (field prediction) 7 when the motion compensation mode is the field mode, and the motion compensation unit (frame prediction) when the motion compensation mode is the frame mode. 8, motion compensation is performed. Then, based on the reference data in the video memory 9, the motion compensating unit (field prediction) 7 or the motion compensating unit (frame prediction) 8 performs 1/4 pixel accuracy in the horizontal direction and 1/2 pixel accuracy in the vertical direction. An interpolation process is performed. The adder 6 combines the predicted image generated by this and data obtained by inverse discrete cosine transform by the reduced inverse discrete cosine transform unit (4 × 8) 8 and outputs the result to the video memory 9.

The pixel values stored in the video memory 9 are converted to an image frame size suitable for the display device by the image frame conversion unit 10 and output.

In the following, the reduced inverse discrete cosine transform unit (4
× 8) The operation principle in 5 will be described.

In the vertical direction, an 8th-order inverse discrete cosine transform is performed in the same manner as in a normal MPEG2 image information decoding apparatus. In the horizontal direction, four low-frequency coefficients among the eight-order discrete cosine transform coefficients are extracted and subjected to fourth-order inverse discrete cosine transform. FIG. 7 shows this processing. In the figure, iD ₈ is an 8th-order inverse DCT matrix, and iD ₄ ^t is 4
Inverse of a next reverse DCT, 0 ₄ is the zero matrix of 4 × 4.

It should be noted that realization by a high-speed algorithm is possible in both the horizontal and vertical directions. Wang's algorithm (Reference: Zhong de Wang., “Fast Algorithms fo
r the Discrete W Transform and for the Discrete Fo
urier Transform ", IEEE Tr.ASSP-32, N0.4, pp.803-81
6, Aug. 1984) shows a configuration of a method for realizing the fourth-order inverse discrete cosine transform based on FIG.

In FIG. 8, the 0th output element f (0)
Is obtained by adding the value s2 and the value s5 in the adder 43.

Here, the value s2 is the 0th input element F (0)
And a second input element F (2) added by the adder 31 is multiplied by A in the multiplier 34. The value s5 is the first
An adder 4 adds a value obtained by multiplying the input element F (1) by C by the multiplier 37.
At 0, the value s1 is added. The value s1 is the third
The first input element F (1) is converted from the input element F (3) to an adder 3
The value reduced by 3 is multiplied by D in the multiplier 38.

The first output element f (1) has a value s3 and a value s
4 in the adder 41.

Here, the value s3 is the 0th input element F (0)
And the second input element F (2) is subtracted by the adder 32 and multiplied by A in the multiplier 35. The value s4 is obtained by subtracting the value s1 in the adder 39 from the value obtained by multiplying the third input element F (3) by B in the multiplier 36.

The second output element f (2) has a value s4 to a value s
It is obtained by subtracting 3 in the adder 42.

The third output element f (3) is calculated from the value s2 to the value s
5 in the adder 44.

As described above, implementation with nine adders and five multipliers is possible. The following quantities are used in the figures.

A = 1 / √2 B = −C _1/8 + C _3/8 C = C _1/8 + C _3/8 D = C _{3/8 In} the above equation, the following numbers are used. Can be Others are the same.

C _3/8 = cos (3π / 8) The operation principle of the motion compensator (field mode) 7 and the motion compensator (field mode) 8 is the same as that of a normal image information decoding apparatus in the vertical direction. It is. However, in the horizontal direction, the reduced inverse discrete cosine transform unit (4 × 8) 5 reduces the resolution by half, and
It is necessary to perform interpolation processing. That is, first, a pixel equivalent to 精度 precision is created by a double interpolation filter such as a half-band filter, and a pixel equivalent to ４ precision is created by linear interpolation based on the created pixels. At this time, when a half-band filter is used to output a pixel value having the same phase as a pixel extracted from the frame memory as a predicted image, it is not necessary to perform a multiply-accumulate operation according to the number of taps. Is possible. When a half-band filter is used, its coefficient is x
Since it can be expressed in the form of / ²ⁿ , high-speed execution is possible by realizing division necessary for filtering by a shift operation. Alternatively, it is also conceivable to directly create pixels required for motion compensation by filtering with quadruple interpolation.

As an actual process, a two-stage interpolation realized by a double interpolation filter and a linear interpolation as described above is prepared in advance so that a coefficient is prepared so as to be performed at a time, and is a one-stage interpolation. The processing is performed as if. Also, only necessary pixel values are created according to the value of the motion vector in the input image compression information (bit stream).

When performing double interpolation filtering,
Depending on the value of the motion vector, it may be necessary to refer to outside the image frame in the video memory. in this case,
Either wrap around the end point symmetrically for the required number of taps (hereinafter referred to as mirror processing) or have pixels with the same value as the pixel value of the end point outside the image frame by the required number of taps (This is hereinafter referred to as a hold process). FIG. 9A shows the mirror processing. The symbol p in the figure is a pixel in the video memory 9, and the symbol q is a virtual pixel outside the image frame required for interpolation. FIG. 9B shows the hold processing. Alternatively, the pixel value outside the image frame may be a fixed value such as 128, for example.

Next, the operation principle of the image frame conversion unit 10 will be described.

For example, if the image frame of the input image compression information (bit stream) is 1920 × 1080 pixels, the image output from the video memory 9 is 960 × 1
080 pixels. When this is output to a display device having 720 × 480 pixels (aspect ratio 16: 9), thinning processing of 3/4 in the horizontal direction and 4/9 in the vertical direction is required. When the image compression information to be input is interlaced scanning, it is necessary to perform a thinning process divided into a first field and a second field in the vertical direction.

FIG. 10 shows an example of processing in the vertical direction. In the figure, the symbol a1 indicates an input pixel in the first field, and the symbol a2 indicates an input pixel in the second field. The symbol b1 represents the output pixel of the first field, and the symbol b2
Indicates output pixels of the second field.

Specifically, for the first field,
For example, based on the vertical position 0 of the input pixel, 0- (5/1
Given in 6). For the second field, for example, 0+ (5/16) based on the vertical position 0 of the input pixel
Given by

As described above, the MPEG2 image compression information (bit stream) has been targeted for input, and the MPEG4 image compression information (bit stream) has been targeted for output. However, the input and output are not limited to this. For example, MPEG-1 or H.264.
H.263 or other image compression information (bit stream).

As described above, in the present embodiment, a high-resolution image and a standard-resolution image coexist, and high-resolution image information is thinned out while minimizing image quality deterioration. It is possible to build.

It is considered that the coexistence of a high-resolution image and a standard-resolution image occurs not only in transmission media such as digital broadcasting but also in storage media such as optical disks and flash memories.

[0070]

As described above, the present invention provides MPEG2 image compression information (bit stream) for interlaced scanning.
, And a progressively scanned MPEG4 image compression information having a 1/4 × 1/4 picture frame of the input image compression information (bit stream) by a circuit configuration using a smaller amount of arithmetic processing and video memory capacity. (Bit stream) and outputs the converted data.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an image information conversion device according to an embodiment.

FIG. 2 is a block diagram showing an image information decoding apparatus (4 × 8 down decoder) according to the present embodiment, in which all information is used in the vertical direction, but only fourth-order low-frequency information is used in the horizontal direction. FIG. 2 is a block diagram showing a device configuration for performing the above.

FIG. 3 is a diagram showing an operation principle in a scan conversion unit 19;

FIG. 4 is a diagram showing an operation principle in a down sampler (horizontal direction) 20;

FIG. 5 is a diagram showing an operation principle in a down sampler (vertical direction) 21.

FIG. 6 is a diagram showing an operation principle in a variable length decoding unit.

FIG. 7 is a diagram showing an operation principle in a reduced inverse discrete cosine transform unit (4 × 8) 5;

FIG. 8 is a diagram showing a method of realizing a fourth-order inverse discrete cosine transform using a Wang high-speed algorithm.

FIG. 9 is a diagram illustrating an example of mirror processing and hold processing in a motion compensation unit (field mode) 7 and a motion compensation unit (frame mode) 8;

FIG. 10 is a diagram illustrating an example of processing in the vertical direction in the image frame conversion unit 10;

FIG. 11 is a diagram showing a configuration of a conventional technique of an image information conversion device (transcoder) that receives MPEG2 image compression information (bit stream) as input and outputs MPEG4 image compression information (bit stream).

[Explanation of symbols]

17 picture type discriminator, 18 MPEG2 image information decoder, 19 scan converter, 20 downsampler (horizontal direction), 21 downsampler (vertical direction), 2
2 MPEG4 image information encoder, 23 motion vector synthesizer, 24 motion vector detector

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Teruhiko Suzuki 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5C059 KK08 KK15 LA05 LB05 LB18 MA00 MA03 MA05 MA23 MC11 MC22 MC23 MC38 ME01 PP05 PP06 SS02 SS08 SS11 UA02 UA05 UA13 5J064 AA01 AA02 BA01 BA16 BB03 BC01 BC02 BD02

Claims (28)

[Claims] 1. An image information conversion apparatus for converting the resolution of image compression information obtained by performing discrete cosine transform on a macroblock consisting of eight-component pixels in both the horizontal and vertical directions as a unit, wherein the image obtained by interlaced scanning is encoded. Of the eight discrete cosine transform coefficient coefficients in both the horizontal and vertical directions of the macroblocks constituting the input image compression information, an interlaced scan image is decoded using only four low-frequency components in the horizontal direction and eight components in the vertical direction. Decoding means for selecting one of a first field and a second field constituting an interlaced image decoded by the decoding means, and generating a progressively scanned image; Horizontal downsampling means for downsampling an image generated by the conversion means in the horizontal direction; Vertical downsampling means for vertically downsampling the image downsampled by the downsampling means; and an output image having a lower resolution than the input image compression information by downsampling the image downsampled by the vertical downsampling means. An image information conversion apparatus, comprising: an encoding unit that encodes compressed information. 2. The compressed image information according to the MPEG2 standard, and the compressed image information according to the MPEG4 standard.
2. The image information conversion device according to claim 1, wherein the image information conversion device conforms to a standard. 3. The horizontal downsampling means performs downsampling that is 1/2 times in the horizontal direction of the image, and the vertical downsampling is performed by 1 in the vertical direction of the image.
2. The image information conversion method according to claim 1, wherein downsampling of / 2 times is performed, and the output image compression information has a resolution of 1/4 in both horizontal and vertical directions with respect to the input image compression information. apparatus. 4. The input image compression information includes an intra-coded image coded in a frame, a forward prediction coded image predicted and coded by referring to another image in a forward direction between frames, and And a bidirectional predictive coded image that is predictively coded with reference to another image in the forward and backward directions between frames, and determines the type of image that constitutes the input image compression information. And a discriminating means for passing the coded image and the forward predictive coded image but discarding the bidirectional predictive coded image, wherein the decoding means receives image compression information through the discriminating means. Claim 1
The image information conversion device described in the above. 5. The image information conversion apparatus according to claim 4, wherein said decoding means decodes only the intra-coded image and the forward prediction coded image. 6. The input image compression information has been subjected to variable-length encoding. The decoding means includes a variable-length decoding means for performing variable-length decoding on the image compression information, and a variable-length decoding means for performing variable-length decoding on the variable-length decoding means. An inverse discrete cosine transform unit for performing an inverse discrete cosine transform on the long-decoded image compression information, wherein the variable-length decoding unit includes only discrete cosine transform coefficients required for the inverse discrete cosine transform in the inverse discrete cosine transform unit. 2. The image information conversion device according to claim 1, wherein the image information is decoded. 7. The inverse discrete cosine transform means includes: for the four low-frequency components in the horizontal direction and the eight components in the vertical direction, among the discrete cosine transform coefficients having eight components in both the horizontal and vertical directions that constitute the macroblock. 7. The image information conversion device according to claim 6, wherein the inverse discrete cosine transform is performed. 8. The image information conversion device according to claim 7, wherein said inverse discrete cosine transform means executes an operation using a predetermined high-speed algorithm. 9. The input image compression information has been motion-compensated using a motion vector, and the decoding means has a motion compensation means for compensating an image using a motion vector. Performs motion compensation with 1/2 pixel precision in the vertical direction, but performs motion compensation with 1/4 pixel precision in the horizontal direction based on the motion vector of the input image compression information. 2. The image information conversion device according to 1. 10. The motion compensating means performs a horizontal interpolation process by a two-fold interpolation digital filter with a half-pixel accuracy, and performs a vertical interpolation process by a linear interpolation with a quarter pixel. 10. The image information conversion device according to claim 9, wherein the interpolation is performed with accuracy. 11. The image information conversion device according to claim 10, wherein said digital filter is a half-band filter. 12. The digital filter calculates a coefficient equivalent to a series of interpolation processing in advance, and directly calculates the coefficient on a pixel value according to a value of a motion vector of a macro block constituting the input compressed image information. The image information conversion device according to claim 11, wherein: 13. The image processing apparatus according to claim 1, wherein the motion compensating unit is configured to perform a double-interpolation filter process on pixels existing outside an image frame of the image constituting the input image compression information. 11. The image information conversion apparatus according to claim 10, wherein the filter processing is performed by virtually creating a necessary pixel outside. 14. The motion compensating means wraps a required pixel outside the image frame by folding back at a predetermined position of an existing pixel array, extending the existing pixel array, or using a predetermined value. 14. The image information conversion device according to claim 13, wherein the image information is created. 15. The image forming apparatus according to claim 1, wherein the scan conversion unit is configured to perform an interlaced scan image having an image frame of 1/2 in the horizontal direction and 1 in the vertical direction with respect to the input image compression information decoded by the decoding unit. By selecting one of the first field and the second field, 1 / both in the horizontal and vertical directions are selected.
2. The image information conversion device according to claim 1, wherein the image information is converted into a progressively scanned image having two image frames. 16. The horizontal downsampling means includes:
By performing down-sampling by a factor of に in the horizontal direction on a progressively scanned image having a 枠 image frame in both the horizontal and vertical directions on the input image compression information converted by the scan conversion means. And generating a progressively scanned image having an image frame of 1/4 in the horizontal direction and 1/2 in the vertical direction with respect to the input image compression information.
The image information conversion device described in the above. 17. The horizontal down-sampling means,
17. The image information conversion apparatus according to claim 16, wherein downsampling is performed using a low-pass filter having several taps. 18. The vertical downsampling means,
The input image compression information downsampled by the horizontal downsampling means is divided by 1 / in the horizontal direction.
By performing downsampling of 1/2 times in the vertical direction on an image of progressive scanning having an image frame of 4 in the vertical direction and 1/2 in the vertical direction, the input image compression information is reduced by 1/4 in both the horizontal and vertical directions. 2. The image information conversion apparatus according to claim 1, wherein a progressive scan image having an image frame is generated. 19. The vertical downsampling means includes:
19. The image information conversion apparatus according to claim 18, wherein downsampling is performed using a low-pass filter having several taps. 20. The encoding means, wherein the horizontal downsampling means adjusts the number of pixels for encoding in macroblock units in the horizontal direction and the vertical downsampling means in the vertical direction. The image information conversion device according to claim 1, wherein 21. The output image compression information includes an intra-coded image coded in a frame, a forward prediction coded image predicted and coded by referring to another image in a forward direction between frames, and A bidirectional predictive coded image that is predictively coded with reference to another image in the forward and backward directions between frames, and a sprite coded image, wherein the coding unit includes the intra coded image. 2. The image information conversion apparatus according to claim 1, wherein said forward prediction encoded image is encoded for each image. 22. The image information, which has been motion-compensated by a motion vector, further comprising a motion vector synthesizing means for synthesizing the motion compensation vector, and based on the motion vector of the input image compression information, 2. The image information conversion apparatus according to claim 1, wherein a motion vector corresponding to the image output from the sampling unit is synthesized, and the encoding unit performs encoding based on the motion vector. 23. The apparatus according to claim 22, further comprising a motion vector detecting means for detecting a motion vector based on the motion vector synthesized by said motion vector synthesizing means.
The image information conversion device described in the above. 24. An image information conversion method for converting the resolution of image compression information obtained by performing discrete cosine conversion on an image in units of macroblocks composed of eight-component pixels in both the horizontal and vertical directions, wherein an image obtained by interlaced scanning is encoded. Of the eight discrete cosine transform coefficient coefficients in both the horizontal and vertical directions of the macroblocks constituting the input image compression information, an interlaced scan image is decoded using only four low-frequency components in the horizontal direction and eight components in the vertical direction. A scanning step of selecting one of the first field and the second field constituting the interlaced image decoded in the decoding step to generate a progressively scanned image; A horizontal down-sampling step of horizontally down-sampling the image generated by A vertical downsampling step of vertically downsampling an image downsampled in the downsampling step; and an output image having a lower resolution than the input image compression information. A coding step of coding the compressed information. 25. The input image compression information according to the MPEG2 standard, and the output image compression information
The image information conversion method according to claim 24, wherein the method is based on four standards. 26. The horizontal downsampling step performs downsampling by 1/2 times in the horizontal direction, the vertical downsampling performs downsampling by 1/2 times in the vertical direction, and the output image compression information is The image information conversion method according to claim 24, wherein the input image compression information has a resolution of 1/4 in both the horizontal and vertical directions. 27. The input image compression information includes an intra-coded image coded in a frame, a forward prediction coded image predicted and coded by referring to another image in a forward direction between frames. And a bidirectional predictive coded image that is predictively coded with reference to another image in the forward and backward directions between frames, and determines the type of image that constitutes the input image compression information. And a discarding step of discarding the bidirectionally coded image while passing the coded image and the forward prediction coded image, and image decoding information through the discrimination step is input to the decoding step. The image information conversion method according to claim 24, wherein 28. The image information conversion method according to claim 27, wherein said decoding step decodes only an intra-coded image and a forward prediction coded image.