JP2001346214A - Image information conversion apparatus and method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To perform stable code amount control. An interlaced scanning MPEG2 image compression information (bit stream) is converted into a progressive scanning MPEG4 image compression information (bit stream).
The PEG2 image information decoding unit (bit stream) 8 is M
To extract information from PEG2 compressed image information (bit stream), the complexity calculator 15 calculates the complexity, K _p / K _b calculation unit 16 parametric K _p on the basis of the complexity, K _b Is calculated, and the MPEG4 image information encoding unit (I / P-VOP) 11 performs encoding using the parameters _Kp and _Kb .

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information conversion apparatus and method for converting image information, and more particularly, to image information (bit stream) such as MPEG compressed by orthogonal transform such as discrete cosine transform and motion compensation. And a method for converting image information used when receiving an image via a network medium such as satellite broadcasting, cable TV, or the Internet, or when processing the same on a storage medium such as an optical disk or a magnetic disk. .

[0002]

2. Description of the Related Art In recent years, image information is handled as digital data, and for the purpose of transmitting and storing information with high efficiency, compression is performed by orthogonal transform such as discrete cosine transform and motion compensation using redundancy inherent in image information. An image information compression system such as MPEG is provided. Then, an apparatus conforming to such an image information compression method is used for information distribution of a broadcasting station or the like,
It is becoming widespread in both information reception in general households.

In particular, MPEG2 (ISO / IEC 13
818-2) is defined as a general-purpose image encoding method that covers both interlaced scan images and progressive scan images, as well as standard resolution images and high definition images.

That is, according to the MPEG2 encoding and compression system, for example, a code amount (bit rate) of 4 to 8 Mbps is assigned to a standard resolution interlaced scan image having 720 × 480 pixels, and a high resolution having 1920 × 1088 pixels is assigned. By assigning a code amount (bit rate) of 18 to 22 Mbps to the interlaced scan image, a high compression rate and good image quality can be realized.

[0005] For these reasons, it is expected that MPEG2 will be used in a wide range of applications for professional use and consumer use. However, MPEG2 is mainly intended for high-quality coding suitable for broadcasting, and does not support, for example, a coding amount (bit rate) lower than that of MPEG1, that is, a coding method with a higher compression rate.

[0006] On the other hand, with the spread of portable terminals in recent years, it is expected that the need for an encoding system with a high compression ratio will increase in the future, and in response to this, standardization of the MPEG4 encoding system with a high compression ratio has been carried out. Have been done. Regarding this image coding method, ISO / IEC 14
International standard was approved as 496-2.

By the way, MPEG2 image compression information (bit stream) once encoded for digital broadcasting
There is a need to convert image compression information (bit stream) having a lower code amount (bit rate), which is more suitable for processing on a mobile terminal or the like.

In order to achieve such an object, “Field-to
-Frame Transcoding with Spatialand Temporal Downsa
mpling ”(Susie L Wee, John G. Apostolopoulos, and N
ickFeamster, ICIP 99, hereafter referred to as Document 1).
And an image information conversion device (transcoder) is provided.
I have.

As shown in FIG. 8, the image information conversion device (transcoder) provided in Document 1 includes a picture type discriminating unit 1, an MPEG2 image information decoding unit (I / P picture) 2, a thinning-out unit. It comprises a unit 3, an MPEG4 image information encoding unit (I / P-VOP) 4, a motion vector synthesizing unit 5, and a motion vector detecting unit 6.

[0010] The image information conversion apparatus includes an intra-coded image (I picture; I) encoded in a frame,
A forward predictive coded image (P picture; P) predictively coded by referring to the forward direction in the display order, and a bidirectional predictive coded image predictively coded by referring to the forward and reverse directions in the display order MPEG-2 image compression information (bit stream) of interlaced scanning composed of (B picture; B) is input.

[0011] The MPEG2 image compression information (bit stream) is supplied to an I / P
It is determined whether the picture is related to a picture or a B picture.
Output to the image information decoding unit (I / P picture) 2
The picture is discarded.

The processing in the MPEG2 image information decoding section (I / P picture) 2 is to decode the MPEG2 image compression information (bit stream) into an image signal, as in a normal MPEG2 image information decoding device.

A pixel value output from the MPEG2 image information decoding unit (I / P picture) 2 is input to a thinning unit 3. The thinning section 3 performs a 1/2 thinning process in the horizontal direction, and leaves only one of the data of the first field and the second field in the vertical direction, and discards the other. By such thinning, a progressively scanned image having a size of ４ of the input image information is generated.

The progressively scanned image generated by the thinning section 3 is converted to an MPEG4 image information encoding section (I / P-VOP) 4
Is encoded into an I-VOP encoded in the frame and a P-VOP predicted and encoded with reference to the forward direction in the display order, and output as MPEG4 image compression information (bit stream). VOP is Video object Pla
ne, which corresponds to a frame in MPEG2.

At this time, the motion vector information in the input MPEG2 image compression information (bit stream) is mapped to a motion vector for the decimated image information in the motion vector synthesizing unit 5 and the motion vector detecting unit 6
In, a highly accurate motion vector is detected based on the motion vector value synthesized by the motion vector synthesizing unit 5.

Reference 1 discloses an image information conversion apparatus for generating MPEG4 image compression information (bit stream) of a progressively scanned image having a size of 1/2 × 1/2 of the input MPEG 2 image compression information (bit stream). Is described. That is, for example, the input MPEG2 image compression information (bit stream) is transmitted using NTSC (National Telev.
If it complies with the standards of the ISion Committee, the output MPEG4 image compression information has an SIF size (352 × 240 pixels).

In the image information conversion apparatus shown in FIG. 8, an MPEG4 image information encoding unit (I / P-V
OP) 4 is to control the amount of code to output MPEG4
This is a major factor in determining the image quality of the image compression information (bit stream). ISO / IEC 14496-
In No. 2, the code amount control method is not specified, and each vendor has
It is possible to use a method that is considered optimal in terms of the amount of calculation and the output image quality. In the following, MPEG2 Test Model 5 (IS
O / IEC JTC1 / SC29 / WG11 N040
The method described in 0) will be described.

The flow of this code amount control will be described with reference to the flow shown in FIG. In the first step S11,
The image information encoding unit (I / P-VOP) 4 includes a target code amount (target bit rate) and a GOP (groove).
f pictures) configuration, and allocates bits to each picture. Here, a GOP is a set of pictures that can be randomly accessed.

That is, in step S11, the image information encoding unit (I / P-VOP) 4 determines the amount of bits allocated to each picture in the GOP, including the picture to be allocated, of the picture not yet decoded in the GOP. Are allocated based on the amount of bits allocated to (hereinafter referred to as R). This distribution is repeated in the order of the coded pictures in the GOP. At this time, the code amount is assigned to each picture using the following two assumptions.

First, it is assumed that the product of the average quantization scale code used when encoding each picture and the generated code amount becomes a constant value for each picture type unless the picture changes. Therefore, after encoding each picture, a variable X indicating the complexity of the screen is set for each picture type.
_i, X _p, updated by X _b (grobal complelxity measure) the following equation (1).

[0021]

(Equation 4)

Where S_i, S_p, S_bIs for picture encoding
Is the amount of generated code bits, and Q_i, Q _p, Q_bIs a picture
This is an average quantization scale code at the time of key coding. Also,
The initial value is the target code amount (target bit rate) bi
Using t_rate [bits / sec], equation (2)
The value is indicated by

[0023]

(Equation 5)

Second, when the ratios K _p and K _b of the quantized scale codes of the P and B pictures with respect to the quantized scale code of the I picture are equal to the values defined in the equation (3), the whole is always obtained. Assume that the image quality is optimized.

[0025]

(Equation 6)

That is, the quantization scale code of the B picture is always 1.4 times the quantization scale code of the I and P pictures. This is because the picture quality of the I and P pictures is improved by adding the code amount that can be saved in the B picture to the I and P pictures by coding the B picture somewhat coarsely compared to the I and P pictures. It is assumed that the image quality of the B picture that refers to this is also improved.

Based on the above two assumptions, the bit amount (T _i , T _p , T _b ) allocated to each picture of the GOP is a value shown in equation (4).

[0028]

(Equation 7)

Here, N _p and N _b are the numbers of P and B pictures which have not been encoded in the GOP.

Each time each picture is coded according to steps S11 and S12 based on the allocated code amount obtained in this manner, the bit amount R allocated to the uncoded picture in the GOP is calculated by the equation (5). ) To update.

[0031]

(Equation 8)

When encoding the first picture of the GOP, R is updated by equation (6).

[0033]

(Equation 9)

N is the number of pictures in the GOP. Also,
The initial value of R at the beginning of the sequence is 0.

Next, in step S12, the image information encoding device (I / P-VOP) 4 performs rate control using the virtual buffer. That is, in step S12, the image information encoding device (I / P-VOP) 4 allocates bits (T _i , T _p , T _b ) for each picture obtained by equation (4) in step S11, In order to match the actual generated code amounts, the quantization scale code is obtained by macroblock-based feedback control based on the capacity of three types of virtual buffers independently set for each picture.

First, prior to encoding the j-th macroblock, the occupancy of the virtual buffer is determined by equation (7).

[0037]

(Equation 10)

Here, d ₀ ⁱ , d ₀ ^p , and d ₀ ^b are the initial occupancy of each virtual buffer, B _j is the amount of generated bits from the head of the picture to the j-th macroblock, and MB_cnt is the number of bits in one picture. This is the number of macro blocks. Virtual buffer occupancy at the end of each picture encoding (d _{MB_cnt} ⁱ , d _{MB_cnt} ^p ,
d _{MB_cnt} ^b ) is used as an initial value (d ₀ ⁱ , d ₀ ^p , d ₀ ^b ) of the virtual buffer occupancy for the next picture of the same picture type.

Next, the quantization scale code for the j-th macroblock is calculated by equation (8).

[0040]

[Equation 11]

Here, r is a variable called a reaction parameter that controls the response of the feedback loop, and is given by equation (9).

[0042]

(Equation 12)

The initial value of the virtual buffer at the start of encoding is given by equation (10).

[0044]

(Equation 13)

Finally, in step S13, the image information encoding device (I / P-VOP) 4 performs adaptive quantization for each macroblock in consideration of visual characteristics. That is,
In step S13, the image information encoding unit (I / P-
VOP) 4 is to quantize the quantized scale code obtained in step S12 more finely in a flat portion where the deterioration is conspicuous visually and coarsely in a complicated portion of the picture where the deterioration is relatively inconspicuous. , And is changed by a variable called an activity for each macroblock.

The activity is calculated by using the pixel value of the luminance signal of the original picture and the pixel values of a total of 8 blocks of 4 blocks in the frame discrete cosine transform mode and 4 blocks in the field discrete cosine transform mode. Given by (11).

[0047]

[Equation 14]

Here, P _k is the pixel value in the luminance signal block of the original image. The minimum value in the equation (11) is
This is because the quantization is made fine when there is a flat portion even in only a part of the macro block.

Further, according to equation (12), the value is 0.5 to
A normalization activity Nact _j having a range of 2 is obtained.

[0050]

(Equation 15)

Here, avg_act is the average value of act _j in the picture coded immediately before.

The quantized scale code mquant _j taking into account the visual characteristics is given by equation (13) based on the quantized scale code Q _j obtained in step S12.

[0053]

(Equation 16)

It is known that the above-mentioned code amount control method defined in the MPEG2 Test Model 5 has the following restrictions, and when performing actual control, it is necessary to take measures against these restrictions. That is, the first limitation is that the first step S11 cannot respond to a scene change, and the third step S1
This means that the parameter avg_act used in step 3 has an incorrect value. The second limitation is that MPEG2 and MP
VBV (VideoBuffer Ve) specified in EG4
rifier) is not guaranteed.

By the way, the document "Theoretical analysis of MPEG compression efficiency and its application to code amount control" (IEICE Technical Report, IE-9).
5, DSP95-10, May 1995, hereinafter referred to as Reference 2).
The code amount control method defined in model 5 is MP
The EG-2 image encoding device does not always provide good image quality.

This document 2 proposes the following method as a method for giving an optimal code amount distribution for each frame in a GOP, particularly for giving a good image quality.
That is, let N _I , N _P , and N _B be the numbers of I, P, and B pictures that have not been encoded in the GOP, and let R _I , R _P , and R _B be the code amounts assigned to these.
Also, under the fixed rate condition given by equation (14),
Q _I , Q _P , Q
_B, and m is an order relating the quantization step size and the reproduction error variance (that is, the quantization step size is m
It is assumed that minimizing the average value of the powers will minimize the reproduction error variance). Then, consider minimizing equation (15).

[0057]

[Equation 17]

[0058]

(Equation 18)

The average quantization scale Q and the code amount R in each frame are represented by Test Model.
5 is related to the complexity X of each frame, which is also a medium variable used in Expression 5, as shown in Expression (16).

[0060]

[Equation 19]

The formula (1) is also taken into consideration while considering the relationship of the formula (16).
R _I , which minimizes equation (15) under the constraint of 4)
When R _P and R _B are calculated using the Lagrange's undetermined multiplier method, the following values are obtained as optimal R _I , R _P and R _B.

[0062]

(Equation 20)

When α = 1, equation (17) and MPEG2
It can be said that the relationship with Expression (4) in the code amount control method defined in Test Model 5 is as follows. That is, equation (17) uses the parameters K _p and K _b that are the code amount control parameters as the complexity X _I ,
X _P, according to X _B, nothing but that they are adaptively calculated as in Equation (18).

[0064]

(Equation 21)

In Reference 2, as the value of 1 / (1 + m),
It is shown that good image quality can be obtained by setting the value to about 0.6 to 1.2.

[0066]

In the image information conversion apparatus shown in FIG. 8, an MPEG4 image information encoding apparatus (I / P
-VOP) 4, the MPEG2 Test Mod
When the code amount control is performed using the same method as that defined in e15,
Since it is impossible to cope with a change in complexity within a GOP, stable code amount control becomes difficult,
It is conceivable to cause image quality deterioration.

The present invention has been proposed in view of the above-mentioned circumstances, and in an image information conversion apparatus and method for converting image information, a GOP caused by a scene change or the like.
It is an object of the present invention to provide an image information conversion apparatus and method capable of performing stable code amount control in response to a change in complexity within the image.

[0068]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides an interlaced scan input image compression information compressed by a first compression encoding system by a second compression encoding system. The method converts the compressed output image compression information into compressed output image compression information, extracts information for each frame of the input image compression information, decodes the input image compression information, and sets a parameter based on the extracted information. Is determined, and the output is encoded into the compressed information using the parameters.

According to the present invention, MPEG2 image compression information (bit stream) of interlaced scanning is used as input image compression information, and MPEG4 image compression information (bit stream) of sequential operation is used as output image compression information. These MPEG2 image compression information (bit stream) and MPEG4 image compression information are composed of a pixel block composed of a plurality of pixels, that is, a macroblock.

That is, the present invention provides an interlaced scanning MP
EG2 image information compression device (bit stream) is input, picture type discrimination unit, MPEG2 image information decoding unit (I / P picture), thinning unit, delay buffer, MP
EG4 picture information encoding unit (I / P-VOP), the motion vector synthesis unit, the motion vector detecting section, information buffer, complexity combines city calculator and K _p / K _b calculation unit, a code amount allocation for each frame It is intended to provide means for outputting progressively scanned MPEG4 image compression information (bit stream) in a state optimized for the image.

In the above configuration, the picture type discriminating section stores only the data relating to the I / P picture in the input MPEG2 image compression information (bit stream).
Discard the pictures. The MPEG2 image information decoding unit (I / P picture) converts the compression information (bit stream) relating to the I / P picture, which is the output of the picture type discrimination unit, into all the 8th-order discrete cosine coefficients in both the horizontal and vertical directions. A decoding process using the used or low-frequency component alone is performed. The thinning-out unit extracts only the first field or the second field of the image information output from the MPEG2 image information decoding unit (I / P picture) and converts it into a sequentially scanned image. Down-sampling to convert to
The delay buffer is for one frame or one GOV (G
Image information for realizing a delay of (loop of VOP) is accumulated. MPEG4 image information encoding unit (I
/ P-VOP) encodes the image information to be output from the delay buffer by the MPEG4 encoding method. The motion vector synthesizing unit performs a motion for the scan-converted image data based on a motion vector value in the input image compression information (bit stream) detected by the MPEG2 image information decoding unit (I / P picture). Perform mapping to vector values. The motion vector detection unit performs highly accurate motion vector detection based on the motion vector value output from the motion vector synthesis unit. The information buffer stores the code amount (the number of bits) assigned to each frame and the average quantization scale in each frame, which are obtained when the MPEG2 image information decoding unit (I / P picture) performs the decoding process. . The complexity calculating unit calculates a code amount (the number of bits) assigned to each frame in the input image compression information (bit stream) stored in the information buffer and an average quantization scale in each frame. Calculate the complexity for the frame,
K _p / K _b calculation unit, the complexity for each frame calculated in complexity calculation unit, M
Parametric K _p used PEG4 image information encoding unit in the (I / P-VOP) in the code amount control, to calculate the K _b.

[0072]

Embodiments of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, an image information conversion apparatus to which the present invention is applied includes a picture type discriminating section 7 and an MPEG
2 picture information decoding section (I / P picture) 8, thinning section 9, delay buffer 10, MPEG4 picture information coding section (I / P-VOP) 11, motion vector synthesis section 12
, A motion vector detecting unit 13, an information buffer 14,
A complexity calculating unit 15, and a K _p / K _b calculating unit 16.. As will be described later, a device configuration that does not require the delay buffer 10 is also possible.

The image information conversion apparatus includes an intra-coded image (I picture; I) encoded in a frame,
A forward predictive coded image (P picture; P) predictively coded by referring to the forward direction in the display order, and a bidirectional predictive coded image predictively coded by referring to the forward and reverse directions in the display order MPEG-2 image compression information (bit stream) of interlaced scanning composed of (B picture; B) is input.

The MPEG2 image compression information (bit stream) is transmitted to the I / P
It is determined whether the picture is related to a picture or a B picture.
Output to the image information decoding unit (I / P picture) 8
The picture is discarded.

The MPEG2 image information decoding section (I / P picture) 8 decodes MPEG2 image compression information (bit stream) into an image signal. Here, since the data relating to the B picture is discarded in the picture type discriminating unit 7, the MPEG2 image information decoding unit (I / P picture) 1 only needs to have a function capable of decoding only the I / P picture. .

Further, the MPEG2 image information decoding device (I
/ P picture) 8 extracts information from the MPEG2 image compression information (bit stream) as extraction means, and outputs the average value Q over the entire frame of the quantization scale used in the decoding process, and the MPEG2 image compression information (bit The total number of bits (number of bits) B allocated to the frame in the stream) is stored in the information buffer 14.

The pixel value output from the MPEG2 image information decoding section (I / P picture) 8 is input to the thinning section 9. The thinning section 9 performs 1/2 thinning processing in the horizontal direction, and leaves only data of one of the first field and the second field in the vertical direction, and discards the other. By such thinning, a progressively scanned image having a size of ４ of the input image information is generated.

The image output from the thinning section 9 is converted to an MPEG4 image information encoding section (I / P-VOP) 11
In order to perform encoding in units of macroblocks composed of 16 × 16 pixels, in both the horizontal and vertical directions,
The number of pixels must be a multiple of 16. Thinning section 9
In, the supplement or discard of the pixels for this is performed simultaneously with the thinning.

For example, if the input MPEG2 image compression information (bit stream) is NTSC (National Televis)
In the case of an image conforming to the standards of the Ion System Committee, that is, an interlaced scan image of 720 × 480 pixels and 30 Hz, the image frame after thinning out has an SIF (360 × 240 pixels) size. For this image, the thinning unit 3 discards, for example, the eight lines at the right end or the left end in the horizontal direction to obtain 352 × 240 pixels.

By changing the operation of the thinning section 9, the other image frames, for example, about 1 /
It is also possible to convert the image into a QSIF (176 × 112 pixel) size image, which is a 4 × １ ／ image frame.

Further, the above-mentioned document 1 discloses that the processing in the MPEG2 image information decoding section (I / P picture) 8 includes, in each of the horizontal and vertical directions, the MPEG2 image compression information (bit stream) in the input. 8
An image information conversion apparatus that performs decoding processing using all of the following discrete cosine transform coefficients is described. However, the description is not limited to the apparatus illustrated in FIG. 1. Only the horizontal direction, or both the horizontal and vertical directions, By performing decoding using only low-frequency components of the eighth-order discrete cosine transform coefficients, it is possible to reduce the amount of computation and video memory capacity involved in decoding while minimizing image quality degradation.

The progressively scanned image generated by the thinning section 9 is delayed by one frame by a delay buffer 10, and is then encoded by an MPEG4 image information encoding section (I / P-VOP) 1.
1 and an I-VOP coded in a frame and a P-VOP coded predictively with reference to the forward direction in the display order.
And output as MPEG4 image compression information (bit stream).

At this time, the motion vector information in the input MPEG2 image compression information (bit stream) is mapped to the motion vector corresponding to the decimated image information in the motion vector synthesizing unit 12, and the motion vector detecting unit 13 A high-precision motion vector is detected based on the motion vector value synthesized by the motion vector synthesis unit 12.

VOP means Video Object Plane, and corresponds to a frame in MPEG2. Further, I-VOP is an intra-coded VOP corresponding to an I picture, P-VOP is a forward predictive coded VOP corresponding to a P picture, and B-VOP is a bidirectional predicted coded VOP corresponding to a B picture.

Next, the delay buffer 10 and the information buffer 1
4, complexity calculating unit 15, K _p / K _b calculating section 16
Will be described below.

As shown in FIG. 1, when the frame type is converted by using the picture type discriminating unit 7, the parameter K _b is not required, but is similar to the apparatus shown in FIG. Depending on the configuration, a device configuration that does not convert the frame rate but converts only the image frame size can be considered.
In the following, the parameter _Kb will also be described.
In this case, the picture type discrimination unit 7 becomes unnecessary, and the MPEG2 image information decoding unit 8 and the MPEG4 image information encoding unit 11 perform decoding of a B picture and encoding of a B-VOP, respectively. Become.

When the decoding process is executed in the MPEG2 image information decoding unit (I / P picture) 8, the number of bits B assigned to each frame in the input MPEG2 image compression information (bit stream) is Information about the average quantization scale Q is extracted, and the information is stored in the information buffer 14.

In the complexity calculating device 15, in the MPEG2 image compression information (bit stream) stored in the information buffer 14, the information on the number of bits B and the average quantization scale Q allocated to each frame is calculated based on The complexity X for each frame is calculated as in equation (19). Hereinafter, expressed by equation (19), to I, P, respectively X _I the complexity for B-pictures, X _P, and X _B.

[0090]

(Equation 22)

K_p/ K_bIn the calculation device 16, the comp
Each frame calculated by the lexity calculation unit 15
Based on the complexity X for
Therefore, the parameter K for controlling the code amount is_p, K_bIs calculated
MPEG4 image information encoding unit (I / P-VOP)
Output to 11 K calculated in this way_p, K _b
Is the input MPEG2 image compression information (bit stream
Calculated based on the information of the frame in
Therefore, it is possible to respond to scene changes.

The operation principle of calculating K _p and K _b described above will be described with reference to FIG.

In the first step S21, the MPEG
The two-image information decoding unit (I / P picture) 8 extracts the code amount (number of allocated bits) B and the average quantization scale Q of each frame from the input MPEG2 image information compression information (bit stream), The data is stored in the buffer 14.

In step S22, the complexity calculating unit 15 calculates the complexity for each frame using the code amount (number of allocated bits) B and the average quantization scale Q of each frame stored in the information buffer 14. calculate. In step S23, K _p / K _b calculating unit 16, using the complexity obtained in Step S22, calculating parametric K _p, the K _b according to equation (18). In step 24, the _Kp / _Kb calculation unit 16
Transmits the calculated parameters Kp and Kb to the image information encoding unit (I / P-VOP) 11.

Here, the expression (1) in step S23 is used.
In the execution of 8), by setting the value of 1 / (1 + m) to 1.0, exponential calculation becomes unnecessary, and high-speed processing becomes possible. Also, when the value of 1 / (1 + m) is set to a value other than 1.0, high-speed processing can be performed by preparing a table in advance and executing the exponent calculation with reference to the table.

Note that also in the present invention, the initial value of the virtual buffer is given by equation (10), and the values of K _p and K _b used at that time are as defined in equation (3). And

[0097] Updating of K _p and K _b are conceivable three methods as described below. In the following, in the image information conversion device shown in FIG.
, That is, a case where frame rate conversion is also performed. As described above, in this case, the processing of the B picture / B-VOP is unnecessary.

In the following FIGS. 3 to 7, the GOP of the input MPEG2 image compression information (bit stream) is shown.
The structure is n = 12 / m = 3, and its constituent elements are, for example, as shown in FIG. 3A, ｛I ₁ , B ₁ , B ₂ , P ₁ , B ₃ , B ₄ , P ₂ , B ₅ , B ₆ ,
Let P ₃ , B ₇ , B ₈ ｝. At this time, it is assumed that the GOV structure of the output MPEG4 image compression information (bit stream) is {I ₁ , P ₁ , P ₂ , P ₃ }, for example, as shown in FIG. 3D. The GOV is a randomly accessible unit composed of VOPs.

From the I ₁ , P ₁ , P ₂ , and P ₃ constituting the above-described MPEG2 image compression information bit stream, the complexity calculator 15 calculates the complexity X _{i as} shown in FIG. , X _p1 , X _p2 , and X _p3 are calculated.

A first method for updating K _p and K _b is shown in FIG.
In the image information conversion device shown in FIG.
Is not required. The process at this time is shown in FIG. In the first method, the _Kp / _Kb calculator 16 calculates the initial value X of the complexity of the P-picture as shown in Expression (20) according to Expression (2) prior to the processing.
_{p_init} is set.

[0101]

(Equation 23)

Here, bit_rare_MPEG2 is
It represents the bit rate of the input MPEG2 image compression information. If the picture type discriminating unit 7 does not exist and the frame rate conversion is not performed, the initial value of the complexity of the B picture is also required. However, this also corresponds to the equation (2) and the equation (2). It is calculated as in 21).

[0103]

(Equation 24)

At this time, the MPE which is the output shown in FIG.
G4 compressed image information K _p (K _p0) shown in C of FIG. 3 used in coding the first-th I-VOP (I ₁₎ in (bit stream) includes an input shown in A of FIG. 3 It is calculated from the _complexities X ₁ and X _{p_init} shown in FIG. 3B obtained from I ₁ in the MPEG2 image compression information using Expression (18). K _p (K _p1 ) shown in C of FIG. 3 used when encoding the _first I-VOP (I ₁ ) in the MPEG4 image compression information (bit stream) to be output shown in D of FIG. , X _i shown in FIG. 3B and the complexity X _p1 shown in FIG. 3B obtained from P ₁ in the MPEG2 image compression information to be input shown in FIG. It is calculated by the _Kp / _Kb calculator 16. The same applies to K _p (K _p2 , K _p3 ) shown in C of FIG. 3 used when encoding the second and third P-VOPs (P ₂ , P ₃ ) shown in D of FIG. It is. Note that X _i shown in FIG. 3B and P P in the MPEG2 image compression information to be input shown in FIG.
From complexity X _p3 shown in ₃ of FIG. 3 B is, X _{P_init} in the next GOV is calculated.

The second method uses the image information conversion shown in FIG.
The device requires a delay buffer 10 for one VOP
This is the case. FIG. 4 shows a first example of the processing at this time.
MPEG2 image compression information (video data) to be input shown in FIG.
Stream) ₁, P₁Of FIG. 4 obtained from
Complexity X shown in B_I, X_P1From equation (18)
By using the complexity calculating unit 15 in FIG.
K shown in_p1Is calculated, and this K_p1Is the output shown at D in FIG.
MPEG4 image compression information (bit stream)
The first I-VOP (I₁) When encoding
Used. MPEG2 image as input shown in FIG.
I of image compression information (bit stream)₁, P_ThreeObtained from
Complexity X shown in FIG.₁, X_p3From the formula
K using (18)_p/ K_bThe calculation unit 16 calculates C in FIG.
K shown in_p3Is calculated. This K_p3Is shown in D of FIG.
MPEG4 image compression information to be output (bit stream
The second and third P-VOPs (P_Two,
P_Three) Is used for encoding.

Further, the image information conversion apparatus shown in FIG.
And requires a delay buffer 10 for one VOP
FIG. 5 shows a second example of the processing in. The first shown in FIG.
The difference from the example is that the input MPEG2 image shown in FIG.
I in image compression information (bit stream)₁, P₁Profit
Complexity X shown in FIG. 5B₁, X_p1From
K using equation (18)_p/ K_bCalculated by the calculation unit 16
K shown in FIG. 5C _p1Is the output shown in D of FIG.
MPEG4 image compression information (bit stream)
The first I-VOP (I₁) And the first P-VO
P (P₁) Is used for encoding. Also, in FIG.
MPEG2 image compression information (bits
Trim)₁, I_P3In FIG. 5B obtained from
Complexity X shown₁, X_p3From equation (18)
5 calculated by the complexity calculator 15
K shown in C_p3Is an MPEG output as shown in FIG.
Third in 4 image compression information (bit stream)
P-VOP (P_Three) Is used to encode only
You.

The third method is a case where the image information conversion apparatus shown in FIG. 1 requires a delay buffer 10 for one GOV. FIG. 6 shows a first example of the processing at this time.
In the figure, first, the MPEG input as shown in FIG.
Complexity calculation unit 1 from the _first I picture (I ₁ ) in the two-image compression information (bit stream)
5 complexity X _I shown in B of FIG. 6 are calculated at the same time, from the first to third P-picture shown in A of FIG. 6 (P ₁ to P _3), Figure in complexity calculation unit 15 Complexity X _p1 , X _p2 , X _p3 shown in 6B
Is calculated. Next, an average _complexity X _{p_GOP} in the GOP is calculated from the _complexity X _p1 , X _p2 , and X _p3 shown in FIG. 6B as shown in FIG. _6C . And Complexity X _I and Average Complexity X
_The parameter K _{p_GOP} for this GOP (GOV) is calculated from the _{p_GOP} by the K _p / K _b calculating unit 16 using the equation (18).
Find the value of The value of this parameter K _{p_GOP} is MPEG4
It is used for code amount control in the image information encoding unit (I / P-VOP) 11. That is, as shown in D of FIG. 6, when encoding the I-VOP and P-VOP of the MPEG4 image compression information (bit stream) to be output,
The value of the parameter _{Kp_GOP} is used.

FIG. 7 shows a second example of processing in the case where the image information conversion apparatus shown in FIG. 1 requires a delay buffer 10 for one GOV. Similar to FIG. 6, firstly, in Fig. 7 in the first numbered complexity calculation unit 15 from the I-picture I ₁ in the MPEG2 image compression information (bit stream) in which the input shown in A of FIG. 7 B
As shown in complexity X _I is calculated, at the same time, the first to third P-picture shown in A of FIG. 7 (P ₁ ~
From P ₃ ), the complexity calculating unit 15 calculates the complexity X _{p1 to} X _p3 shown in FIG. 7B. Next, Yotsute to K _p / K _b calculation unit 16, X of _I and _{X p1, X p2, X p3} , using equation _{(18), K p1, K} p2 as shown in C in FIG. 7, K _p3 is calculated. Furthermore,
K _p1, average K _{P_GOP} from K _p2, K _p3 been calculated, is used in encoding the GOV. And D in FIG.
As shown in (1), when encoding the I-VOP and P-VOP of the MPEG4 image compression information (bit stream) to be output, the value of the parameter _{Kp_GOP} is used.

As described above, in the present embodiment, the variables K _p and K _b
Is adaptively calculated using the complexities X _I , X _P , and X _B of each frame as shown in Expression (18), instead of the fixed value defined in Expression (3). By using this for code amount control, the image quality of MPEG4 image compression information (bit stream) output from the MPEG4 image information encoding device is improved. That is, in the present embodiment, Expression (18) is used in an MPEG4 image information encoding device (I / P-VOP), so that the MPEG4 image compression information (bit) output from the image information conversion device shown in FIG. Stream).

[0110] That is, in the present embodiment, by performing the calculation of K _p and K _b according to equation (18) using the average quantizer scale and the number of allocated bits for each frame, more stable code amount control is performed , And also responds to scene changes.

As described above, the MPEG2 image compression information (bit stream) has been targeted for input, and the MPEG4 image compression information (bit stream) has been targeted for output. However, input and output are not limited to this. For example, MPEG-1 or H.264.
H.263 or other image compression information (bit stream).

[0112]

As described above, the present invention provides MPEG2 image compression information (bit stream) for interlaced scanning.
Is input, and in a state where the code amount allocation for each frame is optimized for the image, it is possible to provide means for converting the image data into progressively scanned MPEG4 image compression information (bit stream) and outputting the information. The present invention not only can perform more stable code amount control, but also supports scene changes.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image information conversion device.

FIG. 2 is a flowchart illustrating an operation principle of calculating K _p and K _b .

In the case of Figure 3 there is no delay buffer is a diagram showing an example of an update parametric K _p.

[4] in the case where one frame of delay buffers are present, it is a diagram showing a first example of updating the parametric K _p.

[5] in the case where one frame of delay buffers are present, it is a diagram showing a second example of updating the parametric K _p.

In the case of Figure 6 the delay buffer 1GOV component is present, is a diagram showing a first example of updating the parametric K _p.

[7] in the case where the delay buffer 1GOV component is present, it is a diagram showing a second example of updating the parametric K _p.

FIG. 8 is a block diagram showing a configuration of a conventional image information conversion device.

FIG. 9 is a flowchart showing the operation principle of the code amount control method.

[Explanation of symbols]

7 picture type discriminating unit, 8 MPEG2 image information decoding unit (I / P picture), 9 thinning unit, 10 delay buffer, 11 MPEG4 image information encoding unit (I / P
-Vop), 12 motion vector synthesis unit, 13 motion vector detection unit, 14 information buffer, 15 complexity calculation unit, 16 K _p / K _b calculator

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Teruhiko Suzuki 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Yoichi Yagasaki 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F term (reference) 5C059 LA07 LB05 MA00 MA03 MA05 MA23 MC22 NN01 PP05 PP06 PP07 SS02 SS08 SS11 SS13 TA07 TA08 TA25 TA46 TB03 TB08 TC19 TC38 TD03 UA02 UA05 5J064 AA01 BA16 BB03 BB12 BC01 BC26 BD02

Claims (33)

[Claims] An image information conversion apparatus for converting input image compression information of interlaced scanning compressed by a first compression encoding method into output image compression information of progressive scanning compressed by a second compression encoding method. , An extraction unit for extracting information of each frame of the input image compression information, a decoding unit for decoding the input image compression information, and a medium for determining a parameter based on the information extracted by the extraction unit. An image information conversion apparatus, comprising: a variable determining unit; and an encoding unit that performs encoding on the output image compression information using the parameter. 2. An intra coded image coded in a frame, a forward prediction coded image predicted and coded by referring to the forward direction in a display order, and a display prediction order constituting the input image compression information. The type of the bidirectionally coded image predictively coded with reference to the forward direction and the backward direction is determined, and the intra coded image and the forward coded image are passed but the bidirectional coded image is passed. 2. The image information conversion apparatus according to claim 1, further comprising a conversion unit for converting the frame rate by discarding the frame rate. 3. The image information conversion apparatus according to claim 2, wherein said decoding means decodes only the intra-coded image and the forward prediction coded image. 4. The image information conversion apparatus according to claim 2, wherein said encoding means encodes only the intra-coded image and the forward prediction coded image. 5. A thinning means for discarding or supplementing pixels in order to perform processing in units of a predetermined pixel block composed of a plurality of pixels when encoding the compressed output image information. The image information conversion device according to claim 1. 6. The input image compression information is composed of a discrete cosine transform block composed of 8 × 8 discrete cosine transform coefficients, and the decoding means extracts only low-frequency components of the discrete cosine transform block. 2. The image information conversion apparatus according to claim 1, wherein the used partial decoding is performed. 7. The input image compression information is composed of a first field and a second field, and has thinning means for converting interlaced scanning into sequential scanning by extracting one of the first field and the second field. 2. The image information conversion device according to claim 1, wherein: 8. The decoding means, when decoding the input image compression information by the decoding means, is given as a product of an average quantization scale and a generated code amount in the frame in the input image compression information. The image information conversion device according to claim 1, wherein the complexity is extracted. 9. The input image compression information includes an intra-coded image coded in a frame, a forward prediction coded image predicted and coded by referring to a forward direction in a display order, and forward and backward directions. , And the extraction means comprises a compressor corresponding to each of the intra-coded image, the forward predicted coded image, and the bidirectional predicted coded image. 9. The image information conversion apparatus according to claim 8, wherein the codeness is extracted, and the encoding unit controls the code amount by using a parameter calculated using the complexity. 10. The image information conversion apparatus according to claim 9, wherein said parameter is given by the following equation. (Equation 1) 11. The image information conversion apparatus according to claim 10, wherein the value of the parameter 1 / (1 + m) of the parameter is 0.6 to 1.2. 12. The image information conversion apparatus according to claim 11, wherein a value of an exponent 1 / (1 + m) of the parameter is set to 1.0. 13. The image information conversion apparatus according to claim 11, wherein the operation of the parameter is performed by referring to a table prepared in advance. 14. The image information conversion apparatus according to claim 9, wherein a delay buffer for delaying image information to be encoded by said encoding means is not provided in order to control a code amount in said encoding means. 15. The image information according to claim 14, wherein an initial value of complexity corresponding to the forward prediction coded image is given by the following equation based on the code amount of the input image compression information. Conversion device. (Equation 2) However, the code amount of the input image compression information is set to bit_rate_MPEG2
When the initial value of complexity X _P corresponding to the forward predictive coded picture and X _{P_init.} 16. The image information according to claim 14, wherein an initial value of the complexity corresponding to the bidirectionally predicted coded image is given by the following equation based on the code amount of the input image compression information. Conversion device. (Equation 3) However, the code amount of the input image compression information is set to bit_rate_MPEG2
When the initial value of complexity X _B corresponding to the bidirectional predictive coded picture and X _{B_init.} 17. The method of claim 1, wherein the first compression encoding method is MPEG.
2. The second compression encoding method is MPEG4, and has a delay buffer for delaying image information to be encoded by the encoding means over a period of 1 VOP corresponding to an image of the output image compression information. 10. The image information conversion device according to claim 9, wherein: 18. A frame rate is determined by discriminating the type of the image of the input image compression information and passing the intra-coded image and the forward predicted coded image but discarding the bidirectional predicted coded image. A conversion unit for converting the GOP including n pictures corresponding to the image of the input image compression information when discarding the bidirectionally coded image in the input image compression information. Is the first parameter obtained from the complexity for the first intra-coded image of the input image compression information and the complexity for the first forward predictive coded image. Used when encoding the first intra-coded image of the above, and the complexity of the input image compression information with respect to the first intra-coded image. And when using the second parameter obtained from the complexity for the second forward prediction coded image when encoding the first forward prediction coded image of the output image compression information, The nth parameter obtained from the complexity of the first intra-coded image of the input image compression information and the complexity of the last P picture of the GOP is the (n-1) th parameter of the output image compression information. 18. The image information conversion apparatus according to claim 17, wherein the image information conversion apparatus is used when encoding an nth forward prediction encoded image. 19. For a GOP composed of n pictures corresponding to an image of the input image compression information, the encoding means includes: a complexity for the first intra-encoded image of the input image compression information; The first parameter obtained from the complexity for the first forward prediction coded image and the complexity for the first bidirectional prediction coded image is the first intra of the output image compression information. Used when encoding an encoded image, the complexity of the input image compression information for the first intra-encoded image, both the complexity for the second forward-encoded image and the second The second parameter obtained from the complexity for the forward prediction coded image is the first forward prediction coding of the output image compression information. Used when encoding an image, the complexity of the input image compression information for the first intra-coded image, the complexity for the forward predictive encoded image at the end of the GOP, and the bidirectional predictive encoding at the end of the GOP The n-th parameter obtained from the complexity of the image is represented by (n-
18. The image information conversion device according to claim 17, wherein the image information conversion device is used when encoding the 1) -th and n-th forward prediction encoded images. 20. A frame rate is determined by discriminating the type of the image of the input image compression information and passing the intra-coded image and the forward predicted coded image, but discarding the bidirectional predicted coded image. Having conversion means for converting,
When discarding the bidirectionally coded image in the input image compression information, for the GOP composed of n pictures corresponding to the image of the input image compression information, the coding unit performs processing of the input image compression information. The first parameter obtained from the complexity for the first intra-coded image and the complexity for the first forward-coded image is the first intra code of the output image compression information. Used when encoding the first encoded image and the first forward-predicted encoded image, and the complexity and the second forward-predicted encoded image for the first intra-coded image of the input image compression information. The second parameter obtained from the complexity of the output image compression information is used to encode the second predicted coded image of the output image compression information. Complexity and GO for the first-th intra-coded image of the input image compression information
P is used when encoding the nth parameter obtained from the complexity for the last forward prediction coded image into the nth forward prediction coded image of the output image compression information. 18. The image information conversion device according to claim 17, wherein: 21. For a GOP composed of n pictures corresponding to an image of the input image compression information, the encoding means includes: a complexity of the input image compression information for a first intra-encoded image; The first parameter obtained from the complexity for the first forward prediction coded image and the complexity for the first bidirectional prediction coded image is the first intra of the output image compression information. Used when encoding the coded image and the first forward prediction coded image, the complexity of the input image compression information for the first intra coded image, the second forward prediction coding The second parameter obtained from the complexity for the image and the complexity for the second bidirectionally predicted coded image is the output image pressure. The second predictive coded image of the compressed information is used when coding the coded image, and the complexity of the first intra coded image of the input compressed image information and the forward predicted coded image of the last GOP are used. 18. The image information conversion according to claim 17, wherein the n-th parameter obtained from the complexity of the output image compression information is used to encode the n-th forward prediction encoded image of the output image compression information. apparatus. 22. The image information according to claim 9, further comprising a delay buffer for one GOP composed of a VOP corresponding to the output image compression information, in order to perform code amount distribution in said encoding means. Conversion device. 23. The complexity of an intra-coded image included in a GOP composed of a picture corresponding to the image of the input image compression information, and the complexity X of a forward prediction coded image included in the GOP. Average and G
The image information conversion device according to claim 9, wherein the parameter for the GOP is calculated using the average value of the complexity of the forward prediction coded image included in the OP. 24. The complexity X _I of an intra-coded image included in a GOP composed of a picture corresponding to the image of the input image compression information and the complexity of a forward prediction coded image included in a GOP. The obtained average value of the parameters corresponding to each picture of the forward prediction coded image is represented by a parameter K _P , the complexity of the intra-coded image, and the complexity of the bidirectional predicted coded image included in the GOP. 10. The image information conversion apparatus according to claim 9, wherein an average value of the parameters corresponding to each picture of the bidirectionally coded image obtained from the parameter is used as a parameter. 25. The image according to claim 7, further comprising a virtual buffer for determining a quantization scale code, wherein a parameter value for providing an initial value of the virtual buffer is set to 1. Information conversion device. 26. An image information conversion method for converting input image compression information of interlaced scanning compressed by a first compression encoding method into output image compression information of progressive scanning compressed by a second compression encoding method. A parameter extracting means for extracting and extracting information for each frame of the input image compression information, decoding the input image compression information, and determining a parameter based on the extracted information; Coding means for performing coding on the output image compression information using a variable. 27. An intra-coded image coded in a frame, a forward prediction coded image predictively coded by referring to the forward direction in a display order, and a display prediction order constituting the input image compression information. The type of the bidirectionally coded image predictively coded with reference to the forward direction and the backward direction is determined, and the intra coded image and the forward coded image are passed but the bidirectional coded image is passed. 28. The image information conversion method according to claim 26, wherein the frame rate is converted by discarding the frame rate. 28. The method according to claim 27, wherein only the intra-coded image and the forward prediction-coded image are decoded.
The described image information conversion method. 29. The method according to claim 27, wherein only the intra-coded image and the forward prediction coded image are coded.
The image information conversion device described in the above. 30. The image information conversion method according to claim 26, wherein in order to perform processing in units of a predetermined pixel block composed of a plurality of pixels, thinning is performed by discarding or supplementing pixels. . 31. The input image compression information is composed of a discrete cosine transform block composed of discrete cosine transform coefficients of 8 × 8 order, and the decoding means determines only low-frequency components of the discrete cosine transform lock. 27. The image information conversion method according to claim 26, wherein the used partial decoding is performed. 32. The input image compression information is composed of a first field and a second field, and one of the first field and the second field is extracted to convert interlaced scanning into sequential scanning. The image information conversion method according to claim 26, wherein 33. The method according to claim 33, wherein, when the input image compression information is combined, a complexity given as a product of an average quantization scale and a generated code amount in the frame in the input image compression information is extracted. Claim 2
6. The image information conversion method according to 6.