JPH11252566A - Image signal decoding method and decoding device, image signal encoding method and encoding device, image signal processing method and processing system - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To suppress image quality degradation that occurs in the process of decoding of compression encoding and re-encoding of decoded image signals. SOLUTION: A bit stream is decoded by an MPEG decoder 1, and a decoded image signal is obtained. Multiplexer 2
, The decoded image signal is converted into a transmission image signal, and the control information Ic and the encoded feature point information Ip are transmitted to the re-encoder side. The control information Ic indicates a spatial and temporal relationship of the decoded image in the transmission image signal. The encoding feature point information Ip includes a picture coding type and the like. The MPEG encoder 5 receives the information Ic and Ip, cuts out the encoding target area, and forms a frame structure. Then, a stream is output by encoding the frame in the encoding target area. This stream has the same picture coding type, spatial and temporal relationship as the original stream before decoding, and can minimize image quality deterioration due to decoding and re-encoding.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal decoding method and apparatus, which are applied when decoding and re-encoding an image signal encoded by, for example, MPEG. The present invention relates to an encoding device, an image signal processing method, and a processing system.

[0002]

2. Description of the Related Art In recent years, MPEG (Moving Picture Experts Group), which is one of the inter-image compression coding systems, is being widely put into practical use. By using compression coding such as MPEG, the recording medium can be effectively used. As a system using MPEG, an MPEG stream is decoded, a decoded image signal is converted into a format of a transmission image signal such as NTSC, transmitted via a digital VTR or the like, and the transmission image signal is re-encoded by MPEG. There is something. Also, by connecting the image signal reproduced and decoded from the recording medium and the image signal from the outside, M
An editing system that encodes by PEG and records a stream on a recording medium can be configured. Further, even in the case of dubbing in which a video signal recorded on another recording medium is recorded by a recorder, a reproduced signal of another recording medium is decoded and re-encoded.

[0003] As in these cases, when re-encoding a decoded image signal, in order to improve the image quality when decoding a stream generated by the re-encoding, it is necessary to re-encode the stream (output of the encoder). ) Preferably matches the picture coding type of the stream before decoding (input of the decoder). In the case of MPEG, there are three types of picture types, I, P, and B.

[0004] An I-picture (Intra-coded picture) uses information that is closed only in one image when it is encoded. Therefore, at the time of decoding, decoding can be performed using only the information of the I picture itself. A P-picture (Predictive-coded picture) uses a temporally previous decoded I-picture or P-picture as a predicted picture (a reference picture for taking a difference). Whether to encode the difference from the motion-compensated predicted image, to encode without taking the difference,
The more efficient one is selected for each macroblock. A B picture (Bidirectionally predictive-coded picture) is a temporally previous I-picture or P-picture which is temporally preceding, and a temporally backward I-picture, We use three types of I-pictures or P-pictures already decoded, as well as interpolated pictures made from both. Among the three types of difference coding after motion compensation and intra coding, the most efficient one is selected for each macroblock.

[0005] Therefore, as a macroblock type,
Intra-frame coding (Intra) macroblock, forward (Fward) inter-frame prediction macroblock that predicts the future from the past, and backward (Backwrd) interframe prediction macroblock that predicts the future from the future, There is an interpolative macroblock to predict. I
All macroblocks in a picture are intra-coded macroblocks. The P picture includes an intra-frame coded macro block and a forward inter-frame prediction macro block. The B picture includes all four types of macroblocks described above.

Generally, an image decoded from an I or P picture has a lower image quality than an image decoded from a B picture. At the time of re-encoding, if a B picture is created by using a decoded image that was originally a B picture as an I or P picture, the quality of the B picture will be greatly degraded. In this sense, by matching the picture coding type of the re-coded output with the original one before decoding, it is possible to suppress image quality degradation due to decoding and re-coding. In order to reduce image quality deterioration, it is preferable to save information (referred to as codec information) such as a motion vector and a quantization scale used at the time of decoding, and to reuse the codec information at the time of re-encoding.

[0007]

Generally, a decoded image is often converted into a format of a transmission image signal such as NTSC and supplied to a re-encoding encoder. Therefore, the re-encoder cuts out a decoded image (a region originally MPEG-encoded) in the transmission image signal, and re-encodes a decoded image forming one frame from two fields. The transmission image signal includes information other than the decoded image, such as a blanking period and header information. The position of the spatially and temporally decoded image in the image signal for transmission is not fixed to a known position. Here, the decoded image means an image obtained as a result of decoding each picture in the original bit stream.

The spatial position of the decoded image differs depending on, for example, the application format. The image signal for transmission is DVD, IRD (Integrated Receiver / Decorde
r), the starting position in the vertical direction and the starting position in the horizontal direction are different depending on which one is derived from the digital VTR or the like. Also, depending on the format, 720
In some cases, only an effective image of 352 pixels × 240 lines exists in the area of pixels × 480 lines. In the temporal direction, in the case of MPEG, the relationship between the odd cycle and the even cycle in which each field of the transmission image signal is transmitted and the top field and the bottom field of the decoded image are not uniquely defined.

Therefore, for re-encoding, when extracting an image to be encoded, an area different from the original decoded image area is extracted, or when one frame is composed of two fields, a different combination from the original frame is used. Or had it. This reduces the effect of suppressing picture quality degradation by making the picture coding type the same. At the same time, at the time of re-encoding, the macroblock boundary is different from the original bit stream. Therefore, even if the image quality degradation is suppressed by reusing the codec information, the effect of reducing the effect occurs. .

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to solve the above-mentioned problems and to provide an image signal decoding method and apparatus, an image signal encoding method, and an image signal capable of re-encoding a decoded image with high image quality. An apparatus, an image signal processing method, and a processing system are provided.

[0011]

According to a first aspect of the present invention, there is provided an image signal decoding method for decoding data encoded by inter-picture predictive encoding, in order to achieve the above-mentioned object. And converts the decoded image signal into a transmission image signal, and outputs control information together with the transmission image signal. The control information indicates an indicator of a display start field of the decoded image and a display start line of the decoded image. An image signal decoding method characterized by being data. In this case, the control information is superimposed on the transmission image signal or output to a signal line different from the transmission image signal.

According to an eleventh aspect of the present invention, there is provided an image signal encoding method for encoding a decoded image of data encoded by inter-picture predictive encoding, wherein a transmission image signal composed of the decoded image, An indicator of a display start field and control information including data indicating a display start line of a decoded image are received, and based on the control information, a coding target area in a transmission image signal is configured, and the configured coding target An image signal encoding method characterized by performing inter-picture predictive encoding on a region.

A nineteenth aspect of the present invention is a video signal processing method for decoding data encoded by inter-picture predictive encoding and encoding the decoded data. The signal is converted into a transmission image signal, and control information is output together with the transmission image signal.The control information is an indicator of a decoded image display start field and data indicating a decoded image display start line. Receiving an image signal and control information, configuring an encoding target region in the transmission image signal based on the control information, and performing inter-picture predictive encoding on the configured encoding target region. Image signal processing method.

According to a twentieth aspect of the present invention, in a decoding apparatus for decoding data encoded by inter-picture prediction encoding,
A decoder for decoding the inter-picture prediction coding, and a means for converting the decoded image signal into a transmission image signal and outputting control information together with the transmission image signal, wherein the control information includes:
An image signal decoding apparatus characterized by comprising an indicator of a display start field of a decoded image and data indicating a display start line of the decoded image.

According to a twenty-first aspect of the present invention, in an image signal encoding method for encoding a decoded image of data encoded by inter-picture predictive encoding, a transmission image signal composed of a decoded image, Means for receiving control information including an indicator of a display start field and data indicating a display start line of a decoded image, and configuring an encoding target area in a transmission image signal based on the control information; An image signal encoding apparatus comprising: an encoder that performs inter-picture prediction encoding on an encoding target area.

According to a twenty-second aspect of the present invention, there is provided an image signal processing system for decoding data encoded by inter-picture prediction encoding and encoding the decoded data, a decoder for decoding inter-picture prediction encoding, Means for converting the decoded image signal into a transmission image signal and outputting control information together with the transmission image signal, wherein the control information indicates an indicator of a display start field of the decoded image and a display start line of the decoded image. The decoder side, which is data, the image signal for transmission, and control information are received, and based on the control information,
Means for configuring the encoding target area in the transmission image signal,
An image signal processing system comprising: an encoder configured to perform inter-picture predictive encoding on the configured encoding target region;

When the decoded image is converted into a transmission image signal and the transmission image signal is re-encoded, the temporal and spatial information of the decoded image is transmitted to the re-encoder as control information. Therefore, on the re-encoder side, it is possible to re-encode the decoded image with the same temporal and spatial relationship as the original bit stream. This makes it possible to effectively suppress image quality degradation due to decoding and re-encoding.

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. In FIG. 1A, 1 is M
PEG decoder, 2 is a multiplexer, 3 is a recording / reproducing device (or communication path, but will be simply a recording medium in the following description) including a recording medium, 4 is a separator, 5 is MPEG for re-encoding. It is an encoder. The video bit stream is decoded by the MPEG decoder 1, and a decoded image signal (digital signal) is output. The encoding feature point information Ip included in the video bit stream is extracted from the MPEG decoder 1 and supplied to the multiplexer 2. The multiplexer 2 is also supplied with control information Ic. The multiplexer 2 converts the decoded image signal into a transmission image signal, for example, an NTSC format transmission image signal, and multiplexes the control information Ic and the encoding feature point information Ip with the decoded image signal.

As the recording / reproducing apparatus 3, 525/60
(Or 625/50) component system digital VTR, disk recorder, etc. can be used. In addition, the present invention can be applied to a case where the signal is transmitted through a wireless or wired transmission path such as a broadcast.

A transmission image signal is recorded on the recording medium 3,
It is reproduced from the recording medium 3. The encoding feature point information Ip and the control information Ic are multiplexed in the transmission image signal. These pieces of information Ip and Ic are superimposed on a region where no effective pixel exists, such as a horizontal blanking period and a vertical blanking period. Therefore, the recording medium 3, the recording processing system, and the reproduction processing system are configured such that when the transmission image signal passes therethrough, the information Ip and Ic are stored as the original signals. Further, information Ip and Ic
Can be embedded in the least significant bit of the effective image signal.

In the separator 4 on the encoder side, an encoding target area is cut out from the transmission image signal in accordance with the control information Ic. At the same time, the encoding feature point information Ip is separated from the transmission image signal according to the control information Ic.
The separated encoded feature point information Ip is used for re-encoding in the MPEG encoder 5. The MPEG encoder 5 configures a frame structure using an indicator that is temporal information of the control information Ic. Separator 4
, A frame structure may be configured. From the MPEG encoder 5, a video bit stream formed by re-encoding is output.

The coding feature point information Ip includes a picture coding type, a flag top-field-first (abbreviated as TFF), and a flag repeat-first-field (abbreviated as RFF). The picture coding type indicates I, P, and B picture types. The flag TFF is one bit, and if TFF = 1, the top field is displayed first in time. RFF is one bit, RF
If F = 1, the MPEG decoded frame is displayed in three field times. That is, after the field that is displayed second in time, 1
The second field appears repeatedly.
Using such encoded feature point information Ip, MPEG
Encoder 5 is the original, ie, MPEG decoder 1
Can be generated with the same picture coding type and the same order as the bit stream input to the.

The control information Ic is information for transmitting the spatial and temporal relationship between the decoded image and the corresponding coding target area to the re-encoder side. The control information Ic includes
I frame, P frame or B in transmission image signal
Start of display of the MPEG decoded image of the frame (also referred to as start of transmission, but in the following description, start of display) field indicator and display of the top field of the MPEG decoded image in the transmission image signal The start line number (display start line number in the case of the bottom field = display start line number of the top field + 1 line), and the start position of the spatially horizontal pixel of the MPEG decoded image in the transmission image signal are included.

The indicator of the display start field is 1
Indicates the field that appears earlier in the frame,
For example, it is a 1-bit flag. With reference to this indicator, the re-encoder side can correctly compose one frame from two fields. The spatial information (the line number and the start position in the horizontal direction) in the control information Ic is defined by the application format. Therefore, instead of transmitting the value of the line number as the control information, a code signal indicating the application format may be used as the control information. In addition, since this spatial information does not change frequently, only the information corresponding to the changed portion is transmitted. With reference to the spatial information of the control information Ic, the re-encoder side can correctly cut out the encoding target area.

FIG. 1A shows a configuration for transmitting a decoding image signal via the recording medium 3. However, as shown in FIG. 1B, a configuration in which the MPEG decoder 1, the multiplexer 2, the demultiplexer 4, and the MPEG encoder 5 are connected by a signal line without including the recording medium 3 is also possible. In the configuration shown in FIG. 1B, the encoding feature point information Ip is multiplexed into the transmission image signal in the multiplexer 2 based on the control information Ic, but the control information Ic is a signal different from the transmission image signal. It is transmitted to the re-encoder side via the line.

Although omitted in FIG. 1, codec information (motion vector, quantization scale, control information of generated data amount, etc.) used for decoding in the MPEG decoder 1 is reused in the MPEG encoder 5. You. Thereby, image quality degradation due to decoding and re-encoding can be minimized. Further, in the case of an application such as an editing process, it is possible to perform decoding and re-encoding only near the edit point, and to edit the stream in other cases.

FIG. 2 shows an example of the MPEG decoder 1, and FIG. 3 shows an example of the MPEG2 encoder 5. First, the MPEG encoder 5 will be described with reference to FIG. Although not shown, the MPEG encoder 5 is provided with an encoding control unit. The encoding control unit controls each component of the MPEG2 encoder on a frame or field basis, and performs a compression process on the input image data to any one of I picture, P picture, and B picture types. Further, it controls the quantization characteristic and controls the rate of the encoded output. The encoding feature point information Ip and the codec information transmitted from the decoder side are input to the encoding control unit, and a bit stream similar to that before decoding is generated.

The (4: 2: 0) component video data is supplied to the frame memory 51 and the motion prediction section 52 of the MPEG2 encoder 5. Frame memory 51
And a motion vector is detected by the motion prediction unit 52. The input image signal is supplied to the picture order rearranging unit 53, and the order of the pictures is rearranged so as to be suitable for the encoding process. That is, I and P pictures are encoded first, and then B pictures are encoded.

The input image signals whose order has been rearranged are supplied to the subtraction section 54. The subtraction unit 54 includes a motion compensation unit 63
Supplies the motion-compensated local decoded data.
The subtraction unit 54 generates difference data between the input data and the local decoded data. The difference data is supplied to the DCT unit 55. A control signal indicating one of the field DCT and the frame DCT is supplied to the DCT unit 55.

The output of the DCT unit 34 is supplied to a quantization unit 56. The quantization unit 56 is supplied with a control signal for specifying a quantization characteristic from the encoding control unit. The local decoding unit including the inverse quantization unit 59, the inverse DCT unit 60, the addition unit 61, the frame memory 62, and the motion compensation unit 63 is connected to the quantization unit 56. That is, the inverse quantization unit 59 performs a process reverse to that of the quantization unit 56, and
0 performs the reverse process of the DCT unit 55. Motion compensator 63
Is supplied to the subtraction unit 54, and a prediction error is detected. In the motion compensation unit 63, forward prediction,
Reverse prediction and bidirectional prediction are possible. In the case of intra coding, the subtraction unit 54 does not perform the subtraction processing,
Data simply passes through.

The variable length coding unit 57 is provided for the quantization unit 56.
Are connected and subjected to variable length coding. In the variable length coding unit 57, additional information to be transmitted is inserted as a picture header. The information inserted into the header is information such as a flag and a motion vector. The encoded output of the variable length encoding unit 57 is supplied to a buffer 58. Buffer 58
Is provided to output a variable-length coded output as a constant-rate bit stream.

The MPEG decoder 1 shown in FIG. 2 will be described. A bit stream encoded by an encoder similar to the above-described MPEG encoder is stored in a buffer 11.
Supplied to On the encoder side, virtual buffer control is performed so that the amount of generated information is controlled so that the buffer 11 does not underflow or overflow. The output of the buffer 11 is supplied to the variable-length code decoding unit 12.
In the variable length code decoding unit 12, coding feature point information such as a picture coding type is separated. Although not shown, codec information is also separated.

The encoding feature point information and the codec information are used to control the processing of the decoder 1 and, as described above, are supplied to the multiplexer 2 and embedded in the image signal for transmission or separately. Is transmitted to the re-encoder side using the signal line of. For re-encoding of the MPEG encoder 5, the encoded feature point information Ip is used. In order for the encoder 5 to re-encode the encoding feature point information and codec information generated by the decoder 5, the bit stream output from the encoder 5 is
Can hardly be degraded in image quality as compared with the bit stream input to the bit stream.

The output of the variable length code decoding section 12 is supplied to an inverse quantization section 14. The output of the inverse quantization unit 14 is added to the addition unit 15
Supplied to The local decoding image data obtained by the local decoding unit including the frame memory 17 and the motion compensating unit 18 is supplied to the adding unit 15. In the case of intra coding, the addition process is not performed in the adding unit 15 and the data passes through the adding unit 15. Then, the decoded data from the adding unit 15 is supplied to the picture order rearranging unit 16, and a decoded image signal in the order of the original image is obtained. As described above, the encoding feature point information Ip and the control information Ic are multiplexed by the multiplexer 2 on the decoded image signal.

FIG. 4 shows an example of the timing of input / output of the decoder 1 and input / output of the encoder 5 when decoding and re-encoding a bit stream. FIG. 4A is a bitstream input to the decoder 1. Here, I
Alternatively, the period M at which the P picture appears is set to 3. The order of the pictures input to the decoder 1 is the same as the arrangement on the recording medium or the transmission path. In the decoder 1,
An I picture and a P picture are decoded, and then a B picture is decoded, and a decoded image in the order shown in FIG. 4B is output. This decoded image is input to the encoder 5 for re-encoding in the order shown in FIG. 4C. The bit stream output shown in FIG. 4D is generated by the encoder 5. A delay DL (DL = 3 frames in this example) occurs from the input of the decoded image signal to the output of the bit stream to the encoder 5.

FIG. 5 shows another example of the timing of the input / output of the decoder 1 and the input / output of the encoder 5 when decoding and re-encoding the bit stream. FIG. 5A is a bitstream input to the decoder 1. M = 3. The decoder 1 decodes an I picture and a P picture, then decodes a B picture, and outputs decoded images in the order shown in FIG. 5B. This decoded image is input to the encoder 5 for re-encoding in the order shown in FIG. 5C. The encoder 5 generates the bitstream output shown in FIG. 5D. A delay DL (DL = 2 frames in another example) occurs from the input of the decoded image signal to the output of the bit stream to the encoder 5.

In one embodiment of the present invention, the encoding feature point information Ip and the control information Ic from the MPEG decoder 1 are
Are multiplexed by the multiplexer 2 and transmitted to the MPEG encoder side. Preferably, the codec information is also transmitted to the re-encoder side. The encoding feature point information Ip and the control information Ic will be described below.

FIG. 6 shows the data structure of MPEG for convenience of explanation. I, P, and B indicate picture coding types, respectively. I picture (Intra-coded pic
(ture: intra-coded image) uses information that is closed only in one image when it is coded.
Therefore, at the time of decoding, decoding can be performed using only the information of the I picture itself. A P-picture (Predictive-coded picture) uses a temporally previous decoded I-picture or P-picture as a predicted picture (a reference picture for taking a difference). Either encoding the difference from the motion-compensated predicted image, encoding without taking the difference, or selecting the more efficient one for each macroblock. B picture (Bidirectionally predictive-coded pi
cture: a bidirectionally coded image) is a predicted image (a reference image for taking a difference) as a temporally preceding already decoded I picture or P picture, a temporally backward already decoded I picture or Three types of P pictures and interpolated images made from both are used. Among the three types of difference coding after motion compensation and intra coding, the most efficient one is selected for each macroblock.

Therefore, as the macroblock type,
Intra-frame coding (Intra) macroblock, forward (Fward) inter-frame prediction macroblock that predicts the future from the past, and backward (Backwrd) interframe prediction macroblock that predicts the future from the future, There is an interpolative macroblock to predict. I
All macroblocks in a picture are intra-coded macroblocks. The P picture includes an intra-frame coded macro block and a forward inter-frame prediction macro block. The B picture includes all four types of macroblocks described above.

Further, one or more I pictures,
GOP (Group of Group) from zero or more non-I pictures
picture). FIG. 6 shows an example in which the number of frames in a GOP is N = 12 and the period of an I or P picture is M = 3. The order of the pictures of the illustrated GOP is the order of the original images, and the order of BBI or BBP is rearranged to IBB or PBB upon encoding. Then, even on the recording medium, the rearranged array is used, decoding is performed using the rearranged array, and after decoding, the sequence is returned to the original image order.

FIG. 6 shows that each picture is divided into a number of macroblocks, each macroblock is composed of four luminance blocks adjacent to the left, right, up and down, and two chrominance blocks spatially located at the same position. It shows that the block is composed of six blocks in total with Cb and Cr. These blocks are composed of (8 × 8) pixels. At the time of transmission, transmission is performed in the order of Y1, Y2, Y3, Y4, Cb, and Cr.

FIG. 7 shows an MPEG frame structure picture format (MPEG decoded image format). The dark field indicates the top field (t
op-field), and the white band indicates the bottom-field line. In the format of the transmission image signal for NTSC, one frame has 240 lines each of a top field and a bottom field, and a total of 480 lines are included. The number of pixels in the horizontal direction is 704 pixels. The one-bit flag TFF in the header information of the picture layer determines which of the top field and the bottom field is displayed earlier in time. If TFF = 1, the top field is displayed earlier in time.

FIG. 8 shows the spatial relationship between the MPEG decoded image and the transmission image format. Here, the transmission image format is intended for NTSC. Effective pixel area in one frame (pixel area of MPEG decoded image)
Is 780 pixels × 480 lines, as described above. The transmission image format includes a non-effective area of a horizontal blanking area and a vertical blanking area. FIG.
Shows the start positions of the pixels in the spatially horizontal direction of the MPEG decoded image in the transmission image. The start position is represented by the number of pixels or the like, but differs depending on the application format or the like and is not fixed. In one embodiment of the present invention, the information on the start position is included in the control information Ic. On the decoder side, when receiving the transmission image signal, the start position of the pixel in the horizontal direction can be reliably known based on the control information Ic.

Along with the flag TFF, another flag RFF is transmitted. RFF is a flag indicating the presence of a repeat field. A film material such as a movie is data of 24 frames per second, while a video signal such as N
The TSC video signal is 30 frames per second.
Therefore, when converting a film material into a video signal, a process of generating 30 frames from 24 frames is required. Such a process includes a process of converting two fields into three fields using a predetermined conversion pattern, and is therefore generally called 2: 3 pull-down. In other words, by automatically generating the repetition of the first field twice every five frames, 30 frames are reduced to 30 frames.
A frame conversion is performed. Telecine devices are known as devices for converting film material to television material.

When the video signal obtained by the above-described 2: 3 pull-down processing is compressed by MPEG, the information of the field (repeated field) inserted to increase the number of frames is redundant, so that the repeated field is removed. To improve the efficiency of compression. As described above, the process of detecting the repetition field of the image data in which the number of frames is increased to 30 frames per second by the 2: 3 pull-down processing, removing the repetition field, and reducing the number of frames to 24 frames again is the inverse 2: 3 pull-down. Called processing.

A process of converting a film material of 24 frames per second into a television material of the NTSC system of 30 frames per second, that is, a 2: 3 pull-down process will be described with reference to FIG. The film material is 24 frames per second, and an image of two fields (first and second fields) of the same image is formed from each frame, and an image signal of 48 fields per second is formed. Next, 4 frames (8 fields) of the film material are converted into a video signal, for example, 5 frames (10 fields) of an NTSC video signal.

In FIG. 9, the temporally last field among the three fields surrounded by a substantially triangle is a field repeated for increasing the number of fields, that is, a repeat first field. The repeat first field occurs twice in five frames. As described above, two flags, TFF and RFF, are transmitted along with the video signal subjected to the 2: 3 pull-down processing. TFF is a flag indicating whether the first field is a top or a bottom in the case of a frame structure. RFF is a flag indicating the presence of a repeat field.

Next, an example of the temporal relationship between the MPEG decoded image and the transmission image format will be described. Here, the transmission image format is NTSC, and for simplicity of description, the display start line (the vertical position of the decoded image) is defined to be a predetermined one. FIG. 10 shows M of TFF = 1 and RFF = 0.
The top field of the PEG decoded image is an odd cycle (odd
cycle) and the bottom field is an even cycle
An example of transmission in (even cycle) is shown. FIG. 10 shows a vertical cross section of each field, and ○ and × indicate lines of each field. As described with reference to FIG.
Pixel occupation area of the MPEG decoded image of each field (FIG. 1
A region surrounded by a frame at 0) includes 240 lines. However, FIG. 10 shows a small number of lines for simplicity. These notations for FIG. 10 are:
The same applies to the other FIGS. 11 and 12.

In the example of FIG. 10, since TFF = 1,
The top field of the odd cycle is displayed first, and the bottom field of the even cycle is displayed later. NTS
Since C has an interlace structure, in FIG.
The line indicated by A is a top field display start line, and the line indicated by B is a bottom field display start line. That is, the display start line is the line start position of the MPEG decoded image in the spatially vertical direction. The bottom field display start line is a position spatially one line below the top field display start line.
C is a field where the display of the MPEG decoded frame is started. Information on the display start lines A and B and the display start field C is included in the control information Ic.
The display start field C is represented by a 1-bit indicator.

FIG. 11 shows, similarly to FIG. 10, TFF = 1,
An example is shown in which an MPEG decoded image with RFF = 0 is transmitted.
However, unlike the case of FIG. 10, the top field is transmitted in an even cycle, and the bottom field is transmitted in an odd cycle. In the case of MPEG, the relationship between the top field, the bottom field, the odd cycle, and the even cycle is not uniquely determined, so any of the cases of FIG. 10 and FIG. 11 is possible. In FIG. 11, a line indicated by A is a top field display start line, and a line indicated by B is a bottom field display start line. C is a field where the display of the MPEG decoded frame is started. As can be seen by comparing FIGS. 10 and 11, the display start lines A and B in FIG. 11 are spatially one line higher than the display start lines A and B in FIG. The display start field C is also 1 between FIGS.
There is a field phase difference. Display start field C
Can be known on the re-encoder side by the indicator corresponding to

Further, the example of FIG. 12 shows the temporal relationship between the MPEG decoded image subjected to the above-described 2: 3 pull-down processing (see FIG. 9) and the transmission image format. FIG.
In the case of 2, there are four possible combinations of values of the flags TFF and RFF. In FIG. 12, first, T
The top field of an MPEG decoded image with FF = 1 and RFF = 0 is transmitted in odd cycles, and the bottom field is transmitted in even cycles. Next, in order, TFF = 1,
An MPEG decoded image with RFF = 1 is transmitted, and then TF
An MPEG decoded image with F = 0 and RFF = 0 is transmitted, and then an MPEG decoded image with TFF = 0 and RFF = 1 is transmitted.

In FIG. 12, a line indicated by A is a top field display start line, and a line indicated by B is a bottom field display start line. The bottom field display start line is a position spatially one line below the top field display start line. Also, M
As a field where the display of the PEG decoded frame is started, there are four types of indicators corresponding to the combinations of the flags described above. C1 is (TFF = 1, RFF =
0) indicates a field in which the display of the MPEG decoded frame is started. MPE where C2 is (TFF = 1, RFF = 1)
The field where the display of the G decoded frame is started is shown. C
3 indicates a field where the display of an MPEG decoded frame of (TFF = 0, RFF = 0) is started. C4 is (TFF
= 0, RFF = 1) indicates a field where display of an MPEG decoded frame is started. These display start lines A,
B and the information of the display start fields C1 to C4 are included in the control information Ic.

The processing according to the embodiment of the present invention will be further described. The multiplexer 2 controls the control information I described above.
An MPEG decoded frame is converted into a transmission image signal (moving image signal) based on c and the encoding feature point information Ip.
In this case, the control information Ic and the encoded feature point information Ip are transmitted to the encoder together with the transmission image signal. In the encoding feature point information Ip, flags TFF and RFF
Need not necessarily be transmitted to the encoder side. As a transmission method, a method of embedding such information in an empty area of an image signal such as a blanking period can be used.

On the re-encoder side, the separator 4
The control information Ic and the encoded feature point information Ip are separated, and the input image signal is re-encoded based on the separated information. More specifically, the MPEG encoder 5 understands the temporal display start field of the MPEG decoded image in the transmission image signal from the control information Ic, and displays each field of the MPEG decoded image in the transmission image. Understand the starting line number, and
Understand the spatial start position of the PEG decoded image in the horizontal direction. From the coding feature point information Ip, the picture coding type of the MPEG decoded image is understood.

Then, the MPEG encoder 5 creates an encoding target frame from the temporal display start field of the MPEG decoded image in the transmission image signal and the next one field, and cuts out a macroblock. Do. By such processing, when encoding the input image, the picture coding type of the encoding target image can be the same as that of the original bit stream, and the spatial region of the encoding target image can be It can be the same as the bit stream. Therefore, it is possible to minimize image quality deterioration due to decoding and re-encoding due to a mismatch of picture coding types, a mismatch of frame structures, a mismatch of macroblock boundaries, and the like.

When the flags TFF and RFF are transmitted to the encoder side as the encoding feature point information Ip, an encoded frame at the time of re-encoding may be formed using these flags. That is, if RFF = 0,
An encoded frame is composed of the temporal display start field of the MPEG decoded image in the transmission image signal and the next one field. On the other hand, when RFF = 1, an encoded frame is formed from the temporal display start field of the MPEG decoded image in the transmission image signal and the next field F. The field following this field F is removed and not coded.

As described above, the picture feature type, flag TFF, R
In addition to FF, temporal-reference (abbreviated as TPR) and MPEG from one picture to the next I picture
It may include the number of decoded frames (abbreviated as NUMi). TPR is the number of the display order of the MPEG decoded frames, and is reset for each GOP. TPR of MPEG decoded frame displayed first in GOP
Is zero. NUMi is preferably transmitted in a field of a time at which an indicator of a display start field of an I frame in a transmission image is transmitted. The control information Ic is the same as described above.

As inputs to the MPEG encoder 5 for re-encoding, for example, I0, B1, B2, P3, B
4, B5, P6, B7, B8, P9 / I0, B1, B
2, P3, B4, B5, P6, B7, B8, P9 / are considered. This is similar to that shown in FIG. 4C,
In this example, NUMi = 10 information is transmitted. Also,
/ Indicates the boundary of the GOP. The subscript number indicates the number of the display order of the MPEG decoded image in the GOP. This number is the TPR assigned to the coded picture when the MPEG decoded image is the original coded bit stream.
Matches.

It is preferable that the decoder can transmit the flag TPR together with the image signal for transmission to the re-encoder. By reading the flag TPR, the re-encoder side can know the number of decoded images transmitted until the decoded image of the next I picture arrives. Flag T
The PR is transmitted while being embedded in an empty area such as a blanking period together with other encoded feature point information.

The re-encoder understands the temporal display start field of the MPEG decoded image in the transmission image signal from the control information Ic.
Understand the display start line number of the top field of the G decoded image, and further understand the spatially horizontal start position of the MPEG decoded image in the transmission image. From the encoding feature point information Ip, it is understood whether or not the MPEG decoded image is an I picture.

As described above, the MPEG encoder 5
The image to be encoded as an I-picture when encoding the input image can be the same as that of the original bit stream, and the inputted MPEG decoded image can be spatially and spatially identical to the original bit stream. It can be re-encoded in a temporal relationship.

The transmission image signal is not limited to a digital signal, but may be an analog signal. In such a case, the spatially horizontal pixel start position of the MPEG decoded image in the transmission image cannot be defined by the number of pixels, so that it may be difficult to accurately transmit the decoded image to the re-encoder side. Absent. In this case, other control information is transmitted from the decoder side to the re-encoder side. However, there is still a great advantage in performing high-quality re-encoding.

More specifically, the present invention converts an input analog video signal into a digital video signal and compresses the digital video signal, or compresses a directly input digital video signal and then converts the digital video signal into an optical disc as a recording medium. The present invention can be applied to a video signal recording / reproducing apparatus as shown in FIG. 13, which records and expands and reproduces a compressed digital video signal recorded on the optical disc.

First, the configuration and operation of the recording processing system will be described. In FIG. 13, a digital video signal is directly supplied to an input terminal 81. An analog video signal is supplied to an input terminal indicated by 82. The analog video signal is an image signal, a broadcast video signal received by an antenna, or the like. The analog video signal is converted into a digital video signal by the A / D converter 83. The digital video signal from the input terminal 81 and the digital video signal from the A / D conversion unit 83 are transmitted to the MPEG
It is supplied to the encoder 65.

The video signal control section 64 controls the A / A in accordance with the control of the system controller 75 based on the information obtained via the recording control signal input section 74 in accordance with the settings of the user.
Digital video input from D conversion unit 83, input terminal 81
, And supplies the selected digital video input to the MPEG encoder 65. The MPEG encoder 65 performs MPEG compression coding on the video signal from the video signal control unit 64.

The digital video signal compressed by the MPEG encoder 65 is transmitted to a system controller via a bus.
The address is designated by the memory control unit 67 controlled by the memory 75 and is stored in the recording buffer memory unit 66 a of the integrated buffer memory 66.

The digital video signal stored in the recording buffer memory 66a is supplied to an optical disk drive via a bus, a data processor 68 and a recording / reproduction switch 69. The data processing section 68 comprises a recording signal processing section 68a and a reproduction signal processing section 68b. The recording signal processing section 68a performs processing such as error correction encoding and digital modulation, and the reproduction signal processing section 68b performs processing such as error correction and demodulation of digital modulation. The optical disk drive irradiates an optical disk 71 with a laser beam for recording to record a signal and irradiates a laser beam for reproduction to reproduce a signal, and an optical head 70 for irradiating a laser beam for reproduction.
1 is provided with a spindle motor 72 for rotationally driving the motor 1. The head 70 and the spindle motor 72 are
It is controlled by the head controller 73. By the optical head 70, the output signal of the recording signal processing unit 68a is
1 is recorded. The optical disk 70 is rewritable, and may be an MO (magneto-optical) disk, a phase change disk, or the like.

The system controller 75 controls the optical disk drive via the disk / head control unit 73 and also manages the state of the optical disk drive. The system controller 75 transmits the information to the memory control unit 67 and transmits the information to the integrated buffer memory. The control of the supply of data from 66 is performed.

Next, the reproduction processing system will be described. The reproduction processing system includes an MPEG decoder 77 for decoding a reproduction signal supplied from the reproduction system buffer memory unit 66b of the integrated buffer memory 66 via the bus, and an MPEG decoder 7
Video signal controller 64 for switching the decoded video signal from
And a D / A converter 79 for converting the video signal switched by the video signal controller 64 into an analog video signal.

In the reproduction mode, the servo / head movement of the optical disk drive is controlled by the disk / head control unit 73, and the reproduction signal is transmitted via the reproduction signal processing unit 68b of the data processing unit 68 and the bus. Reproduction buffer memory unit 6
6b. The reproduction-system buffer memory unit 66b includes:
The reproduction video signal is supplied to the MPEG decoder 77 while balancing the writing and reading of the reproduction signal.
The MPEG decoder 77 performs an MPEG decoding process on the reproduced signal. The decoded video signal is supplied to the video signal control unit 64.

The video signal control section 64 is controlled by a system controller 75 based on information obtained via a reproduction control signal input section 76 in accordance with the user's settings.
Switching processing is performed on the decoded video signal from the decoder 77, and the result is output to the D / A converter 79 or the output terminal 78. The D / A converter 79 converts the digital video signal switched and controlled by the video signal controller 64 into an analog video signal, and outputs the analog video signal to the output terminal 78.

The video signal control section 64 comprises a changeover switch SW1 and a changeover switch SW2. The changeover switch SW1 has an input terminal a to which a digital video signal input from the input terminal 81 is supplied, and an A / D converter 83
An input terminal b to which a digital video signal from
An input terminal c to which a decoded video signal from the MPEG decoder 77 is supplied and an output terminal d to input a video signal to the MPEG encoder 65 are provided. The changeover switch SW2 includes an input terminal e to which a digital video signal is supplied from an input terminal 81, an input terminal f to which a decoded video signal is supplied from an MPEG decoder 77, and an output terminal
8 and an output terminal g for outputting a video signal to the D / A converter 79.

The switching of the switches SW1 and SW2 in the video signal controller 64 is controlled by the system controller 75. More specifically, if the command from the user supplied to the system controller 75 via the recording control signal input unit 74 specifies the external digital video signal from the input terminal 81, the switch S
The input terminal d of W1 is connected to the output terminal a. If the video signal from the input terminal 82 is designated, the output terminal d is connected to the input terminal b.

Further, if a command to connect one of the video input and the video data reproduced by the optical disk drive to edit and record it on the optical disk 71 again, the system controller 75 connects the output terminal d to the input terminal. Control the timing of connection to c. That is, the MPEG decoder 7
7 is supplied directly to the MPEG encoder 65 for re-encoding.

In this video signal recording / reproducing apparatus, the integrated buffer memory 66 for changing the allocation of storage areas for the recording system and the reproducing system, and the storage of the integrated buffer memory 66 according to the recording mode or the reproduction mode. The area allocation processing is controlled by the system controller 75. That is, the areas of the recording system buffer memory unit 66a and the reproduction system buffer memory unit 66b are made variable by the control of the system controller 75 via the memory control unit 67. For example, at the time of recording, the recording-system buffer memory unit 66a occupies all of the integrated buffer memory 66. At the time of reproduction, the reproduction system buffer memory unit 66b occupies all. In addition, at the time of simultaneous recording and reproduction, half of the memory capacity may be secured.

[0076]

According to the present invention, when inter-picture predictive encoding, for example, data encoded by MPEG is decoded, the decoded image signal is transmitted as an image signal for transmission, and re-encoded, On the encoding side, the temporal and spatial relationship of the encoding target image can be set to the same relationship as when encoding the original bit stream before decoding. Therefore, it is possible to minimize the image quality deterioration due to decoding and re-encoding.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration and a modification of an embodiment of the present invention.

FIG. 2 is a block diagram of an example of an MPEG decoder that can be used in an embodiment of the present invention.

FIG. 3 is a block diagram of an example of an MPEG encoder that can be used in an embodiment of the present invention.

FIG. 4 is a timing chart showing an example of input and output of a decoder and input and output of an encoder.

FIG. 5 is a timing chart showing another example of the input and output of the decoder and the input and output of the encoder.

FIG. 6 is a schematic diagram used to explain the data structure of MPEG.

FIG. 7 is a schematic diagram illustrating a format of a frame structure.

FIG. 8 is a schematic diagram showing a spatial relationship between an MPEG decoded image and a transmission image format.

FIG. 9 is a timing chart used for describing 2: 3 pull-down processing.

FIG. 10 is a schematic diagram illustrating an example of a temporal relationship between an MPEG decoded image and a transmission image format.

FIG. 11 is a schematic diagram illustrating another example of a temporal relationship between an MPEG decoded image and a transmission image format.

FIG. 12 is a schematic diagram illustrating still another example of a temporal relationship between an MPEG decoded image and a transmission image format.

FIG. 13 is a block diagram of an example of a recording / reproducing apparatus to which the present invention can be applied.

[Explanation of Codes] 1 ... MPEG decoder, 2 ... Multiplexer, 4 ...
Separator, 5: MPEG encoder, Ic: Control information, Ip: Encoding feature point information

Claims (22)

[Claims] 1. An image signal decoding method for decoding data encoded by inter-picture prediction encoding, wherein the inter-picture prediction encoding is decoded, the decoded image signal is converted into a transmission image signal, and the transmission An image signal decoding method, comprising outputting control information together with an image signal, wherein the control information is an indicator of a display start field of the decoded image and data indicating a display start line of the decoded image. 2. The decoding method according to claim 1, wherein the control information is superimposed on the transmission image signal. 3. The decoding method according to claim 1, wherein the control information is output to a signal line different from the transmission image signal. 4. The decoding method according to claim 1, wherein the control information further includes data indicating a start position of a spatially horizontal pixel of the decoded image. 5. The decoding method according to claim 1, wherein data of encoded feature point information is output together with the control information. 6. The decoding method according to claim 1, wherein the inter prediction coding is MPEG. 7. The decoding method according to claim 6, wherein the data indicating a display start line in the control information is data indicating one of a top field and a bottom field. 8. The decoding method according to claim 6, wherein data of encoded feature point information is output together with the control information, and the encoded feature point information is of a picture coding type. 9. The method according to claim 6, wherein data of encoded feature point information is output together with the control information, wherein the encoded feature point information is a picture coding type,
And a flag indicating which of the top field and the bottom field is displayed first in time. 10. The method according to claim 8, wherein information used for decoding is output together with the control information and the encoding feature point information, and the information used for decoding is at least one of a motion vector and a quantization scale. A decoding method characterized by the above-mentioned. 11. An image signal encoding method for encoding a decoded image of data encoded by inter-picture predictive encoding, comprising: a transmission image signal composed of a decoded image; Receiving control information including an indicator and data indicating a display start line of the decoded image; forming an encoding target area in the transmission image signal based on the control information; An image signal encoding method characterized by performing inter-picture predictive encoding on an image signal. 12. The encoding method according to claim 11, wherein the control information further includes data indicating a start position of a spatially horizontal pixel of the decoded image. 13. The encoding method according to claim 11, wherein data of encoded feature point information is received together with the control information. 14. The encoding method according to claim 11, wherein the inter-picture prediction encoding is MPEG. 15. The encoding method according to claim 14, wherein the data indicating the display start line in the control information is data indicating one of a top field and a bottom field. 16. The apparatus according to claim 14, further comprising: receiving data of encoded feature point information together with the control information, wherein the encoded feature point information is a picture coding type; A coding method characterized by matching a picture coding type in a stage before decoding. 17. The method according to claim 14, further comprising: receiving data of encoded feature point information together with the control information, wherein the encoded feature point information is a picture coding type;
And a flag indicating which of the top field and the bottom field is displayed first in time. 18. The information processing apparatus according to claim 16, wherein information used for decoding is received together with the control information and the encoding feature point information, and the information used for decoding is at least one of a motion vector and a quantization scale. An encoding method characterized by performing re-encoding using information used for decoding. 19. An image signal processing method for decoding data encoded by inter-picture prediction encoding and encoding the decoded data, wherein the inter-picture prediction encoding is decoded, and the decoded image signal is transmitted to a transmission image. The control information is output together with the transmission image signal, and the control information is an indicator of a decoded image display start field and data indicating a decoded image display start line. And the control information, configure an encoding target area in the transmission image signal based on the control information, and perform inter-picture predictive encoding on the configured encoding target area. Characteristic image signal processing method. 20. A decoding device for decoding data coded by inter-picture prediction coding, comprising: a decoder for decoding inter-picture prediction coding; converting a decoded picture signal into a picture signal for transmission; An image signal decoding apparatus, comprising: means for outputting control information together with an image signal, wherein the control information is data indicating an indicator of a display start field of the decoded image and a display start line of the decoded image. 21. An image signal encoding method for encoding a decoded image of data encoded by inter-picture prediction encoding, comprising: a transmission image signal composed of a decoded image; and a display start field of the decoded image. Means for receiving control information including an indicator and data indicating a display start line of the decoded image, and for configuring an encoding target area in a transmission image signal based on the control information; An image signal encoding apparatus, comprising: an encoder that performs inter-picture prediction encoding on a target area. 22. An image signal processing system for decoding data encoded by inter-picture prediction encoding and encoding the decoded data, comprising: a decoder for decoding inter-picture prediction encoding; and transmitting a decoded image signal. Means for outputting control information together with the transmission image signal, wherein the control information is an indicator of a display start field of the decoded image and data indicating a display start line of the decoded image. A decoder side, receiving the transmission image signal, the control information, and, based on the control information, means for configuring an encoding target area in the transmission image signal; and for the configured encoding target area, An image signal processing system comprising: an encoder configured to perform inter-picture predictive encoding.