JP2010041408A - Moving image encoding apparatus, moving image decoding apparatus, moving image encoding method and moving image decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide moving image encoding technologies with which high-definition progressive images are highly efficiently encoded together with standard interlaced images. <P>SOLUTION: A first interlaced image and a second interlaced image are extracted from a moving image, wherein with the first interlaced image, odd-numbered fields and even-numbered fields are alternately extracted in a predetermined frequency and with the second interlaced image, odd-numbered fields and even-numbered fields are extracted with a phase which is inverse to that of the first interlaced image. A first interlaced moving image encoder 107 encodes the first interlaced image to produce first interlaced encoded data. A second interlaced moving image encoder 109 encodes the second interlaced image while using at least a first interlaced decoded image, that is obtained by decoding the first interlaced encoded data, as a reference image for predictive coding to produce the second interlaced encoded data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving image encoding device and a moving image encoding method for encoding a moving image, and a moving image decoding device and a moving image decoding method for decoding encoded moving image data.

  A video has an interlaced image and a progressive image. If 60 frames per second (exactly 59.94 Hz), the progressive image display has a larger amount of information and a higher quality image can be expressed. However, interlaced images are basically used in current DVDs (Digital Versatile Disks) and broadcasts. That is, although it is 60 frames, every other line of the image is thinned out and displayed alternately. In recent DVD players and DTV (Digital Television) devices, in order to compensate for the shortcomings of interlaced images, on the display side, the interlaced images are simulated into progressive images using the characteristics of the images and temporal movements. Many forms have been converted and displayed progressively on the screen.

  As an image encoding technique, Patent Document 1 discloses a method for continuously encoding two odd and even fields constituting one frame on a field basis as the most basic encoding method for interlaced images. It is disclosed.

  Patent Document 2 discloses a method for efficiently encoding a video signal in a scalable manner that allows a video image to be decoded in various resolution scales and image formats.

  MPEG (Moving Pictures Expert Group) can be cited as an internationally standardized moving picture coding technique. MPEG is an abbreviation of the name of an organization that examines the video coding standard established in 1988 by ISO / IEC JTC1 / SC2 (International Organization for Standardization / International Electrotechnical Standards Meeting Technical Committee 1 / Technical Committee 2, currently SC29). It is. MPEG1 (MPEG Phase 1) is a standard for storage media of about 1.5 Mbps, and JPEG for the purpose of still image coding, and video compression for low transfer rates of ISDN video conferences and videophones. The target H.C. It inherits the basic technology of H.261 (CCITT SGXV, standardized by the current ITU-T SG15), and introduces a new technology for storage media. These were established in August 1993 as ISO / IEC11172.

  MPEG2 (MPEG Phase 2) was established as ISO / IEC 13818, H.262 in November 1994 for the purpose of general-purpose standards so as to be compatible with various applications such as communication and broadcasting.

  MPEG is realized by combining several technologies. An example of a general MPEG encoding apparatus will be described with reference to FIG. The differentiator 910 reduces the time redundant portion by taking the difference between the input image and the image decoded by the motion compensation predictor 909. There are three prediction directions: past only, only future, and both past and future. The prediction direction can be switched for each MB (macroblock) of 16 pixels × 16 pixels. The prediction direction is determined by the picture type given to the input image. There are two modes: a mode that predicts from the past and a mode that independently encodes the MB without prediction. There are four types of B pictures: a mode predicting from the future, a mode predicting from the past, a mode predicting from both the past and the future, and a mode encoding independently. An I picture encodes all MBs independently.

  Motion compensation is performed by performing pattern matching on the motion region for each MB, detecting a motion vector with half-pel accuracy, and performing a prediction after shifting the motion vector. The motion vector has horizontal and vertical components, and is transmitted as additional information of the MB together with the MC (Motion Compensation) mode indicating the prediction direction. A group from an I picture to a picture before the next I picture is called a GOP (Group Of Picture). When used in storage media, about 15 pictures are generally used as a GOP.

  The difference image obtained by the difference unit 910 is subjected to orthogonal transformation in a DCT (Discrete Cosine Transform) unit 901. DCT is an orthogonal transformation that discretely transforms an integral transformation with a cosine function as an integral kernel into a finite space. In MPEG, MB is divided into four and two-dimensional DCT is performed on 8 × 8 DCT blocks. In general, a video signal has many low-frequency components and few high-frequency components. Therefore, when DCT is performed, coefficients are concentrated in a low frequency.

  The image data (DCT coefficient) orthogonally transformed by the DCT unit 901 is quantized by the quantizer 902. For quantization, a value obtained by multiplying a value obtained by weighting an 8 × 8 two-dimensional frequency called a quantization matrix with a visual characteristic and a value called a quantization scale for multiplying the whole by a scalar is used as a quantized value, and a DCT coefficient is quantized. This is done by dividing by the digitized value. When inverse quantization is performed on the decoder side, a value approximate to the original DCT coefficient is obtained by multiplying by the quantized value.

  The quantized data is subjected to variable length coding by a VLC (Variable Length Coding) unit 903. Of the quantized values, a direct current (DC) component is encoded by DPCM (differential pulse code modulation) which is one of predictive encoding. The alternating current (AC) component is zigzag scanned from low to high, with zero run length and effective coefficient value as one event, and Huffman coding that assigns codes with a short code length from those with a high probability of appearance. Done.

  The variable-length encoded data is stored in the temporary buffer 904 and output as encoded data at a predetermined transfer rate. The generated code amount for each macroblock of the encoded data to be output is transmitted to the code amount controller 905. The code amount controller 905 feeds back the error of the generated code amount with respect to the target code amount to the quantizer 902. The quantizer 902 adjusts the quantization scale based on the fed back error code amount. Thereby, the code amount controller 905 can control the code amount of the generated encoded data to an appropriate rate.

  The quantized image data is inversely quantized by the inverse quantizer 906, inversely DCTed by the inverse DCT unit 907, temporarily stored in the image memory 908, and then calculated by the motion compensation predictor 909. It is used as a reference decoded image.

Next, an example of a general MPEG decoding apparatus will be described with reference to FIG. The encoded moving image data is buffered in the buffer 1001, and the data from the buffer is input to the VLD unit 1002. The VLD unit 1002 performs variable length decoding on the image data to obtain a direct current (DC) component and an alternating current (AC) component. The alternating current (AC) component data is arranged in an 8 × 8 matrix in the zigzag scan order from the low range to the high range. This data is input to an inverse quantizer 1003 and inversely quantized by a quantization matrix. The inversely quantized data is input to the inverse DCT unit 1004, subjected to inverse DCT conversion, and output as image data (decoded data). The decoded data is temporarily stored in the image memory 1006 and then used as a reference decoded image for calculating a difference image in the motion compensation predictor 1005.
JP-A-6-98314 JP-A-7-162870

  Since there are so many applications that employ standardized MPEG2 interlaced video encoding, standard interlaced images must be encoded in order to maintain compatibility when encoding video. is there. However, since there is a need to reproduce high-quality moving images at the same time, there is a need for a moving image coding technique that can efficiently encode high-quality progressive images along with standard interlaced images. . If progressive video is encoded in addition to standard interlaced images in one moving image data, either interlaced images or progressive images can be selected and used according to the needs of the application. become.

  However, methods such as interlace coding and progressive coding, or hierarchical coding by taking differences in images with different resolutions have been developed, and while maintaining coding efficiency, Currently, it is not possible to encode progressive video together while maintaining complete compatibility.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a moving image encoding technique and a moving image decoding technique capable of achieving both interlaced reproduction and progressive reproduction of moving image data. is there.

  In order to solve the above-described problem, a moving picture coding apparatus according to an aspect of the present invention includes an odd field composed of odd-numbered scanning lines and an even field composed of even-numbered scanning lines alternately from a moving picture at a predetermined frequency. An extraction unit that extracts the extracted first interlaced image, and a second interlaced image in which the odd and even fields are extracted from the moving image in an opposite phase to the first interlaced image; and the first interlaced image is encoded A first encoding unit that generates first interlaced encoded data and a first interlaced decoded image obtained by decoding the first interlaced encoded data at least as a reference image for predictive encoding A second interlaced encoded data is generated by encoding a two interlaced image. And a No. unit.

  Another aspect of the present invention is also a moving image encoding apparatus. The apparatus includes a first interlaced image obtained by alternately extracting an odd field composed of odd-numbered scan lines and an even field composed of even-numbered scan lines from a moving image at a predetermined frequency, and odd-numbered and even-numbered scan lines. Obtained by extracting a progressive image including both, a first encoding unit that encodes the first interlaced image to generate first interlaced encoded data, and decoding the first interlaced encoded data Using at least the first interlaced decoded image and the first interlaced image included in the progressive image supplied from the extraction unit as a reference image for predictive coding, the odd field and the even field are the first interlaced image. By encoding the second interlaced image that is in anti-phase, Progressive coding is performed by generating interlace coded data and combining the first interlace coded data obtained by coding the first interlace image included in the progressive image and the generated second interlace coded data. A progressive encoder for generating data.

  Yet another embodiment of the present invention is a video decoding device. This apparatus encodes a first interlaced image obtained by encoding, from moving image encoded data, an odd field consisting of odd-numbered scanning lines and an even field consisting of even-numbered scanning lines alternately extracted at a predetermined frequency. A separation unit that separates and extracts one interlace encoded data and second interlace encoded data obtained by encoding a second interlace image in which odd and even fields are in opposite phases to the first interlace image; A first decoding unit that decodes the first interlaced encoded data to generate a first interlaced decoded image; and the second interlaced encoding by performing prediction using at least the first interlaced decoded image as a reference image. A second decoding unit for decoding data to generate a second interlaced decoded image; By combining the second interlace decoded picture and the first interlace decoded picture, and a progressive image generating unit for generating a progressive image.

  Yet another embodiment of the present invention is also a moving picture decoding apparatus. This apparatus encodes a first interlaced image obtained by encoding, from moving image encoded data, an odd field consisting of odd-numbered scanning lines and an even field consisting of even-numbered scanning lines alternately extracted at a predetermined frequency. A separation unit that separates and extracts one interlace encoded data and progressive encoded data obtained by encoding a progressive image including both odd-numbered and even-numbered scan lines; and decoding the first interlace encoded data A first decoding unit that generates a first interlaced decoded image; and a first interlaced encoded data included in the first interlaced decoded image and progressive encoded data decoded by the first decoding unit. A first interlaced decoded image obtained by using at least the reference image as a reference image Accordingly, the second interlaced encoded data obtained by decoding the second interlaced image in which the odd field and the even field are opposite in phase to the first interlaced image is decoded to generate a second interlaced decoded image, Progressive decoding that generates a progressive decoded image by combining the first interlace decoded image obtained by decoding the first interlace encoded image included in the progressive encoded data and the generated second interlace decoded image. Including

  Yet another aspect of the present invention is a video encoding method. This method includes a first interlaced image in which an odd field consisting of odd-numbered scanning lines and an even-numbered field consisting of even-numbered scanning lines are alternately extracted from a moving image at a predetermined frequency, and an odd field and an even number from the moving image. Extracting a second interlaced image whose field is extracted in an opposite phase to the first interlaced image; encoding the first interlaced image to generate first interlaced encoded data; and the first interlaced image Generating the second interlaced encoded data by encoding the second interlaced image using at least a first interlaced decoded image obtained by decoding the encoded data as a reference image for predictive encoding. Including.

  Yet another aspect of the present invention is a video decoding method. In this method, a first interlaced image obtained by encoding a first interlaced image obtained by alternately extracting an odd field composed of odd-numbered scanning lines and an even-numbered field composed of even-numbered scanning lines from moving image encoded data at a predetermined frequency is encoded. Separating and extracting one interlace encoded data and second interlace encoded data obtained by encoding a second interlace image in which an odd field and an even field are opposite in phase to the first interlace image; Decoding the interlaced encoded data to generate a first interlaced decoded image; and decoding the second interlaced encoded data by predicting at least using the first interlaced decoded image as a reference image Generating a second interlaced decoded image; By combining the one interlace decoded picture second interlace decoded picture, and generating a progressive image.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to achieve both interlaced playback and progressive playback of moving image data.

Embodiment 1
FIG. 1 is a configuration diagram of a video encoding apparatus according to Embodiment 1. In this embodiment, a configuration in which a moving image is shot and recorded on a recording medium will be described. However, a frame of a moving image that has already been shot may be input, and encoded moving image data may be recorded on the recording medium. The data may be output to the outside as stream data without being recorded.

  An image is picked up by the moving image light receiving element 102 through the lens 101. The light receiving element 102 is, for example, a CCD (Charge Coupled Device) or a CMOS, and can be taken out as a progressive image as a moving image pickup element. Data output from the light receiving element 102 is A / D converted by the A / D converter 103.

  The A / D converter 103 outputs two types of interlaced images A and B that are complementary (complementary to each other) as digital data, and the first interlaced image A is output to the first interlaced video encoder 107. The second interlaced image B is input to the second interlaced video encoder 109.

  FIG. 2 is a diagram for explaining the relationship between the two complementary interlace images A and B and the progressive image C input to the first interlace moving image encoder 107 and the second interlace moving image encoder 109.

  FIG. 2A shows a first interlaced image A input to the first interlaced video encoder 107. Scan line images (for example, horizontal 720 pixels and vertical 240 pixels) shown in white are input every approximately 1/60 seconds (exactly 1 / 59.94 seconds). The first sheet is an image of an odd field composed of odd-numbered scanning lines, and the second sheet is an image of an even-numbered field composed of even-numbered scanning lines. Thereafter, the odd-numbered field and even-numbered field are alternately repeated.

  FIG. 2B shows the second interlaced image B input to the second interlaced video encoder 109. Similarly, a scanning line image (720 pixels wide and 240 pixels high) shown in white is input approximately every 1/60 seconds (accurately 1 / 59.94 seconds), but the second interlaced image B is The interlaced image A has a complementary relationship, and in the second interlaced image B, the odd field and the even field are in opposite phases to the first interlaced image A. That is, the first image of the second interlaced image B is an even field image composed of even-numbered scan lines, and the second image is an odd-field image composed of odd-numbered scan lines. The field repeats alternately.

  When the first interlaced image A in FIG. 2 (a) and the second interlaced image B in FIG. 2 (b) are combined, they are complemented by odd and even lines, so that the progressive image C ( 720 pixels horizontally and 480 pixels vertically).

  Refer to FIG. 1 again. The encoding of the first interlaced image A by the first interlaced video encoder 107 may be any encoding method as long as it is a standard encoding method compatible with interlace, such as ordinary MPEG video. Here, in consideration of compatibility, it is more preferable to perform encoding according to the standard. For example, in the case of MPEG, a standard such as H264 is used, or in the case of VC1, a media standard such as a DVD or a Blu-ray disc (trademark or registered trademark) or a standard standard adopted in a digital broadcasting standard is used. Is important in the sense of performing encoding in consideration of compatibility in the present embodiment.

  The moving image local decoder 108 decodes the first interlaced image A encoded by the first interlaced moving image encoder 107 to generate a first interlaced decoded image A ′. The moving image local decoder 108 supplies the first interlaced decoded image A ′ to the second interlaced moving image encoder 109. Here, the first interlaced image A after encoding is locally decoded in the moving image encoding device because the first interlaced image A before encoding cannot be used in the moving image decoding device described later. This is because the second interlaced image B must be decoded using the first interlaced decoded image A ′.

  The second interlaced video encoder 109 of FIG. 1 uses the second interlaced image B supplied from the A / D converter 103 and the first interlaced decoded image A ′ supplied from the video local decoder 108. The second interlaced image B is predictively encoded. When the second interlaced video encoder 109 predictively encodes the second interlaced image B, the second interlaced video B is used as a time-series reference for the second interlaced image B in the past. In addition, the first interlace decoded image A ′ supplied from the moving image local decoder 108 is also used as a reference image. Details of encoding by the second interlaced video encoder 109 will be described later.

  On the other hand, the audio data output from the audio input device 104 is converted to digital data by the A / D converter 105 and input to the speech encoder 106. The audio encoder 106 performs compression such as MPEG audio encoding and Dolby AC3. Since the content of the encoding method is standardized, description thereof is omitted here.

  The first interlace encoded data output from the first interlace video encoder 107, the second interlace encoded data output from the second interlace video encoder 109, and the audio output from the audio encoder 106 The encoded data is input to the multiplexer 110, respectively.

  The multiplexer 110 multiplexes these three types of data using a packet multiplexing method. The multiplexer 110 may be replaced with a picture user data generator using a user data of a picture. Either can be built as a system. Details of the multiplexer 110 or the picture user data generator will be described later. In any case, the three types of encoded data are collected into one stream data and recorded on the recording medium 112 by the recording medium writer 111.

  FIG. 3 is a diagram for explaining unidirectional predictive encoding by the first interlace moving image encoder 107 and the second interlace moving image encoder 109.

  FIG. 3A shows a state in which the first interlaced video encoder 107 performs one-way predictive encoding of the first interlaced image A. This is an explanatory diagram of one-way predictive coding used in MPEG and the like. Since it is an interlaced moving image, one field image is input every 1/60 seconds (exactly 1 / 59.94 seconds) when the horizontal axis is the time axis.

  The first interlaced image A is an image in which field images are arranged in the time direction in the order of odd fields and even fields. An odd field is shown in the upper row and an even field image is shown in the lower row.

  First, intra coding which is completed within one MPEG screen is performed in a predetermined unit. For example, it is appropriate to intracode about 2 frames in 30 fields (the two frames pair odd and even fields) into an I picture. Otherwise, predictive coding is performed as a P picture. When a P picture is unidirectionally predictive encoded, it is encoded with reference to an intra-coded I picture or a past P picture.

  In FIG. 3A, the reference relationship in the unidirectional predictive coding defined by the MPEG2 main profile is indicated by arrows. Since the encoding syntax is defined in MPEG, details are omitted.

  For example, the P picture of the third odd field is predictively encoded with reference to the I picture of the first odd field and the I picture of the second even field. The fourth even field P picture is predictively encoded in the forward direction with reference to the second even field I picture and the third odd field P picture.

  Here, when performing predictive coding, instead of taking a simple difference on the screen, which one is a unit of a macroblock (for example, a block of 16 pixels horizontally, 8 pixels vertically, 16 pixels horizontally, 16 pixels vertically) If the content included in the macro block is moved with an accuracy of 1 pixel, 1/2 pixel, or 1/4 pixel, the difference for each macro block is an absolute value sum or an absolute value square sum. The smallest evaluation value is found and the direction of the motion vector (MV) and the amount of motion are obtained. The macro block is moved in accordance with the obtained motion vector, and a difference value with respect to the predicted reference image is taken and encoded.

  In order to select a reference image with the smallest difference for each macroblock, identification information for specifying the selected reference image is set for each macroblock, and is recorded in the encoded data together with the motion vector information.

  FIG. 3B shows a state in which the second interlace video encoder 109 unidirectionally encodes the second interlace video B. The second interlaced moving image B indicated by the solid line is arranged in the time direction in the order of the even field and the odd field, and has a complementary relationship with the first interlaced moving image A indicated by the dotted line.

  Unlike normal MPEG encoding, the predictive encoding of the second interlaced video B by the second interlaced video encoder 109 is not only the field image of the second interlaced video B but also the local part of the first interlaced video A. The decoding field image is also referred to. Since the image encoded by the second interlaced video encoder 109 is neither an I picture nor a P picture, it will be referred to as “M picture” as a new image type.

  Since the first even-field image of the second interlaced image B is intra-coded, it becomes an I picture. The image of the second odd field of the second interlaced image B includes the decoded image I ′ of the first odd field of the first interlaced image A, the image I of the first even field of the second interlaced image B, Predictive coding is performed with reference to three types of decoded images I ′ in the second odd field of the first interlaced image A, and an M picture is obtained.

  The image of the third even field of the second interlaced image B includes the decoded image I ′ of the second even field of the first interlaced video A, the image M of the second odd field of the second interlaced image B, and Then, prediction encoding is performed with reference to the three types of the decoded image P ′ of the third odd field of the first interlaced image A, and an M picture is obtained.

  Thereafter, the k (> 3) th field image of the second interlaced image B is the (k−1) th decoded field image P ′ of the first interlaced image A and (k−1) of the second interlaced image B. ) The third field image M and the k-th decoded field image P ′ of the first interlaced image A are referred to and predictively encoded to form an M picture.

  Thus, the reference image used for the one-way predictive encoding of the M picture of the second interlaced image B at time t is the local decoded field image of the first interlaced image A at time (t−1), time (t -1) is a field image M of the second interlaced image B and a locally decoded field image of the first interlaced image at time t, and the reference relationship is indicated by three arrows. Here, one unit of time is 1/60 seconds (more precisely, 1 / 59.94 seconds).

  When only the field image of the second interlaced image B is used as a reference image in the one-way predictive coding of the second interlaced image B, as shown in FIG. Only images can be used. On the other hand, in the present embodiment, the field image of the first interlaced image A is also adopted as a reference image, and the second interlaced image B is predictively encoded with reference to the three most recent field images in terms of time. The Therefore, it is expected that the prediction error is reduced, and the encoding efficiency is increased.

  Predictive coding is performed between the reference image and the predicted image indicated by the arrows in FIGS. 3 (a) and 3 (b). (1) When there are a plurality of reference images, an image having the smallest difference value is selected. (2) In the case of bi-directional prediction, an image obtained by selecting two future images in the future and taking a difference value using an averaged predicted image is also an option. (3) A difference value obtained using two images in the forward direction and the reverse direction is also an option. These are all MPEG2 predictive coding methods, and one of the three methods may be used, or two or three methods may be used in combination. When there are a plurality of combinations, identification information indicating a selected one of the combinations is put in syntax bits indicating a macroblock prediction method. Thereafter, the obtained difference image is subjected to DCT transform of 8 pixels in the horizontal direction and 8 pixels in the vertical direction, and further quantized. Variable length coding (VLC) is performed by scanning the quantized DCT transform coefficients in a predetermined order.

  In the predictive encoding of the second interlaced image B, there are a maximum of three reference images, so a 2-bit identifier is used to identify the reference image used for predictive encoding. Since this identification bit can be changed for each macroblock, it is desirable to describe the identification bit in an area for storing a motion vector for each macroblock.

  In the above description, the case where the first interlace video encoder 107 and the second interlace video encoder 109 unidirectionally encode the first interlace image A and the second interlace image B has been described. May be used.

  FIG. 4 is a diagram for explaining bi-directional predictive encoding by the first interlace moving image encoder 107 and the second interlace moving image encoder 109.

  FIG. 4A shows a state in which the first interlace video encoder 107 bi-directionally encodes the first interlace video A. This is bidirectional predictive coding used in MPEG and the like. The difference from FIG. 3A is that there is a B picture that is bidirectionally predicted. Since it is an interlaced image, the past and future field images are referred to by odd and even pairs, and prediction coding is performed using four past and future reference images. Since this method is the same as MPEG, detailed description is omitted.

  FIG. 4B shows a state in which the second interlace video encoder 109 performs bi-directional predictive encoding on the second interlace video B.

  Bidirectional predictive encoding of the second interlaced image B by the second interlaced video encoder 109 is performed with reference to the local decoded field image of the latest first interlaced video A. An image that is bi-directionally predictively encoded by the second interlace video encoder 109 will be referred to as an “N picture” as a new image type.

  Since the first even-field image of the second interlaced image B is intra-coded, it becomes an I picture. The second odd field image of the second interlaced image B includes the decoded image I ′ of the first odd field of the first interlaced image A and the decoded image I of the second even field of the first interlaced image A. 'And three types of decoded images B' of the third odd field of the first interlaced image A are predictively encoded to become N pictures.

  The third even field image of the second interlaced image B includes the decoded image I ′ of the second even field of the first interlaced image A and the decoded image B of the third odd field of the first interlaced image A. 'And the three types of the fourth even-field image B' of the first interlaced image A are subjected to predictive encoding to become an N picture.

  Thereafter, the k (> 3) th field image of the second interlaced image B is the (k-1) th decoded field image of the first interlaced image A and the kth encoded field of the first interlaced image A. Predictive encoding is performed with reference to three types of images and the (k + 1) -th decoded field image of the first interlaced image A, resulting in an N picture.

  Thus, the reference image used for bidirectional predictive coding of the N picture of the second interlaced image B at time t is the local decoded field image of the first interlaced image at time (t−1), the first image at time t. There are three local encoded field images of one interlaced image A and local decoded field images of the first interlaced image A at time (t + 1), and the reference relationship is indicated by three arrows. Here, one unit of time is 1/60 seconds (more precisely, 1 / 59.94 seconds).

  Assuming that only the field image of the second interlaced image B is used as a reference image during bi-directional predictive coding of the second interlaced image B, as shown in FIG. Only available. On the other hand, in the present embodiment, the field image of the first interlaced image A is adopted as the reference image, and the prediction of the second interlaced image B is performed by referring to the three adjacent field images from the past and the future. Encoding is done. As a result, the prediction error is reduced and the coding efficiency is improved.

  FIG. 5 is a diagram illustrating a method in which the multiplexer / picture user data generator 110 merges the first interlace encoded data and the second interlace encoded data.

  FIG. 5A shows a method in which the picture user data generator 110 merges two types of encoded data according to a user data scheme. MPEG has an area where user data can be described in units of pictures. The area in which user data can be described is not limited to the picture layer, but also in other layers such as the GOP layer. If the layer is a group of a plurality of pictures such as the GOP layer, only a predetermined number of second interlace encoded data is stored. Collected and stored in the user data area. Here, an example in which the second interlace coded data is inserted for each picture will be described.

  Second interlace encoded data having a complementary relationship is inserted for each picture in the user data area of the picture layer of the first interlace encoded data. Basically, the data from the picture header to the front of the next picture header may be regarded as one piece of picture data. However, if there is a GOP header or sequence header at the head of the GOP of the intra picture part, 1 of the intra picture is used. What is necessary is just to handle as picture data.

  As shown in FIG. 5A, the second interlace coded data is stored in the user data area in the MPEG picture data. As user data, user_data () in the syntax of the video layer of MPEG2 is used. user_data () starts from a byte-aligned start code that can be uniquely determined as user_start_code, and can continue user data until the next 3 bytes of 0x000001 are received. At that time, there is a possibility that user_data () is used in other applications, so after user_start_code of user_data (), describe a unique code of about 4 bytes, for example 0x22220204, indicating that it is data of this method To do. This can prevent confusion with user data used for other purposes.

  FIG. 5B shows a method in which the multiplexer 110 merges two types of encoded data by a packet multiplexing method. Two types of video encoded data and one type of audio encoded data are multiplexed in accordance with the MPEG2 system layer standard. Audio encoded data, first interlace encoded data, and second interlace encoded data are packetized together with a packet header. In each packet header, an identification ID for identifying whether it is audio 1, video 1 or video 2 is described. The identifier of video 1 indicates first interlace encoded data, the identifier of video 2 indicates second interlace encoded data, and the identifier of audio 1 indicates audio encoded data.

  These encoded data are multiplexed into one stream so that a buffer overflow does not occur in each decoder of two types of video and one type of audio. Since details are described in the MPEG2 multiplexing standard, the description is omitted here.

  FIG. 6 is a configuration diagram of the video decoding apparatus according to Embodiment 1. The recording medium 201 is a recording medium 112 on which moving image and audio encoded data are recorded by the recording medium writer 111 of the moving image encoding device of FIG. The recording medium 201 records three types of first interlace encoded data, second interlace encoded data, and audio encoded data in one stream. As a method of grouping into one stream, there are a packet multiplexing method shown in FIG. 5B and a user data method of picture data shown in FIG.

  The recording medium reader 202 reads the encoded data from the recording medium 201 and supplies it to the demultiplexer / user data separator 203. When the encoded data is recorded by the packet multiplexing method shown in FIG. 5B, the multiplexing / separating device 203 uses the packet header identification data for audio 1 and video 1 based on the packet header identification ID. , Video 2 is identified. The demultiplexer 203 determines that if the identification ID is audio 1, it is audio encoded data, if it is video 1, it is first interlace encoded data, and if it is video 2 it is second interlace encoded data. Each encoded data is supplied to the audio decoder 204, the first interlace video decoder 208, and the second interlace video decoder 207, which are decoders.

  When the encoded data is recorded by the user data method of FIG. 5A, the user data separator 203 separates and extracts the second interlace encoded data inserted in the user data area. User data is not necessarily described in a picture layer in which user data can be described in units of pictures, but may be described in a GOP layer in which groups of pictures are collected. When picture user data is used, user_start_code of user_data () in the MPEG2 video layer syntax is detected, and a unique code of about 4 bytes (for example, 0x22220204) indicating the data of this method is detected after user_start_code. To detect. The information stored after this unique code can be extracted by determining that it is the second interlace encoded data.

  The first interlaced video decoder 208 is means for decoding interlace encoded data such as MPEG. As described above, since the first interlace moving picture encoder 107 of the moving picture coding apparatus performs the interlace coding according to the standard, the first interlace moving picture decoder 208 corresponds to a normal MPEG video or the like. The first interlaced video encoded data can be decoded by a standard decoding method that supports interlace. In this embodiment, it is important to adopt a standard encoding method in order to guarantee compatibility.

  The second interlace video decoder 207 decodes the second interlace encoded data using the first interlace decoded image decoded by the first interlace video decoder 208 as a reference image.

  As described in FIG. 3B, the second interlaced image B is predictively encoded using the first interlaced decoded image as a reference image. In general, the M picture of the second interlaced image B at time t is the first interlace decoded picture I ′ (or P ′) at time (t−1), the M picture at time (t−1), and the first picture at time t. Predicted from the three most recent reference images of one interlaced decoded picture I ′ (or P ′). When the second interlace video decoder 207 decodes the M picture of the second interlace image B at time t, the second interlace video decoder 207 receives the first interlace decoding at time (t−1) and time t from the first interlace video decoder 208. Is received as a reference image from among the first interlaced decoded image at time (t-1), the first interlaced decoded image at time t, and the M picture at time (t-1). The M picture at time t is decoded by adding the difference image to the image. As shown in FIG. 4B, when bi-prediction is used, when decoding the N picture at time t, decoding is performed using three adjacent reference images from both the past and future directions.

  The first interlace decoded image A ′ decoded by the first interlace video decoder 208 is supplied to the D / A converter 211 and the progressive image composer 209. The first interlace decoded image A ′ is converted from digital to analog by the D / A converter 211, and the image output unit 212 outputs the image.

  The second interlace decoded image B ′ decoded by the second interlace video decoder 207 is supplied to the progressive image composer 209. The progressive image composer 209 generates a progressive decoded image C ′ by synthesizing the first interlace decoded image A ′ and the second interlace decoded image B ′ and supplies it to the D / A converter 210. The progressive decoded image C ′ is converted from digital to analog by the D / A converter 210, and the image output unit 212 outputs the image.

  Here, although the D / A converters 210 and 211 are described separately from the image output unit 212, they may be mounted in the image output unit 212. In some cases, an image-encoded digital signal is input as it is like a device such as a DTV.

  On the other hand, the audio data output from the demultiplexer 203 is input to the speech decoder 204. The audio decoder 204 decodes audio data encoded by MPEG audio encoding or Dolby AC3. Since the content of the encoding method is standardized, description of decoding is omitted. The decoded audio data is converted from digital to analog by the D / A converter 205 and is output as audio by an audio output unit 206 such as a speaker.

  If only the information of the first interlaced image output from the first interlaced video decoder 208 is used and output by the image output unit 212, the interlaced image can be reproduced, and the first interlaced video decoder 208 is used. The first interlaced image output from the second interlaced video decoder 207 and the second interlaced image output from the second interlaced video decoder 207 are synthesized using the progressive image composer 209, converted into a progressive signal, and output from the image output unit 212. Progressive images can be reproduced.

Embodiment 2
FIG. 7 is a configuration diagram of a video encoding apparatus according to Embodiment 2. The second interlaced video encoder 109 of the first embodiment is replaced with a progressive video encoder 119 in the second embodiment, and the image supplied from the A / D converter 103 to the progressive video encoder 119 is The difference from Embodiment 1 is that it is a progressive image C in FIG. Since other than this is the same as in the first embodiment, common description is omitted, and different configurations and operations will be described.

  The progressive video encoder 119 receives the first interlace decoded image A ′ output from the video local decoder 108 and the progressive image C output from the A / D converter 103 as inputs, and encodes the progressive image C. Progressively encoded data is generated and provided to the multiplexer 110.

  FIG. 8 is a diagram for explaining unidirectional predictive encoding of a progressive image C by the progressive video encoder 119.

  For comparison, FIG. 8A shows the one-way predictive encoding of the interlaced image A by the first interlaced video encoder 107, which is the same as in the first embodiment.

  FIG. 8B shows unidirectional predictive encoding of the progressive image C by the progressive video encoder 119. FIG. 8B shows two types of stream data for convenience of explanation. In the upper stream data, the M picture of the second interlaced image B is predictively encoded using the past M pictures as well as the field images I ′ and P ′ of the first interlaced decoded image A ′ as reference images. This is the same as in the first embodiment.

  In the second embodiment, since the progressive video encoder 119 is provided with the progressive image C, the original first interlaced image A can also be used as a further reference image. The lower stream data in FIG. 8B indicates that the M picture of the second interlaced image B is predictively encoded using the field images I and P of the original first interlaced image A as reference images. ing. When the upper stage and the lower stage of FIG. 8B are combined, the M picture of the second interlaced image B can be predictively encoded using a total of five reference images.

  As described above, the progressive video encoder 119 uses the second interlaced image to be encoded as a time-series reference, and uses the first interlace generated by the video local decoder 108 in addition to the past M pictures. Using the decoded image A ′ and the first interlaced image A included in the progressive image C supplied from the A / D converter 103 as a reference image, the M picture of the second interlaced image B is predicted, and the progressive image C is Encode. The progressive video encoder 119 outputs the encoded I picture, P picture, and M picture shown in FIG. 8B in a progressive state. For example, a P picture of an odd field and an M picture of an even field, which are in a complementary relationship, are synthesized and output as a progressive image.

  FIG. 9 is a diagram for explaining bidirectional predictive encoding of a progressive image C by the progressive video encoder 119.

  For comparison, FIG. 9A shows bi-directional predictive encoding of the interlaced image A by the first interlaced video encoder 107, which is the same as in the first embodiment.

  FIG. 9B shows bidirectional predictive encoding of the progressive image C by the progressive video encoder 119. FIG. 9B shows two types of stream data for convenience of explanation. The upper stream data shows that N pictures of the second interlaced image B are predictively encoded using the field images of the first interlaced decoded image A ′ of the past, current, and future as reference images. The steps up to here are the same as those in the first embodiment.

  In the second embodiment, since the progressive video encoder 119 is provided with the progressive image C, the original first interlaced image A can also be used as a further reference image. In the lower stream data of FIG. 9B, the N picture of the second interlaced image B is predictively encoded using the past, current, and future field images of the original first interlaced image A as reference images. It is shown that. When the upper stage and the lower stage in FIG. 9B are combined, the N pictures of the second interlaced image B can be predictively encoded using a total of six reference images.

  As described above, the progressive video encoder 119 includes the first interlace decoded image A ′ generated by the video local decoder 108 and the first interlace included in the progressive image C supplied from the A / D converter 103. Using the image A as a reference image, the N picture of the second interlaced image B is predicted, and the progressive image C is encoded. The progressive video encoder 119 outputs the encoded I picture, P picture, B picture, and N picture of FIG. 9B in a progressive state. For example, a B picture of an odd field and an N picture of an even field, which are in a complementary relationship, are synthesized and output as a progressive image.

  FIG. 10 is a configuration diagram of a moving picture decoding apparatus according to Embodiment 2. The second interlaced video decoder 207 of the first embodiment is replaced with a progressive video decoder 217 in the second embodiment, and an image supplied from the demultiplexer 203 to the progressive video decoder 217 is: This is progressive encoded data obtained by encoding the progressive image C in FIG. In the second embodiment, the progressive image composer 209 of the first embodiment is unnecessary, and the output of the progressive video decoder 217 is directly given to the D / A converter 210. Since other than this is the same as in the first embodiment, common description is omitted, and different configurations and operations will be described.

  The progressive video decoder 217 receives the input of the first interlace decoded image decoded by the first interlace video decoder 208, decodes the progressive encoded data, and outputs a progressive image.

  As described in FIG. 8B, the second interlaced image B is predictively encoded using the first interlaced decoded image and the first interlaced image included in the progressive image as a reference image. In general, the M picture of the second interlaced image B at time t is the first interlace decoded picture I ′ (or P ′) at time (t−1), the M picture at time (t−1), and the first picture at time t. 1 interlaced decoded picture I ′ (or P ′), first interlaced picture I (or P) included in the progressive picture at time (t−1), and first interlaced picture I (included in the progressive picture at time t) Or P) is predicted from the five latest reference images. When the progressive video decoder 217 decodes the M picture of the second interlaced image B at time t, the progressive video decoder 217 receives the first interlaced decoded picture at time (t−1) and time t from the first interlaced video decoder 208. The first interlace picture I (or P) obtained by decoding the first interlace encoded data included in the progressive encoded data supplied from the demultiplexer 203 in response to the input of I ′ (or P ′) Obtaining and decoding the M picture at time t by adding the difference image to the image designated as the reference image among these pictures. The progressive video decoder 217 outputs the decoded I picture, P picture, and M picture in a progressive state. For example, a P picture of an odd field and an M picture of an even field, which are in a complementary relationship, are synthesized and output as a progressive image. When bi-prediction is used as shown in FIG. 9B, when decoding the N picture at time t, decoding is performed using six adjacent reference images from both the past and future directions. The progressive video decoder 217 outputs the decoded I picture, P picture, B picture, and N picture in a progressive state. For example, a B picture of an odd field and an N picture of an even field, which are in a complementary relationship, are synthesized and output as a progressive image.

  In the moving image encoding apparatus, a second moving image encoder having both the function of the second interlaced moving image encoder 109 of the first embodiment and the function of the progressive moving image encoder 119 of the second embodiment may be provided. Good. When the second interlaced image B is input from the A / D converter 103, the second moving image encoder executes the function of the second interlaced moving image encoder 109, and the A / D converter 103 performs the progressive image C. Is input, the second video encoder performs the function of the progressive video encoder 119.

  Similarly, in the video decoding device, a second video decoder having both the function of the second interlace video decoder 207 of the first embodiment and the function of the progressive video decoder 217 of the second embodiment is provided. It may be provided. When the second interlaced image B encoded from the demultiplexer / separator 203 is input, the second moving picture decoder executes the function of the second interlaced moving picture decoder 207, When the converted progressive picture C is input, the second moving picture decoder executes the function of the progressive moving picture decoder 217.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the present embodiment, the type of recording medium is not particularly limited. For example, any recording medium such as a hard disk, an optical disk, a memory, and a tape may be used. In addition, data can be transmitted via any transmission medium such as communication and broadcasting without recording data on the recording medium. In that case, the recording apparatus can also be used as a transmission apparatus. The playback device can also be used as a receiving device. The definition of medium includes not only a narrowly-defined medium that can record data, but also electromagnetic waves and light for transmitting signal data. The information recorded on the recording medium includes data itself such as an electronic file in an unrecorded state.

1 is a configuration diagram of a video encoding apparatus according to Embodiment 1. FIG. It is a figure explaining the relationship between two complementary interlaced images and progressive images. It is a figure explaining the one-way prediction encoding by the 1st interlace moving image encoder of FIG. 1, and a 2nd interlace moving image encoder. It is a figure explaining the bi-directional predictive encoding by the 1st interlace moving image encoder of FIG. 1, and a 2nd interlace moving image encoder. FIG. 3 is a diagram illustrating a method in which the multiplexer / picture user data generator of FIG. 1 merges first interlace encoded data and second interlace encoded data. 1 is a configuration diagram of a moving picture decoding apparatus according to Embodiment 1. FIG. [Fig. 10] Fig. 10 is a configuration diagram of a video encoding device according to Embodiment 2. It is a figure explaining the unidirectional prediction encoding of the progressive image by the progressive moving image encoder of FIG. It is a figure explaining the bidirectional | two-way predictive coding of the progressive image by the progressive moving image encoder of FIG. It is a block diagram of the moving image decoding apparatus which concerns on Embodiment 2. It is a block diagram of the conventional MPEG encoding apparatus. It is a block diagram of the conventional MPEG decoding apparatus.

Explanation of symbols

  106 audio encoder, 107 first interlace video encoder, 108 video local decoder, 109 second interlace video encoder, 110 multiplexer, 119 progressive video encoder, 203 demultiplexer, 204 audio A decoder; 207 a second interlace video decoder; 208 a first interlace video decoder; 209 progressive image composer; and 217 progressive video decoder.

Claims (10)

A first interlaced image in which odd fields consisting of odd-numbered scanning lines and even-numbered fields consisting of even-numbered scanning lines are alternately extracted from a moving image at a predetermined frequency, and odd-numbered fields and even-numbered fields are extracted from the moving image. An extraction unit that extracts a second interlaced image extracted in an opposite phase to the one interlaced image;
A first encoding unit that encodes the first interlaced image to generate first interlaced encoded data;
The second interlaced encoded data is generated by encoding the second interlaced image using at least the first interlaced decoded image obtained by decoding the first interlaced encoded data as a reference image for predictive encoding. And a second encoding unit.   The second encoding unit, when predictively encoding the second interlaced image currently being encoded, uses the second interlaced image that is the encoding target as a time series reference, Three types of interlaced decoded images, past second interlaced images, and current first interlaced decoded images are used as reference images for unidirectional predictive coding, and identifiers for identifying the three types of reference images are used as first identifiers. The moving picture coding apparatus according to claim 1, wherein the moving picture coding apparatus is included in the two-interlace coded data.   The second encoding unit, when predictively encoding the second interlaced image currently being encoded, uses the second interlaced image that is the encoding target as a time series reference, An identifier for identifying the three types of reference images by using three types of interlace decoded images, a current first interlace decoded image, and a future first interlace decoded image as reference images for bidirectional predictive coding. The moving picture encoding apparatus according to claim 1, wherein the second interlace encoded data is included in the second interlace encoded data. Progressive including a first interlaced image in which an odd field composed of odd-numbered scan lines and an even-numbered field composed of even-numbered scan lines are alternately extracted from a moving image at a predetermined frequency, and both odd-numbered and even-numbered scan lines. An extractor for extracting images;
A first encoding unit that encodes the first interlaced image to generate first interlaced encoded data;
Using at least a first interlace decoded image obtained by decoding the first interlace encoded data and a first interlace image included in a progressive image supplied from the extraction unit as a reference image for predictive encoding, The second interlaced image is generated by encoding the second interlaced image in which the odd field and the even field are opposite in phase to the first interlaced image, and the first interlaced image included in the progressive image is encoded. And a progressive encoding unit that generates progressive encoded data by combining the generated first interlace encoded data and the generated second interlace encoded data. apparatus.   The progressive encoding unit, when predictively encoding the second interlaced image that is the current encoding target, uses the second interlaced image that is the encoding target as a chronological reference to the past first interlaced image. Five types of decoded images, past second interlaced images, past progressive images, current first interlaced decoded images, and current progressive images are used as reference images for unidirectional predictive coding, and the five types of references are used. 5. The moving image encoding apparatus according to claim 4, wherein an identifier for identifying an image is included in the progressive encoded data.   The progressive encoding unit, when predictively encoding the second interlaced image that is the current encoding target, uses the second interlaced image that is the encoding target as a chronological reference to the past first interlaced image. Six types of decoded images, past progressive images, current first interlace decoded images, current progressive images, future first interlace decoded images, and future progressive images are used as reference images for bidirectional predictive coding. 5. The moving picture coding apparatus according to claim 4, wherein an identifier for identifying the six types of reference pictures is included in the progressive coded data. First interlaced encoding in which a first interlaced image in which odd fields consisting of odd-numbered scanning lines and even-numbered fields consisting of even-numbered scanning lines are alternately extracted at a predetermined frequency from encoded video data A separation unit that separates and extracts data and second interlace encoded data obtained by encoding a second interlace image in which an odd field and an even field are opposite in phase to the first interlace image;
A first decoding unit that decodes the first interlace encoded data to generate a first interlace decoded image;
A second decoding unit that generates a second interlaced decoded image by decoding the second interlaced encoded data by predicting at least using the first interlaced decoded image as a reference image;
A moving picture decoding apparatus comprising: a progressive image generating unit that generates a progressive image by combining the first interlace decoded image and the second interlace decoded image. First interlaced encoding in which a first interlaced image in which odd fields consisting of odd-numbered scanning lines and even-numbered fields consisting of even-numbered scanning lines are alternately extracted at a predetermined frequency from encoded video data A separation unit that separates and extracts data and progressive encoded data obtained by encoding a progressive image including both odd-numbered and even-numbered scanning lines;
A first decoding unit that decodes the first interlace encoded data to generate a first interlace decoded image;
At least the first interlace decoded image decoded by the first decoding unit and the first interlace decoded image obtained by decoding the first interlace encoded data included in the progressive encoded data are used as reference images. The second interlaced encoded data obtained by encoding the second interlaced image in which the odd field and the even field are opposite in phase to the first interlaced image is generated to generate a second interlaced decoded image. Then, a progressive decoded image is generated by combining the first interlace decoded image obtained by decoding the first interlace encoded image included in the progressive encoded data and the generated second interlace decoded image. Including a progressive decoding unit Moving picture decoding apparatus according to symptoms. A first interlaced image in which odd fields consisting of odd-numbered scanning lines and even-numbered fields consisting of even-numbered scanning lines are alternately extracted from a moving image at a predetermined frequency, and odd-numbered fields and even-numbered fields are extracted from the moving image. Extracting a second interlaced image taken out of phase with the one interlaced image;
Encoding the first interlaced image to generate first interlaced encoded data;
The second interlaced encoded data is generated by encoding the second interlaced image using at least the first interlaced decoded image obtained by decoding the first interlaced encoded data as a reference image for predictive encoding. A video encoding method comprising the steps of: First interlaced encoding in which a first interlaced image in which odd fields consisting of odd-numbered scanning lines and even-numbered fields consisting of even-numbered scanning lines are alternately extracted at a predetermined frequency from encoded video data Separating and extracting data and second interlace encoded data obtained by encoding a second interlaced image in which odd and even fields are in opposite phases to the first interlaced image;
Decoding the first interlace encoded data to generate a first interlace decoded image;
Decoding the second interlace coded data to generate a second interlace decoded image by predicting at least using the first interlace decoded image as a reference image;
A method for decoding a moving image, comprising: generating a progressive image by combining the first interlaced decoded image and the second interlaced decoded image.