JP2008011117A - Reference picture determination method for interlaced coding in image coding - Google Patents

Abstract

In a conventional image encoding apparatus, only three reference pictures are encoded when encoding a B picture, which is insufficient to achieve further reduction in memory amount and power consumption. In addition, although the motion is not large, the determination of the reference picture based only on the magnitude of the conventional motion has a problem that the compression efficiency is lowered for an image having a very strong correlation between the top field and the bottom field constituting the same frame. is there.
A method for performing a reference picture in interlaced video coding. When the motion is small, the same parity field is included in the reference picture. When the motion is large, a field with a short distance is included in the reference picture. When the correlation between the fields constituting the same frame is strong, the second field selects the reference picture to include the first field in the reference picture.
[Selection] Figure 6

Description

  The present invention relates to a method for determining a reference picture at the time of interlace coding in compression coding of a video signal, and provides a reference picture selection method according to the characteristics of the video signal, while maintaining high-efficiency coding, This is to reduce the amount of processing and memory usage during encoding.

  In recent years, the multimedia era of voice, image, and other information has been integrated, and conventional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, telephones, etc. to people are targeted for multimedia. It has come to be taken up as. In general, multimedia refers to an information transmission medium in which not only characters but also figures, sounds, images, and the like are associated with each other at the same time. It must be expressed in digital form.

  However, when the information amount of each information medium is estimated as a digital information amount, the information amount per character in the case of characters is 1 to 2 bytes, whereas in the case of voice, 64 Kbits (phone quality) In addition, for a moving image, an information amount of 100 Mbits (current television reception quality) or more per second is required, and it is not realistic to handle the information medium with such an enormous amount of information as it is in a digital format. For example, videophones have already been put into practical use by an integrated services digital network (ISDN) having a transmission rate of 64 Kbps to 1.5 Mbps, but it is not possible to send TV and camera images as they are using ISDN. Is possible.

  Therefore, what is required is information compression encoding technology. For example, in the case of a videophone, H.264 recommended by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). 261 and H.264. A compression coding standard of H.263 is used. Further, by using the MPEG-1 compression coding standard, it is possible to put image information together with audio information on a normal music CD (Compact Disc).

  Here, the MPEG (Moving Picture Experts Group) standard is an international standard regarding a compression method of a moving picture signal established by ISO / IEC (International Electrotechnical Commission). The MPEG-1 standard is a standard for compressing moving picture signals up to 1.5 Mbps, that is, information of television signals to about 1/100. Further, in the MPEG-1 standard, the target quality is a moderate quality that can be realized mainly at a transmission rate of about 1.5 Mbps, so that MPEG-standardized to meet the demand for higher image quality. In the 2 standard, the TV broadcast quality is realized at 2 to 15 Mbps for the moving image signal.

  After that, the working group (ISO / IEC JTC1 / SC29 / WG11), which has been standardizing with MPEG-1 and MPEG-2, achieved a compression ratio higher than MPEG-1 and MPEG-2, and further encoded in object units. MPEG-4 standard has been established to enable new functions necessary in the multimedia era. In the MPEG-4 standard, it was originally aimed at standardizing a low bit rate encoding method, but now it has been extended to a higher bit rate and more general purpose encoding including interlaced images. Furthermore, at present, MPEG-4 AVC and ITU H.264 have been standardized as a next-generation image encoding method with a higher compression rate jointly by ISO / IEC and ITU-T.

  In general, in the encoding of moving images, the amount of information is compressed by reducing signal redundancy in terms of time and space. Therefore, in inter-picture predictive coding for the purpose of reducing temporal redundancy, motion is detected and a predicted image is created in units of blocks with reference to the forward or backward picture, and the resulting predicted image and the encoded image are encoded. Encoding is performed on the difference value from the target picture. Here, “picture” is a term representing one screen, which means a frame in a progressive image and a frame or field in an interlaced image. Here, an “interlaced image” is an image in which one frame is composed of two fields having different times. The two fields are alternately positioned in rows.

  FIG. 7 is a diagram for explaining an interlace signal. The first field is called the first field, the second field is called the second field, and the first field is spatially located in the first row as shown in FIG. If the second field is spatially located in the second row, it is called top field first. When the first field is spatially located in the second row and the first field is spatially located in the first row as shown in FIG. 7B, it is called bottom field first. In the description in this specification, a top field first drawing is used, but the present invention can also be applied to a bottom field first video signal.

  In encoding and decoding of interlaced images, one frame can be processed as a frame, processed as two fields, or processed in a frame structure or a field structure for each block in the frame.

  An image obtained by performing intra prediction encoding without a reference image (that is, without referring to another picture) is called an I picture. In addition, an image obtained by performing inter-frame predictive coding with reference to only one picture in front or rear is referred to as a P picture. In addition, an image obtained by performing inter-frame predictive coding with reference to two pictures at the same time is referred to as a B picture. The B picture uses any two pictures whose display time is either forward or backward as reference images. Here, the prediction that refers to the picture whose display time is the forward prediction is the forward prediction, the prediction that refers to the backward picture is the backward prediction, and the prediction that refers to two pictures of any combination from the front or the rear is bidirectional prediction. Call. A reference image (reference picture) can be specified for each block that is a basic unit of encoding and decoding. As a condition for encoding and decoding these pictures, a reference picture has already been encoded. And need to be decrypted.

  FIG. 8 shows a reference field in a bidirectional prediction picture (B picture) in the MPEG-2 field structure. The reference pictures of the MPEG-2 bi-predictive picture are temporally forward and backward I or P pictures. Since the number of consecutive B frames is usually two, it will be described here also based on this assumption. In the case of two B frames, when interlace coding is performed, a total of four fields of B pictures are included. The description about the reference of each B picture is described in each of (a) to (d). In the case of the B field of the top field of the first B frame among the two consecutive B frames (FIG. 8A), the picture to be referenced is a temporally forward picture and is an I picture or P picture ( In this case, I1 and P1b) and I picture or P picture (in this case, P2t and P2b) are referred to in the temporally backward picture. Similarly, in the case of the bottom field of the first B frame, the case of the top field of the second B frame, and the case of the bottom field of the second B frame, respectively, FIG. 8 (c) and FIG. 8 (d).

  9 shows MPEG-2 and H.264. 2 shows differences between H.264 (MPEG-4AVC) reference pictures. H. In the case of H.264 (MPEG-4AVC), unlike MPEG-2, it is possible to refer to a B picture. FIG. The reference picture of the bottom field of the second B frame in H.264 is shown. FIG. 9B shows a reference picture of the bottom field of the second B frame in MPEG-2 that does not refer to the B picture. As can be seen from the figure, since the distance between the picture to be encoded and the reference picture is closer when referring to the B picture than when referring to the IP picture, the correlation between pictures becomes stronger, and the compression efficiency in inter-picture prediction Will improve.

  FIG. 10 shows an H.264 format in the case where a reference picture is used by referring to four fields as in MPEG-2 and a referenceable B picture is used. 2 shows an example of a H.264 reference picture.

  A processing unit obtained by dividing a picture into 16 × 16 pixel units is called a macroblock. In H.264, the macroblock prediction method is significantly increased as compared with MPEG-2. It is possible to properly select the method with the best compression rate from among many prediction methods, but the processing amount for that is enormous, so it is processed for products that require real-time processing such as movies and recorders. There is a problem that the amount must be reduced. Especially for intangible battery-operated devices such as movies and mobile phones, the demand for reducing the amount of processing is even more severe. There are several methods for reducing the processing amount. In H.264, a B picture that can be referred to can be used, and the correlation between screens is strengthened. Therefore, a method of not performing screen prediction on a reference picture having a relatively small correlation is conceivable.

  Conventionally, a method of reducing the number of reference pictures from four to three has been proposed (see Patent Document 1). In the prior art, the size of motion is detected, and when the motion is large, the picture with near time is prioritized, and when the motion is small, the picture of the same parity is prioritized to reduce the number of four to three. However, the compression efficiency is not reduced.

  However, there is a strong demand for power reduction and memory reduction more than expected by the prior art, and it is necessary to reduce the number of reference pictures from three to two. The prior art is based on the premise that the number of reference pictures is three, and the selection method in the case of two is not disclosed. In addition, some videos show a tendency that does not belong to either the case of large movement or the case of small movement classified by the prior art, and a new method is required.

  FIG. 11 is a block diagram showing an example of the conventional image encoding device. As illustrated in FIG. 11, the image encoding device 200 includes an encoded picture memory 102, an orthogonal transform quantization unit 103, a variable length encoding unit 104, an inverse variable length encoding unit 106, an inverse quantization / inverse orthogonal transform unit. 107, a reference picture decoding unit 108, a reference picture memory 109, and an inter-screen prediction unit 110.

  The input video signal 101 is first stored in the encoded picture memory. When bi-predictive pictures are used, the order of pictures to be encoded may be changed by the order of display and several frames, but this rearrangement process is assumed to be performed in advance. The inter-screen prediction unit 110 searches for and obtains the reference picture with the highest correlation from the reference pictures stored in the reference picture memory 109. There are many types of search methods, but the amount of processing tends to increase as the number of reference pictures increases. The difference between the image with the highest correlation searched by the inter-screen prediction 110 and the input video signal 101 is taken, and the difference signal is output to the orthogonal transform / quantization unit 103. The difference signal is subjected to orthogonal transformation, and the transformed difference signal is further quantized. The quantized difference signal is modulated and encoded by the variable length encoding unit and output as an output stream 105. Further, since inter-picture prediction is prediction with a past encoded picture, it is necessary to decode the encoded picture and return it to an image signal in the encoding device. That is, the output stream is input to the inverse variable length coding unit 106, further input to the inverse quantization / inverse orthogonal transform unit, and returned to the difference signal. The reference picture decoding unit 108 adds the difference signal to any (or a plurality) of reference pictures stored in the reference picture memory 109 in the past, and updates the reference picture memory.

In the encoding device of the prior art, when encoding a B picture, the memory of three field pictures is stored in the reference picture memory, and is a search target of the inter-screen prediction unit 110.
Japanese Patent Application No. 2004-318489, Name: Method for determining reference picture in predictive coding of video signal, Inventors: Arakawa, Tsunono, Shibahara

  However, in the conventional image coding apparatus 200 described above, only three reference pictures at the time of coding a B picture are reduced, which is insufficient to achieve further reduction in memory amount and power consumption. . In addition, although the motion is not large, the determination of the reference picture based only on the magnitude of the conventional motion reduces the compression efficiency for video with a very strong correlation between the top field and the bottom field constituting the same frame. is there.

  In view of the above problems, the present invention makes it possible to maintain a high compression rate by reducing the number of reference pictures of B pictures to two and by switching the reference picture structure more finely according to the video characteristics. It is an object to provide a reference picture determination method.

  In order to solve the above problems, a reference picture selection method according to the present invention is described in [Claim 1], and specific reference picture determination methods are described in [Claim 2] to [Claim 4]. It is described.

  Note that the present invention can be realized as an image encoding method using characteristic constituent means in the image encoding apparatus as steps, or as a program for causing a personal computer or the like to execute these steps. Needless to say, the program can be widely distributed via a recording medium such as a DVD or a transmission medium such as the Internet.

  Furthermore, the present invention can also be realized as an LSI or the like in which the main functions of the image encoding device are integrated circuits.

  According to the present invention, it is possible to reduce the number of reference pictures to two or less while maintaining a high compression rate based on the characteristics of the video signal. As a result, it is possible to reduce the amount of memory required to store the reference picture, and to reduce the amount of calculation and data transfer for inter-screen prediction for the reference picture.

  Embodiments of the present invention will be described below with reference to the drawings. In addition, although this invention is demonstrated using the following embodiment and attached drawing, this is for the purpose of illustration and this invention is not intended to be limited to these.

(Embodiment 1)
FIG. 1 is a block diagram showing an example of an image encoding device 100 according to Embodiment 1 of the present invention. As shown in FIG. 1, the image encoding apparatus includes an encoded picture memory 102, an orthogonal transform quantization unit 103, a variable length encoding unit 104, an inverse variable length encoding unit 106, an inverse quantization / inverse orthogonal transform unit 107. A reference picture decoding unit 108, a reference picture memory 109, and an inter-screen prediction unit 110. In FIG. 1, the same reference numerals are used for the same components as in FIG. 11, and the description thereof is omitted. Characteristic components in the present embodiment are a reference picture selection unit 111, a reference picture determination unit 112, a video signal feature amount 113, field position information 114, and a control signal 115.

  The video signal feature amount information 113, which is information indicating characteristics relating to the input video signal 101, is information representing a state where the movement is large or the movement is small, for example. It can be obtained from the input video signal 101 stored in the coded picture memory 102, can be obtained by statistically processing past results of the inter-screen prediction unit 110, or can be coded. It is also possible to give from the outside of the apparatus 101. In the present invention, based on the information indicating the magnitude of the motion, the reference picture determination unit 112 determines the reference picture of the bi-predictive picture and controls which of the pictures stored in the reference picture memory is to be referred to The signal 115 is sent to the reference picture selection unit 111. Based on the control signal 115, the reference picture selection unit 111 outputs only the selected picture to the inter-screen prediction unit 110. The field position information 114 is the number of B frames in the display order (counted from the P frame) and whether the encoded picture is the first field or the second field constituting the frame. This information is shown.

  The reference picture selection unit 111 and the reference picture determination unit 112 are described as independent components, but may be a single component having both functions.

  FIG. 2 shows how to select a reference picture when the motion of the input video signal 101 is small. When the motion is small, when the encoded picture is the first field, the reference picture also refers to the first field, and when the encoded picture is the second field, the reference picture also refers to the second field. Since the correlation is often higher, the selection is made as shown in FIG.

  As shown in FIG. 2A, the first field B1t of the first B frame refers to the first field I1 of the temporally forward frame and is the first field of the temporally backward frame. Refer to P2t. In addition, since P2t of time back is far, reference is not important compared with I1. It is also possible to refer to another picture by omitting the reference to P2t.

  As shown in FIG. 2B, the second field B1b of the first B frame refers to P1b, which is the second field of the temporally forward frame, and refers to P2b of the temporally backward frame. Similarly, reference to P2b can be omitted.

  As shown in FIG. 2C, the first field B1t of the second B frame refers to the temporally forward B1t and the temporally backward frame Pt2. Since both are short distances and there is a concern that the compression rate will be reduced if omitted, there are many cases where the picture quality of a P picture is better than that of a B picture.

  As shown in FIG. 2D, the second field B2b of the second B frame refers to the temporally forward B1b and the temporally backward P2b. As in the case of Bt2, since both are close to each other, there is a concern that the compression rate will be reduced if omitted. Is also possible.

  FIG. 3 shows how to select a reference picture when the motion of the input video signal 101 is large. When the motion is large, it is possible to maintain a high compression rate by selecting a reference picture with the shortest distance. H. In the case of H.264, the use of the referenceable B picture makes the distance from the temporally forward reference picture closer to that of the temporally backward reference picture, so selection of the temporally forward reference picture is important. .

  As shown in FIG. 3A, the first field B1t of the first B frame refers to the second field P1b of the temporally forward frame. Since the distance from the reference picture to the temporal rear is long, it is relatively less important and can be omitted. However, in the temporal backward reference picture, it is desirable to refer to the P2t having a short distance. .

  As shown in FIG. 3B, the second field B1b of the first B frame refers to the first field B1t of the frame, which is the reference picture with the shortest distance. Since the temporally backward reference picture is relatively insignificant because it is far away, it can be omitted. However, it is desirable to refer to P2t having a short distance in the temporally backward reference picture.

  As shown in FIG. 3C, the first field B2t of the second B frame refers to B1b, which is the second field of the temporally forward frame, which is the reference picture with the shortest distance. Since the reference picture has a relatively long distance and is relatively low in importance, it can be omitted. However, it is desirable to refer to P2t having a short distance among the reference pictures that are temporally backward.

  As shown in FIG. 3 (d), the second field B2b of the second B frame is a reference picture having the shortest distance, B2t being the first field of the frame, and the first field behind in time. Reference is made to Pt2. Since both are short distances and there is a concern that the compression rate will be reduced if omitted, there are many cases where the picture quality of the P picture is better than the B picture, so it is conceivable to refer to P2t and not to B2t.

  FIG. 4 shows the input video signal 101 classified according to three characteristics. FIG. 4A is a picture with small motion, and shows a video signal having a high vertical resolution but a high correlation between spatially adjacent rows. FIG. 4B is a picture with a large motion, and is a video signal in which the correlation between fields in a short distance is stronger than the correlation between spatially adjacent rows. FIG. 4C shows a video signal in which the correlation between the first field and the second field constituting the frame is very strong. When an interlaced video signal is converted to a progressive video by applying a vertical filter, and converted to an interlaced signal again for the convenience of the transmission and storage system, a video signal having the characteristics shown in FIG. is there. Alternatively, even when a once encoded video signal is decoded and input to the encoding device again, the amount of data assigned to the second field is reduced for the purpose of reducing the amount of data during the first interlaced encoding. Some video signals have characteristics similar to those of the first field.

  For a video signal having the characteristics as shown in FIG. 4C, it is indispensable that the second field constituting the frame refers to the first field constituting the same frame. In the case of the first field constituting the frame, the reference picture may be determined according to the magnitude of the motion as described above.

  In FIGS. 5A and 5C, the fields constituting the same frame cannot be referred to, so the contribution to the compression rate is not relatively high. As described above, the reference picture may be determined based on the magnitude of motion.

  FIG. 5B shows that it is very important for the second field of the first B frame to refer to the first field constituting the same frame. In this figure, Pt2 is referred to as a temporally backward reference picture, but P2t or Ptb may be referred to depending on the magnitude of motion, and if it is desired to further reduce the number of reference pictures, the distance Because it is far away, there is no need to refer to it.

  FIG. 5D shows the reference picture of the second field of the second B frame. As in the case of the first B frame, it is very important to refer to the first field constituting the same frame.

  FIG. 6 shows an example of a flow for determining the reference picture structure. First, in step S101, it is checked whether the fields constituting the frame have a strong correlation. If the correlation is strong, the first field constituting the same frame is included in the reference picture. If the reference picture is the first field, it can be considered that the processing in step S103 and after is similarly performed. In step S103, it is checked whether the motion is large. If the motion is large, the closest picture is included in the reference picture in step S104. If it is determined in step S105 that the motion is small, in step S105, a picture that is the same field as the encoded picture and has the closest distance is included in the reference picture. In FIG. 6, the process of step 101 is first performed. However, the same result can be obtained as the operation of determining the magnitude of the motion in step S103 first. In FIG. 6, the aim is to indicate that there are three types of reference pictures to be selected based on the video characteristics, and there is no premise of the temporal relationship between the determination processes S101 and S103.

  Each component in each block diagram of the first embodiment (FIGS. 2 and 5, etc.) is typically realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them (for example, functional blocks other than the memory may be made into one chip).

  The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. There is a possibility of adaptation of biotechnology.

  In addition, among the functional blocks, only the means for storing the data to be encoded or decoded may be configured separately instead of being integrated into one chip.

  In the present embodiment, the bidirectional prediction picture has been described. It is also applicable to bi-predictive pictures such as H.264. The bi-predictive picture is the same as the bi-predictive picture in that a predictive picture is generated from two reference pictures. However, not only bi-directional pictures but also two reference pictures in the same direction (forward or backward) are used. The point that can be predicted is different.

  The image coding method according to the present invention has means for compressing image data with high efficiency, and is useful for long-term recording of images for storage purposes and for distributing images with a small amount of data for transmission purposes. is there. It can also be applied to applications such as videophone and TV broadcast recording.

FIG. 1 is a block diagram illustrating a functional configuration of an image encoding device including a reference picture determination device according to the invention. FIG. 2 illustrates the inclusion of a picture of the same parity as a reference picture when the motion of the invention is small. FIG. 3 is a conceptual diagram illustrating a reference picture selection method according to the invention that includes a picture with the closest time as a reference picture when the motion is large. FIG. 4 is a diagram for explaining classification of video signal features. FIG. 5 is a conceptual diagram illustrating a reference picture selection method according to the invention in which the second field includes at least the first field as a reference picture when the correlation between fields of the same frame is strong. FIG. 6 is a flowchart for explaining the operation of the reference picture selection method of the invention. FIG. 7 is a conceptual diagram of an interlace signal. FIG. 8 is a conceptual diagram of a conventional MPEG-2 reference picture. FIG. It is a conceptual diagram of H.264 B picture which can be referred. FIG. 2 is a conceptual diagram of H.264 reference pictures. FIG. 11 is a block diagram of an encoding apparatus including a conventional reference picture determination method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Conventional image coding apparatus 101 Input video signal 102 Encoded picture memory 103 Orthogonal transformation / quantization part 104 Variable length coding part 105 Output stream 106 Inverse variable length coding part 107 Inverse quantization / inverse orthogonal transformation part 108 Reference Picture decoding unit 109 Reference picture memory 110 Inter-picture prediction unit 200 (comprising the reference picture determination method of the invention) Image encoding device 111 Reference picture selection unit 112 Reference picture determination unit 113 Video signal feature 114 Field position information (B1 and B2) , Distinction between top and bottom)
115 Control signal

Claims (4)

  A method of determining a reference picture in field predictive coding of a video signal, wherein a picture (bidirectional prediction picture) that refers to the front in display order and the back in display order is referred to in units of pictures according to the characteristics of the video signal A reference picture determination method for determining a picture.   In the reference picture determination method, when the motion of the entire space of the bi-predictive picture is small, if the picture is the n-th field (n is 1 or 2) constituting the frame, it moves forward in the display order. 2. The reference picture determining method according to claim 1, wherein the nth field constituting each of the rear frames in the display order is referred to.   In the reference picture determination method, when the motion of the entire space of the bi-predictive picture is large, if the picture is the first field constituting the frame, the second frame constituting the forward frame in the display order is used. 2. The reference picture determination according to claim 1, wherein, when the picture is a second field constituting a frame, the picture refers to the first field constituting the frame. Method.   In the reference picture determination method, when the picture is the second field constituting the frame and the first field and the second field constituting the frame including the picture have a strong correlation, the picture 2. The reference picture determining method according to claim 1, wherein the first field constituting the frame including the picture is referred to.

Images