JP2013128188A - Image encoder - Google Patents

Abstract

【Task】
The load on the data bus when the input image is encoded by two encoding methods having different image qualities is reduced.
[Solution]
Each image data of the input moving image is stored in the DRAM 104. The encoder (101) reads the image data of the DRAM 104 and encodes it with HD image quality. The encoder (102) reads out image data to be encoded other than the B picture from the DRAM 104, and encodes it with SD image quality simultaneously with the encoding of the encoder (101). After completing the simultaneous encoding of the input moving image, the encoder (101) decodes the encoded image corresponding to the B picture that has not been encoded by the encoder (102), and the encoder (102) The image decoded by the encoder (101) is encoded as a B picture.
[Selection] Figure 1

Description

  The present invention relates to an image encoding apparatus that compresses and encodes moving image data, and more particularly to an image encoding apparatus that simultaneously records an input image in two systems having different image sizes or formats.

  In recent years, digital video camcorders that can shoot and record a moving image of a so-called full HD (High Definition) size of 1920 × 1080 pixels have become widespread. Also, with the development and spread of broadband infrastructure on the Internet, it is common to distribute moving images over the Internet and upload moving images to video sharing sites.

  Usually, since a moving image to be stored is desired to be kept with high image quality, it is often recorded at a full HD size and a high bit rate. However, when uploading to a video sharing site on the Internet, the file size is too large and difficult to handle at full HD size and high bit rate. In such a case, in order to reduce the file size, a recorded image may be reproduced using a personal computer or the like, the image size may be reduced, and re-encoding may be performed at a low bit rate.

  Thus, when it is desired to perform two different encoding processes, it takes time to re-encode the recorded image. Further, re-encoding an image including deterioration due to the first encoding promotes deterioration.

  On the other hand, it has been proposed to encode and record a captured image simultaneously in two systems of high image quality and low image quality (Patent Documents 1 and 2). Thereby, the trouble of re-encoding can be saved and image quality deterioration due to re-encoding does not occur.

JP 2004-343576 A JP 2006-33393 A

  When two systems of encoding processing are performed simultaneously, the amount of data flowing on the data bus is greatly increased compared to when only one system of encoding processing is performed. Therefore, even when each encoder has sufficient processing performance, there may occur a case where the encoding process does not end with a predetermined number of processing cycles due to a bus bottleneck. In such a case, frames that have not been encoded are dropped, and a correct image cannot be encoded.

  An object of the present invention is to present an image encoding device that realizes two moving image recordings having different image quality with a small load.

  An image encoding apparatus according to the present invention includes a first encoding unit that encodes an input image as a moving image having a first image size, and a second image that is smaller than the first image size. Second encoding means for encoding as a moving image of a size, and recording means for recording the moving image encoded by the first encoding means on a recording medium, wherein the second encoding means When the simultaneous operation with the first encoding means is performed, a part of the image constituting the input image is encoded, and after the completion of the simultaneous operation, other than the part of the image constituting the input image. The image is encoded by decoding the image encoded by the first encoding unit recorded on the recording medium by the first encoding unit.

  According to the present invention, it is possible to reduce the amount of data flowing on the data bus by limiting the encoding target images of the second encoding unit when two different encoding processes are performed simultaneously. By reducing the bus bottleneck, the encoding process can be normally performed without dropping frames.

It is a schematic block diagram of one Example of this invention. It is a schematic diagram which shows the example of an encoding sequence of each picture by two encoding parts. It is an operation | movement flowchart of the encoding process of a present Example. It is a schematic diagram which shows another example of an encoding sequence of each picture by two encoding parts. It is a schematic block diagram of another embodiment of the present invention. 6 is an operation flowchart of encoding control of the embodiment shown in FIG. 5. It is a schematic diagram of the example of an encoding sequence of each picture in the embodiment shown in FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of an image encoding apparatus according to the present invention. In the present embodiment, full HD size H.264 is used as the two encoding processes. The H.264 encoding method and the SD size MPEG-2 encoding method are employed, but the encoding method and the image size are not limited to this example. That is, any encoding scheme and image size may be used as long as the conditions described later are satisfied.

  The first encoder 101 converts a full HD size (first image size) image into an H.264 format. This is an encoder capable of performing an encoding process and a decoding process using the H.264 encoding method. The second encoder 102 can encode and decode an SD (Standard Definition: 720 × 480 pixels) size (second image size) image using the MPEG-2 encoding method. It is. Further, the second encoder 102 has a function of reducing the full HD size (first image size) to the SD size (second image size).

  The camera signal processing 103 performs image processing such as filter processing on the input image data, and stores the image data in the DRAM 104 through the data bus 107 and the memory interface 105. The image data stored in the DRAM 104 is the encoding target image data. Hereinafter, this encoding target image is referred to as an original image. The control CPU 106 controls the overall processing of the image encoding device.

  The encoders 101 and 102 read the original image data stored in the DRAM 104 via the memory interface 105 and the data bus 107, and perform encoding processing respectively. It is assumed that the encoders 101 and 102 are capable of performing processing within a predetermined time when operated independently. For example, in the case of an image of 30 frames / second, the processing can be completed within 33 ms per frame.

  The moving image data encoded by the encoders 101 and 102 is recorded on the recording medium 108. The recording medium 108 includes a recording medium including a hard disk or a nonvolatile semiconductor memory and an interface (recording unit) of the recording medium.

  When performing simultaneous recording of two systems, it is necessary to operate both the encoders 101 and 102 simultaneously. The encoders 101 and 102 have sufficient processing performance, but image data is read through the common data bus 107, and bus contention occurs. That is, when the encoder 101 reads image data from the DRAM 104, the encoder 102 cannot access the image data in the DRAM 1-4. The reverse is also true. Due to this bus contention, the image data read operation is delayed, and there may occur a case where the encoding process does not end in a predetermined number of processing cycles.

  In this embodiment, such a problem is solved by reducing the data access amount via the data bus 107 as described below.

  H. In the H.264 encoding method and the MPEG-2 encoding method, there are three types of picture types: an I picture (Intra picture), a P picture (Predictive picture), and a B picture (Bidirectional Predictive picture). The I picture is a picture of intra-picture coding that performs coding only with information in the picture. A P picture is a picture that is encoded by forward prediction. A B picture is a picture that is encoded by bidirectional prediction of forward prediction and backward prediction. Normally, an I picture and a P picture are reference pictures for pictures that are encoded later, but a B picture is not a reference picture.

  When performing simultaneous recording of two systems, the encoder 101 encodes all original images, while the encoder 102 encodes only I pictures and P pictures from the original images. That is, the encoder 102 does not perform encoding from an original image of a B picture. Note that the encoding of the image corresponding to the B picture in the encoder 102 is performed by decoding the encoded stream encoded by the encoder 101 after the simultaneous recording encoding operation is completed, and inputting the decoded image. This is realized by re-encoding as an image. Such processing can reduce the amount of data access via the data bus 107 during simultaneous recording. Since the B picture does not become a reference image, it does not affect the image quality of other pictures even if it is not encoded from the original image.

  In general, when the reference image is degraded, the degradation is propagated to an image encoded in the subsequent stage, and thus the influence of the image quality degradation is increased. Compared to the case where all the images are re-encoded, the I picture and the P picture that can be the reference images are encoded using the original original image. There is no. In this embodiment, since only re-encoding is performed for B pictures, it is possible to obtain good encoded image data with minimal image quality degradation with respect to SD images.

  In the first place, since B pictures are bi-directionally predicted, the amount of data access for reading a reference picture is very large compared to I pictures and P pictures. Therefore, the amount of data access via the data bus 107 can be greatly reduced by the encoder 102 not encoding the B picture during simultaneous recording.

  FIG. 2 shows an example of an encoding sequence of the present embodiment. For the original images 201 to 206 arranged in the encoding order, the encoders 101 and 102 generate an I picture, a B picture, and a P picture by a GOP (Group Of Picture) having an IBBPBB. That is, the encoder 101 creates an encoded stream of I picture 211, B pictures 212 and 213, P picture 214, and B pictures 215 and 216 from the original images 201-206. On the other hand, the encoder 102 generates an I picture 221 from the original image 201 and a P picture 224 from the original image 204 at the time of simultaneous recording. The moving image data encoded by the encoders 101 and 102 is recorded on the recording medium 108.

  After the simultaneous recording is completed, the encoder 101 decodes the B pictures 212, 213, 215, and 216, and the encoder 102 generates B pictures 222, 223, 225, and 226 by re-encoding. As a result, an encoded stream of SD image quality is generated. The encoded image data of the B pictures 222, 223, 225, and 226 generated here is merged into an encoded stream including the encoded image data recorded on the recording medium 108.

  FIG. 3 shows an operation flowchart of the encoding process of the present embodiment. As two simultaneous recording processes, the encoder 101 encodes the original images 201 to 206, and the encoder 102 encodes the original images 201 and 204 (S31). The encoded stream is stored in the DRAM 104.

  After the simultaneous recording is completed, the encoder 102 reads out the encoded image data of the I picture 221 that is the reference image of the insufficient B picture from the recording medium 108 and decodes it (S32). The encoder 101 reads out the B pictures 212 and 213, which are input images of the encoder 102, from the recording medium 108 and decodes them (S33). The encoder 102 encodes the B pictures 222 and 223 using the images of the B pictures 212 and 213 decoded by the encoder 101 as input images (S34). S32, S33, and S34 are repeated until necessary B picture decoding by the encoder 101 and encoding by the encoder 102 are completed (S35).

  Through the above processing, an encoded stream of all images by the encoder 102 is completed.

  In the present embodiment, the encoder 102 encodes only a part of the images constituting the input image when performing simultaneous operation or simultaneous encoding with the encoder 101. Then, after the end of the simultaneous operation or the simultaneous encoding, the encoder 102 applies the encoded image by the encoder 101 to an image other than the partial image of the input image, that is, an image that has not been encoded yet. Re-encode.

  By such processing, the amount of data transferred on the data bus 107 during simultaneous recording can be greatly reduced, and as a result, frame dropping due to a bus bottleneck can be prevented. Since only the B picture is re-encoded, image quality degradation due to re-encoding can be minimized and the processing time can be shortened as compared with the case where all the images are re-encoded.

  At the time of simultaneous recording, the encoder 102 may not encode not only the B picture but also the P picture. In this way, data bus contention during simultaneous recording can be further alleviated. However, as the number of pictures to be re-encoded increases, the image quality is relatively deteriorated as compared with the first embodiment.

  FIG. 4 shows an example of an encoding sequence of the present embodiment corresponding to FIG. 4, for the original images 401 to 409 arranged in the encoding order, the encoders 101 and 102 generate an I picture, a B picture, and a P picture with a GOP having a configuration of IBBPBB. Specifically, the encoder 101 generates I picture 411, B pictures 412, 413, P picture 414, B pictures 415, 416, P picture 417, B pictures 418, 219,. Create an encoded stream. On the other hand, the encoder 102 generates an I picture 421 from the original image 401 and an I picture 427 from the original image 407 during simultaneous recording. After the simultaneous recording is completed, the encoder 101 decodes the B pictures 212, 213, 215, and 216, and the encoder 102 generates B pictures 222, 223, 225, and 226 by re-encoding. As a result, an encoded stream of SD image quality is generated.

  Compared to the case of the first embodiment, the encoder 102 does not encode not only the B picture but also the P picture at the time of simultaneous recording, so that the data access amount via the data bus 107 can be further reduced.

  FIG. 5 shows a schematic block diagram of a third embodiment of the present invention. The same components as those in FIG. 1 are denoted by the same reference numerals. Differences from FIG. 1 will be mainly described.

  The encoding control unit 501 changes each encoding method according to the number of processing cycles for each picture of the encoder 101a and the second encoder 102a. More specifically, the encoding control unit 501 indicates the number of processing cycles of the encoder 102a (hereinafter referred to as “first processing cycle number”) at the time of simultaneous recording. Hereinafter, it is referred to as “second processing cycle number”). The encoding control unit 501 controls two types of encoding methods using these processing cycle numbers. One is a method in which both the encoders 101a and 102a encode all original images as usual during simultaneous recording. This is referred to as a normal encoding processing mode. The other is that the encoder 101a performs encoding processing of all original images at the time of simultaneous recording, while the encoder 102a performs encoding processing of the original images only for I pictures and P pictures, and the encoding of B pictures. This is a method that does not perform the conversion process. This is referred to as a special encoding processing mode. The special encoding processing mode is the encoding method described as the first embodiment. In the special encoding processing mode, the encoder 102a does not perform the B picture encoding processing, and therefore, the average number of processing cycles can be reduced as compared with the normal encoding processing mode.

  FIG. 6 shows an operation flowchart of encoding mode control by the encoding control unit 501. The encoding control unit 501 receives the first processing cycle number from the encoder 101a and the second processing cycle number from the encoder 102a for each frame (S61). The encoding control unit 501 selects the larger one of the first processing cycle number and the second processing cycle number and sets it as the processing number of the current frame (S62). The encoding control unit 501 calculates the average processing cycle number Cavg in the current frame unit from the processing cycle number in the current frame and the average processing cycle number in the previous frame unit (S63).

  Processing branches depending on whether the current mode is the normal encoding processing mode or the special encoding processing mode (S64). When the current mode is the normal encoding processing mode (S64), the encoding control unit 501 compares the average processing cycle number Cavg with the predetermined cycle number α (S65). When the average processing cycle number Cavg is smaller than the predetermined cycle number α (S65), the encoding control unit 501 continues the normal encoding processing mode (S66). When the average processing cycle number Cavg is equal to or greater than the predetermined cycle number α (S65), the encoding control unit 501 changes to the special encoding mode (S67).

  When the current mode is the special encoding processing mode (S64), the encoding control unit 501 compares the average processing cycle number Cavg with the predetermined cycle number β (S68). When the average processing cycle number Cavg is larger than the predetermined cycle number β (S68), the encoding control unit 501 continues the special encoding processing mode (S69). When the average processing cycle number Cavg is equal to or less than the predetermined cycle number β (S68), the encoding control unit 501 changes to the normal encoding processing mode (S70). Here, β is smaller than α.

  S61 and subsequent steps are repeated until encoding of all images is completed (S71). When the encoding of all the images is completed (S71), decoding by the encoder 101a and re-encoding by the encoder 102a are performed on the image that has not been encoded by the encoder 102a in the special encoding processing mode. (S72). Thereby, encoding of all the images by the encoder 102a is completed.

  In the flow shown in FIG. 6, since the special encoding processing mode is entered when the average processing cycle number exceeds α, the subsequent average processing cycle number becomes smaller. When the average processing cycle number is sufficiently small, that is, when the average processing cycle number is smaller than β, the normal encoding processing mode is restored. By appropriately setting α and β, the average number of processing cycles can be kept below the desired number of cycles. In addition, since only the necessary part is re-encoded, a better image can be obtained as compared with the case where all the B pictures are re-encoded.

  After all, the encoding control unit 501 controls the presence / absence or range of an image to be encoded by the encoder 102a after the end of the simultaneous encoding according to the average number of processing cycles.

  In the configuration shown in FIG. 1, the data transfer amount of the data bus 107 is reduced by making the GOP configuration in the encoding of the encoder 101 different from the GOP configuration in the encoding of the encoder 102 during simultaneous recording of two systems. it can.

  Differences from the first embodiment will be described in detail. FIG. 7 shows a GOP configuration example of each of the encoders 101 and 102. In the example shown in FIG. 7, the GOP configuration in the encoding of the encoder 101 is BIBPBP..., And the GOP configuration in the encoding of the encoder 102 is IBPBPB. As an image to be encoded by the encoder 102 as a B picture, a decoded image of the encoder 101 is used after the simultaneous recording, instead of the original image. An image to be substituted for the original image is preferably one with little deterioration, and in this respect, an I picture and a P picture are preferred as substitutes for the original image.

  In other words, in this embodiment, the GOP configurations of the two are made different so that the I picture or P picture of the encoder 101 corresponds to the B picture of the encoder 102. As a result, the B picture of the encoder 102 can be re-encoded from the I picture or P picture of the encoder 101, and image quality degradation due to re-encoding can be reduced.

Claims (5)

First encoding means for encoding an input image as a moving image of a first image size;
Second encoding means for encoding the input image as a moving image having a second image size smaller than the first image size;
Recording means for recording the moving image encoded by the first encoding means on a recording medium,
When the second encoding means operates simultaneously with the first encoding means, it encodes a part of the image constituting the input image, and configures the input image after the simultaneous operation ends. Encoding an image other than the partial image, which is obtained by decoding the image encoded by the first encoding unit recorded on the recording medium by the first encoding unit. An image encoding device characterized by the above.   The image coding apparatus according to claim 1, wherein the partial image constituting the input image is an image that is encoded by the second encoding unit.   Further, a code that receives information on the number of processing cycles from each of the first encoding unit and the second encoding unit, and controls whether or not the partial image is encoded by the second encoding unit. The image coding apparatus according to claim 1, further comprising: a coding control unit.   4. The image according to claim 1, wherein the partial image is an image that becomes a reference image when the second encoding unit encodes another image. 5. Encoding device.   The GOP configuration representing the coding sequence of the first coding means is different from the GOP configuration representing the coding sequence of the second coding means, and does not become a reference image by the coding by the second coding means. 5. The image encoding device according to claim 1, wherein an image corresponds to an image that can be a reference image by encoding by the first encoding unit. 6.

Images