JP2018050150A - Image coding apparatus, image coding method, and computer program - Google Patents

Abstract

Kind Code: A1 To determine whether adaptation of intra-smoothing processing of the HEVC method is adaptive depending on an input image, and maintain encoded image quality. A gradation detection unit detects whether an input image is a gradation image in a predetermined pixel block unit. Intra-smoothing processing determining means 104 instructs encoding section 105 to validate linear interpolation of reference pixels at the coding block boundary in the intra-screen prediction mode when a predetermined number or more of gradation images are included in the screen. . [Selection diagram] Fig. 1

Description

  The present invention relates to an image encoding device, an image encoding method, and a computer program, and more particularly, to an image encoding device that uses intra prediction, an image encoding method, and a computer program.

  2. Description of the Related Art Conventionally, a digital video camera is well known as a camera-integrated moving image recording apparatus that compresses and records moving image data obtained by taking a moving image of a subject.

  As the moving image compression method, MPEG2 method and H.264 which can be compressed at a high compression rate using motion prediction between frames. The H.264 system is generally used. In recent years, the HEVC method (ISO / IEC 23008-2) that can be compressed with high efficiency using a more complicated prediction method has been used.

  In the HEVC system, H.264 is used. Various improvements have been adopted to improve the compression efficiency with respect to the H.264 system. As one of them, a technique is known in which a filtering process is performed on a reference pixel adjacent to a coding block in order to increase the coding efficiency by using the correlation between adjacent pixels during the intra prediction process. In particular, in this filter processing, special filter processing for improving visual quality can be applied to a gradation region in which color shading and lightness change little by little depending on conditions.

  With reference to FIG. 14, the necessity of the filter process for gradation area will be briefly described. FIG. 14A shows an example of a change in pixel value when there is a block boundary in the gradation area. The horizontal axis represents pixels, and the vertical axis represents luminance values. H. In the H.264 system, compression processing is performed in units of blocks, so that pixels at the block boundary are discontinuous between encoded pixel blocks adjacent to the periphery of the encoding target block. As a result, in a coded image block that employs intra prediction, a predicted image is generated while the pixel values remain discontinuous between reference pixels at the block boundary. As a result, a pseudo contour is generated as shown in FIG. FIG. 14B shows a state in which a pseudo contour is generated in a line connecting from the upper right to the lower left at the center of the encoding target block.

  For such a problem, Patent Document 1 describes a method of generating a predicted value by defining a reference pixel in a wider range than a pixel position defined in the standard. Patent Document 2 describes that gradation detection is performed in units of coding blocks, and the pixel size for intra prediction is set to the minimum size in the standard for an image block determined to be a gradation image.

  Therefore, in the HEVC method, when the absolute value of the difference between the pixel value at both ends of the reference pixel at the adjacent block boundary and the average value is less than a threshold value determined according to the luminance bit depth, the reference pixel is linearly changed. Intra smoothing processing to be interpolated is defined. However, in order to apply this processing, it is necessary to satisfy all of the following three conditions in addition to the difference in pixel values between block boundaries. First, the pixel block size (Transform Unit size: TU size) for frequency conversion is 32 × 32 pixels. Second, the mode value (predModeIntra) of directionality prediction is neither 1 (DC prediction), 10 (horizontal direction prediction), or 26 (vertical direction prediction). Third, the variable strong_intra_smoothing_enabled_flag of the sequence parameter set (hereinafter referred to as “SPS”) is 1.

JP 2008-245088 A JP2011-239365A

  The technique described in the cited document 1 is described in H.C. It is limited to the H.264 system and cannot be applied to a moving picture encoding apparatus that employs the HEVC system.

  The technique described in the cited document 2 is a method that does not depend on the HEVC type intra smoothing process. However, since the processing is performed in units of macroblocks, it is not always possible to increase the encoding efficiency for the entire encoded picture and the entire moving image sequence.

  In the method employing the HEVC method of intra smoothing, when the variable strong_intra_smoothing_enabled_flag in the SPS is always set to 1, the linear interpolation filter process is applied to all the encoded sequences. In this method, depending on the design of the input image, the resolution of the image may be impaired, and visual characteristics may not necessarily be improved.

  An object of the present invention is to present an image encoding device, an image encoding method, and a computer program that can apply the HEVC method, and that can adaptively determine whether or not to apply an intra smoothing process according to an input image.

  In order to solve the above problems, an image encoding device according to the present invention is an image encoding device that encodes an input image, and is an encoding unit that encodes the input image in units of encoding blocks. Encoding means having a mode for linearly interpolating reference pixels at the encoding block boundary in the intra prediction mode, and gradation in a predetermined pixel block unit with respect to the encoding block to be encoded by the encoding means Gradation detecting means for detecting whether an image is present; and intra-smoothing process determining means for determining whether or not the linear interpolation process of the reference image in the intra prediction mode is applied to the encoding target block image by the encoding means. When the number of pixel blocks determined to be the gradation image by the gradation detecting means exceeds a predetermined number , Characterized by having a intra smoothing processing determining means for instructing activation of the linear interpolation to the encoding means.

  According to the present invention, since the applicability of the intra smoothing process is determined based on the gradation detection result of the input image, it is possible to maintain or improve the image quality of the entire encoded picture and the moving image sequence.

It is a schematic block diagram of Example 1 of the present invention. It is a schematic block diagram of an encoding part. 3 is an operation flowchart according to the first embodiment. It is an example of the encoding data structure of Example 1. FIG. It is an example of a link of the parameter set of Example 1. FIG. 6 is a schematic configuration block diagram of Embodiment 2. FIG. 6 is an operation flowchart of the second embodiment. It is a figure which shows the input image of Example 2, and the example of a histogram. FIG. 10 is a schematic configuration block diagram of Embodiment 3. 10 is an operation flowchart of the third embodiment. FIG. 9 is an operation explanatory diagram of Embodiment 3. FIG. 10 is a schematic block diagram of a fourth embodiment. 10 is an operation flowchart of the fourth embodiment. FIG. 10 is an operation explanatory diagram of the fourth embodiment. It is explanatory drawing of the pseudo contour which generate | occur | produces with a prior art.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an imaging apparatus that employs an embodiment of an image encoding apparatus according to the present invention. The imaging apparatus shown in FIG. 1 includes an imaging unit 101, a gradation detection unit 102, an intra smoothing process determination unit 104, and an encoding unit 105. The gradation detection unit 102 includes a variance value calculation unit 103.

  The imaging unit 101 is an image sensor such as a CCD sensor, an optical system that inputs a subject optical image to the image sensor, and an image processing unit that digitizes an output image signal of the image sensor and outputs it as moving image data in a predetermined video format. Consists of.

  The gradation detection unit 102 divides each frame image of moving image data output from the imaging unit 101 into a plurality of pixel blocks, and determines and detects whether or not the image is a gradation image in pixel block units. The variance value calculation unit 103 calculates the variance of the pixel value distribution in each image area. The gradation detection unit 102 determines that the image is a gradation image when the dispersion value calculated by the dispersion value calculation unit 103 is not a single-color dispersion value and indicates a value less than a certain value.

  The intra smoothing processing determination unit 104 accumulates the number of pixel blocks determined to be a gradation image included in the frame image according to the detection result of the gradation detection unit 102. When the number of pixel blocks of the gradation image is greater than or equal to a predetermined number in one frame image, the intra-smoothing process determination unit 104 determines whether or not to apply linear interpolation of the reference pixel at the encoding block boundary in the intra prediction mode of the frame image. decide. When applying, the intra smoothing process determination unit 104 outputs an instruction signal (smoothing enable instruction signal) for enabling the intra smoothing process to the encoding unit 105.

  The encoding unit 105 compresses and encodes the moving image data output from the imaging unit 101 with the HEVC method while referring to the notification from the intra-smoothing process determination unit 104, and the obtained encoded data is in a predetermined bit stream format. To output.

  In this embodiment, the size of the pixel block for gradation detection is the same size as the processing unit in the encoding unit 105, that is, CTU (Coding Tree Unit) defined in the HEVC standard, but may be different. .

  FIG. 2 shows a schematic block diagram of the encoding unit 105. The encoding unit 105 stores the moving image data from the imaging unit 101 in the frame memory 201 in units of frames. In addition, the frame memory 201 stores reference image data used in motion prediction processing.

  The intra prediction unit 202 reads the image data of the encoding target block from the frame image data stored in the frame memory 201, and calculates correlations with a plurality of intra prediction images generated from the reference pixel data around the encoding target block. To do. The intra prediction unit 202 also receives the output signal of the intra smoothing process determination unit 104. When the smoothing activation instruction signal is input from the intra smoothing process determination unit 104, the intra prediction unit 202 enables the linear interpolation process for the reference pixel at the block boundary in generating a predicted image for intra prediction. The intra prediction unit 202 selects an intra prediction method having the highest correlation and notifies the intra / inter determination unit 204 of the intra prediction method.

  The inter prediction unit 203 performs pattern matching between pixel data in units of encoding blocks between the original image data of the encoding target block stored in the frame memory 201 and the encoded image data, and obtains a motion vector. calculate. The inter prediction unit 203 supplies the calculated motion vector to the intra / inter determination unit 204.

  The intra / inter determination unit 204 determines a prediction mode to be applied based on the output results of the intra prediction unit 202 and the inter prediction unit 203, and outputs the prediction mode to the predicted image generation unit 205. For example, the intra / inter determination unit 204 generates a prediction image for the encoding target image block according to the output of the intra prediction unit 202, and calculates a prediction error between the prediction image and the encoding target block image. The intra / inter determination unit 204 also generates a prediction image from the reference image according to the motion vector obtained by the inter prediction unit 203, and calculates a prediction error between the prediction image and the encoding target block image. Then, the intra / inter determination unit 204 compares both prediction errors, and sets the smaller prediction method as a prediction mode to be applied to encoding. In the case where the intra prediction unit 202 and / or the inter prediction unit 203 are executing up to the calculation of the prediction error, the intra / inter determination unit 204 calculates a prediction error that is insufficient, compares them, and applies a prediction mode to be applied. Can be determined.

  The predicted image generation unit 205 generates a predicted image according to the prediction mode determined by the intra / inter determination unit 204. The predicted image generation unit 205 also receives the output signal of the intra smoothing process determination unit 104. When the intra-smoothing process determination unit 104 outputs the smoothing validation instruction signal, the encoding target image is a gradation image as described above. In the case of a gradation image, the prediction image generation unit 205 should apply the three conditions that the intra prediction mode, the TU size is 32 × 32 pixels, and the directionality prediction mode value is neither 1, 10, nor 26. It is determined whether or not the encoding condition is satisfied. When these three conditions are satisfied, the predicted image generation unit 205 generates a predicted image from the reference pixels subjected to the linear interpolation process.

  The subtracter 214 subtracts the predicted image from the predicted image generation unit 205 from the image data to be encoded from the frame memory 201. The difference image data generated by the subtraction in the subtracter 214 is input to the integer conversion unit 206. The integer conversion unit 206 converts the difference image data from the subtracter 214 into a spatial frequency domain in units of encoded blocks. The integer conversion of the integer conversion unit 206 is an example of frequency conversion.

  The quantization unit 207 calculates a quantization coefficient based on the target code amount, and quantizes the spatial frequency domain coefficient data output from the integer transform unit 206. The quantized coefficient data is input to the entropy encoding unit 208 and the inverse quantization unit 210.

  The entropy encoding unit 208 entropy encodes the quantized coefficient data from the quantization unit 207. For example, in the case of inter prediction, the entropy encoding unit 208 compresses information on a vector value used in motion prediction using a bias in the appearance probability of bit data such as a CABAC (context adaptive arithmetic coding) method. The entropy encoding unit 208 supplies the encoded data after compression to the bit stream generation unit 213.

The bit stream generation unit 213 adds header information including parameters necessary for decoding to the encoded data from the entropy encoding unit 208, and outputs the data in a predetermined streaming format. Parameters necessary for the decoding process include, for example, an SPS and a picture parameter set (hereinafter referred to as “PPS”). The bit stream generation unit 213 also associates the picture to which the intra smoothing process is applied in the encoded image so as to refer to a header having a strong_intra_smoothing_enabled_flag field value of 1 in the SPS.
What are the details of the method of associating the SPS and PPS headers for the coded picture to which the intra-smoothing process is applied, and the relationship between the data structure of the bit stream? It will be described later. The output of the bit stream generation unit 213 is recorded on a recording medium (not shown) such as a memory card or a hard disk.

  The code amount control unit 209 obtains the code amount of the encoded data output from the entropy encoding unit 208, calculates the target code amount per picture based on the bit rate and the buffer model, and Feedback control of the quantization factor.

  The part related to the local decoding process will be described. The inverse quantization unit 210 performs inverse quantization by multiplying the quantized coefficient data from the quantization unit 207 by a quantization coefficient. Thereby, the coefficient data is reconstructed. The inverse integer transform unit 211 performs inverse integer transform of the coefficient data output from the inverse quantization unit 210 into pixel data. The adder 215 adds the predicted image data from the predicted image generation unit 205 to the output of the inverse integer conversion unit 211. Thereby, the image data is restored. The output image data of the adder 215 is supplied to the predicted image generation unit 205 and the loop filter 212 as reference image data.

  The loop filter 212 performs filter processing to reduce the coding distortion generated at the block boundary on the output image data of the adder 215 and stores the filter image in the frame memory 201.

  FIG. 3 shows a flowchart of the encoding process of the present embodiment. The processing illustrated in FIG. 3 is executed by the gradation detection unit 102, the variance value calculation unit 103, the intra smoothing processing determination unit 104, and the bit stream generation unit 213. The processing shown in FIG. 3 is repeatedly executed for each input frame image. From the viewpoint of reducing the calculation burden, the processing shown in FIG. 3 may be executed for a luminance component that easily detects a change in human visual characteristics.

  In S301, the gradation detection unit 102 determines whether or not the variance value VAR for all CTU blocks has been calculated for the input frame image data by the variance value calculation unit 103. If the variance value has been calculated for all CTU blocks (TRUE in S301), the process proceeds to S305. If a CTU block image for which the variance value has not been calculated remains (FALSE in S301), the process proceeds to S302.

分散値ＶＡＲは、画像ブロックにおける画素値の散らばり具合を示す指標である。分散値が大きいと画素のバラツキが大きく複雑な絵柄であると判断できる。他方、分散値ＶＡＲが小さい場合、平坦な絵柄であると判断できる。 In S302, the variance value calculation unit 103 calculates the variance value VAR of each CTU block image one by one. Here, the variance value calculation unit 103 calculates, as the variance value VAR, the value obtained by subtracting the square of the average value of all the pixels from the average value of the square sum of the pixel values for all the pixels in the predetermined image block. To do. That is,

The variance value VAR is an index indicating the degree of dispersion of pixel values in the image block. If the variance value is large, it can be determined that the pixel variation is large and the pattern is complicated. On the other hand, when the variance value VAR is small, it can be determined that the pattern is flat.

  In S303 following S302, the gradation detection unit 102 determines whether or not the variance value calculated in S302 is less than a threshold value. This threshold value is determined in advance as a value that can be taken in the case of a flat pattern such as a gradation image. This threshold value depends on the size of the block image for which the variance value is calculated, but is preferably set to about 10 to 100 in the case of a macroblock size of 32 × 32 pixels or 64 × 64 pixels. If the variance value of the block image is less than the threshold value (TRUE in S303), the process proceeds to S304. On the other hand, when the variance value of the block image is equal to or larger than the threshold value (FALSE in S303), the process returns to S301 and the process is repeated for the next CTU block image.

  In S304, the gradation detection unit 102 increments the value of the number of CTU blocks estimated to be a gradation image included in the input image. Thereafter, the process returns to S301, and the process is repeated for the next CTU block image.

  In step S305, the gradation detection unit 1102 determines whether the number of CTU blocks estimated as a gradation image is greater than or equal to a predetermined threshold value. This threshold value is used to determine whether or not the input frame image of interest includes a large number of flat patterns having a small variance value, that is, a gradation image. This threshold value is set, for example, as a guideline for the number of 80 to 90% or more of all CTU blocks in the screen that are flat. When the input frame image includes many gradation images (TRUE in S305), the process proceeds to S306. When the input frame image is an image with a large variance value including many patterns such as many objects and textures (FALSE in S305), the process proceeds to S308.

  In S306, the intra smoothing process determining unit 104 outputs the intra smoothing process enabling signal to the encoding unit 105, and the encoding unit 105 encodes the input frame image data in accordance with the intra smoothing process enabling signal. In step S307, the bit stream generation unit 213 generates a bit stream in which the SPS and the slice header associated with the strong_intra_smoothing_enabled_flag are set to 1 for the encoded picture data.

  In S308, the encoding unit 105 compresses and encodes the frame image data with the intra smoothing process disabled. In step S309, the bit stream generation unit 213 generates a bit stream of the encoded picture data by associating the SPS with strong_intra_smoothing_enabled_flag set to 0 and a slice header.

  The structure of the bit stream data output from the bit stream generation unit 213 will be described. FIG. 4A shows an example of a bit stream structure when the intra smoothing process is enabled in steps S307 and S309, and FIG. 4B shows a reference relationship of parameter sets.

  As shown in FIG. 4A, the HEVC encoded bitstream uses the H.264 standard. As in the H.264 system, a data structure in which a plurality of access units (hereinafter referred to as “AU”) are consecutive in decoding order is employed. Each AU consists of encoded data that can be decoded on a screen-by-screen basis. The AU includes an access unit delimiter (AUD) indicating a start position, a video parameter set (VPS) having various parameters necessary for decoding, and a plurality of NAL units called SPS or PPS. Data encoded in actual macroblock units is stored as slice data for each picture following the header parameter.

  The header information called SPS or PPS can be changed in the middle of the stream according to the standard. For example, parameters such as a quantization matrix and deblocking filter strength can be reloaded in AU units according to the pattern. In order to realize this function, at the stream output timing, the encoded slice data and the header information necessary for decoding the encoded slice data are associated by the identifier (ID) included in each header information. .

  FIG. 4B shows a reference relationship between VPS, SPS, PPS, and slice data. The VPS has vps_video_parameter_set_id for identifying the VPS. The SPS has sps_video_parameter_set_id with the ID of the VPS referred to by the SPS, and sps_seq_parameter_set_id for identifying the SPS. Similarly, the PPS has pps_seq_parameter_set_id with the ID of the SPS referred to by the PPS, and pps_pic_parameter_set_id for identifying the PPS. The slice header contains a PPS ID referenced by the slice and slice_pic_parameter_set_id. Based on the links of the identification information, as shown in the reference relationship indicated by arrows in FIG. 4B, the header information having the same ID is associated from the slice data to PPS, from PPS to SPS, and from SPS to VPS. Thereby, a different header can be referred for each AU.

  In the example shown in FIG. 4B, the input image corresponding to slice 0 and slice 1 is a non-gradation image, and the portion corresponding to slice 2 is a gradation image. As a result, these slices 0 and 1 are configured to refer to SPS0 having a strong_intra_smoothing_enabled_flag value of 0. For slice 2, SPS2 having strong_intra_smoothing_enabled_flag value of 1 is referred to. Thereby, at the time of decoding of slice 2, it is possible to generate a predicted image using smoothing processing and obtain a reproduced image without block boundary distortion.

  The setting for enabling or disabling the intra-smoothing process performed in steps S307 and S309 may be performed after preparing SPS having a corresponding flag value in advance after the encoded slice data is generated.

  As described above, in the first embodiment, before performing the encoding process, it is determined whether or not the input image data is a gradation image based on the variance value, and the intra smoothing process at the time of intra prediction is controlled based on the determination result. To do.

  A second embodiment in which a gradation image is determined using the dynamic range of input image data will be described. FIG. 5 is a block diagram illustrating a schematic configuration of an imaging apparatus that employs the second embodiment. Components having the same functions as those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted.

  A gradation detection unit 502 that replaces the gradation detection unit 102 includes a dynamic range calculation unit 503 that replaces the variance value calculation unit 103. The gradation detection unit 502 detects the gradation image by determining whether the target portion of the input image is a gradation image based on the dynamic range of the frame image calculated by the dynamic range calculation unit 503.

  FIG. 6 shows an operation flowchart of the second embodiment instead of FIG. Note that the processes in S605 to S609 are the same as the processes in S305 to S309, respectively.

  In step S <b> 601, the gradation detection unit 502 determines whether the dynamic range calculation unit 503 has calculated the dynamic range for all CTU blocks for the input frame image data. If the dynamic range has been calculated for all CTU blocks (TRUE in S601), the process proceeds to S605. If a CTU block image whose dynamic range has not been calculated remains (FALSE in S601), the process proceeds to S602.

  In step S602, the dynamic range calculation unit 503 calculates the dynamic range of the CTU block of interest. For example, the dynamic range calculation unit 503 calculates the difference between the maximum pixel value and the minimum pixel value in all the pixels in the CTU block, and sets it as the dynamic range. FIG. 7 shows an example of an image and its histogram for a non-gradation image and a gradation image. FIG. 7A shows an example of a non-gradation image, and FIG. 7B shows an example of a gradation image. FIG. 7C shows an example of a histogram for the pixel values of the non-gradation image shown in FIG. FIG. 7D shows an example of a histogram for the pixel values of the gradation image shown in FIG. 7C and 7D, the horizontal axis indicates the pixel value of the pixel included in the region of interest or the image. As shown in FIG. 7, in the case of the image as shown in FIG. 7A including the subject / object, the histogram shape has a wide pixel value distribution with a plurality of mountains as shown in FIG. 7C. The dynamic range becomes wider. On the other hand, in the case of a flat gradation image such as the sky as shown in FIG. 7 (b), the histogram for the pixel value has no specific peak or one specific pixel value as shown in FIG. 7 (d). A narrow dynamic range that concentrates on the range.

  In step S603 following step S602, the gradation detection unit 502 determines whether the dynamic range calculated in step S602 is less than a threshold value. This threshold value is determined in advance as a value that can be taken in the case of a flat pattern such as a gradation image. For example, when the input image is expressed by 8 bits, the threshold value is set from several percent to several tens of percent of the dynamic range with a maximum of 255 gradations. The gradation detection unit 502 detects an image with a flat color tone with little or no change in luminance.

  When the dynamic range of the block image is less than the threshold (TRUE in S603), the process proceeds to S604. On the other hand, if the dynamic range of the block image is greater than or equal to the threshold (FALSE in S603), the process returns to S601 and the process is repeated for the next CTU block image.

  The processing after S605 is the same as the processing after S305 in FIG.

  As described above, in the second embodiment, it is determined whether the image is a gradation image using the dynamic range of the input image data, and the intra smoothing process at the time of intra prediction is controlled based on the determination result.

  Embodiment 3 will be described in which an input image is divided into a plurality of small block images, and when a high frequency component does not appear in the frequency conversion coefficient of each block image, it is determined whether or not the image is a gradation image based on a DC component value. FIG. 8 shows a schematic block diagram of an imaging apparatus that employs the third embodiment. Components having the same functions as those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted.

  A gradation detection unit 802 that replaces the gradation detection unit 102 includes a frequency conversion unit 806 and a DC component acquisition unit 807 instead of the variance value calculation unit 103. The frequency conversion unit 806 divides the input image into block images having a size smaller than the CTU block, and frequency-converts the pixel values of each block image. The DC component acquisition unit 807 extracts the DC component value within the CTU block size from the frequency conversion result by the frequency conversion unit 806, and calculates the difference between the maximum value and the minimum value of the DC component. This difference corresponds to the dynamic range in the second embodiment. The gradation detection unit 502 detects a gradation image by determining whether the image is a gradation image based on the difference between the maximum value and the minimum value of the DC component calculated by the DC component acquisition unit 807.

  FIG. 9 shows an operation flowchart of the third embodiment shown in FIG. The operation of the third embodiment will be described with reference to FIG.

  In step S901, the gradation detection unit 802 determines whether all CTU blocks of the input frame image data have been determined as gradation images. If it is determined whether all CTU blocks are gradation images (TRUE in S901), the process proceeds to S908, and if an undetermined CTU block image remains (FALSE in S901), the process proceeds to S902.

  In step S902, the gradation detection unit 802 divides the CTU block image of interest into a plurality of smaller block images in order to perform frequency conversion. If the CTU size is 32 × 32 pixels or 64 × 64 pixels and the unit for performing the gradation determination is set, the divided block size is set to a size that can be expressed by a divisor such as 16 × 16 pixels or 8 × 8 pixels. .

  In step S903, the gradation detection unit 802 causes the frequency conversion unit 806 to perform frequency conversion on the block image divided in step S902. As the frequency conversion method, any of known techniques such as Hadamard transform, integer precision discrete cosine transform (DCT), and real number precision DCT may be applied. The present invention is not limited to a specific frequency conversion method.

  In step S <b> 904, the gradation detection unit 802 obtains a frequency conversion coefficient value from the output of the frequency conversion unit 806, and determines whether the coefficient value of the high frequency component is greater than or equal to the writing constant. Having a lot of high-frequency component signals indicates that the image characteristics include edges and textures, and it can be determined that the image is not a gradation image suitable for performing the smoothing process. On the other hand, when the divided small block image in the CTU block has no high frequency component or a small amount, the possibility of a gradation image is high. Therefore, when the high-frequency component does not exist or is small in the divided small block image in the CTU block (FALSE in S904), the process proceeds to S905. If the block image includes a lot of high-frequency components (TRUE in S904), the process returns to S901, and the process is repeated for the next CTU block image.

  In S905, the DC component acquisition unit 807 of the gradation detection unit 802 calculates the DC component in the frequency domain derived in units of divided small blocks, as shown in FIG. 10, among the coefficients of all frequency components in the CTU block obtained in S903. get. FIG. 10A shows the relationship of orthogonal transformation of CTU block images. FIG. 10B shows a distribution example of DC component values of a non-gradation image, and FIG. 10C shows a distribution example of DC component values of a gradation image.

  The DC component is an average value of pixel values in the divided small block and is an index indicating the brightness of the divided small block. Since the DC component is an index indicating the image brightness, in a flat image that does not include gradation, a change in brightness exists even within the CTU size. In this case, as shown in FIG. 10B, the difference between the minimum value and the maximum value of the DC component becomes large. On the other hand, in the case of a gradation image, the brightness in divided small block units hardly changes even in CTU units. As a result, as shown in FIG. 10C, the difference between the minimum value and the maximum value of the DC component is reduced.

  In step S906, the gradation detection unit 802 determines whether the image is a gradation image based on such image characteristics. That is, the gradation detection unit 802 calculates the difference between the maximum value and the minimum value of the DC component, determines whether the difference value is less than a predetermined value, and determines whether the gradation image is a gradation image. If the difference value is less than the predetermined value (TRUE in S906), the process proceeds to S907. On the other hand, if the difference value is equal to or greater than the predetermined value (FALSE in S906), the process returns to S901 and the process is repeated for the next CTU block image.

  In S907, as in S304, the gradation detection unit 802 increments the value of the number of CTU blocks estimated as a gradation image included in the input image. Thereafter, the process returns to S901, and the process is repeated for the next CTU block image.

  The processing of S908 to S912 is the same as the processing of S305 to S309 in FIG.

  As described above, in the third embodiment, the gradation image is determined based on the difference between the maximum value and the minimum value of the DC component, and the intra smoothing process is controlled to be valid or invalid based on the determination result.

  Embodiment 4 will be described in which the gradation smoothing processing is switched between valid / invalid when gradation is continuously detected a predetermined number of times or not. In the fourth embodiment, it is possible to prevent the image quality from being deteriorated by invalidating or enabling the intra smoothing process when an object appears on the screen for a moment, such as flashing or blinking in an operation sequence.

  FIG. 11 is a block diagram illustrating a schematic configuration of an imaging apparatus that employs the fourth embodiment. Components having the same functions as those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted.

  A gradation detection unit 1102 instead of the gradation detection unit 102 includes a continuous detection number counting unit 1107 in addition to the variance value calculation unit 103. The continuous detection number counting unit 1107 calculates the continuous number of gradation detections and the continuous number of non-gradation detections. In the present embodiment, the validity / invalidity of the encoding intra-smoothing process is controlled in accordance with the number of continuous detections of the gradation image or the non-gradation image. Also in the configurations shown in the second and third embodiments, the processing of the continuous detection number counting unit 1107 can be added.

  The CPU 1116 generally controls the imaging apparatus illustrated in FIG. In particular, the CPU 1116 controls the start and end of encoding of the encoding unit 105 in accordance with a user operation by the operation unit 1117. The CPU 1116 may realize the functions of the gradation detection unit 1102 and the intra smoothing process determination unit 104 by a program operating on the CPU 1116.

  FIG. 12 shows an operation flowchart of the fourth embodiment. The process shown in FIG. 12 is repeatedly executed until there is no input image data to be encoded.

  In step S1201, the CPU 1116 determines whether the frame image input to the encoding unit 105 is the final frame to be encoded. This can occur, for example, when the user instructs the CPU 1116 to stop recording using the operation unit 1117. In the case of the final frame (S1201: TRUE), the CPU 1116 ends the flow shown in FIG. If it is not the last frame, that is, if there is an input image to be encoded (FALSE in S1201), the process proceeds to S1202. If the variance value has been calculated for all CTU blocks of the input frame image data (TRUE in S1202), the process proceeds to S1206, and if the CTU block image for which the variance value has not been calculated remains (FALSE in S1202), Transition. Steps S1202 to S1205 are the same as the processes of S301 to S305, respectively, and thus description thereof is omitted.

  In step S1206, the gradation detection unit 1102 determines whether the number of CTU blocks estimated as a gradation image is equal to or greater than a predetermined threshold, as in step S305. When the input frame image includes a lot of gradation images (TRUE in S1205), the process proceeds to S1207. When the input frame image is an image with a large variance value including many patterns such as many objects and textures (FALSE in S1206), the process proceeds to S1214.

In step S <b> 1207, the gradation detection unit 1102 initializes the non-gradation image continuous frame number C _NGR of the continuous detection number counting unit 1107 to zero. In step S1208, the gradation detection unit 1102 performs an increment process of adding 1 to the gradation image continuous frame number C _GRD , and the process proceeds to step S1209.

In step S1209, the gradation detection unit 1102 determines whether the gradation image continuous frame number C _GRD is equal to or greater than a predetermined value. If the image is a gradation image continuously for a predetermined frame period or longer (TRUE in S1209), the process proceeds to step S1210. When the gradation image continuous frame number C _GRD is less than the predetermined frame period (FALSE in S1209), the process proceeds to step S1212.

  In S1210, the intra smoothing process determination unit 104 outputs an intra smoothing process enable signal to the encoding unit 105, as in S306. The encoding unit 105 encodes the input frame image data in accordance with the intra smoothing processing enabling signal. In step S1211, the bit stream generation unit 213 generates a bit stream in the same manner as in step S307. That is, the bit stream generation unit 213 generates a bit stream in which the SPS and the slice header associated with the strong_intra_smoothing_enabled_flag are set to 1 for the encoded picture data. After S1211, the process returns to S1201.

  In S1212, the encoding unit 105 compresses and encodes the frame image data with the intra smoothing process disabled as in S308. In step S1213, the bit stream generation unit 213 generates a bit stream for the encoded picture data by associating the SPS with strong_intra_smoothing_enabled_flag set to 0 and the slice header for the encoded picture data. After S1213, the process returns to S1201.

In step S1214, the gradation detection unit 1102 initializes the gradation image continuous frame number C _GRD of the continuous detection number counting unit 1107 to zero. In step S1215, the gradation detection unit 1102 performs an increment process of adding 1 to the non-gradation image continuous frame number _CNGR , and the process proceeds to step S1216.

In step S1218, the gradation detection unit 1102 determines whether the non-gradation image continuous frame number C _NGR is equal to or greater than a predetermined value. If it is a non-gradation image continuously for a predetermined frame period or longer (TRUE in S1218), the process proceeds to step S1212. When the non-gradation image continuous frame number C _NGR is less than the predetermined frame period (FALSE in S1218), the process proceeds to step S1210.

The threshold value of the gradation image continuous frame number C _GRD (S1209) is set to, for example, the number of frames from the intra-picture coded picture (I picture) to be subjected to random access reproduction to the next I picture. Alternatively, it may be the number of frames corresponding to the so-called GOP length or the number of frames corresponding to the distance between the reference frames of the I picture and the forward reference picture (P picture). The threshold value of the non-gradation image continuous frame number _CNGR (S1216) is also preferably set to any one of these.

  FIG. 13 shows an example of image and smoothing control in the fourth embodiment. FIG. 13A shows an example in which the control based on the continuous number according to the fourth embodiment is not applied for comparison, and FIG. 13B shows an example according to the fourth embodiment. FIG. 13 shows a scene in which a small object moves from the lower left to the upper right of the screen as time elapses as an input image example with gradation as the background. Then, an object containing many edges and textures suddenly appears in the third frame in the middle of the scene. On the lower side of the image example, the valid (ON) / invalid (OFF) state of the intra smoothing process is shown.

  In the first to third embodiments, as illustrated in FIG. 13A, when a scene of a sudden input image is switched, the smoothing process is invalidated only for that frame. On the other hand, when the filter processing is switched using the continuous detection number counting unit 1107 according to the fourth embodiment, the smoothing processing is enabled (ON) for the third frame as shown in FIG. Then, encoding is performed. By such control, it is possible to suppress a sudden change in image quality visually for the user, and maintain or improve the image quality as a moving image.

  Part or all of the functions of the gradation detection units 102, 502, 802, and 1102, the intra smoothing processing determination unit 104, and the encoding unit 105 can be realized by a computer program that runs on a computer. Moreover, although the Example applied to the encoding of a moving image was demonstrated, the function limited to the prediction in a screen is applicable also to the encoding of a still image.

Claims (10)

An image encoding device for encoding an input image,
Coding means for coding the input image in coding block units, the coding means having a mode for linearly interpolating reference pixels at the coding block boundary in the intra prediction mode;
Gradation detecting means for detecting whether or not the gradation image is in a predetermined pixel block unit with respect to the encoded block to be encoded by the encoding means;
The encoding means is an intra-smoothing process determining means for determining whether or not to apply a linear interpolation process of the reference image in the intra prediction mode to the block image to be encoded, the gradation detecting means and the gradation image An image encoding apparatus comprising: an intra-smoothing process determining unit that instructs the encoding unit to validate the linear interpolation when the determined number of pixel blocks exceeds a predetermined number. The gradation detection means includes a dispersion value calculation means for calculating a dispersion value from a pixel value of the image data of the predetermined pixel block unit,
2. The image encoding apparatus according to claim 1, wherein if the variance value is less than a predetermined value, it is determined that the image data in the predetermined pixel block unit includes a gradation image. The gradation detection means includes dynamic range calculation means for calculating a difference between a maximum value and a minimum value of pixel values for the image data in the predetermined pixel block unit,
2. The image encoding apparatus according to claim 1, wherein when the dynamic range is less than a predetermined value, it is determined that the image data of the predetermined pixel block unit includes a gradation image. The gradation detecting means includes
Frequency conversion means for dividing the image data of the predetermined pixel block unit into smaller pixel blocks and performing frequency conversion;
DC component acquisition means for extracting only the DC component from the coefficient value of the frequency conversion by the frequency conversion means,
2. The image data of the predetermined pixel block unit is determined to include a gradation image when the difference between the maximum and minimum DC component values acquired by the DC component acquisition means is less than a predetermined value. The image encoding device described in 1. The gradation detection means has continuous detection number counting means for counting the number of continuous detections of the result of gradation detection,
The intra-smoothing process determining means switches whether to apply the intra-smoothing process to the encoding target block image when the gradation image continues for a predetermined frame period according to the continuous detection frequency counting means. The image encoding device according to any one of 1 to 6.   The intra-smoothing process determining means switches whether to apply the intra-smoothing process to the encoded image when the non-gradation image that is not the gradation image continues for a predetermined frame period according to the continuous detection frequency counting means. The image encoding device according to claim 5. The gradation detection means has continuous detection number counting means for counting the number of continuous detections of the result of gradation detection,
The intra-smoothing process determining means switches whether to apply the intra-smoothing process to the encoding target block image when the non-gradation image that is not the gradation image continues for a predetermined frame period according to the continuous detection number counting means. The image encoding device according to any one of claims 1 to 6.   8. The image encoding device according to claim 1, wherein a size of the predetermined pixel block is a coding tree unit defined by the HEVC standard (ISO / IEC 23008-2). 9. . An image encoding method for encoding an input image, comprising:
A gradation detection step of detecting whether the input image is a gradation image in units of a predetermined pixel block with respect to an encoding block to be encoded;
An intra-smoothing process determining step for enabling linear interpolation of reference pixels at the encoding block boundary in the intra prediction mode when the number of pixel blocks determined as the gradation image by the gradation detection step exceeds a predetermined number; ,
An encoding step for encoding the input image in units of the encoding block, and an encoding step for linearly interpolating reference pixels at the boundary of the encoding block in the intra prediction mode according to the validation of the linear interpolation; An image encoding method characterized by comprising:   A computer program for causing a computer to function as the image encoding device according to claim 1.

Images