TWI533708B - Method and image processing device for image enhancement - Google Patents

Description

Image enhancement processing method and image processing device

The present invention relates to an image processing method and apparatus, and more particularly to an image enhancement processing method and an image processing apparatus.

A pixel is the smallest basic element that makes up an image, and each image is composed of a plurality of pixels of different colors. "Dynamic range" is defined as the ratio of the highest and lowest brightness of all pixels in an image. The so-called high dynamic range scene is a scene indicating that the brightness varies greatly between dark, such as sunrise, sunset, fireworks, and the like.

The human eye can feel the brightness of the scene changes very much, and the so-called visual adaptation ability can make the human eye adapt to the change of the external brightness in a short time, and the range of the change is perceived to be greater. The dynamic range of the human eye is much larger than that of a general image capturing device. In the case of a display, when the ambient light source is uneven, the image displayed on the display may cause overexposure in the bright area and underexposure in the dark area, which is caused by insufficient dynamic range of the display. Therefore, the higher the dynamic range of the image, the more detail and information the image can present in the original scene; in other words, the image is closer to the original scene seen by the human eye.

High dynamic range scenes can reproduce high contrast and information on the display by dynamic range compression. Dynamic range compression reduces the dynamic range of the image while preserving the details of the highlights and shadows of the image. However, most of the prior art needs to implement a real-time processing function or a high-performance processor through a cost-effective hardware acceleration design to perform an algorithm to achieve dynamic range compression and contrast enhancement of the image. Therefore, how to display high dynamic range images on electronic products such as low-cost displays, and at the same time realize the performance of instant image signal processing and preserve the color information of the original images has become one of the problems to be solved.

The invention provides an image enhancement processing method and an image processing device, which utilize dynamic range compression and local contrast enhancement to improve image quality.

The invention provides an image enhancement processing method. The method first receives a plurality of input pixels of an image and obtains input color pixel data for each of the input pixels, wherein the input pixels comprise a first input pixel. Then, according to the input color pixel data of the input pixel, an input luminance pixel value and an input luminance component data of each of the input pixels are obtained. A filter operation is performed based on the input luminance pixel value to obtain a filter result of the first input pixel. Adjusting the gain operation according to the filter result of the first input pixel to obtain an adjusted gain value of the first input pixel. The adjustment update operation is performed according to the input luminance component data of the first input pixel to obtain an adjustment update value of the first input pixel. Then, according to the input color pixel data of the first input pixel and the adjusted gain value of the first input pixel and the adjusted update value The product is used to obtain the output color pixel data of the first input pixel.

In an embodiment of the invention, the step of adjusting the gain operation according to the filter result of the first input pixel to obtain the adjusted gain value of the first input pixel comprises filtering the first input pixel according to the brightness transfer function. The result is converted to a luminance transfer output value of the first input pixel; and an adjustment gain value of the first input pixel is obtained according to a ratio between the luminance transfer output value of the first input pixel and the filter result.

In an embodiment of the invention, the step of performing an adjustment update operation according to the input luminance component data of the first input pixel to obtain an adjustment update value of the first input pixel includes: inputting color pixel data according to the first input pixel, The input luminance component data of the first input pixel and the color saturation parameter obtain an adjustment update value of the first input pixel.

In an embodiment of the invention, the calculation formula for obtaining the output color pixel information of the first input pixel according to the input color pixel data of the first input pixel and the product of the adjusted gain value of the first input pixel and the adjusted update value includes: Equation (1)~(3): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel data of the input pixel of the xth column and the yth row in the image, and g ( x , y ) is the adjustment gain value of the input pixel of the xth column and the yth row in the image, D ^RGB ( x , y ) is the adjustment update value of the input pixel of the yth row of the xth column in the image, L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, and T is the luminance transfer function. w is the color saturation parameter and matches w [0,1], E is a constant vector; where Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and Y _in ( x , y ) is defined as the equation (4) Or the approximation of the result of equation (4) obtained by performing at least one bit shift operation on R _in ( x , y ), G _in ( x , y ), and B _in ( x , y ): Y _in ( x , y )= r ₀ R _in ( x , y )+ g ₀ G _in ( x , y )+ b ₀ B _in ( x , y )+ N Equation (4) where R _in ( x , y ), G _In ( x , y ) and B _in ( x , y ) are the input color pixel elements of the input color pixel data of the input pixel of the xth column and the yth row in the image, and r ₀ , g ₀ and b ₀ are input brightness The ratio of multiple input luminance components of the component data, N is a constant, r ₀ [0,1], g ₀ [0,1] and b ₀ [0,1].

In an embodiment of the invention, after the step of obtaining the adjustment gain of the first input pixel and the adjustment update value of the first input pixel, the method further comprises performing an integer operation on the adjustment gain value of the first input pixel. Obtaining an adjusted gain integer value of the first input pixel; performing an integer operation on the adjusted update value of the first input pixel to obtain an adjusted update integer value of the first input pixel; and inputting color pixel data according to the first input pixel and The product of the adjusted gain integer value of the first input pixel and the adjusted updated integer value obtains the output color pixel approximation of the first input pixel.

In an embodiment of the invention, the input color pixel data of the first input pixel and the adjusted gain integer value of the first input pixel and the adjusted update integer value are used to obtain the output color pixel approximation of the first input pixel. The calculus formula includes equations (5)~(7): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel approximation of the input pixel of the xth column and the yth row in the image, and g _n ( x , y ) is the integer value of the adjustment gain of the input pixel of the yth row of the xth column in the image, g ( x , y ) is the adjusted gain value of the input pixel of the yth row of the xth column in the image. ( x , y ) is the adjustment of the input pixel of the xth row and the yth row of the image to update the integer value, and D ^RGB ( x , y ) is the adjusted update value of the input pixel of the yth row of the xth column in the image, L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, E is a constant vector; T is a luminance transfer function, and for any L _avg ( x , y ) > 0, T must satisfy the following conditions : Operator Represents the largest integer less than or equal to X; w _n = as well as =255- w _n , where w is the color saturation integer parameter and conforms to w [0,1]; Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and is defined as R _in ( x , y ), G _in ( x , y ) And the approximate approximation value of the equation (8) obtained by performing at least one bit shift operation on B _in ( x , y ) is Y _in ( x , y ) = r ₀ R _in ( x , y ) + g ₀ G _in ( x , y ) + b ₀ B _in ( x , y )+16 Equation (8) where R _in ( x , y ), G _in ( x , y ) and B _in ( x , y ) are the xth column in the image The input pixel of the input pixel of the yth row is a plurality of input color pixel elements of the color pixel data, r ₀ , g ₀ and b ₀ are the ratios of the input luminance components of the input luminance component data, r ₀ [0,1], g ₀ [0,1] and b ₀ [0,1].

In an embodiment of the invention, the image enhancement processing method further includes establishing an adjustment gain lookup table according to equation (6) to obtain an adjustment gain integer value of the first input pixel, wherein the input index of the adjustment gain lookup table is The filter result of an input pixel.

The invention provides an image processing device, which comprises: a color capture circuit, a brightness capture circuit, a filter operation circuit and a pixel adjustment circuit. The color capture circuit is configured to receive a plurality of input pixels of the image and obtain input color pixel data of each of the input pixels, wherein the input pixel comprises a first input pixel. The brightness capture circuit is coupled to the color capture circuit for obtaining the input luminance pixel value and the input luminance component data of each of the input pixels according to the input color pixel data of the input pixel. The filter operation circuit is coupled to the brightness extraction circuit for performing a filter operation according to the input brightness pixel value to obtain a filter result of the first input pixel. The pixel adjustment circuit is coupled to the color capture circuit, the brightness extraction circuit, and the filter operation circuit for performing an adjustment gain operation according to the filter result of the first input pixel to obtain an adjustment gain value of the first input pixel, and Adjusting and updating the operation according to the input brightness component data of the first input pixel to obtain an adjustment update value of the first input pixel, and then according to the input color pixel data of the first input pixel And the product of the adjusted gain value of the first input pixel and the adjusted update value, and the output color pixel data of the first input pixel is obtained.

In an embodiment of the invention, the pixel adjustment circuit converts the filter result of the first input pixel into a brightness transfer output value of the first input pixel according to the brightness transfer function, and then shifts the output according to the brightness of the first input pixel. The ratio of the value to the filter result of the first input pixel obtains an adjusted gain value for the first input pixel.

In an embodiment of the invention, the pixel adjustment circuit obtains an adjustment update value of the first input pixel according to the input color pixel data of the first input pixel, the input luminance component data of the first input pixel, and the color saturation parameter.

In an embodiment of the invention, the pixel adjustment circuit obtains the output color pixel information of the first input pixel according to the input color pixel data of the first input pixel and the product of the adjusted gain value of the first input pixel and the adjusted update value. The calculus formula includes equations (1)~(3): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel data of the input pixel of the xth column and the yth row in the image, and g ( x , y ) is the adjustment gain value of the input pixel of the xth column and the yth row in the image, D ^RGB ( x , y ) is the adjustment update value of the input pixel of the yth row of the xth column in the image, L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, and T is the luminance transfer function. w is the color saturation parameter and matches w [0,1], E is a constant vector; where Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and Y _in ( x , y ) is defined as the equation (4) Or the approximation of the result of equation (4) obtained by performing at least one bit shift operation on R _in ( x , y ), G _in ( x , y ), and B _in ( x , y ): Y _in ( x , y )= r ₀ R _in ( x , y )+ g ₀ G _in ( x , y )+ b ₀ B _in ( x , y )+ N Equation (4) where R _in ( x , y ), G _In ( x , y ) and B _in ( x , y ) are the input color pixel elements of the input color pixel data of the input pixel of the xth column and the yth row in the image, and r ₀ , g ₀ and b ₀ are input brightness The ratio of multiple input luminance components of the component data, N is a constant, r ₀ [0,1], g ₀ [0,1] and b ₀ [0,1].

In an embodiment of the invention, the pixel adjustment circuit further performs an integer operation on the adjustment gain value of the first input pixel to obtain an adjustment gain integer value of the first input pixel, and then updates the adjustment of the first input pixel. The value is integerized to obtain an adjusted update integer value of the first input pixel, and the first is obtained according to the input color pixel data of the first input pixel and the product of the adjusted gain integer value of the first input pixel and the adjusted update integer value. The output color pixel approximation of the input pixel.

In an embodiment of the invention, the pixel adjustment circuit obtains the output color of the first input pixel according to the input color pixel data of the first input pixel and the product of the adjustment gain integer value of the first input pixel and the adjusted update integer value. The calculation formula of the pixel approximation information includes equations (5)~(7):

Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel approximation of the input pixel of the xth column and the yth row in the image, and g _n ( x , y ) is the integer value of the adjustment gain of the input pixel of the yth row of the xth column in the image, g ( x , y ) is the adjusted gain value of the input pixel of the yth row of the xth column in the image. ( x , y ) is the adjustment of the input pixel of the xth row and the yth row of the image to update the integer value, and D ^RGB ( x , y ) is the adjusted update value of the input pixel of the yth row of the xth column in the image, L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, E is a constant vector; T is a luminance transfer function, and for any L _avg ( x , y ) > 0, T must satisfy the following conditions : Operator Represents the largest integer less than or equal to X; w _n = as well as =255- w _n , where w is the color saturation integer parameter and conforms to w [0,1]; Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and is defined as R _in ( x , y ), G _in ( x , y ) And the approximate approximation value of the equation (8) obtained by performing at least one bit shift operation on B _in ( x , y ) is Y _in ( x , y ) = r ₀ R _in ( x , y ) + g ₀ G _in ( x , y ) + b ₀ B _in ( x , y )+16 Equation (8) where R _in ( x , y ), G _in ( x , y ) and B _in ( x , y ) are the xth column in the image The input pixel of the input pixel of the yth row is a plurality of input color pixel elements of the color pixel data, r ₀ , g ₀ and b ₀ are the ratios of the input luminance components of the input luminance component data, r ₀ [0,1], g ₀ [0,1] and b ₀ [0,1].

In an embodiment of the invention, the pixel adjustment circuit further includes an adjustment gain lookup table. The adjustment gain lookup table is established according to equation (6), wherein the input index of the adjustment gain lookup table is the filter result of the first input pixel, and the output of the adjustment gain lookup table is the adjusted gain integer value of the first input pixel.

Based on the above, the image enhancement processing method and the image processing apparatus proposed by the present invention, combined with any continuously differentiable luminance transfer function and related calculations for controlling color saturation, can simultaneously improve image brightness, local contrast, and preserve original color information. To avoid distortion of the image. In addition, by adjusting the index operation of the gain lookup table, the integerization, and the bit movement, the processing procedure of the entire dynamic range compression and local contrast enhancement calculation can be greatly accelerated.

The above described features and advantages of the invention will be apparent from the following description.

100, 400‧‧‧ image processing device

110,410‧‧‧Color capture circuit

120, 420‧‧‧ Brightness capture circuit

125‧‧‧ filter

130, 430‧‧‧ filter arithmetic circuit

140, 440‧‧‧ pixel adjustment circuit

442‧‧‧Adjustment gain lookup table

444‧‧‧Adjustment update circuit

446, 447a~447c, 448a~448c‧‧‧ multiplier

447d~447f, 449a~449c‧‧‧Adder

445‧‧‧Non-volatile memory

450‧‧‧Color output circuit

A, A'‧‧‧ Input image, output image

A _C ‧‧‧first input pixel

S201~S211‧‧‧ Flow of dynamic range compression and local contrast enhancement method

1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention.

2 is a flow chart of an image enhancement processing method according to a first embodiment of the present invention.

3 is a schematic diagram of an image according to a first embodiment of the present invention.

4 is a block diagram of an image processing apparatus according to a second embodiment of the present invention; schematic diagram.

The components of the present invention will be described in detail in the following description in conjunction with the accompanying drawings. These examples are only a part of the invention and do not disclose all of the embodiments of the invention. Rather, these embodiments are merely examples of devices and methods within the scope of the patent application of the present invention.

[First Embodiment]

1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention, but is for convenience of description and is not intended to limit the present invention. First, all components and configuration relationships of the image processing apparatus will be described first in FIG. 1. The detailed functions will be disclosed in conjunction with FIG.

Referring to FIG. 1, the image processing apparatus 100 of the present embodiment receives an image A and outputs an image A'. The image processing apparatus 100 performs dynamic range compression and local contrast enhancement on the image A to enhance the detail information of the image A, and the output result is the output image A'. The image processing apparatus 100 can be an electronic device such as a personal computer, a notebook computer, a digital camera, a digital camera, a network camera, a smart phone, a tablet computer, a scanner, a driving recorder, a projector, a car audio and video system, and the like. Not limited to this.

The image processing device 100 includes a color capture circuit 110 and a brightness capture circuit. 120, the filter operation circuit 130 and the pixel adjustment circuit 140, wherein the color capture circuit 110, the brightness capture circuit 120, the filter operation circuit 130, and the pixel adjustment circuit 140 can be implemented as one or more integrated circuits, each of which The integrated circuit includes a plurality of logic gates.

The color capture circuit 110 receives a plurality of input pixels in the image A, and each of the input pixels includes at least one color pixel value.

The brightness capture circuit 120 is coupled to the color capture circuit 110 to perform color space conversion on the color pixel values of each input pixel to obtain luminance pixel values of each input pixel in different color gamuts.

The filter operation circuit 130 is coupled to the brightness extraction circuit 120 for performing a filter operation on the luminance pixel values. This filter operation is spatial low-pass filtering, but the invention is not limited thereto.

The pixel adjustment circuit 140 is coupled to the color capture circuit 110, the brightness capture circuit 120, and the filter operation circuit 130. The color pixel values in the image A can be adjusted to generate the image A'. In the embodiment, the pixel adjustment circuit 140 is an integrated circuit composed of a plurality of adders and multipliers. In addition, the pixel adjustment circuit 140 may further include a non-volatile memory for storing a plurality of parameters.

FIG. 2 is a flowchart of an image enhancement processing method according to a first embodiment of the present invention. Referring to FIG. 2, the method of the present embodiment is applied to the image processing apparatus 100 of FIG. 1. Hereinafter, the detailed steps of the image enhancement processing method of the present invention will be described with reference to various elements in the image processing apparatus 100.

First, the color capture circuit 110 receives a plurality of input pixels in the image and obtains input color pixel data of the respective input pixels, wherein the input pixels include the first input pixels (step S201). In detail, please refer to the schematic diagram of the image A shown in FIG. 3 at the same time. The image A includes a plurality of input pixels, and the input pixels are arranged in a matrix in a row and a row. For example, the input pixel A ₁₁ is the input pixel of the first row of the first row in the image A, the input pixel A ₁₂ is the input pixel of the second row of the first row of the image A, and the input pixel A ₂₁ is the second of the image A. The input pixels in row 1 of the column, and so on. Each input pixel will include at least one pixel value. In this embodiment, the image A is a color image in the RGB color gamut, where each pixel includes a red pixel value (R pixel value), a green pixel value (G pixel value), and a blue pixel value (B pixel value). ), which is the aforementioned "input color pixel data". However, in other embodiments, the input color pixel data may also include Y pixel values, Cb pixel values, and Cr pixel values in the YCbCr color gamut, and the present invention is not limited thereto. In addition, for convenience of explanation, only the step flow of the image enhancement processing will be described for the first input pixel A _{C in the} image A.

Next, the brightness capture circuit 120 obtains the input luminance pixel value and the input luminance component data of each input pixel according to the input color pixel data of each input pixel (step S203). In detail, the brightness capture circuit 120 obtains the luminance pixel values of each input pixel in different color gamuts according to the input color pixel data of each input pixel. In this embodiment, the aforementioned input luminance pixel value is a V pixel value in the HSV (hue, saturation, value) color gamut, and the aforementioned input luminance component data is a Y pixel value in the YCbCr color gamut. In other embodiments, however, the luma pixel value may also be a pixel value that converts RGB pixel values to other criteria. In the following description, the input color pixel data can be expressed in the following mathematical formula: among them ( x , y ) is the input color pixel data of the input pixel of the xth column and the yth row in the image, which is composed of R _in ( x , y ), G _in ( x , y ), and B _in ( x , y ) , RGB pixel value, defined here as "input color pixel element". In addition, in the following description, L _in ( x , y ) can be used to represent the input luminance pixel value of the input pixel of the xth row and the yth row in the image A, and Y _in ( x , y ) can be used to represent the image A The input luminance component data of the input pixel of the xth row of the xth column.

The filter operation circuit 130 performs a filter operation on the input luminance pixel values of the input pixels to obtain a filter result of the first input pixel A _C (step S205). In detail, the filter operation circuit 120 performs filter calculation based on the filter 125. In this embodiment, the filter 125 is centered on the first input pixel A _C and utilizes a spatial low-pass filter kernel function to input luminance pixels of all input pixels in the range of the filter 125. The value is subjected to a convolution operation. Here, the filter result of the first input pixel A _C can be obtained by the following equation: Where x and y are positive integers, Represents a two-dimensional convolution operation, F _L ( x , y ) represents the spatial low-pass filter kernel function, and all x and y are satisfied conditions of. In this embodiment, the filter 125 may be a Gaussian blur filter, a mean filter, an edge-preserving filter or other kinds of low-pass filters, and the present invention Not limited to this.

The image enhancement processing method of the present invention is based on the concept of dynamic range compression and local contrast enhancement, and uses input color pixel data plus its pixel adjustment data to generate output color pixel data. In this embodiment, equation (1) is used to generate output color pixel data: among them ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, (X, y) of the image x-th column of an output color pixel data input pixel y-th row, g (x, y) of the image x-th column to adjust the gain value of the input pixel y-th row (adjustment gain), D ^RGB ( x , y ) is the adjustment innovation of the input pixels of the yth row of the xth column in the image. In the embodiment, the pixel adjustment data is g ( x , y ) D ^RGB ( x , y ).

Specifically, the pixel circuit 140 according to the adjustment result of the first input pixel filter A _C, the gain adjustment operation, in order to obtain an adjusted gain value of the first input pixel A _C (step S207). The pixel adjustment circuit 140 converts the filter result of the first input pixel A _C into a luminance transfer output value of the first input pixel according to a continuously differentiable luminance transfer function, and then shifts the output value according to the brightness of the first input pixel. the ratio between the first input pixel filter a _C the results obtained first input pixel value adjustment gain of a _C. In this embodiment, the pixel adjustment circuit 140 may obtain the adjusted gain value of the first input pixel A _C according to equation (2): Where L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, T is the luminance transfer function, and g ( x , y ) is the input pixel of the yth row of the xth column in the image. Adjust the gain value.

For example, in order to increase the input luminance pixel value of the low-intensity input pixel in the image A while preserving the contrast of the high-brightness input pixel, the above-described luminance transfer function T may be Equation (2.1): Where m ( x , y )= L _avg ( x , y ) s + m _min

s is the scaling factor and s >0, m _min is a constant and m _min >0, The maximum value for all input luminance pixel values. Using the transfer function of equation (2.1) Under the architecture, equation (2) can be rewritten as equation (2.2): Where ε is a very small positive value to avoid any operation divided by zero.

And adjusting the pixel circuit 140 according to the input luminance component of the pixel data of the first input A _C, the update operation is adjusted to obtain an adjusted input pixel value of the first update of the A _C (step S209). In detail, the pixel adjustment circuit 140 obtains an adjustment update value of the first input pixel A _C according to the input color pixel data, the input luminance component data, and the color saturation parameter of the first input pixel A _C . In this embodiment, the pixel adjustment circuit 140 may obtain an adjustment update value of the first input pixel A _C according to equation (3): Where w is the color saturation parameter and conforms to w [0,1], E is a constant vector [1 1 1] ^T , D ^RGB ( x , y ) is the adjusted update value of the input pixel of the yth row of the xth column in the image, Y _in ( x , y ) is the image Input luminance component data of the input pixel of the xth column and the yth row.

In one embodiment, the so-called "input luminance component data" is NTSC luminance component information of RGB input pixels, and Y _in ( x , y ) is defined as equation (4): Y _in ( x , y ) = r ₀ R _In ( x , y )+ g ₀ G _in ( x , y )+ b ₀ B _in ( x , y )+ N Equation (4) r ₀ , g ₀ and b ₀ are the multiple input luminances of the input luminance component data Composition ratio, r ₀ [0,1], g ₀ [0,1] and b ₀ [0,1], N is a constant. For example, in one embodiment, the parameters in equation (4) may be r ₀ =0.257, g ₀ =0.5044, b ₀ =0.0977, and N =16, and equation (4) may be rewritten as equation (4.1). ): Y _in ( x , y )=0.257 R _in ( x , y )+0.5044 G _in ( x , y )+0.0977 B _in ( x , y )+16 Equation (4.1)

In another embodiment, in order to speed up the calculation of the speed of equation (4), the brightness capture circuit 120 may further include a plurality of bit registers, which may be respectively for R _in ( x , y ), G _In ( x , y ) and B _in ( x , y ) perform an accumulation operation after performing at least one bit-shift operation, and obtain a result approximation value of equation (4). Taking equation (4.1) as an example, it can be approximated by one of equation (4.2) or equation (4.3):

Where << and >> respectively represent the operator of bit-shift left and bit-shift right. The input luminance component data Y _in ( x , y ) obtained by using equations (4.2) and (4.3) is similar to the result of equation (4.1), because it uses bit shifting and accumulating operations instead of multiplication, so Increase the speed of inputting the luminance component data Y _in ( x , y ) and reduce the cost of the hardware circuit.

It should be noted that the order of step S207 and step S209 in this embodiment can be reversed, and the present invention is not limited thereto.

Adjusting the pixel circuit 140 after obtaining the updated adjustment value and an adjustment gain value, i.e., the product of the input color adjustment data of the first pixel and the input pixel A _C A _C of the first input pixel value and the gain adjustment value is updated, to obtain a first input The color pixel data of the pixel A _C is output (step S211). That is, the pixel adjustment circuit 140 generates the output color pixel data using the aforementioned equation (1): The output color pixel data can be expressed in the following mathematical formula: Thereby, the output color pixel data of the image A can constitute the image A', which can achieve the effect of dynamic range compression and preserve the detail information in the image A.

[Second embodiment]

The second embodiment is similar to the first embodiment, and only the differences will be described herein. The approach taken in the first embodiment involves a variety of non-linear operations, which are more suitable for image processing devices with high computational efficiency. The method employed in the second embodiment is more suitable for an image processing apparatus of a low cost hardware design, in addition to speeding up the processing of color images.

4 is a block diagram of an image processing apparatus according to a second embodiment of the present invention. Referring to FIG. 5 , the image processing apparatus 400 includes a color capture circuit 410 , a brightness capture circuit 420 , a filter operation circuit 430 , a pixel adjustment circuit 440 , and a color output circuit 450 . The color capture circuit 410 and the filter operation circuit 430 . The color capture circuit 110 and the filter operation circuit 130 in the first embodiment have the same functions, and will not be described again. Similarly, the brightness capture circuit 420 is coupled to the color capture circuit 410; the filter operation circuit 430 is coupled to the brightness capture circuit 420; the pixel adjustment circuit 440 is coupled to the color capture circuit 410, the brightness capture circuit 420, and Filter operation circuit 430. The color output circuit 450 is coupled to the pixel adjustment circuit 440 for outputting an image.

In this embodiment, the brightness capture circuit 420 includes a plurality of 16-bit registers and an adder (not shown). The pixel adjustment circuit 440 includes an adjustment gain lookup table 442, an adjustment update circuit 444, multipliers 448a to 448c, and adders 449a to 449c. The adjustment and update circuit 444 further includes an integrated circuit composed of a non-volatile memory 445, a multiplier 446, multipliers 447a to 447c, and adders 447d to 447f.

In the present embodiment, image A includes a plurality of input pixels and is in the RGB color gamut, wherein the input pixels include a first input pixel A _C . The color capture circuit 410 receives the plurality of input pixels in the image A, and obtains the input color pixel data of each input pixel, that is, the R pixel value, the G pixel value, and the B pixel value, and transmits the same to the brightness capture circuit 420. Next, the brightness capture circuit 420 converts the input color pixel data into V pixel values of the HSV color gamut, that is, input luminance pixel values. Here, the input color pixel data is still expressed in the following mathematical formula: among them ( x , y ) is the input color pixel data of the input pixel of the xth row and the yth row in the image, which are the input color pixel elements R _in ( x , y ), G _in ( x , y ), and B _in ( x , y ) is composed. In addition, L _in ( x , y ) can be used to represent the input luminance pixel value of the input pixel of the xth column and the yth row in the image A, which can be expressed by the following mathematical expression: L _in ( x , y )=max( R _in ( x , y ), G _in ( x , y ), B _in ( x , y )) wherein the operator max( x , y , z ) represents the maximum value among x, y, and z. In addition, the brightness capture circuit 420 also converts the input color pixel data into Y pixel values in the YCbCr color gamut, that is, inputs luminance component data. It should be noted that since the present embodiment is applicable to the image processing apparatus 400 of a low-cost hardware design, the input luminance component data Y can be approximated by using one of the equations (4.2) or (4.3) in the first embodiment. _In ( x , y ) greatly increases the speed at which the input luminance component data Y _in ( x , y ) is obtained and reduces the hardware cost of the luminance capture circuit 420. For related description, please refer to the first embodiment, and details are not described herein again.

The luminance capture circuit 420 transmits the input luminance pixel value to the filter operation circuit 430, and the filter operation circuit 430 calculates the filter result of the first input pixel A _C and transmits the filter result to the adjustment gain of the pixel adjustment circuit 440. Look up table 442. For the manner in which the filter operation circuit 430 generates the filter result, refer to the related description of the first embodiment, and details are not described herein again.

The concept of dynamic range compression and local contrast enhancement in this embodiment also produces output color pixel data by inputting color pixel data plus its pixel adjustment data. In this embodiment, equation (5) is used to generate output color pixel data: Where x and y are positive integers, ( x , y ) is the output color pixel approximation of the input pixel of the xth column and the yth row in the image, and g _n ( x , y ) is the integer value of the adjustment gain of the input pixel of the yth row of the xth column in the image, ( x , y ) is the adjustment of the input pixel of the xth column and the yth row in the image to update the integer value. In this embodiment, the pixel adjustment data is . In the image processing apparatus 400 of the present embodiment, the main function of the pixel adjustment circuit 440 is to calculate the pixel adjustment data. Similarly, in the present embodiment, only the input luminance pixel value of the first input pixel A _C in the image A will be described.

In this embodiment, the adjustment gain lookup table 442 is a 1-dimensional adjustment gain lookup table (1D adjustment gain LUT), which is established according to the filter result and the adjustment gain integer value. In detail, the adjustment gain integer value is defined as equation (6): For the filter result of any L _avg ( x , y )>0, the following conditions must be met: In addition, the operator Represents the largest integer less than or equal to X. Brightness transfer function The image A can be pre-calculated before any processing is performed, and the adjusted gain lookup table 442 is previously established using Equation (6). The input index of the adjustment gain lookup table 442 is the filter result L _avg ( x , y ) obtained from the filter operation circuit 430, and the output of the adjustment gain lookup table 442 is the adjustment gain integer value g _n ( x , y ).

On the other hand, the adjustment and update circuit 444 of the pixel adjustment circuit 440 can obtain the input color pixel elements and the input brightness component data of the input color pixel data of the first input pixel A _C from the color capture circuit 410 and the brightness capture circuit 420, respectively. The adjustment and update integer values are calculated by the multiplier 446, the multipliers 447a to 447c, the adders 447d to 447f, and the 16-bit register. In particular, the adjustment update integer value is defined as equation (7): Where w _n = as well as =255- w _n are integer parameters that control the color saturation of image A, respectively. w is the color saturation parameter, which is consistent with w [0, 1] and may be stored in advance in the non-volatile memory 445.

It should be noted that in another embodiment, the input luminance component data Y _in ( x , y ) can be directly operated by the adjustment update circuit 444. In this architecture, the pixel adjustment circuit 420 does not need to be coupled to the brightness capture circuit 420, and the adjustment update circuit 444 in the pixel adjustment circuit 440 further includes a plurality of bit registers.

After the adjustment gain lookup table 442 and the adjustment update circuit 444 of the pixel adjustment circuit 440 respectively generate the adjustment gain integer value and adjust the update integer value, the multipliers 448a to 448c and the adders 449a to 449c can be calculated according to the above equation (5). Output color pixel approximation The output color pixel approximation data can be expressed in the following mathematical formula: Then, the output color pixel approximation data of the image A can be transmitted from the pixel adjustment circuit 440 to the color output circuit 450 to generate an image A', which can achieve the effect of dynamic range compression and preserve the detailed information in the image A.

In this embodiment, due to the establishment of the adjustment gain lookup table 442, the operations required for the image enhancement processing method flow can be simplified to the operation of inputting the luminance pixel value, the filter operation, the index operation of adjusting the gain integer value, and the adjustment and update. Numerical calculations and pixel adjustment data operations and outputs greatly speed up the overall dynamic range compression and local contrast enhancement processing.

In summary, the image enhancement processing method and image processing apparatus provided by the present invention combines any continuously differentiable luminance transfer function and related calculations for controlling color saturation, thereby simultaneously improving image brightness, local contrast, and retaining original color. Information to avoid image distortion. In addition, by adjusting the index operation of the gain lookup table, the integerization, and the bit shifting operation, the processing procedure of the overall dynamic range compression and the local contrast enhancement calculation can be greatly accelerated. The image enhancement processing method and the image processing device can not only achieve the effect of color image enhancement, but also achieve the performance of instant image or video processing, and can be applied to low-cost consumer electronic products, and the invention is enhanced. Applicability in inter-applications.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

S201~S211‧‧‧ Flow of dynamic range compression and local contrast enhancement method

Claims (14)

An image enhancement processing method includes: receiving a plurality of input pixels of an image, and obtaining an input color pixel data of each of the input pixels, wherein the input pixel includes a first input pixel; according to the input pixel Inputting color pixel data, obtaining an input luminance pixel value of each input pixel and an input luminance component data; performing a filter operation according to the input luminance pixel value to obtain a filter of the first input pixel As a result, an adjustment gain operation is performed according to the filter result of the first input pixel to obtain an adjusted gain value of the first input pixel; and an adjustment update is performed according to the input luminance component data of the first input pixel. Computing to obtain an adjusted update value of the first input pixel; and the input color pixel data of the first input pixel and the adjusted gain value of the first input pixel and the adjusted update value of the first input pixel The product is used to obtain an output color pixel data of the first input pixel. The image enhancement processing method of claim 1, according to the filter result of the first input pixel, performing the adjustment gain operation to obtain the adjusted gain value of the first input pixel comprises: a brightness transfer function, the filter result of converting the first input pixel is a brightness transfer output value of the first input pixel; and the brightness transfer output value according to the first input pixel and the first input pixel The ratio of the filter results is obtained for the adjusted gain value of the first input pixel. The image enhancement processing method of claim 2, wherein the step of performing the adjustment update operation to obtain the adjustment update value of the first input pixel according to the input luminance component data of the first input pixel comprises: And obtaining, according to the input color pixel data of the first input pixel, the input luminance component data of the first input pixel, and a color saturation parameter, the adjusted update value of the first input pixel. The image enhancement processing method of claim 3, wherein the input color pixel data of the first input pixel and the adjusted gain value of the first input pixel and the adjusted update value of the first input pixel are The calculation formula for obtaining the output color pixel information of the first input pixel includes the equations (1) to (3): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel data of the input pixel of the yth row of the xth column in the image, and g ( x , y ) is the adjusted gain value of the input pixel of the yth row of the xth column in the image, D ^RGB ( x , y ) is the adjustment update value of the input pixel of the yth row of the xth column in the image, and L _avg ( x , y ) is the filter of the input pixel of the yth row of the xth column in the image. As a result, T is the luminance transfer function, w is the color saturation parameter and conforms to w [0,1], E is a constant vector, operator Represents the largest integer less than or equal to X, where Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and Y _in ( x , y ) is defined as the equation (4) Or the approximation of the result of equation (4) obtained by performing at least one bit shift operation on R _in ( x , y ), G _in ( x , y ), and B _in ( x , y ): Y _in ( x , y )= r ₀ R _in ( x , y )+ g ₀ G _in ( x , y )+ b ₀ B _in ( x , y )+ N Equation (4) where R _in ( x , y ), G _In ( x , y ) and B _in ( x , y ) are a plurality of input color pixel elements of the input color pixel data of the input pixel of the yth row of the xth column in the image, r ₀ , g ₀ and b ₀ are The ratio of the input luminance components of the input luminance component data, N is a constant, r ₀ [0,], g ₀ [0,] and b ₀ [0,1]. The image enhancement processing method of claim 3, wherein after the step of obtaining the adjustment gain of the first input pixel and the adjustment update value of the first input pixel, the method further comprises: the first input Performing an integer operation on the adjusted gain value of the pixel to obtain an adjusted gain integer value of the first input pixel; performing the integer operation on the adjusted update value of the first input pixel to obtain the first input pixel And adjusting the integer value; and obtaining the first according to the input color pixel data of the first input pixel and the product of the adjusted gain integer value of the first input pixel and the adjusted update integer value of the first input pixel An output color pixel approximation of the input pixel. The image enhancement processing method of claim 5, wherein the input color pixel data of the first input pixel and the adjustment gain integer value of the first input pixel and the adjustment update of the first input pixel are The product of the integer value, the calculation formula for obtaining the approximate color pixel of the first input pixel includes equations (5)~(7): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel approximation of the input pixel of the yth row of the xth column in the image, and g _n ( x , y ) is the adjustment gain of the input pixel of the yth row of the xth column in the image. The integer value, g ( x , y ), is the adjusted gain value of the input pixel of the yth row of the xth column in the image, ( x , y ) is an integer value of the adjustment of the input pixel of the yth row of the xth column in the image, and D ^RGB ( x , y ) is the adjusted update value of the input pixel of the yth row of the xth column in the image. , L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, E is a constant vector, where T is the luminance transfer function, and for any L _avg ( x , y )>0, T must meet the following conditions: Operator Represents the largest integer less than or equal to X, where as well as , where w is the integer parameter of the color saturation and conforms to w [0,], where Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and is defined as R _in ( x , y ), G _in ( x , y ) and B _in ( x , y ) The result approximation of the equation (8) obtained by performing at least one bit shift operation, respectively, Y _in ( x , y ) = r ₀ R _in ( x , y ) + g ₀ G _In ( x , y )+ b ₀ B _in ( x , y )+16 Equation (8) where R _in ( x , y ), G _in ( x , y ) and B _in ( x , y ) are in the image a plurality of input color pixel elements of the input color pixel data of the input pixel of the xth row and the yth row, r ₀ , g ₀ and b _{0 are} a plurality of input luminance component ratios of the input luminance component data, r ₀ [0,], g ₀ [0,1] and b ₀ [0,1]. The image enhancement processing method of claim 6, further comprising: establishing an adjustment gain lookup table according to equation (6) to obtain the adjusted gain integer value of the first input pixel, wherein the adjustment gain lookup table The input index is the filter result of the first input pixel. An image processing device includes: a color capture circuit that receives a plurality of input pixels of an image, and obtains an input color pixel data of each of the input pixels, wherein the input pixel includes a first input pixel; The capture circuit is coupled to the color capture circuit, and obtains an input luminance pixel value and an input luminance component data of each of the input pixels according to the input color pixel data of the input pixel; and a filter operation circuit And coupled to the brightness extraction circuit, performing a filter operation according to the input brightness pixel value to obtain a filter result of the first input pixel; and a pixel adjustment circuit coupled to the color capture circuit The brightness capture circuit And the filter operation circuit, performing an adjustment gain operation according to the filter result of the first input pixel to obtain an adjusted gain value of the first input pixel, and according to the input brightness component of the first input pixel Data, performing an adjustment update operation to obtain an adjustment update value of the first input pixel, and then according to the input color pixel data of the first input pixel and the adjustment gain value of the first input pixel and the first input The product of the adjusted update value of the pixel obtains an output color pixel data of the first input pixel. The image processing device of claim 8, wherein the pixel adjustment circuit converts the filter result of the first input pixel to a brightness transfer output value of the first input pixel according to a brightness transfer function, and then And obtaining the adjusted gain value of the first input pixel according to a ratio of the brightness transfer output value of the first input pixel to the filter result of the first input pixel. The image processing device of claim 9, wherein the pixel adjustment circuit is configured according to the input color pixel data of the first input pixel, the input luminance component data of the first input pixel, and a color saturation parameter. Obtaining the adjustment update value of the first input pixel. The image processing device of claim 10, wherein the pixel adjustment circuit is configured according to the input color pixel data of the first input pixel and the adjustment gain value of the first input pixel and the adjustment of the first input pixel The calculation formula of the output color pixel information of the first input pixel including the product of the updated value includes equations (1) to (3): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel data of the input pixel of the yth row of the xth column in the image, and g ( x , y ) is the adjusted gain value of the input pixel of the yth row of the xth column in the image, D ^RGB ( x , y ) is the adjustment update value of the input pixel of the yth row of the xth column in the image, and L _avg ( x , y ) is the filter of the input pixel of the yth row of the xth column in the image. As a result, T is the luminance transfer function, w is the color saturation parameter and conforms to w [0,1], E is a constant vector, operator Represents the largest integer less than or equal to X, where Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and Y _in ( x , y ) is defined as the equation (4) Or the approximation of the result of equation (4) obtained by performing at least one bit shift operation on R _in ( x , y ), G _in ( x , y ), and B _in ( x , y ): Y _in ( x , y )= r ₀ R _in ( x , y )+ g ₀ G _in ( x , y )+ b ₀ B _in ( x , y )+ N Equation (4) where R _in ( x , y ), G _In ( x , y ) and B _in ( x , y ) are a plurality of input color pixel elements of the input color pixel data of the input pixel of the yth row of the xth column in the image, r ₀ , g ₀ and b ₀ are The ratio of the input luminance components of the input luminance component data, N is a constant, r ₀ [0,1], g ₀ [0,] and b ₀ [0,1]. The image processing device of claim 10, wherein the image The adjusting circuit further performs an integer operation on the adjusted gain value of the first input pixel to obtain an adjusted gain integer value of the first input pixel, and then performs the adjusted update value of the first input pixel to the integer. An operation to obtain an adjusted update integer value of the first input pixel, and according to the input color pixel data of the first input pixel and the adjusted gain integer value of the first input pixel and the adjustment of the first input pixel Updating the product of the integer values to obtain an output color pixel approximation of the first input pixel. The image processing device of claim 12, wherein the pixel adjustment circuit is based on the input color pixel data of the first input pixel and the adjusted gain integer value of the first input pixel and the first input pixel The adjustment updates the product of the integer values, and the calculation formula for obtaining the output color pixel approximation information of the first input pixel includes equations (5)~(7): Where x and y are positive integers, ( x , y ) is the input color pixel data of the input pixel of the yth row of the xth column in the image, ( x , y ) is the output color pixel approximation of the input pixel of the yth row of the xth column in the image, and g _n ( x , y ) is the adjustment gain of the input pixel of the yth row of the xth column in the image. The integer value, g ( x , y ), is the adjusted gain value of the input pixel of the yth row of the xth column in the image, ( x , y ) is the integer value of the adjustment of the input pixel of the yth row of the xth column in the image, ( x , y ) is the adjusted update value of the input pixel of the yth row of the xth column in the image, and L _avg ( x , y ) is the filter result of the input pixel of the yth row of the xth column in the image, E is a constant vector, where T is the luminance transfer function, and for any L _avg ( x , y ) > 0, T must satisfy the following conditions: Operator Represents the largest integer less than or equal to X, where as well as Where w is the color saturation parameter and conforms to w [0,], where Y _in ( x , y ) is the input luminance component data of the input pixel of the yth row of the xth column in the image, and is defined as R _in ( x , y ), G _in ( x , y ) and B _in ( x , y ) result of approximating the result of equation (8) obtained by performing at least one bit shift operation: Y _in ( x , y )= r ₀ R _in ( x , y )+ g ₀ G _qq ( x , y )+ b ₀ B _in ( x , y )+16 Equation (8) where R _in ( x , y ), G _in ( x , y ) and B _in ( x , y ) are a plurality of input color pixel elements of the input color pixel data of the input pixel of the xth column and the yth row in the image, r ₀ , g ₀ and b _{0 are} ratios of the plurality of input luminance components of the input luminance component data, r ₀ [0,1], g ₀ [0,1] and b ₀ [0,1]. The image processing device of claim 13, wherein the pixel adjustment circuit further comprises: an adjustment gain lookup table, wherein the adjustment gain lookup table is established according to equation (6), wherein the adjustment gain lookup table is The input index is the filter result of the first input pixel, and the output of the adjusted gain lookup table is the adjusted gain integer value of the first input pixel.