CN111311502A - Method for processing foggy day image by using bidirectional weighted fusion - Google Patents

Abstract

A method for processing foggy images using bidirectional weighted fusion, the method includes the following steps: S1. On the one hand, the foggy image I to be processed is processed by a dehazing algorithm to obtain an output image J1, and on the other hand, the foggy image to be processed is processed. Invert the sky image I, and then use the dehazing algorithm for processing, and then perform the inversion operation after processing to obtain the output image J2; S2, weighted average of the output image J1 and the output image J2 to obtain the final output image J. The advantages of the invention are: for the original foggy image, combined with image inversion, the existing image dehazing technology is used bidirectionally and weighted fusion is performed. The image processed by the technology of the present invention has a better image sharpening effect, can better improve the color of bright areas such as the sky, and can retain more detailed information and edge features. The invented technology also has the characteristics of real-time processing and the method is simple, and can be used in the defogging processing of general electronic consumer products.

Description

A method for processing foggy images using bidirectional weighted fusion

technical field

The invention relates to the field of computer application and information processing, in particular to a method for processing foggy images using bidirectional weighted fusion.

Background technique

Many important night vision or low-light scenes, such as military bases, security centers and traffic fortresses, mainly use infrared images to achieve comprehensive monitoring. Since the actual image acquisition will be affected by the external environment such as insufficient lighting, uneven lighting, fog, haze, rain and other bad weather, which seriously affects the visual quality of the image, it is necessary to dehaze the image.

In the prior art, the dehazing method basically directly adopts the adaptive histogram enhancement algorithm, Retinex theory, and dark channel prior dehazing algorithm, but the above methods are used to achieve image dehazing through enhancement, but will remove scene details from the image, resulting in image distortion.

In order to reduce the degree of image distortion, the prior art discloses a wavelet domain Retinex image dehazing method (CN201510694867) and an improved Retinex and Welsh near-infrared image enhancement and colorization algorithm (CN201711005353), but these two methods are more complex, so how to use a simple way to solve image distortion is a technical problem that needs to be solved urgently.

SUMMARY OF THE INVENTION

In order to overcome the above-mentioned deficiencies in the prior art, the present invention provides a method for processing foggy images using bidirectional weighted fusion.

To achieve the above object, the present invention adopts the following technical solutions:

A method for processing foggy images using bidirectional weighted fusion, including the following steps:

S1, on the one hand, the foggy image I to be processed is processed using a dehazing algorithm to obtain an output image J1, on the other hand, the foggy image I to be processed is reversed, and then processed using a dehazing algorithm, processing After completion, the inversion operation is performed to obtain the output image J2;

S2. Perform a weighted average on the output image J1 and the output image J2 to obtain the final output image J.

The advantages of the present invention are:

(1) For the original foggy image, combined with image inversion, the existing image dehazing technology is used bidirectionally and weighted fusion is performed. The image processed by the technology of the present invention has better image sharpening effect, can better improve the color of bright areas such as the sky, and can retain more detailed information and edge features. The invented technology also has the characteristics of real-time processing and the method is simple, and can be used in the defogging processing of general electronic consumer products.

(2) The present invention proposes the idea of two-way weighted fusion of image sharpening algorithms in foggy days. Combined with image inversion, the dehazing algorithm is used in both forward and reverse directions, and the weighted average of two processes can effectively suppress the amplification of noise; Two-time weighted fusion can improve the lack of traditional single-time enhancement for contrast enhancement; two-time weighted fusion can better restore scene details covered by fog; at the same time, the technical solution has a certain real-time performance and can be applied to general electronic consumption product demand.

Description of drawings

The three pictures from top to bottom in FIG. 1 are respectively the effect comparison pictures of the original foggy image, the image using the traditional MSR algorithm, and the image obtained by the present invention based on the MSR algorithm.

The three pictures from top to bottom in FIG. 2 are respectively the effect comparison diagrams of the original foggy image, the image using the traditional dark channel algorithm, and the image obtained by the present invention based on the dark channel algorithm.

Detailed ways

Example 1

A method for processing foggy images using bidirectional weighted fusion, including the following steps:

S1. On the one hand, the fog image I to be processed is processed using the multi-scale Retinex algorithm to obtain the output image J1; on the other hand, the fog image I to be processed is inverted, and then processed using the multi-scale Retinex algorithm , and then perform the inversion operation after processing to obtain the output image J2;

S2. Perform a weighted average on the output image J1 and the output image J2 to obtain the final output image J. Specifically, the dehazing algorithm is the MSR algorithm in the multi-scale Retinex algorithm. In this example, the specific steps of the MSR algorithm are:

SA1: Take the original image I or the image after the inversion operation as the image to be processed; obtain the grayscale values S _i (x, y) corresponding to the three color channels of the image to be processed at the position (x, y), and i takes 1 ,2,3;

SA2: Use the wraparound function and the convolution of the image to be processed to estimate the brightness value of the image to be processed and perform a weighted average as shown in formula (1) to obtain the outputs r _i (x, y), i corresponding to the three color channels respectively Take 1,2,3;

此处取w_n＝1/N；F_n(x,y)为权值w_n对应的第n个环绕函数，服从式(2)：Among them, N is the number of scales; _wn is the weight of the nth scale, satisfying

Here w _n =1/N; F _n (x, y) is the nth wraparound function corresponding to the weight w _n , obeying formula (2):

其中，F′(x,y)为F(x,y)的一阶导数。其中N取值为3，c₁＝15、c₂＝80、c₃＝240。Among them, c _n is the nth scale parameter, and k _n is the normalization factor. Since the wrapping function obeys ∫∫F(x,y)dxdy=1, so

Among them, F'(x,y) is the first derivative of F(x,y). The value of N is 3, c ₁ =15, c ₂ =80, and c ₃ =240.

Since N=3 in this scheme, the MSR algorithm can be weighted corresponding to the large, medium and small scales. The small-scale Retinex algorithm can realize the dynamic range compression of the image, the large-scale Retinex algorithm can reproduce the tone of the image, and the medium-scale Retinex algorithm The algorithm takes into account the balance between dynamic range compression and color fidelity of the image. The MSR algorithm can make up for the shortcomings of the SSR algorithm (single-scale weighted average Retinex algorithm), so that the color fidelity and dynamic compression ability of the image are greatly improved; however, the image processed by the simple MSR algorithm has color distortion and more noise. Shortcomings.

Through the above steps, the effect of the rightmost picture in Figure 1 is obtained. Compared with the left and middle ones in Figure 1, the effect is better, and the parameter indicators of the defogging performance are described in Table 1:

Parameter index The original image Traditional MSR Bidirectional MSR Gray standard deviation 146.5862 113.5934 114.1452 Information entropy value 6.0415 7.0654 7.0709 average gradient 3.2938 6.8074 6.8181

Table 1

The foggy image is selected to test the actual effect of the two-way weighted fusion in the foggy image sharpening process. The average gradient is used to reflect the image's ability to express the contrast of small details, and the image grayscale standard deviation is used to reflect the suppression effect of the dehazing algorithm on noise amplification. , and the image information entropy value is used to reflect the clarity of image details after dehazing.

For the subjective evaluation of Figure 1, the performance of dehazing is judged by human eye perception. In addition to restoring the scene details covered by fog, the traditional MSR algorithm in Figure 1 also needs to improve the global contrast of the image, and also needs to take into account the maintenance of foggy images. The realism of the two-way weighted fusion is applied to the sharpening of foggy images, which can effectively prevent the phenomenon of over-enhancement and suppress the amplification of noise while achieving dehazing.

For the objective evaluation in Table 1, it is not difficult to find through the statistical data in the table that the dehazing performance of the algorithm can be effectively improved after bidirectional weighted fusion. This processing can be used in many fields such as drone identification and vehicle-mounted video recording, and has certain commercial value.

Example 2

A method for processing foggy images using bidirectional weighted fusion, including the following steps:

S1. On the one hand, the fog image I to be processed is processed using the dark channel prior dehazing algorithm to obtain the output image J1; on the other hand, the fog image I to be processed is inverted, and then the dark channel Check the defogging algorithm for processing, and then perform the inversion operation after processing to obtain the output image J2;

S2. Perform a weighted average on the output image J1 and the output image J2 to obtain the final output image J.

In this example, the dehazing algorithm is a dark channel prior image dehazing processing algorithm based on guided filtering, and also includes image enhancement technology, and the dehazing image obtained from the dark channel is fused with the image processed by the image enhancement technology. .

The steps of the dark channel prior image dehazing algorithm based on guided filtering are:

SB1: The original foggy image I or the image after inversion operation is used as the image to be processed; the dehazing model of the image to be processed is described as:

Among them, M(x) is the image to be processed; J11(x) is the processed haze-free image; x is any pixel in the corresponding image; A is the global parameter atmospheric light intensity, which is the brightest in the dark channel 0.1% average value of pixels; t(x) is the transmittance of the scene reflected light of the corresponding pixel point;

SB2. Obtain the dark channel image M ^dark (x) of the to-be-processed image M (x), which is calculated by formula (4):

为Ω(x)中任一像素点；

为像素点

的三个颜色通道中的最小值，i＝1，2，3，表示R,G,B三个颜色通道；where Ω(x) represents the local neighborhood centered on pixel x,

is any pixel in Ω(x);

for pixels

The minimum value of the three color channels, i=1, 2, 3, represents the three color channels of R, G, and B;

SB3. Obtain the preliminary estimated value t ₁ (x) of the transmittance t(x), and the calculation is as in formula (5):

ω is the set value, which is 0.95 in this scheme, indicating a small amount of fog reserved for the far depth of field; A _i is the global parameter atmospheric light intensity of the corresponding color channel;

SB4. Taking the dark channel image M ^dark as a guide, a finer transmittance t ₂ is obtained through guided filtering; a local linear model based on a two-dimensional window is established between M ^dark and t ₂ :

为Ω(k)中任一像素点；(a_k,b_k)在此局部邻域中为常数；Among them, k is any pixel in M ^dark , Ω(k) represents the local neighborhood centered on pixel k,

is any pixel in Ω(k); ( _ak , b _k ) is constant in this local neighborhood;

之间的差异为代价函数，确定a_k和b_k的值：SB5, to minimize t ₁ (x) and

The difference between is the cost function, which determines the values of a _k and b _k :

在公式(7)中取值相同；ε为调整参数；求得

后，最后由依据式(1)所得的式(8)计算去雾图像J11：where x and

The value is the same in formula (7); ε is the adjustment parameter;

Finally, the dehazing image J11 is calculated by the formula (8) obtained according to the formula (1):

The specific steps of image enhancement technology are as follows:

中的变量

替换为变量x，采用式(9)与式(10)进行归一化；SB6, the transmittance in formula (6)

variables in

Replace with variable x, and use formula (9) and formula (10) for normalization;

t ₂ ^* (x) is the normalized transmittance of pixel x; max(t ₂ (x)) and min(t ₂ (x)) are the maximum and minimum values of t ₂ (x), respectively; In the scheme, min(t ₂ (x))=0.1, max(t ₂ (x))=0.9.

SB7. Perform clipping processing on t ₂ ^* (x) to obtain the corrected transmittance T(x):

The specific steps of fusing the dehazed image obtained from the dark channel with the image processed by the image enhancement technology are as follows:

SB8: Use the histogram equalization algorithm based on the HSV color space to process the image: first convert the input image M from the RGB color space to the HSV color space, and then perform histogram equalization processing on the luminance component V in the HSV space to obtain enhanced luminance Component V*; hue component H and saturation component S remain unchanged; convert back to RGB color space according to H, S, V* of HSV space, and obtain output image J12(x);

SB9: According to formula (11), the dark channel dehazing image J11 and the enhanced image J12 are fused to obtain the output image D(x);

D(x)=(1-T(x))·J11(x)+T(x)·J12(x) (11).

For the pixels with larger corrected transmittance T(x), the weight of the dehazed image J11(x) is relatively small, while the weight of the enhanced image J12(x) is larger; conversely, for the corrected transmittance T(x) Pixels with smaller x) J11(x) are weighted more and J12(x) are less weighted. Therefore, the algorithm can automatically fuse the results of dehazing processing and contrast enhancement processing, and automatically assign reasonable weight factors according to the transmittance values of different pixels.

The parameter indicators of the existing de-performance are described in Table 2:

Table 2

Select fog images to test the actual effect of bidirectional weighted fusion in fog image sharpening processing, use the average gradient to reflect the image's ability to express the contrast of tiny details, and use the image grayscale standard deviation to reflect the dehazing algorithm's suppression effect on noise amplification , and the image information entropy value is used to reflect the clarity of image details after dehazing.

For the subjective evaluation of Figure 2, the performance of dehazing is judged by human eye perception. In addition to restoring the scene details covered by fog, the traditional dark channel algorithm in Figure 2 also needs to improve the global contrast of the image. In addition, it is necessary to maintain the foggy weather. The authenticity of the image, the bidirectional weighted fusion is applied to the sharpening of the foggy image, and it can effectively prevent the phenomenon of over-enhancement and suppress the amplification of noise while achieving dehazing.

For the objective evaluation in Table 2, it is not difficult to find through the statistics in the table that the dehazing performance of the algorithm can be effectively improved after bidirectional weighted fusion.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (10)

1. the method for processing foggy images using bidirectional weighted fusion, is characterized in that, comprises the following steps: S1, on the one hand, the fog image I to be processed is processed using a dehazing algorithm to obtain an output image J1; After completion, the inversion operation is performed to obtain the output image J2; S2. Perform a weighted average on the output image J1 and the output image J2 to obtain the final output image J. 2 . The method for processing foggy images using bidirectional weighted fusion according to claim 1 , wherein the dehazing algorithm is an arbitrary multi-scale Retinex-based processing algorithm. 3 . 3 . The method for processing foggy images using bidirectional weighted fusion according to claim 1 , wherein the dehazing algorithm is the MSR algorithm (multi-scale weighted average Retinex algorithm) in the multi-scale Retinex algorithm. 4 . 4. the method for using bidirectional weighted fusion to process foggy images according to claim 3, is characterized in that, the step of described MSR algorithm is: SA1: Take the original image I or the image after the inversion operation as the image to be processed; obtain the gray value Si(x, y) corresponding to the three color channels of the image to be processed at the position (x, y), i is 1, 2, 3; SA2: Use the wraparound function and the convolution of the image to be processed to estimate the brightness value of the image to be processed and perform a weighted average as shown in formula (1) to obtain the output ri(x, y) corresponding to the three color channels respectively, i is taken as 1, 2, 3;

Among them, N is the number of scales; wn is the weight of the nth scale, satisfying

Here w _n =1/N; F _n (x, y) is the nth wraparound function corresponding to the weight w _n , obeying formula (2):

Among them, c _n is the nth scale parameter, and k _n is the normalization factor. Since the wrapping function obeys ∫∫F(x, y)dxdy=1, so

Among them, F'(x, y) is the first derivative of F(x, y). 5 . The method for processing foggy images using bidirectional weighted fusion according to claim 4 , wherein N is 3, c ₁ =15, c ₂ =80, and c ₃ =240. 6 . 6 . The method for processing foggy images using bidirectional weighted fusion according to claim 1 , wherein the dehazing algorithm is a dark channel prior image dehazing processing algorithm based on guided filtering. 7 . 7. The method for using bidirectional weighted fusion to process foggy images according to claim 6, wherein the step of the dark channel prior image dehazing algorithm based on guided filtering is: SB1: The original foggy image I or the image after inversion operation is used as the image to be processed; the dehazing model of the image to be processed is described as:

Among them, M(x) is the image to be processed; J11(x) is the processed haze-free image; x is any pixel in the corresponding image; A is the global parameter atmospheric light intensity; t(x) is the corresponding pixel The transmittance of the scene reflected light; SB2. Obtain the dark channel image M ^dark (x) of the to-be-processed image M (x), which is calculated by formula (4):

where Ω(x) represents the local neighborhood centered on pixel x,

is any pixel in Ω(x);

for pixels

The minimum value of the three color channels of , i=1, 2, 3, representing the three color channels of R, G, B; SB3. Obtain the preliminary estimated value t ₁ (x) of the transmittance t(x), and the calculation is as in formula (5):

ω is the set value, indicating a small amount of fog reserved for the far depth of field; A _i is the global parameter atmospheric light intensity of the corresponding color channel; SB4. Taking the dark channel image M ^dark as a guide, a finer transmittance t ₂ is obtained through guided filtering; a local linear model based on a two-dimensional window is established between M ^dark and t ₂ :

Among them, k is any pixel in M ^dark , Ω(k) represents the local neighborhood centered on pixel k,

is any pixel in Ω(k); ( _ak , b _k ) are constants in this local neighborhood; SB5, to minimize t ₁ (x) and

The difference between is the cost function, which determines the values of a _k and b _k :

where x and

The value is the same in formula (7); ε is the adjustment parameter;

Finally, the dehazing image J11 is calculated by the formula (8) obtained according to the formula (1):

8 . The method for processing foggy images using bidirectional weighted fusion according to claim 7 , wherein the dehazing algorithm further comprises an image enhancement technique after the dark channel prior image dehazing processing algorithm based on guided filtering. 9 . , and fuse the dehazed image obtained from the dark channel with the image processed by image enhancement technology. 9. The method for processing foggy images using bidirectional weighted fusion according to claim 8, wherein the specific steps of the image enhancement technique are as follows: SB6, the transmittance in formula (6)

variables in

Replace with variable x, and use formula (9) and formula (10) for normalization;

t ₂ ^* (x) is the normalized transmittance of the pixel point x; max(t ₂ (x)) and min(t ₂ (x)) are the maximum and minimum values of t ₂ (x), respectively; SB7. Perform clipping processing on t ₂ ^* (x) to obtain the corrected transmittance T(x):

10. The method for processing foggy images using bidirectional weighted fusion according to claim 9, wherein the specific steps of merging the dehazed image obtained by the dark channel and the image processed by the image enhancement technology are: SB8: The histogram equalization algorithm based on HSV color space is used. Image to be processed: first convert the input image M from RGB color space to HSV color space, and then perform histogram equalization processing on the luminance component V in HSV space to obtain enhanced luminance Component V*; hue component H and saturation component S remain unchanged; convert back to RGB color space according to H, S, V* of HSV space, and obtain output image J12(x); SB9: According to formula (11), the dark channel dehazing image J11 and the enhanced image J12 are fused to obtain the output image D(x); D(x)=(1-T(x))·J11(x)+T(x)·J12(x) (11).

Images