TW201101814A - Method of compensating for backlight image and photograph apparatus with backlight image compensation system - Google Patents

Abstract

A photograph apparatus with backlight image compensation system is disclosed. The system includes an image-retrieving module, a Functional-Link-Based Neural Fuzzy (FNFN) process module and a backlight compensating module. The image-retrieving module is took at least one picture. The FNFN process module includes a pre-process unit, an input unit, an operation unit and an output unit. The pre-process unit calculates at least one backlight factor. The input unit receives the backlight factor. The operation unit uses the backlight factor to calculate a compensation value. The output unit outputs the compensation value. The backlight compensating module receives the compensation value and compensates the picture.

Description

201101814 VI. Description of the invention: [Technical field to which the invention pertains] ^Daily has a 1 image repairing method and a camera device having an image repairing mechanism, and particularly relates to a photographing device having a backlight image compensating mechanism. ~^贝贝 4 [Previous technology] ❹ ❹ With the development of technology, 'human requirements for quality of life are gradually increasing, := The electronic products of the 'requires the same - an electronic product month b,,, 〇S Month b to improve the convenience of use. In recent years, built-in photographic devices have been a must-have feature for mobile phones. However, the camera or PDA can help (A), such as anti-shake, fill light, and object-to-camera, causing the user to use the mobile phone or PDA to shoot = not to repair through the computer's retouching software, but cause use, sex 'The original intention of building a camera device on a mobile phone or PDA was lost. SUMMARY OF THE INVENTION Therefore, one aspect of the present invention is to provide a photographic device that provides a singularity mechanism, which can be directly used at the same time; ❿ 上传 将 将 将 将 将 将 将 将 将 ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® ® Group, a type of neuro-fuzzy network 4 201101814 road processing module and a backlight compensation module. The image capturing module is used to record at least one image. The neuro-fuzzy network processing module is used to calculate a compensation value, and is backlit. The compensation module receives the compensation value, thereby repairing the brightness of the image. The neuro-fuzzy network processing module includes a pre-processing unit, an input unit, an operation unit, and an output unit. The pre-processing unit is configured to At least one backlight factor. The input unit is configured to receive a backlight factor, and the operation unit calculates a compensation value by using a backlight factor, and outputs a compensation value through the output unit. The backlight image can be repaired directly in the photographic device to increase the convenience of shooting. Another aspect of the present invention is to provide a backlight image compensation method, which is applied to a photographic device that can capture images taken by a user. The image brightness correction is performed directly in the photographing device, and the image quality captured by the photographing device is improved. According to another embodiment of the present invention, a backlight image compensation method is provided, which is applied to a photographing device, and the steps are as follows: using a photographing device Shooting an image and grouping the images into brightness. Using brightness grouping, at least one backlight factor can be proposed, and the backlight factor is input into a type of neural fuzzy network to calculate a compensation value. The compensation value is converted into a compensation curve. The compensation curve is used to compensate the brightness of the image. Thereby, the backlight image compensation method is applied to a mobile phone or a PDA with a photographing function, thereby improving the photographing performance of the photographing device and increasing the convenience of photographing the image. 】 5 201101814 Please refer to FIG. 1 , which shows a real according to the present invention. A block diagram of a photographic device having a backlight image compensation mechanism. The photographic device 100 having a backlight image compensation mechanism may be a digital mobile device such as a mobile phone or a PDA, and includes an image capturing module 2 〇〇, a type of neuro-fuzzy network processing module 300 and a backlight compensation module 4. The image capturing module 200 can be a lens of the photographic device 100 for recording and capturing an image; The processing module 300 receives the image captured by the image capturing module 2〇〇0, and analyzes the degree of backlighting of the image, and then calculates a compensation value; then, the backlight compensation module 400 and the neuro-fuzzy network processing module The 300 connection 'uses the compensation value to perform backlight compensation processing on the image. Thereby, the backlight problem of the image caused by the position of the photographing device 100 and the light source at the time of shooting is improved. It is worth mentioning that the aforementioned neuro-fuzzy network processing module 300 can be a processing chip embedded in a photographic device; and the other can be an image processing program, and the photographic device 1 has an operation. System to execute the image processing program. The neuro-fuzzy network processing module 3 〇〇 includes a pre-processing unit t 31 〇 , and the neuro-fuzzy network 3: 2 and the factor 322 and the output unit 323 : 3 = single 70 321 , and the different units *. Really produced eight hall color secret layer to convert the color of the image: two =: 33; 32 for the brightness of each pixel in r! The input unit 321 calculates the two backs for receiving the two moonlight factors using the results of the group. The arithmetic unit 322 6 201101814 calculates a compensation value by using two backlight factors. The output unit 323 is for outputting a compensation value. In addition, the backlight compensation module 400 is configured to receive a compensation value and modify the intensity of the image, and includes a curve conversion unit 41 and an image repair unit 420. The curve conversion unit 41 is configured to convert the compensation value into a compensation curve. The image repairing unit 420 repairs the brightness of the image using the compensation curve. Thereby, the user can perform backlight compensation on the image directly on the photographing device. The specific principles and functions of this embodiment are explained as follows: ❹ Generally, s, the captured images are all color images, and each pixel in the image is composed of RGB color space. However, the color image is more complicated and difficult to classify. ,deal with. Therefore, through the color conversion layer 311, each pixel in the image is converted from RGB to a color space of YIQ, that is, the color image is converted into a grayscale image. In grayscale images, γ represents brightness, while I and Q represent color information. Each pixel contains only one grayscale, and γ can be used to represent the grayscale value of each pixel in the grayscale image. Although separately separating the brightness from the color information can improve the convenience of image analysis, using only Y to represent the gray scale value of the Q table may lose the color information of some images. Therefore, the color conversion layer 311 'uses a feature-weighted transformation algrothm to convert a color image into a grayscale image, as shown in the following formula (1): F(iJ) = R(iJ)(l + ^ ^Mi))

N - +G(i,j)(1+^EM) (1) N , ' - +B(i;j)(1+Am〇uiffl(iJ)) 7 201101814 where ' R(1,J), G(i,j) and B(i,j) represent the RGB values of the pixels at the (i,j) position in the image, respectively, AmountR(i,j), Am_tG(i,j), and

Am〇UntB(i,j) represents the total number of pixels of the image having the same RGB value for the pixel of the entire image and image position (i, j). Finally, after calculating the _value F of each pixel, normalize its value to be between 0 and 255. Thereby, the grayscale value of each pixel in the image is obtained. The party clustering layer 312 is primarily used to separate objects in the image from background objects or backgrounds. The gray scale value F of each pixel 〇 obtained in the color conversion layer 311 represents the luminance of each pixel. Therefore, in the luminance grouping layer, the operation of luminance grouping is performed using the Fuzzy C-mean (fcm) algorithm. The grouping formula is as follows (2):

FCM V. Z|;uk(ij)q|F(iJ)-vk|2 F(i,j)-vn ZyUk(i,j)qxF(i,j) Σ

l<:k<C (2)

l<k^C

Attachment 1 is a technical image of a backlight image compensation method and a photographic device having a backlight image compensation mechanism. Referring to FIG. 3 on page 5 of Annex 1, the left backlight image is calculated by the brightness grouping layer 312, and the right group can be remitted. schematic diagram. It can be seen from the grouping diagram that the object located in the middle of the image is the object of the backlight, and the brightness of the backlight object is much smaller than the brightness of the background compared to the brightness of the background of the back side of the backlight object. The factor calculation layer 313 calculates the two backlight factors B_hist 313a and 201101814 B_fcm 313b by using the result of image grouping by the above-described luminance grouping layer 312. Hist is defined as the ratio of pixels above a certain brightness threshold to the total pixels, which detects the state of the picture's brightness distribution. B_hist313a is the average of the brightness Hist of each level of the entire image. :Bj*cm3l3b is used to represent the degree of backlighting of the image. Please refer to Fig. 2, which is a Hist histogram drawn by the result of luminance grouping in the embodiment of Fig. 1. In the factor calculation layer 313, the luminance grouping of the image by the luminance grouping layer 312 can be first used to continue a Hist histogram, wherein the horizontal axis of the Hist straight square graph is luminance, and the vertical axis is Hist value. To measure the gradient of the background and the luminance group histogram of the backlight object in Figure 4 on page 6 of the attachment t, the factor calculation layer 312 uses the Sliding Window (SW) to capture the current B-Caf. In conjunction with Figure 2, when the cumulative amount of Hist is less than 〇2, I am the largest SW, ie swmax. Through observation we can see that the degree of backlighting will increase SWmax. So we define B-hist via the following equation (3) Ο

SW B-hiSt = Thist^) ......Ο) Where T* is converted to listen, B~hist can be converted to a modulus. The conversion function T is the following function (4): , if 0.6 > χ > Q j , if x >= 0.6 , otherwise (4) (X - 0.3) / (0.6 - 0.3) T (X) = i 1 Hist 0 Please refer to FIG. 3, which is a Hist histogram of another embodiment using the luminance grouping result 9 201101814. As shown in Fig. 3, the horizontal axis of the mst histogram is the brightness 'the vertical is the Hist value, and the B_hist is the average value of the brightness Hist of the entire image. Please refer to Fig. 4, which is a histogram drawn by the luminance grouping result in the embodiment of Fig. 1. The factor meter nasal layer 313 also uses the results of the grouping of the Fuzzy C-mean (fcm) algorithm in the luminance grouping layer—the background and the backlight object represent the bright and dark areas in the backlight image, respectively. In order to divide the background and 〇 backlight object' factor calculation layer 313, the number of clusters is set to 2. After the Fuzzy C-mean (fcm) algorithm, two clusters C1 and C2 of the luminance group 5〇0, 51〇 are obtained, wherein C1 and C2 respectively represent the two darkest groups after being grouped by the luminance grouping layer 312. Groups of brightness groups 500, 51〇. Obviously, there is no significant accumulation of brightness between the background and the backlight. In other words, the lower the brightness accumulation between the background and the backlight object, the higher the degree of backlighting. Therefore, another backlight factor B-fcm 313b can be defined, which is used to determine the degree of backlighting of an image. In other words, B-fcm 313b can represent the degree of contrast between the backlight object and the background, as shown in the following formula (5): ” ( Σ p(r ) B fcm p(c ) + p(c ) (5) where ~ = ΐ is the percentage of the ith gray scale, which represents the total pixel ❼n i is the total number of i gray scale values in the image 々$ B-fcm can be converted to degree of blur. The conversion function τ is the following two): ' 201101814 (x - 0.3; / (0.6 - 0.3) , if 0.6 > x > 〇 3 ^ (x) ~ < ^ Jf x >= 〇6 (6) hist 〇, otherwise V» where the backlight factor B-fam313b represents the cumulative brightness between the two cluster centers C1 and C2, and the degree of backlighting increases with B_fcm 313b. For the fuzzy C_mean grouping result, the two darkest luminance groups 5〇〇 and 510 are used as the basis for compensation. Therefore, only the absolute distances of the group centers C1 and C2 of the darkest two luminance groups 5(9) and 51〇 are taken as B_fcm 313b. And normalize the 〇 value field between 0 and 1, and its formula (7) is as follows: B fcm II C2 _ Cl _ 255 (7) Please refer to Figure 5, which is the first picture or the like. Schematic diagram of the operational flow of the neuro-fuzzy network 320. Among them, X! represents the backlight factor B_hist 313a, and χ2 represents the backlight factor B_fcm 313b. The two backlight factors B_hist 313a and B_fcm 313b derived from the above-mentioned factor calculation layer 313 are thrown into the nerve-like body. The operation is performed in the fuzzy network 320. The neuro-fuzzy network 320 is a functional neuro-fuzzy network O (FNFN), and is combined with an immune system-based particle swarm optimization algorithm (IPSO). The neural fuzzy network 320 includes an input unit 321, an operation unit 322, and an output unit 323. The operation unit 322 includes a home function layer 322a, a rule layer 322b, and an inference layer 322c. The input unit 321 is configured to receive the backlight factor B_Hist 313a (ie, In the fifth figure, the forks 1) and 3_&1113131) (i.e., \2 in Fig. 5), and in the input unit 321, each node is called an input term node, and each round entry The nodes respectively correspond to a received backlight factor, namely: wf} = xi ...... (8) 201101814 and transmitted to the neuro-fuzzy network 320. Each node in the attribution function layer 322a is called input language awareness. Point (input linguistic node) 'present home function layer 322a corresponding to the main functions of the fuzzy fuzzy control interface, which represents the value of a home input value (i.e., a backlight Factor) belongs to a fuzzy set degrees. In the attribution function layer 322a, we use the Gaussian membership function, namely: , (2) exp ] 2 (9) Ο ο

V, where the gas and the table do not Gaussian attribution function center and width ° Each node in the law layer 322b is called a rule node (_ plus (four), and each rule node represents a fuzzy rule. The rule layer 322b completes the fuzzy logic rule. The front part of the module is called (four) point to complete the fuzzy and (fuzzy AND) or product action: ...do each node in the inference layer 322C is called a consequent node. Each of the posterior nodes is a linear combination of the output of the rule layer 322b multiplied by the output unit and the link weight, as follows: ur=uT(w〇j+p^ ...··.(11) where linear combination In the process 600, the normalization of (~Nonnalizati〇n)6〇l is performed first, and the function expansion (f-deletion-expanSi〇n) 602′ is performed to generate a linear group of the rounded unit and the connected weight. 12 201101814 0 0 each node node in the output unit 323, which completes the defuzzification action, as follows output node (〇Utput

(12) where Be. y is the compensation value B_c, and the compensation value is output by the rounding unit 323. 模糊 In the arithmetic unit 322, the fuzzy law in a nonlinear manner is expressed as follows (13): " is the inference unit, such

IF

lSAiJ and

XN

Is A

(13) ο

Rule-y: x\isAlj^dx2kA2r^ndx.

Among them, Rule-j points out the j-th rule, Aij is a fuzzy set, xi is the network input variable, and the inference part is a local output j> using a functional type of neuro-fuzzy network 32;; represents a series of nonlinear combinations. After the inference of the neuro-fuzzy network 320, the image compensation value B_c can be obtained. The output unit 323 can output the compensation value B_c from the neuro-fuzzy network 320. Please refer to Fig. 6, which is the compensation curve drawn by the curve conversion unit 41〇 of Fig. 1. The backlight compensation module 400 is configured to receive the compensation value B c and repair the brightness of the image ‘which includes the curve conversion unit 410 and the image; 补 13 201101814 unit 420. In order to reduce the difference between the shadows and the curves, the curve conversion · 41Q / ^ 4 \ between the object and the background can be compensated, and the bright place can be compensated, and the complement curve equation (1 sentence is as follows: - If a < Cl f(X) = fly (Xa)2 +bf(X) = S^(x'a)2+b — (14) ...compensation curve wire type (9) towel, lower throw curve The action of ffij on the dark side of the image towel is the action of suppressing the bright ground. 'a and b are the adjustment points of the curve (τρ), which depends on the center of the image brightness group and The compensation value BC. The adjustment point (ΤΡ) is defined as the following formula (15): TP(a, b) = (C1, c 1+B_fcm* B_c) (15), as seen from Fig. 6, curve 412 is - In the comparative example, it represents a curve that the backlight image has not compensated, and the compensation curve 411 obtained by the equation (14) can compensate the curve 412 to improve the brightness of the backlight image. Finally, the image repairing unit 420 utilizes the image in the image. Each pixel is compensated by the curve 2 411 i ' to repair the brightness of the image, and the compensation is obtained. Refer to Figure 12 on page 13 of the Attachment 1 'the PDA image operation interface ^Sang Zuo process. When the user of the PDA camera shoots the image, 7 selects the backlight compensation system from the interface and selects whether the image is a light image. When the user selects the image as a non-backlight image, the image No: Any change; otherwise, 'When using this image as a backlit image, the interface will go to the next step, and the user can choose whether to backlight the image 201101814. When the user chooses not to perform this image, When the backlight is compensated, the image will not be changed. Otherwise, when the user chooses to perform backlight compensation for the image, the image can be transmitted through the neuro-fuzzy network processing module 300 and the backlight compensation module 400 such as the first image. For backlight compensation. Figures n(a), 18(a), 19(a) and 20(a) on page 16-19 are all backlit images' and these images are directly visible in the camera.

❹ The neuro-fuzzy network 320 compensates for the image, and the results are compensated for Figures i7(b), 18(b), 19(b), and 20(b) on pages 16-19 of Annex 1, respectively. Rear image. Obviously, after the backlight-compensated image, the object and the square are more harmonious, and the objects are clearer. In summary, the photographic arrangement with backlight image compensation mechanism and the backlight image compensation method thereof have the following advantages: The neuro-fuzzy network of the 2nd Ϊ function type has less computational complexity and is determined by the invention.任 』 揭 揭 揭 揭 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects of the present invention 15 201101814 can be more clearly understood, and the description of the drawings is as follows: FIG. 1 is a diagram showing an embodiment of the present invention. Block diagram of a photographing apparatus having a backlight image compensating mechanism. Fig. 2 is a Hist histogram drawn by the result of luminance grouping in the embodiment of Fig. 1. Fig. 3 is a diagram showing luminance in another embodiment. Hist histogram drawn by grouping results. Fig. 4 is a histogram plotting 0 using the luminance grouping result in the embodiment of Fig. 1. Fig. 5 is a flow chart of the neuro-fuzzy network 320 such as Fig. 1. Schematic. The figure is a compensation curve drawn by the curve conversion unit 410 of Fig. 1. Annex 1 is a technical image of a backlight image compensation method and a photographic device having a backlight image compensation mechanism 〇 [Main component symbol description] Image capture module preprocessing Module brightness grouping layer 313a: B_hist 320: neuro-fuzzy network 322: arithmetic unit 322b: law layer 323: output unit 100: photographing apparatus 200 300: neuro-fuzzy network processing module 310 311: color conversion layer 313: Factor calculation layer 313b: B fcm 321 : input unit 322a: attribution function layer 322c: inference layer 312 16 201101814 400 : curve conversion unit 410 411 : curve 412 420 : image patch unit 500 510 : luminance group 600 601 : step 602 curve conversion Unit curve brightness group linear combination process step

17

Claims (1)

201101814 VII. Patent application scope: .1. A photographic device having a backlight image compensation mechanism, comprising: an image capture module for recording at least one image; and a type of neural fuzzy network processing module for receiving the image The image includes: a pre-processing unit for extracting at least one backlight factor from the image; an input unit for receiving the backlight factor; and an operation unit, calculating a compensation value by using a backlight factor obtained by the input unit And an output unit for outputting the compensation value; and a backlight compensation module for receiving the compensation value, thereby repairing the brightness of the image. 2. The photographic device having the backlight image compensation mechanism according to claim 1, the preprocessing unit further comprising: 〇 a color conversion layer for converting the color space of the image. 3. The photographic device having the backlight image compensation mechanism according to claim 1, the preprocessing unit further comprising: a brightness grouping layer for grouping the brightness of each pixel in the image. 4. The photographic device 18 201101814 having the backlight image compensation mechanism according to claim 3, the preprocessing unit further comprises: a factor calculation layer, and calculating the two backlights by using the result of the brightness grouping. The photographic device having a backlight image compensation mechanism, the backlight compensation module further comprising: a curve conversion unit for converting the compensation value into a compensation curve; and a first image repairing unit, wherein the compensation curve is used to repair the The brightness of the image. 6. The photographic device having a backlight image compensation mechanism according to claim 1, wherein the neuro-fuzzy network processing module is a processing chip and is embedded in the photographic device. 7. The photographic device having a backlight image compensation mechanism according to claim 1, wherein the neuro-fuzzy network processing module is an image processing program, and the photographic device has an operating system to execute an image processing program. 8. The photographic apparatus having a backlight image compensation mechanism according to claim 1, wherein the photographic apparatus is a mobile phone. 9. A photographic apparatus having a backlight image compensation mechanism as claimed in claim 1, wherein the photographic apparatus is a personal digital assistant (PDA). 19 201101814 10. A backlight image compensation method is applied to a photographing device, comprising the following steps: - taking an image; and performing brightness grouping on the image; using at least one backlight factor by brightness grouping; inputting the backlight factor into a class The neural network is fuzzy to calculate a compensation value; the compensation value is converted into a compensation curve; and 亮度 the image is subjected to brightness compensation using the compensation curve. 11. The backlight image compensation method of claim 10, further comprising: converting the image from a color space to a grayscale space, and obtaining a plurality of grayscale values. 12. The backlight image compensation method according to claim 11, wherein the images are subjected to luminance grouping using the grayscale values. Ο 20