JP2001112019A - Automatic white balance device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve instability in automatic white balancing. SOLUTION: A block division circuit 102 divides the output image from an image pickup device 20 into block units. A representative color calculation circuit 104 calculates a mean value of pixels of each block and converts the mean value into a luminance and color difference expression. A block reliability calculation circuit 110 obtains a distance between a color difference component of a block and a typical color difference of an assumed light source such as a fluorescent light or daylight or the like, and the reliability of the block that is lighted by the estimate light source is obtained on the basis of the distance. The reliability is obtained from a function changed continuously in response to the distance above. An automatic white balancer 10 obtains a color of the lighting light (lighting color) lighting the scene of a picture and the white balance is adjusted to cancel the color. Since the reliability changing continuously according to the color difference of the block is used, the instability in white balancing can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic white balance device used for an electronic still camera, a video camera and the like.

[0002]

2. Description of the Related Art In video cameras and digital still cameras, automatic white balance adjustment is performed to reproduce a white subject in white. As a conventional auto white balance method, a method of adjusting the balance of RGB components (three primary color components of red, green, and blue) of a signal of each pixel so that the average of the entire image becomes an achromatic color is well known. However, this method has a disadvantage that erroneous white balance adjustment is likely to be performed when a chromatic color occupies most of the area of the image.

[0003] Such incorrect white balance adjustment is called color failure. As an automatic white balance adjustment method for reducing the color failure, there is known a technique disclosed in Japanese Patent Application Laid-Open No. 5-292533. In this technique, an image is divided into a plurality of blocks, an average value of RGB of each block is calculated, and only blocks whose average values belong to a predetermined range are extracted. Then, the RGB components are adjusted so that the average value of RGB of the extracted block group becomes an achromatic color.

However, these methods have been effective when the light source for illuminating the subject is limited.
Sufficient white balance adjustment could not be performed when illuminated by unexpected light sources or when simultaneously illuminated by multiple types of light sources.

Accordingly, the applicant of the present invention has disclosed in Japanese Patent Application Laid-Open No. 8-2893.
No. 14 discloses an improved automatic white balance adjusting device. In this device, the image is divided into a plurality of blocks, a group of blocks determined to have taken a white object under a fluorescent lamp, a group of blocks determined to have taken a white object under sunlight or tungsten light, A block group having a color close to the brightest block in the image is extracted. Then, for each of these block groups, RG
The average value of each of the B components is obtained, and a value obtained by mixing them according to a predetermined rule is used as a white balance adjustment signal. In this device, the average value of RGB of a block group of a color similar to the brightest block in an image is reflected in a white balance signal, so that appropriate white balance adjustment can be performed even under an unexpected light source or a plurality of types of light sources. It can be performed.

[0006]

In the above-mentioned conventional method, a binary (i.e., whether or not an average value of the color difference of each block is included in a region (light source region) fixed for each light source) is determined. There is a problem that the behavior of the white balance is likely to be unstable due to a change in the photographing scene because the result of such discrimination is used. For example, a certain block that belonged to a certain light source area only slightly changed the shooting scene,
There is a possibility that the light does not belong to the light source region, and such a sudden change in the binary value may cause instability of the white balance.

The present invention has been made to solve this problem, and has as its object to provide an automatic white balance device capable of performing stable white balance adjustment.

[0008]

An automatic white balance apparatus according to the present invention comprises: a block dividing means for dividing an image to be subjected to white balance processing into a plurality of blocks; and, for each of the divided blocks, a pixel in the block. Representative color calculating means for obtaining a representative color of the block based on the value;
For each of the blocks and for each assumed light source, the distance between the color difference component of the color of the white object under the light source and the color difference component of the representative color of the block is determined. Block reliability estimating means for estimating the reliability of illuminating the scene of the block; and for each light source, the representative color of each block is weighted according to the reliability of the light source for that block, to obtain the light source. Light source contributing component estimating means for estimating a contributing component to the entire image; and, for each of the light sources, the reliability of each block of the light source is summed over all the blocks in the image, so that the An overall reliability estimating means for estimating the reliability of the light source,
The weighted average of the contribution components of the respective light sources estimated by the light source contribution component estimating means to the entire image according to the reliability of the entire image of the respective light sources estimated by the overall reliability estimating means is obtained. Illumination color estimation means for estimating an illumination color for illuminating the entire scene, and white balance means for performing white balance processing on each pixel of the image so as to cancel the estimated illumination color;
Having.

This apparatus uses a color difference of a block representative color and a typical color difference of each light source instead of using a binary discrimination result of whether or not the color difference average value of each block is included in each light area. White balance is adjusted using the reliability obtained from the distance. The distance between the two color difference values takes a continuous value. Therefore, the white balance adjustment processing according to the present invention allows the behavior of the white balance to continuously follow changes in the photographing scene, and the white balance adjustment processing according to the related art has performed the adjustment based on the binary discrimination result. Also allows smooth white balance adjustment.

An apparatus according to a preferred aspect of the present invention includes means for correcting the reliability of each block estimated by the block reliability estimating means according to a luminance component of a representative color of the block.

The degree to which the color of the scene reflects the color of the illumination differs depending on the brightness of the scene. According to this aspect, since the reliability of each light source illuminating the scene can be corrected according to the brightness of the scene, that is, the brightness, the accuracy of estimation of the illumination color is improved, and as a result, the white balance adjustment is performed. Accuracy is also improved.

In this aspect, if the correction of the reliability is performed using a function prepared individually for each light source, each light source illuminates the scene using the knowledge about the brightness of each light source. The reliability can be modified. Further, by setting the function used for the correction to be a continuous function in which the value continuously changes with respect to the luminance, smooth white balance adjustment can be performed.

According to another preferred embodiment of the present invention, as a light source, direct sunlight in the daytime and shaded light in the daytime are distinguished from each other, so that both the portion where the image is exposed to sunlight and the portion where the image is shaded are treated. , The white balance can be controlled flexibly.

[0014]

Embodiments of the present invention (hereinafter referred to as embodiments) will be described below with reference to the drawings.

FIG. 1 is a functional block diagram showing the configuration of the automatic white balance device 10 of the present embodiment. The auto white balance device 10 is typically incorporated in a camera device such as an electronic still camera or a video camera. The automatic white balance device 10 generates a white balance adjustment signal from an image signal input from the imaging device 20 of the camera device, and the garden uses the signal to perform white balance adjustment for the extra signal and outputs the result. In the present embodiment, the color of a light source (referred to as an illumination color) that illuminates a scene (scene) of an image is estimated, and white balance adjustment is performed based on the estimation result.

FIG. 2 is a flowchart showing the procedure of processing performed by the automatic white balance apparatus.

Referring to FIGS. 1 and 2, a detailed configuration of the automatic white balance device 10 and the processing contents thereof will be described in detail.

The image signal output from the imaging device 20 is
The block is divided into blocks by the block dividing circuit 102 (S10). FIG. 3 shows an example of image block division. In this example, the image 200 of one frame is divided into 16 (= 4 × 4) equal blocks 210 each including m × m pixels. The block division shown in FIG. 3 is merely an example, and other division methods may of course be used. The image data of each block divided by the circuit 102 is input to a representative color calculation circuit 104, a variance coefficient calculation circuit 106, a saturation coefficient calculation circuit 108, and a light source contribution calculation circuit 114, respectively.

The representative color calculation circuit 104 obtains a representative color which is an average color of each block based on the image data of each block (S12). The detailed processing contents of this circuit 104 are as follows. That is, the representative color calculation circuit 104 first calculates the average value of the pixel values of all the pixels in one block. When the image signal of the imaging device 20 is RGB (red, green,
(Blue), the representative color calculation circuit 104 averages the RGB values of each pixel over all the pixels of the block for each of the RGB components. This average (R, G, B)
Is the representative color of the block. Next, the representative color calculation circuit 104 linearly converts the average value (R, G, B) into a set (L, u, v) of luminance (L) and two color difference components (u, v). This conversion is represented by the following equation.

[0020]

(Equation 1) The conversion matrix used for the conversion depends on the input color characteristics of the imaging device 20, and the one shown in Expression (1) is only an example. When the image signal is represented by a color system other than RGB, the luminance and color difference components may be obtained using a conversion matrix corresponding to the color system. In the above processing, when calculating the representative color of the block, the average of all the pixels of the block was performed. Instead, the average of the pixels extracted from the pixels of the block according to a predetermined sampling rule is used. You may do so.

Such processing is performed for each block constituting one image. The obtained luminance / color difference expression (L, u, v) of the representative color of each block is input to the block reliability calculation circuit 110. Hereinafter, the identification number of the block is i (i = 1, 2,..., N: n is the total number of blocks of the image), and the representative color of the i-th block (denoted as block i) is (Li, ui, vi). Is expressed as

In parallel with the processing of the representative color calculation circuit 104, the dispersion coefficient calculation circuit 106 and the saturation coefficient calculation circuit 1
08 executes each process.

The variance coefficient calculation circuit 106 receives the image data of each block, and calculates the variance coefficient Cv for each block. The variance coefficient Cv is a coefficient for correcting block reliability described later, and is determined according to the variance of the pixel values of the block. The variance coefficient of block i is denoted by Cvi. The variance coefficient calculation circuit 106 first samples the pixels from the pixel group of the block by a predetermined method, and obtains the variance of the pixel values of the sampled pixel group.
In this case, for example, the circuit 106 obtains the variance in the pixel group sampled for each of the R, G, and B components, and obtains an overall variance value by, for example, averaging the variance of each of the components. Then, the dispersion coefficient calculation circuit 106
Calculates the variance coefficient based on the variance value thus obtained. The dispersion coefficient Cv is obtained using a function prepared in advance. FIG. 4 is a graph of an example of this function. As shown in FIG. 4, in the circuit 106, as the variance increases,
A function that reduces the corresponding dispersion coefficient is used. This function is registered in advance in the circuit 106, for example.

The saturation coefficient calculation circuit 108 receives the image data of each block, and obtains the saturation coefficient Cs for each block. The saturation coefficient Cs is a coefficient for correcting block reliability described later, and is determined according to the number of saturated pixels in the block. A saturated pixel is a pixel in which one or more components of the pixel value (for example, R, G, or B) have reached the upper limit value of that component (for example, 255 in 8-bit representation). The saturation coefficient of block i is denoted by Csi. The saturation coefficient calculation circuit 108 first counts the number of saturated pixels in the block, and obtains a saturation coefficient from the count result. The saturation coefficient Cs is obtained using a function prepared in advance. FIG. 5 is a graph of an example of this function. As shown in FIG. 5, the circuit 108 uses a function such that the larger the number of saturated pixels, the smaller the corresponding saturation coefficient. The function used here is, for example, the circuit 108
Is registered in advance.

The dispersion coefficient and the saturation coefficient obtained by the dispersion coefficient calculation circuit 106 and the saturation coefficient calculation circuit 108 are input to the reliability correction circuit 112.

When the representative color calculation circuit 104 obtains the luminance / color difference expression (L, u, v) of the representative color of the block, the block reliability calculation circuit 110 next calculates the luminance for each light source assumed in advance. The reliability of the light source illuminating the block (referred to as block reliability) is determined (S14). The block reliability of a certain block is obtained for each assumed light source. Hereinafter, the process of S14 will be described in detail.

In this embodiment, it is assumed that sunlight, tungsten light (for example, incandescent light), and fluorescent light are used as light sources. These are common illumination light sources for photographing and video photographing by general users. In the prior art disclosed in Japanese Patent Application Laid-Open No. 8-289314 described above, sunlight and tungsten light are handled collectively, but in the present embodiment, they are handled separately. This enables white processing that better reflects the difference between the spectra of sunlight and tungsten light.

Further, in the present embodiment, the sunlight is divided into daylight (daylight in the daytime) and shade light (daylight in the daylight). The difference between the sun and the shade is that the former has a relatively low color temperature (reddish) and the latter has a relatively high color temperature (bluish), even under the same sunlight. Therefore, if both are handled together, there is a possibility that the sun balance and the shade may have an incomplete white balance, and furthermore, a flexible white balance can be obtained in a scene where the sun shade and the shade are mixed. Control becomes difficult. Therefore, in the present embodiment, such a problem is solved by treating daylight and shade light separately as different light sources. After all, in this embodiment, four types of light sources of daylight, shade light, tungsten light, and fluorescent light are assumed.

The block reliability calculation circuit 110 calculates, for each of these assumed light sources, a color difference component of a color of a white object under the light source (hereinafter referred to as a typical color difference of the light source) and a color difference component of a representative color of the block. The distance D from the component (u, v) is obtained. Here, the identification number of the light source is j (j = 1, 2,...,
m: m is the number of assumed light sources. In the present embodiment, m = 4), and the light source with the identification number j is expressed as a light source j. here,
For convenience, the light source 1 is daylight, the light source 2 is shade light, the light source 3 is tungsten light, and the light source 4 is fluorescent light. The color difference component (ui, vi) of the representative color of the block i and the typical color difference (Uj,
The distance Dij to Vj) is obtained, for example, according to the following equation.

[0030]

Dij = d11 * (ui−Uj) ² + d12 * (ui−Uj) * (vi−Vj) + d22 * (vi−Vj) ² (2) where d11, d12 and d22 are Each is a predetermined constant. For each light source, these constants d1
By selecting a set of 1, d12 and d22, an appropriate distance suitable for the light source characteristics can be defined. In this case, the values of these constants are obtained in advance by experiments or the like for each light source, and are stored in the block reliability calculation circuit 110 or a storage device (such as a ROM, not shown) provided in the camera. In addition, it is also possible to use something other than the above equation (2) for the definition of the distance. In this case, it is also possible to use a different distance definition formula according to the characteristics for each light source.

The above calculation of the distance D is performed for one block i.
For all assumed light sources j.

Next, the block reliability calculation circuit 110 obtains the reliability Rdij that the block i is illuminated by the light source j from the obtained distance Dij. The reliability Rdij is a reliability from the viewpoint of color difference. The reliability Rdij is obtained from the distance Dij using a predetermined function f. Since the relationship between the distance D and the reliability Rd is different for each type of light source, it is preferable that the function f be prepared individually for each light source j. If the reliability function for the light source j is represented by fj, the reliability Rdij at which the block i is illuminated by the light source j is obtained by the following equation.

[0033]

Rdij = fj (Dij) (3) FIG. 6 shows an example of the reliability function fj. As shown in FIG. 6, the function fj is a function whose value decreases as the distance Dij increases. By making this function a function that changes continuously with respect to distance, it is possible to avoid the problems of the related art based on the binary discrimination result. The reliability function fj for each light source is obtained in advance by an experiment or the like and stored in the apparatus.

The block reliability calculation circuit 110 calculates
The reliability Rd from the viewpoint of color difference obtained in this way is modified from the viewpoint of luminance. This is because the brightness distribution of a white object under fluorescent light, the brightness distribution of a white object under direct sunlight (daylight), the brightness distribution of a white object in the shade, etc. are different. is there. Therefore,
According to the brightness (brightness) Li of the block i, the block i
May be illuminated by the light source j. Here, using this possibility as a coefficient, the reliability Rd based on the previously obtained color difference is corrected.

For this reason, the block reliability calculation circuit 110
Calculates a luminance coefficient Clij indicating the possibility that the block i is illuminated by the light source j from the viewpoint of luminance for each block i. The luminance coefficient Clij is obtained by applying the luminance Li of the block i to a function gj predetermined for each light source j. That is,

## EQU4 ## Clij = gj (Li) (4) FIG. 7 shows an example of the function gj. The function gj is a function of the luminance Li. In this example, the coefficient Cl increases as the luminance increases. This takes into account that the brighter the object, the higher the possibility of being affected by the light source. By making this function a function that changes continuously with respect to luminance, the effect of block luminance Li on white balance can be made smooth. This helps stabilize the behavior of white balance. In addition,
The characteristic of the luminance coefficient differs for each light source j. The luminance coefficient function gj for each light source is obtained in advance by an experiment or the like and stored in the apparatus.

Then, the block reliability calculation circuit 110
Calculates the reliability Rij by multiplying the reliability Rd based on the color difference by the luminance coefficient Cl. The reliability Rij is a value indicating the possibility that the block i is illuminated by the light source j from the viewpoint of both the color difference and the luminance. That is, the reliability Rij is

Rij = Clij * Rdij (3)

The block reliability calculation circuit 110 executes the above processing for all the blocks i in the image for each light source j. Information on the obtained reliability Rij is input to the reliability correction circuit 112.

After the reliability Rij is estimated, the reliability correction circuit 112 corrects the reliability (S
16). In this circuit 112, the reliability Rij of the light source j with respect to the block i is determined by the above-described dispersion coefficient Cvi and saturation coefficient Csi for the block i and the position coefficient Cpi determined by the position of the block i in the entire image. Correct the reliability Rij. Specifically, the reliability is corrected by multiplying these coefficients by the reliability Rij. Assuming that the modified reliability (referred to as modified reliability) Rmij,

Rmij = Cvi * Cli * Cpi * Rij (6)

Here, the position coefficient Cpi is determined in advance so that the value increases as the block i is closer to the center of the image. This is because it is generally considered that the closer to the center of the image, the higher the importance in the image. By giving a high position coefficient to a block having a high importance in terms of position, the color of the block having a high importance is strongly reflected by the white balance control. FIG.
FIG. 4 is a diagram showing an example of setting a position coefficient. In this example, the block 210-1 at the center of the image has a position coefficient of 1.
0 and 0.5 are given as position coefficients to the block 210-2 at the periphery of the image. These position coefficients are stored in the apparatus in advance.

In this correction process, the variance coefficient Cv
i and saturation coefficient Csi are also taken into account. This has the following significance.

First, the variance coefficient Cvi is, as described above,
It is determined that the larger the variance of the pixel values in the block i, the smaller the value. Therefore, as the variance of the block i increases, the variance coefficient Cvi decreases, and as a result, the value of the modified reliability Rmij also decreases. The reason for such correction is that the greater the variance of the pixel values of the block i, the lower the reliability of the reliability Rij itself obtained from the representative colors (Li, ui, vi) of the block. . The larger the variance is, the larger the variation of each pixel value of the block is. Therefore, it is considered that the possibility that the representative color obtained from the pixel value indicates the color tendency of the block (that is, the tendency of illumination) is low. Therefore, the reliability Rij obtained from such a low-reliability representative color is considered to be low in reliability. In order to reduce the influence of the reliability Rij on the white balance, the dispersion coefficient Cvi
The value of.

The saturation coefficient Csi is, as described above,
The value decreases as the number of saturated pixels in the block i increases. Therefore, as the number of saturated pixels in the block i increases, the saturation coefficient Csi decreases, and as a result, the reliability Rij is corrected to a small value. This correction is performed because the reliability of the reliability Rij itself obtained from the representative color (Li, ui, vi) of the block is considered to be lower as the number of saturated pixels of the block i is larger. Since there is a possibility that a portion of the saturated pixel exceeding the upper limit value of each of the RGB components has been cut, the saturated pixel may not represent the correct color of the object. Therefore, it is considered that the more saturated pixels, the lower the possibility that the representative color of the block indicates the tendency of the color of the block (and thus the tendency of illumination). Since the reliability Rij obtained from such a low-reliability representative color is considered to be low in its own right, here, in order to reduce the influence of the reliability Rij on the white balance, the saturation coefficient Csi Is made smaller.

In S16, the above-described reliability correction processing is executed for each block i and for each light source j. The obtained corrected reliability Rmij is input to the light source contribution calculation circuit 114 and the overall reliability calculation circuit 116.

When the modified reliability Rmij is obtained for all blocks, the reliability of each light source j for the entire scene of the image and the contribution component of each light source j to the entire image are estimated (S18). .

The reliability (referred to as overall reliability) Rtj of the light source j for the entire image is calculated by the overall reliability calculation circuit 116. The circuit 116 calculates the modified reliability R of each block i.
The total reliability Rtij is determined by summing mij for all blocks in the image. That is,

Rtj = ΣRmij (where Σ is the sum of i = 1 to n) (7)

The light source contribution component is calculated by a light source contribution calculation circuit 116.
Required by The light source contributing component of the light source j is the influence of each light source j on the hue of the entire image. In other words, when the light source j illuminates the scene of the image, it appears in the image. Estimated color trend. The light source contribution component is represented by a set of luminance and color difference. Here, the contribution component of the light source j to the entire image is represented as (Lcj, ucj, vcj).

The light source contribution calculating circuit 116 calculates the light source in accordance with the following equation based on the modified reliability Rmij for each light source for each block, the representative color (Li, ui, vi) of each block, and the overall reliability Rtj for each light source. Estimate the contribution component.

[0048]

Lcj = (ΣRmij · Li) / Rtj ucj = (ΣRmij · ui) / Rtj vcj = (ΣRmij · vi) / Rtj (8) In equation (8), Σ is i = 1 to n (n Indicates the sum of the total number of blocks. This calculation is performed for each block i
Is equivalent to a process of weighting and averaging the luminance and color difference of all the blocks with the correction reliability Rmij of each block i.

When the contribution component of each light source j to the entire scene of the image is estimated in this way, the light source contribution correction circuit 118 corrects the estimated value (S
20). This modification is roughly divided into two stages.

In the first stage, the contribution component of each light source j is weighted and averaged with a standard color predetermined for each light source j. The standard color of the light source j is a standard color of a scene under the illumination of the light source j (that is, a set of luminance and color difference). , The color difference is calculated. The standard colors (Lsj, usj, vsj) of each light source j are stored in the apparatus in advance. The first-stage correction processing is represented by the following equation.

[0051]

Lmj = wsj · Lcj + (1-wsj) · Lsj umj = wsj · ucj + (1−wsj) · usj vmj = wsj · vcj + (1−wsj) · vsj (9) where (Lmj, umj) , Vmj) is the correction result of the contribution component, and wsj (where 0 ≦ ws ≦ 1) is a predetermined weight. The weight wsj is obtained in advance by an experiment or the like for each light source j and stored in the apparatus.

The correction in the first stage is a light source contribution component (Lc
j, ucj, vcj) is a correction for reducing the effect of the object color included in the image data. The color appearing in the image is affected by both the color of the illumination light from the light source (illumination color) and the color of the object itself (object color). White balance is to correct the color so that a white object looks white according to the color temperature of the illumination light. Therefore, if the color of the illumination light can be accurately estimated,
Accurate white balance adjustment can be performed. However, most of the actual images are obtained by photographing scenes including many objects other than white, and the colors of the images largely include the effects of the object colors. By performing a weighted average of the light source contribution component and the standard color of the light source j, the influence of the object color included in the light source contribution component can be relatively reduced, and the light source contribution component is set to a value closer to the illumination color. Can be modified.

In the second stage, the correction result (Lmj, umj, vmj) of the first stage is corrected in consideration of the subject luminance. The subject brightness Lo is detected by a subject brightness detection device 30 provided in the camera device. In the second-stage correction, the correction circuit 118 first obtains a correction coefficient Coj for each light source j. The correction coefficient Coj is
It is obtained by applying the subject luminance Lo to the function hj prepared for each. That is,

## EQU10 ## Coj = hj (Lo) (10)

FIG. 9 shows an example of the correction function hj based on the subject luminance. In this example, the correction coefficient Co decreases as the subject luminance Lo increases, and the correction coefficient becomes 0 when the subject luminance exceeds a certain value. This is an example of the function hj when the light source is a fluorescent lamp. When the shooting scene is under fluorescent lighting, the subject is darker than when the scene is outdoors in the daytime. If the subject brightness is very large, the image is likely to be an outdoor scene and less likely to be fluorescent lighting. Therefore, in the case of a fluorescent light source, the correction coefficient Co is set to a smaller value as the subject luminance increases. Also in the case of a scene of tungsten light or shade, the brightness of the subject is considered to be lower than that of the daylight scene, so a function hj is used such that when the brightness of the subject becomes higher than a certain level, the correction coefficient Co decreases. These correction functions hj are obtained in advance by experiments or the like and stored in the apparatus.

Then, the light source contribution correction circuit 118 multiplies the correction coefficient Co thus obtained by the correction result (Lmj, umj, vmj) of the first stage to obtain the final correction result (Lzj, uzj, vzj). That is,

Lzj = Coj · Lmj uzj = Coj · umj vzj = Coj · vmj (11) The correction result (Lzj, uzj, vzj) is a light source contribution component corrected in consideration of parameters such as subject brightness. This correction result is input to the illumination color estimation circuit 120.

The illumination color estimation circuit 120 calculates the correction results (Lzj, uzj, vzj) of the light source contribution components of each light source j input from the light source contribution correction circuit 118 and the entire scene obtained by the overall reliability calculation circuit 116. Is estimated based on the overall reliability Rtj of each light source j with respect to (S22). This estimation is based on the correction result (Lzj, uzj, vz) of the light source contribution component.
j) is performed by performing weighted averaging for all assumed light sources using the overall reliability Rtj of each light source j as a weight. That is, if the illumination color is (IL, Iu, Iv),

Wj = Rtj / (ΣRtj) IL = Σ (Wj * Lzj) Iu = Σ (Wj * uzj) Iv = Σ (Wj * vzj) (12) In equation (12), Σ represents all assumed light sources This is the sum of j.

That is, in S22, the illumination color of the composite illumination is estimated by the weighted average under the assumption that the scene is illuminated compositely by the assumed light source j. This illumination color corresponds to the color of a white object under the composite illumination. The obtained illumination colors (IL, Iu, Iv)
It is input to the white balance gain calculation circuit 122.

White balance gain calculation circuit 122
Is based on the information of the received illumination colors (IL, Iu, Iv) based on the gain (Rgain,
Ggain, Bgain) are calculated (S24). This calculation is
This is performed based on the following equation.

[0059]

(Equation 13)

Imax = max (IR, IG, IB) (14)

Rgain = IMax / IR Ggain = IMax / IG Bgain = IMax / IB (15) (IR, IG, IB) is an RGB representation of the illumination color.
The required white balance gain (Rgain, Ggain,
Bgain) is a value that corrects the color (ie, (IR, IG, IB) itself) when the illumination of this color is reflected by a white object to gray (ie, R = G = B). The obtained white balance gain is input to the white balance adjustment circuit 124.

The white balance adjustment circuit 124 multiplies each of the pixel values R, G, and B of the image input from the imaging device 20 by the gains Rgain, Ggain, and Bgain obtained by the white balance gain calculation circuit 122, respectively. , Adjust the white balance of the image (S2
6). Therefore, from the output terminal 126 of the auto white balance device 10,

Rout = Rgain * R Gout = Ggain * G Bout = Bgain * B (16) The output (Rout, Gout, Bout) obtained by (16) is output.

The preferred embodiment of the present invention has been described above. As described above, in the present embodiment, the reliability of each block being illuminated by each light source is estimated based on the distance between the representative color of each block and the typical color difference of each light source, and based on the reliability. Perform white balance adjustment. Since the reliability is obtained from a continuous function with respect to the distance, it is compared with the prior art in which white balance adjustment is performed based on a binary discrimination result of whether each block is included in a range corresponding to each light source. The effect of stabilizing the behavior of white balance control is obtained.

In this embodiment, the reliability based on the color difference is corrected in accordance with the luminance of the representative color of the block, so that the white balance control incorporating knowledge on the influence of each assumed light source on the luminance of the scene is performed. Becomes possible. Since the correction according to the luminance is performed according to a continuous function relating to the luminance, there is an advantage that the white balance control is smoothly performed.

Further, in this embodiment, since a mechanism is provided for correcting so that the reliability decreases as the variance of the pixel values in the block increases, the variation of the pixel values is large (that is, the color of the illumination light is expressed. It is possible to reduce the influence of the blocks (which are unlikely to be present) on the white balance control.

In the present embodiment, a mechanism is provided for correcting the reliability so that the reliability decreases as the number of saturated pixels in the block increases. Therefore, the number of saturated pixels is large (that is, the pixel value does not represent a correct color). It is possible to reduce the influence of (possibly) blocks on white balance control.

Further, in the present embodiment, a mechanism for adjusting the white balance according to the brightness of the subject is provided by using the knowledge on the relationship between the light source and the brightness of the subject, so that more appropriate white balance control can be performed.

In the above embodiment, the reliability obtained from the color difference of the representative color of the block is used to calculate the luminance of the representative color, the variance of the pixel values in the block, the number of saturated pixels in the block, and the image of the block. However, even if the white balance control is performed using the reliability that does not perform such correction at all, a certain improvement effect can be obtained with respect to the related art. The degree of improvement is improved by correcting each of the above items with respect to the reliability based on the color difference. Then, the more the items are corrected, the higher the accuracy of the white balance control becomes.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration of an automatic white balance device according to an embodiment.

FIG. 2 is a diagram showing a processing procedure of the automatic white balance device of the embodiment.

FIG. 3 is a diagram illustrating an example of block division.

FIG. 4 is a diagram illustrating an example of a function for obtaining a variance coefficient.

FIG. 5 is a diagram showing an example of a function for obtaining a saturation coefficient.

FIG. 6 is a diagram illustrating an example of a function for obtaining block reliability.

FIG. 7 is a diagram illustrating an example of a function for obtaining a luminance coefficient.

FIG. 8 is a diagram illustrating a setting example of a position coefficient.

FIG. 9 is a diagram illustrating an example of a function for calculating a correction coefficient based on subject brightness.

[Explanation of symbols]

10 auto white balance devices, 20 imaging devices,
Reference Signs List 30 object luminance detection device, 102 block division circuit, 104 representative color calculation circuit, 106 dispersion coefficient calculation circuit, 108 saturation coefficient calculation circuit, 110 block reliability calculation circuit, 112 reliability correction circuit, 114 light source contribution calculation circuit, 116 overall Reliability calculation circuit, 118 light source contribution correction circuit, 120 illumination color estimation circuit, 122 white balance gain calculation circuit, 124 white balance adjustment circuit.

Claims (18)

[Claims] 1. A block dividing means for dividing an image to be subjected to white balance processing into a plurality of blocks, and a representative color calculating means for obtaining, for each of the divided blocks, a representative color of the block based on a pixel value in the block. And for each block and for each assumed light source, determine the distance between the color difference component of the color of the white object under the light source and the color difference component of the representative color of the block, and based on each of these distances, Block reliability estimating means for estimating the reliability of illuminating the scene of each block, for each light source, by weighted averaging the representative color of each block according to the reliability of the light source for the block, A light source contribution component estimating means for estimating a contribution component of the light source to the entire image; and a signal for each block of the light source for each of the light sources. Total reliability over all blocks in the image, thereby estimating the reliability of the light source for the entire image, and an overall image of each light source estimated by the light source contribution component estimating means. An illumination color estimating means for estimating an illumination color for illuminating a scene of the entire image by performing a weighted average according to the reliability on the entire image of each light source estimated by the overall reliability estimating means. An auto white balance apparatus comprising: a white balance unit that performs white balance processing on each pixel of the image so as to cancel the estimated illumination color. 2. The apparatus according to claim 1, wherein the block reliability estimating unit estimates the reliability using a continuous function that monotonically decreases as the distance increases. . 3. The apparatus according to claim 2, wherein said continuous function used to determine said reliability is prepared separately for each of said light sources. 4. The apparatus according to claim 1, further comprising: for each of the blocks, the block reliability estimating means estimates a coefficient determined according to a position of the block in the image. An apparatus comprising means for modifying the reliability. 5. The apparatus according to claim 1, wherein for each of the light sources, the reliability of each block estimated by the block reliability estimating means is determined according to a luminance component of a representative color of the block. Device having means for correcting by means of 6. The apparatus according to claim 5, wherein the means for correcting the reliability of each of the blocks performs correction using a continuous function individually prepared for each of the light sources. Equipment to do. 7. The apparatus according to claim 1, further comprising: means for obtaining a variance of pixel values of pixels sampled from the block; and calculating the reliability estimated by the block reliability estimating means using the pixel value. Means for modifying according to a predetermined function, the value of which decreases as the variance of increases. 8. The apparatus according to claim 1, further comprising: a means for counting the number of saturated pixels included in the block; and a reliability estimated by the block reliability estimation means, Means for modifying according to a predetermined function in which the larger the number, the smaller the value. 9. An apparatus according to claim 1, wherein said light source estimated by said light source contributing component estimating means according to the detected brightness of the subject, and subject brightness detecting means for detecting the brightness of the subject. Correction means for correcting the contribution component of (i), wherein the illumination color estimation means estimates the illumination color based on a result of the correction of the contribution component by the correction means. 10. The apparatus according to claim 1, wherein, for each light source, the contribution component of each light source estimated by the light source contribution component estimating means is a standard color prepared in advance for the light source. The illumination color estimating means estimates the illumination color based on a result of the correction of the contribution component by the second correcting means. 11. The apparatus according to claim 1, wherein each of the assumed light sources includes daytime sunlight and daytime shade. 12. A representative color calculating means for obtaining a representative color of an image to be subjected to white balance processing from each pixel value of the image, a color difference component of a color of a white object under the light source for each assumed light source, and Calculating a distance between the color difference component of the representative color of the image and a reliability estimating means for estimating the reliability of illuminating the scene of the image based on each of the distances; Illumination color estimating means for estimating the color of illumination for illuminating the scene of the image based on the reliability of each light source; and white balance performing white balance processing on each pixel of the image so as to cancel the illumination color. Means, and an auto white balance device comprising: 13. The apparatus according to claim 12, further comprising: a means for correcting, for each of the light sources, the reliability estimated by the reliability estimation means in accordance with a luminance component of a representative color of the image. The illumination color estimating means estimates an illumination color using the corrected reliability. 14. The apparatus according to claim 13, wherein the means for correcting the reliability performs the correction using a continuous function individually prepared for each of the light sources. 15. A method for performing automatic white balance adjustment on a captured image, comprising: (a) dividing an image to be subjected to white balance processing into a plurality of blocks; Calculating a representative color of the block based on pixel values in the block, and (c) for each of the blocks and each assumed light source,
Obtain the distance between the color difference component of the color of the white object under the light source and the color difference component of the representative color of the block, and estimate the reliability that each light source illuminates the scene of each block based on these distances. (D) For each of the light sources, a representative color of each block is weighted average according to the reliability of the light source for the block to estimate a contribution component of the light source to the entire image; For each light source, the reliability of the light source for each block is summed over all blocks in the image to estimate the reliability of the light source for the entire image. (F) An illumination color for illuminating a scene of the entire image by performing a weighted average of the contribution components to the entire image according to the estimated reliability of each light source for the entire image. Estimated, (g) estimated to cancel the illumination color, the white balance processing for each pixel of the image, wherein the. 16. The method of claim 15, wherein (c1) for each of the light sources, the step (c).
The reliability of each block estimated in the above,
A method according to claim 1, wherein correction is performed in accordance with a luminance component of a representative color of the block, and in each of the steps after step (d), processing is performed based on the reliability corrected in step (c1). 17. The method according to claim 16, wherein in the step (c1), the correction is performed using a continuous function prepared individually for each of the light sources. 18. The method of claim 16, wherein each of the assumed light sources includes daytime sunlight and daytime shade.