JP2013161348A - Image correction device, method, program, and recording medium - Google Patents

Abstract

The present invention corrects a shine area without losing the stereoscopic effect while maintaining the original brightness balance.
A face detection unit detects a face image region from an input image. The shine area detection unit detects an area brighter than a predetermined brightness in the face image area as a shine area. The average color acquisition unit 18 acquires an average color obtained by averaging the skin colors around the shine area. The brightness classification unit 19 detects the brightness of all the pixels in the shine area, and classifies each pixel into a plurality of groups according to the brightness. The brightest pixel detection unit 21 acquires the color of the brightest pixel among the pixels not corrected in the shine area. The correction color determination unit 22 creates a correction color by changing the synthesis ratio of the average color and the color of the brightest pixel in the shine area according to the number of corrections. The fill correction unit 23 paints pixels in order from the pixels belonging to the bright group with the created correction color.
[Selection] Figure 2

Description

  The present invention relates to an image correction apparatus, a method, a program, and a recording medium for correcting the shine of a face image.

  Conventionally, the occurrence of so-called “shine” has been a problem in an image in which a person is imaged. The term “shine” means that a part of the skin is imaged white because light such as a strobe emitted at the time of taking a picture is reflected (particularly specularly reflected) on the surface of the skin of the subject person. For this reason, a site where shine occurs (hereinafter referred to as “shine site”) is imaged with higher brightness than the surrounding skin. Such shine gives an unpleasant impression to those who see the image. For this reason, there is a demand for a technique for removing or reducing shine in an image where shine has occurred.

  In response to such demands, conventionally, a correction method has been realized that suppresses shine by manually reducing the brightness and brightness of high-brightness parts by manually finding high-brightness parts and using photo retouching software. Has been.

  Further, an invention is proposed in which a face image area including the eyes of a subject person is detected from an image, a shine area is detected from the detected face image area, and correction is performed to suppress the maximum brightness in the shine area. (Patent Document 1).

JP 2007-190885 A

  However, in the invention described in Patent Document 1, image data in the shine area is simply smoothed as correction for suppressing the maximum brightness. For this reason, the stereoscopic effect of the shine area may be impaired, and the balance of brightness (luminance) of the shine area may be changed.

  The present invention has been made in view of the above points, and is an image correction apparatus, method, program, and recording that maintain the brightness balance of the shine as in the original image without correcting the stereoscopic effect. The purpose is to provide a medium.

  In the present invention, face detection means for detecting a face image area from an input image, and a shine area detection for detecting, as a shine area, an area brighter than a predetermined brightness in the face image area or an area brighter than a surrounding by a predetermined brightness. And a shine correction means for correcting the shine area so as to gradually approach the surrounding color from the brightest area toward the periphery.

  As a shine correction means, the correction color that gradually fills the shine area from the brightest area to the periphery is averaged over the color of the pixels that are the target of filling in the shine area. Create by gradually increasing the ratio of mixing average colors.

  Specifically, an average color acquisition unit that acquires an average color obtained by averaging the colors of pixels around the shine area, a classification unit that classifies the shine area into a plurality of groups according to pixel brightness, and the average color A correction unit that creates a correction color by combining the average pixel color with the color of the pixel of the group to be filled and gradually increases the average color composition ratio. It is preferable to constitute the correcting means.

  Further, as the shine correction means, an average color acquisition unit that acquires an average color obtained by averaging the colors of pixels around the shine area, a brightest pixel detection unit that detects the brightest pixel in the shine area, and the average A correction color is created by combining a color and the color of a pixel to be filled so that the pixel having a shorter distance from the surrounding area gradually increases the composite ratio of the average color. You may comprise by the correction | amendment part which paints a pixel in order toward a surrounding pixel from a pixel.

  In the present invention, an image correction method may be used. Furthermore, the present invention may be a computer-readable program for executing the image correction method and a recording medium on which the computer-readable program is recorded.

  According to the present invention, the shining area is filled from the brightest area toward the surroundings so as to gradually approach the surrounding colors, so that the shining area is corrected without losing the stereoscopic effect while maintaining the original brightness balance. can do.

It is a block diagram which shows the outline of the image correction apparatus of this invention. It is a block diagram which shows the outline of a shine area correction | amendment part. 3 is a flowchart illustrating a processing procedure of the image correction apparatus described in FIG. 1. It is a histogram which shows the some group classified according to the brightness. It is explanatory drawing which shows the area | region of several groups among a shine area and a shine area. It is explanatory drawing which shows an example of the synthetic | combination ratio with respect to the frequency | count of correction | amendment when the number of classification is "5". It is a flowchart which shows the process sequence of the other example which changes a composition ratio according to the distance to the circumference | surroundings of a shine area | region with respect to the pixel to fill. It is explanatory drawing which shows the outline of the shortest distance of the pixel adjacent to the brightest pixel of a shine area | region, and the surrounding pixel of a shine area | region. It is explanatory drawing which shows an example of the synthetic | combination ratio with the apparatus demonstrated in FIG.

  As is well known, the image correction apparatus 10 of the present invention includes a CPU, a ROM, a RAM, a storage unit, an operation unit, an external I / F, and the like connected via a bus. Various programs (OS, applications, etc.) are stored in the storage unit, loaded into the CPU in response to the start operation from the operation unit, and executed by the CPU, as shown in FIG. It functions as an apparatus including an input unit 11, a face detection unit 12, an eye / mouth detection unit 13, a shine area detection unit 14, a shine area correction unit 15, a corrected image output unit 16, and the like.

  The face detection unit 12, the eye / mouth detection unit 13, the shine area detection unit 14, and the shine area correction unit 15 are realized by executing a program by the CPU. The face detection unit 12, the eye / mouth detection unit 13, the shine area detection unit 14, and the shine area correction unit 15 may be stored in a dedicated chip.

  The image input unit 11 functions as an interface for inputting original image data to the image correction apparatus 10. For example, data of an original image recorded on a recording medium (for example, various memories including those incorporated in an electronic camera, a flexible disk, a CD (Compact Disk), a DVD (Digital Versatile Disc, Digital Video Disc)) is converted into the image correction apparatus 10. Is input. In this case, the image input unit 11 is configured using a device (for example, a memory reader, a flexible disk, a CD drive, and a DVD drive) that reads data from a recording medium.

  The face detection unit 12 detects an area including a human face image from the original image as a face image area, and specifies face image information indicating the position and size of the face image area. The face detection unit 12 is configured to detect a face by template matching using a reference template corresponding to the outline of the entire face, for example.

  The eye / mouth detection unit 13 detects an area including an eye and mouth image area from the face image area as an eye and mouth image area, and identifies eye and mouth image information indicating the position and size of the eye and mouth image area. To do.

  The shine area detection unit 14 detects, as a shine area, a remarkably bright portion of the face image area excluding the eye and mouth image areas. As a detection of a remarkably bright part, for example, an area that appears brighter than a predetermined brightness may be detected as a shine area. The predetermined brightness may be a fixed value set in advance, or may be set on the basis of the brightness of the white eye part or the tooth part. In addition, an area that is brighter than a surrounding area by a predetermined brightness or more may be detected as a shine area.

  The shining area correction unit 15 corrects the shining area in order from light pixels to a color that gradually approaches the skin color around the shining area. The corrected image data is output from the corrected image output unit 16. Here, in the case of image data read from a recording unit or the like, it is overwritten or saved with a new name.

  As shown in FIG. 2, the shine area correction unit 15 includes an average color acquisition unit 18, a brightness classification unit 19, and a correction unit 20. The average color acquisition unit 18 acquires an average color a obtained by averaging the skin colors around the shine area.

  The brightness classification unit 19 detects the brightness (luminance) of all pixels in the shine area, and classifies each pixel into a plurality of groups according to the brightness. The degree of brightness is expressed as a numerical value for the gradation that can be expressed by each pixel. In the case of 8-bit 256 gradation, the minimum value of the brightness of each pixel is “0” and the maximum value is “255”. The color image has 0 to 255 values for each of the R, G, and B color components. The average value of the RGB values is the brightness at each pixel.

  The correction unit 20 includes a brightest pixel detection unit 21, a correction color determination unit 22, and a fill correction unit 23, and a correction color c determined according to the number of corrections in order from a bright pixel in the shine area. Fill with. The brightest pixel detection unit 21 acquires the color b of the brightest pixel among the pixels not corrected in the shine area. The correction color determination unit 22 creates a correction color c by changing the composition ratio of the average color a and the color b of the brightest pixel in the shine area according to the number of corrections i. The composition ratio is determined based on the formula shown in [Formula 1].

[Equation 1]
c = a · (i + 1) / 2 · x + b · (i−1) / 2 · x

  “C” described in [Equation 1] is a correction color (filling color), “a” is an average color obtained by averaging skin colors around the shine area, and “b” is not corrected in the shine area. The color of the brightest pixel among the pixels, “x” is the number of classifications, and “i” is the number of corrections.

  The equation described in [Equation 1] is an equation for making the correction color c by mixing the average color a little by little with the color b of the brightest pixel at that time according to the number of corrections i. The fill correction unit 23 fills with the correction color c in order from the pixel group classified by the bright group.

  The classification number x is the same as the maximum number of corrections i. When the correction is performed for the first time, “0” is substituted for the number of corrections i. Here, the color of the brightest pixel in the shining area to be corrected first is “b0”, the correction color c to be filled (corrected) first is “c0”, and the correction in the shining area to be corrected next is not corrected. Assuming that the color of the brightest pixel is “b1” and the correction color to be filled is “c1”, the correction color c1 is the second correction (the number of corrections i is calculated as “1”). Since the average color a is slightly mixed with c0, and the bright pixel color b1 is also slightly darker than the first bright pixel color b0, it approaches the average color a of the surrounding pixels of the shine area. Become a color. By repeating such operations for the number of classifications x, the shine area can be corrected without changing the direction of light and the balance of the shine.

  Next, the operation of the above configuration will be described with reference to FIG. In the image data input by the image input unit 11, a face image region is detected by the face detection unit 12 (S-1). When the face image area is detected, the eye / mouth detection unit 13 detects the eye / mouth image area from within the face image area (S-2).

  When the eye / mouth image area is detected in the face image area, the shine area detection unit 14 determines an area that is brighter than a predetermined brightness in the area other than the eye / mouth image area in the face image area. (S-3). When the eye / mouth image area is not detected, an area that is brighter than a predetermined brightness in the face image area is detected as a shine area (S-4).

  In the correction of the shine area, first, the average color acquisition unit 18 determines the color of the skin around (peripheral area) around the shine area, that is, the color of the pixel 26 adjacent to the boundary of the shine area 25 as shown in FIG. The average color a is acquired (S-5).

  Thereafter, the brightness classification unit 19 examines the brightness of all the pixels in the shine area 25 and temporarily stores it in a storage unit such as a memory. Then, the brightness of all the pixels in the shine area is classified into a plurality of groups based on a predetermined threshold value (S-6). Further, the brightness of the acquired pixel and the pixel acquired immediately before are compared in order, and the color of the brightest pixel is detected (S-7).

  For example, in the case of five classifications, all pixels in the shine area 25 are classified into five groups 27 to 31. The first to fifth groups 27 to 31 are classified according to the brightness range on the horizontal axis shown in FIG. Note that the vertical axis of the graph shown in FIG. 5 indicates the number of pixels. The pixel groups belonging to the first to fifth groups 27 to 31 shown in the histogram of FIG. 5 correspond to the first to fifth regions 27 to 31 in the glossy region 25 shown in FIG.

  The correction unit 20 creates a correction color c by changing the composition ratio of the average color a and the color b of the brightest pixel (the pixel A of the first area 27) according to the number of corrections i (S-8). The pixel A (pixels in the first region 27) is filled with the corrected color c. The correction color c is determined based on the formula shown in [Formula 1].

  For example, as described above, when the classification number x is “5”, the correction color c0 to be filled in by the first correction is obtained based on the formula [Equation 1]. Since this is the first correction, if “0” is substituted for the number of corrections i, and the color b of the brightest pixel (the pixel in the first area 27) in the shine area 25 is “b0”, the correction is performed to fill first. The color c0 is c0 = [1/2 · a + 1/2 · (b0)]. That is, the correction color c0 to be filled in at the first correction is created by combining the average color a and the color b0 of the brightest pixel in half. Therefore, in the first correction, the first area 27 shown in FIG. 4 is filled with the correction color c0 (S-9).

  Thereafter, when there is a pixel that has not been corrected in the shine area 25 (S-10), the correction unit 20 determines the brightest pixel (pixels in the second area 28) among the pixels that have not been corrected in the shine area 25. The color b is acquired as “b1”. Then, using [Equation 1] and substituting “1” for the number of corrections i (S-11), the correction color c1 to be painted second time is C1 = [3/5 · a + 2/5 · (b1)] It becomes. That is, the correction color c1 created by the second correction is obtained by slightly reducing the ratio of the color b1 of the next bright pixel (the pixel in the second region 28) to the correction color c0 created first, and the average color. It is made with a mixing ratio with the ratio of a slightly increased. As a result, the next brightest pixel group (second region 28) shown in FIG. 4 is filled with the created correction color c1.

  In this way, by repeating the above-described operation for the number of classifications x, pixels (groups) classified according to the brightness have a high brightness with the correction color c in which the composition ratio is changed according to the number of corrections. Paint sequentially from the pixel group. Here, the composition ratio of the correction colors c0 to c4 calculated from the equation [Equation 1] according to the number of corrections i is shown in FIG.

  Note that a plurality of shine areas may be detected for the face image area. With respect to the detected shine area, the shine area correction unit 15 sequentially performs the shine correction (S-12).

  In this way, the correction color c is created by gradually increasing the average color a as the number of corrections i increases, and conversely reducing the composition of the color b of the brightest pixel at that time. The shine area 25 is a correction color that gradually approaches the average color a from the bright area 27 toward the dark area 31 and is filled step by step for one pixel. As a result, correction is made with a natural gradation that fits around the shine area without impairing the stereoscopic effect.

  In the shine correction of the above-described embodiment, a plurality of groups in which the shine area is classified according to the brightness of the pixels are filled with the correction color c so as to gradually approach the above-described average color a in order from the brightest pixel group. Although the correction is performed, the present invention is not limited to this, and the average color and the color of the pixel to be filled are determined by combining the average color a with the interval between the brightest pixel and the surrounding pixels of the shine area. It is also possible to paint in order from the brightest pixel in the correction color created by changing the.

  For example, a pixel in a region where the distance between the brightest pixel and the periphery of the shine area is short is filled with a correction color created with a larger amount of the average color a mixed by one correction with respect to the reference correction color. On the contrary, as the pixel is located in the region with the longer distance, the correction color is gradually filled with the correction color created by reducing the amount of mixing the average color a by one correction with respect to the reference correction color. For this reason, even if the brightest pixel area exists at a position biased in the shine area, in the area where the distance between the brightest pixel area and the surrounding area is short, the reference correction color is shifted toward the average color a. A continuous change in color is corrected with a fast gradation, and in a long interval, a continuous change in color is corrected with a slow gradation from the reference correction color toward the average color a. Therefore, it is possible to correct the shine area so that a natural gradation is obtained according to the interval between the brightest pixel area and the surrounding area.

  In this case, the correction unit determines the correction color d for painting the pixel according to the distance using the equation described in [Equation 2].

[Equation 2]
d = a · (0.5 + 0.5 / n) + e · (0.5−0.5 / n)
Here, d is the correction color, e is the color of the pixel adjacent to the filled pixel, n is the shortest distance k / “size of one pixel” between the pixel to be filled and the surroundings, that is, the number of pixels (number of dots) is there.

  As shown in FIGS. 7 and 8, the correction unit acquires an average color a obtained by averaging the skin colors around the shine area 40 (S-13), and acquires the brightness of all pixels in the shine area 40. Then, the color w of the brightest pixel f in the shine area 40 is acquired (S-14). Then, the correction color d is created by synthesizing the average color a and the color w of the brightest pixel f at a ratio of 1: 1 (S-15). Then, the brightest pixel f is filled with the created correction color d (S-16). That is, the brightest pixel f (the pixel to be filled first) in the shine area 40 is filled with the reference correction color d created at a composite ratio of 0.5a + 0.5e.

  In the second correction, the color e of the pixel z1 adjacent to the previously filled pixel f is acquired (S-17), the surrounding pixel p1 closest to the pixel z1 is detected, and the interval between them is detected. The shortest distance k1 is calculated (S-18). A correction color d is created by mixing the average color a at a ratio corresponding to the shortest distance k1 with respect to the reference correction color (S-19).

  For example, in the example shown in FIG. 8, the brightest pixel f group 41 is filled with the reference correction color d described above. In the second correction, the color e of the pixel z1 adjacent to the pixel group 41 is acquired, and then the shortest distance k1 between the pixel z1 and the surrounding pixel p1 is calculated. Then, the correction color d1 is calculated by substituting the values of the color e of the pixel z1 and the distance k1 into the equation described in [Equation 2]. Then, the pixel z1 is painted with the calculated correction color d1 (S-20).

  In the n-th correction, the color e of the pixel zn adjacent to the pixel group 41 is acquired, and then the shortest distance kn between the pixel zn and the surrounding pixel pn is calculated. Then, the correction color dn is calculated by substituting the values of the color e and the distance kn of the pixel zn into the equation described in [Equation 2]. Then, the pixel zn is filled with the calculated correction color dn.

  Thereafter, the above-described procedure is repeated, and the pixels adjacent to the previously painted pixel are painted in order to correct all the pixels in the shine area (S-21).

  The average color a is synthesized at a ratio corresponding to the distance k with respect to the reference correction color d. When using the formula [2], as shown in FIG. 9, the correction color d is created by increasing the ratio of the average color a as the distance k between the pixel to be filled and the surrounding pixels becomes shorter. Along with this, it goes without saying that the composition ratio of the color e of the pixel to be corrected is reduced. When n = 1, the color is painted with the average color a.

  Incidentally, there are cases where it is desirable to change the composition ratio of the average color a that forms the correction color d in accordance with the part other than the mouth and eyes. For example, when there is a shine area in the nose area, the brightest area of the reference correction color in which the average color a is mixed at least half is corrected by using the formula described in [Expression 1] or [Expression 2]. In addition, the ratio of the average color a is gradually increased with respect to the reference correction color from this region toward the periphery, and the correction color is created, so that the composite ratio of the average color is large from the beginning. There is an inconvenience that the stereoscopic effect cannot be obtained.

  Therefore, in this case, the brightest area is filled with the second reference correction color obtained by synthesizing the average color a at a ratio of at least half or less, and the second reference correction color is extended from this area toward the periphery. What is necessary is just to paint with the correction color which gradually increased the ratio of the average color.

  Of course, in this case, a determination unit that determines whether or not the shine area is a nose area is provided, and correction is performed when the shine area exists in the nose area. In addition, a correction coefficient determined in advance is corrected by multiplying the portion for obtaining the ratio of the average color a in the above-described equation [Equation 1] or [Equation 2] so that the composition ratio as described above is obtained. May be. Thereby, it can prevent that a nose three-dimensional effect is lost.

DESCRIPTION OF SYMBOLS 10 Image correction apparatus 12 Face detection part 14 Detective area detection part 15 Detective area correction part

Claims (12)

Face detection means for detecting a face image area from the input image;
A shine area detection means for detecting, as a shine area, an area brighter than a predetermined brightness in the face image area or an area brighter than the surrounding area by a predetermined amount;
Shine correction means for correcting the shine area so as to gradually approach the surrounding color from the brightest area toward the periphery; and
An image correction apparatus comprising: The image correction apparatus according to claim 1,
The shine correction means is
A correction color that gradually fills the shine area from the brightest area to the periphery, and an average color obtained by averaging the colors of the pixels around the shine area with respect to the color of the pixel to be filled in the shine area. An image correction apparatus characterized by gradually increasing the mixing ratio. The image correction apparatus according to claim 1 or 2,
The shine correction means is
An average color acquisition unit that acquires an average color obtained by averaging the colors of pixels around the shine area;
A classification unit for classifying the shine area into a plurality of groups according to pixel brightness;
A correction color is created by combining the average color with the color of the pixel of the group to be filled so that the ratio of the average color gradually increases, and the correction is performed by filling the pixels in order from the brightest group of the created correction color. And
An image correction apparatus comprising: The image correction apparatus according to claim 1 or 2,
The shine correction means is
An average color acquisition unit that acquires an average color obtained by averaging the colors of pixels around the shine area;
The brightest pixel detection unit for detecting the brightest pixel in the shine area;
The correction color is created by combining the average color and the color of the pixel to be filled so that the ratio of the average color gradually increases as the distance from the surrounding area becomes shorter. A correction unit that fills pixels in order from the brightest pixel to surrounding pixels;
An image correction apparatus comprising: A face detection step for detecting a face image area from the input image;
A shining area detecting step for detecting, as a shining area, an area brighter than a predetermined brightness in the face image area or an area brighter than the surrounding area by a predetermined amount;
A shine correction step for correcting the shine area so as to gradually approach the surrounding color from the brightest area toward the periphery,
An image correction method comprising: The image correction method according to claim 5,
The shine correction step includes
A correction color that gradually fills the shine area from the brightest area to the periphery, and an average color obtained by averaging the colors of the pixels around the shine area with respect to the color of the pixel to be filled in the shine area. A method for correcting an image, characterized by gradually increasing the mixing ratio. The image correction method according to claim 5 or 6,
The shine correction step includes
An average color acquisition step of acquiring an average color obtained by averaging the colors of pixels around the shine area;
A classification step of classifying the shine area into a plurality of groups according to pixel brightness;
The average color and the colors of the pixels of the group to be filled are synthesized so that the synthesis ratio of the average color gradually increases to create a correction color, and the correction color thus created is filled in order from the brightest group of pixels. A correction step;
An image correction method comprising: The image correction method according to claim 5 or 6,
The shine correction step includes
An average color acquisition step of acquiring an average color obtained by averaging the colors of pixels around the shine area;
A brightest pixel detection step for detecting the brightest pixel in the shine area;
A correction color is created by combining the average color and the color of the pixel to be filled so that a pixel having a shorter distance from the surrounding area gradually increases the average color combination ratio, A correction step of filling the pixels in order from the brightest pixel to the surrounding pixels;
An image correction method comprising: A face detection step for detecting a face image area from the input image;
A shining area detecting step for detecting, as a shining area, an area brighter than a predetermined brightness in the face image area or an area brighter than the surrounding area by a predetermined amount;
A shine correction step for correcting the shine area so as to gradually approach the surrounding color from the brightest area toward the periphery,
A program that allows a computer to execute. A face detection step for detecting a face image area from the input image;
A shining area detecting step for detecting, as a shining area, an area brighter than a predetermined brightness in the face image area or an area brighter than the surrounding area by a predetermined amount;
An average color acquisition step of acquiring an average color obtained by averaging the colors of pixels around the shine area;
A classification step of classifying the shine area into a plurality of groups according to pixel brightness;
The average color and the colors of the pixels of the group to be filled are synthesized so that the synthesis ratio of the average color gradually increases to create a correction color, and the correction color thus created is filled in order from the brightest group of pixels. A correction step;
A program that allows a computer to execute. A face detection step for detecting a face image area from the input image;
A shining area detecting step for detecting, as a shining area, an area brighter than a predetermined brightness in the face image area or an area brighter than the surrounding area by a predetermined amount;
An average color acquisition step of acquiring an average color obtained by averaging the colors of pixels around the shine area;
A brightest pixel detection step for detecting the brightest pixel in the shine area;
A correction color is created by combining the average color and the color of the pixel to be filled so that a pixel having a shorter distance from the surrounding area gradually increases the average color combination ratio, A correction step of filling the pixels in order from the brightest pixel to the surrounding pixels;
A program that allows a computer to execute.   The computer-readable recording medium which recorded the program for performing the program of any one of Claim 9 thru | or 11.

Images