JP2018151766A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve such a natural correction that color drift is not brought about.SOLUTION: A face detection unit (113) detects a face image from a captured image or the like as an image region of a prescribed subject. An image processing unit (105) detects regions having high brightness from the face image being the image region of the subject and further determines saturation levels of the regions with high brightness on the basis of values of a plurality of color signals in the regions having high brightness and performs a correction corresponding to the saturation levels, on regions having high brightness in which at least one of the plurality of color signals is saturated.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing captured images.

  2. Description of the Related Art Conventionally, in an imaging apparatus such as a digital camera, a photographed image with an appropriate exposure is acquired by controlling the amount of incident light from a subject or the like by adjusting an aperture in the camera or adjusting a shutter speed. However, even if, for example, an image of a person photographed with optimal exposure control is taken, there is a case where shininess occurs in the face area due to the influence of an environmental light source or strobe light. With shine, the environment light source or strobe light is reflected on the skin surface of the subject person, the input signal value captured by the imaging device is saturated, color information is lost, and part of the skin is imaged white (whitening (Flying) phenomenon. If there is a part in which the shine is generated in the skin color region and the brightness is locally high, an unpleasant impression is given as a human image. For this reason, there is a demand for a correction technique that reduces saturation by correcting the saturation of an image having a saturated region (region where shine occurs) where the input signal value is saturated.

  In response to such demands, a technique has been proposed in which an input image signal is separated into a luminance signal and a chromaticity signal, and the luminance signal is increased or decreased to adjust the contrast of the image without losing the color balance. Yes. For example, in Patent Document 1, a limit value for a color saturation signal calculated from an input image is applied to visual characteristics of a display device, a gain amount for correcting each color signal is determined, and the gain amount is based on the gain amount. A technique for reducing shine by correcting each color signal is disclosed.

JP 2007-31349 A

  Here, in a general digital camera, for example, R (red), G (green), and B (blue) color signals are output in an analog signal state from an image sensor such as a CCD or a CMOS image sensor. In this case, as the state of the saturation region (the shining region), the full saturation state in which all color signals of the R signal, the G signal, and the B signal are saturated, and the partial saturation in which only some of the color signals are saturated. State. As an example, when the state of the saturation region is a partial saturation state in which, for example, the R signal and the B signal are saturated, the color of the saturation region becomes bright yellow, which is an unnatural state (so-called shift from the original color) Color bending occurs). When the saturation region is in a partially saturated state in this way, color misregistration is not corrected and color distortion remains only by performing color correction processing using the input luminance signal as in the technique described in Patent Document 1 described above. The image becomes unnatural.

  Accordingly, an object of the present invention is to enable natural correction without causing color misregistration.

  The present invention provides a detection means for detecting an image area of a predetermined subject from an input image, an extraction means for extracting a high brightness area from the image area of the subject, and a plurality of color signals for determining the color of the high brightness area And determining means for determining a saturation level of the high luminance region based on the value of the high luminance region corresponding to the determined saturation level for the high luminance region in which at least one color signal of the plurality of color signals is saturated. Correction means for performing correction.

  According to the present invention, it is possible to realize natural correction without causing color misregistration.

It is a figure which shows the schematic structural example of the digital camera of this embodiment. It is a figure which shows the detailed structural example of an image process part. It is a flowchart of a process until it detects a face area | region and performs shine correction. It is a figure used for description of the detected face area. It is a flowchart of a shine correction process. It is a figure used for description of the saturation level of a shine area. It is a figure which shows the output signal with respect to the input of each color of R, G, B. It is a figure which shows the input / output signal of each color of R, G, B before and after a shine correction. It is a figure which shows the correspondence of each color signal value of R, G, B before and after a shine correction.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The image processing apparatus of the present invention is applied to various cameras such as digital still cameras, digital video cameras, smartphones and tablet terminals having digital camera functions, various information terminals such as personal computers, industrial cameras, in-vehicle cameras, and medical cameras. Applicable. In the present embodiment, an example in which the image processing apparatus of the present invention is applied to a digital still camera (hereinafter simply referred to as a digital camera) will be described.

FIG. 1 is a block diagram illustrating a configuration example of a digital camera 100 according to the present embodiment.
The lens group 101 is a zoom lens including a focus lens. The shutter 102 has a diaphragm function, and adjusts the amount of light incident on the image sensor provided in the imaging unit 103 and the exposure time under the control of the system control unit 50. The imaging unit 103 converts an optical image formed on the imaging surface of the imaging element via the lens group 101 into an electrical signal by photoelectric conversion, and outputs an analog image signal. The imaging device of the imaging unit 103 is a CCD image sensor or a CMOS image sensor. In front of the imaging surface of the imaging device, color filters corresponding to the respective color components of the three primary colors R (red), G (green), and B (blue) are provided for each pixel in an arrangement called a Bayer arrangement. For this reason, the imaging unit 103 outputs an analog image signal composed of color signals of R, G, and B pixels corresponding to the Bayer array. The A / D conversion unit 104 converts the analog image signal output from the imaging unit 103 into digital image data, and outputs the image data to the image processing unit 105.

  The image processing unit 105 performs white balance adjustment, γ (gamma) on the image data captured by the imaging unit 103 and output from the A / D conversion unit 104 or the image data read from the memory 106 by the memory control unit 107. ) Perform various image processing such as correction. Further, the image processing unit 105 performs a predetermined evaluation value calculation process using the face detection result by the face detection unit 113 and the captured image data. The calculated value obtained by the predetermined evaluation value calculation process here is sent to the system control unit 50 as an evaluation value used for exposure control and distance measurement control. In this case, the system control unit 50 performs exposure control and distance measurement control based on the evaluation value, thereby TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing or the like is performed. Further, when there is a shine area in which the color signal value is saturated and the color information is lost in the captured image, the image processing unit 105 also performs a shine correction process on the shine area. Details of the shine correction processing performed by the image processing unit 105 will be described later.

  The codec unit 110 compresses and decodes image data. The codec unit 110 encodes or decodes the image data recorded in the memory 106 in a format compliant with a standard such as MPEG.

  The face detection unit 113 performs predetermined image analysis processing on image data captured by the imaging unit 103 and output from the A / D conversion unit 104 or image data read out from the memory 106 by the memory control unit 107. Thus, an area where a human face is reflected is detected from the image. The data of the face detection result detected by the face detection unit 113 is temporarily stored in the memory 106 via the memory control unit 107, and read out from the memory 106 as necessary and sent to the image processing unit 105. It is done.

  The memory 106 temporarily stores image data when the image processing unit 105 performs various types of image processing, as well as image data read from the recording medium 112 via the recording medium interface (I / F) 111, and display Image data to be displayed on the unit 109 is stored. The memory control unit 107 controls reading / writing of the memory 106. The D / A converter 108 converts, for example, image display image data read from the memory 106 by the memory control unit 107 into an analog video signal and outputs the analog video signal to the display unit 109.

  The display unit 109 has a display device such as an LCD (Liquid Crystal Display), and the captured image, the image read from the recording medium 112, live captured by the imaging unit 103 during the so-called live view display. View images etc. The display unit 109 also displays a user interface image when the user operates the digital camera 100. The operation unit 120 includes a touch panel that displays the above-described user interface image, buttons, and switches, detects an operation by the user, and notifies the system control unit 50 of the detected operation information.

  The system control unit 50 includes a CPU or MPU and executes a program stored in the non-volatile memory 121 by expanding it in the work area of the system memory 122, thereby performing each function of the entire digital camera 100 of the present embodiment. Control. The nonvolatile memory 121 includes a nonvolatile semiconductor memory such as an EEPROM that stores programs, parameters, and the like as an auxiliary storage device. The system memory 122 develops a program read from the nonvolatile memory 121 as a main storage device, and stores constants and variables for operation of the system control unit 50. The recording medium I / F 111 mechanically and electrically connects a detachable recording medium 112 such as a semiconductor memory card or a card type hard disk to the digital camera 100.

Hereinafter, a detailed configuration of the image processing unit 105 and details of the image processing performed by the image processing unit 105 will be described with reference to FIG.
The image signal generation unit 201 of the image processing unit 105 receives image data captured by the imaging unit 103 and output from the A / D conversion unit 104. This image data is made up of R, G, and B color signals configured in a Bayer array as described above. The image signal generation unit 201 performs synchronization processing on the image data of each color of R, G, and B corresponding to the Bayer array, and generates image data including each color of R, G, and B per pixel. The image signal generation unit 201 outputs the image data generated by the synchronization processing to the white balance control unit 202 (hereinafter referred to as the WB control unit 202).

  The WB control unit 202 calculates a white balance correction value (hereinafter referred to as a WB correction value) based on the image data output from the image signal generation unit 201. The WB control unit 202 corrects the white balance of the image data using the calculated WB correction value.

  The shine correction circuit 203 performs the shine correction process based on the image data that has been white balance corrected by the WB control unit 202. Details of the shine correction processing of the shine correction circuit 203 will be described later. The image data that has been corrected by the shine correction circuit 203 is sent to the color conversion matrix circuit 204 and the Y generation circuit 211.

  A color conversion matrix circuit 204 (hereinafter referred to as a color conversion MTX circuit 204) multiplies the color gain so that the image data after the shine correction by the shine correction circuit 203 is reproduced with an optimum color, and then the two color differences. Data is converted into RY and BY. The two color difference data RY and BY output from the color conversion MTX circuit 204 are sent to a low-pass filter (LPF) circuit 205 (hereinafter referred to as an LPF circuit 205). The LPF circuit 205 limits the band of the color difference data RY and BY, and outputs the color difference data RY and BY after the band limitation to the CSUP circuit 206. A CSUP (Chroma Suppress) circuit 206 performs a process of suppressing a false color signal in a saturated portion of the color difference data RY and BY that is band-limited by the LPF circuit 205. The color difference data RY and BY after the false color signal suppression by the CSUP circuit 206 is output to the RGB conversion circuit 207.

  The Y generation circuit 211 generates luminance data Y from the image data after the shine correction by the shine correction circuit 203. The luminance data Y generated by the Y generation circuit 211 is sent to the edge enhancement circuit 212. The edge enhancement circuit 212 performs edge enhancement processing on the luminance data Y, and outputs the luminance data Y after the processing to the RGB conversion circuit 207.

  The RGB conversion circuit 207 performs RGB conversion processing for generating RGB data from the color difference data RY and BY output from the CSUP circuit 206 and the luminance data Y output from the edge enhancement circuit 212. The RGB data output from the RGB conversion circuit 207 is sent to the γ correction circuit 208. The γ correction circuit 208 performs gradation correction processing (γ correction processing) on the RGB data according to a predetermined γ characteristic. The RGB data subjected to the γ correction processing by the γ correction circuit 208 is converted into YUV data by the color luminance conversion circuit 209 and then compressed and encoded by the JPEG compression circuit 210. The compressed and encoded image data is then recorded as an image data file on the recording medium 112 in FIG.

  Hereinafter, details of the shine correction performed in the digital camera 100 of the present embodiment will be described. FIG. 3 shows that the digital camera 100 detects an image area of a predetermined subject from a captured image of the imaging unit 103 or an image stored in the memory 106, extracts a shine area from the image area, and performs a shine correction process. It is a flowchart which shows the flow of a process until. In the flowchart of FIG. 3, steps S301 to S304 are abbreviated as S301 to S304, respectively. 3 may be executed by a hardware configuration, or may be realized by a CPU or the like executing a program. The same applies to other flowcharts described later.

In the present embodiment, an example will be described in which a face image is acquired as an image area of a predetermined subject, a shine area is extracted from the face image, and shine correction is performed on the shine area.
For this reason, in the digital camera 100 of the present embodiment, as the process of S301, the face detection unit 113 first detects the area of the person's face image (hereinafter referred to as the person's face image from the image stored in the image capturing unit 103 or the memory 106). A face area).

  Next, in S302, the face detection unit 113 detects facial organ regions (hereinafter referred to as organ regions) such as eyes, mouths, and eyebrows from the face regions detected in S301. Here, the reason for detecting the organ area is to separate the skin color area from the other color areas in the face area, and the organ areas such as eyes and mouth are generally not skin color, so these organ areas are detected. By doing so, it becomes possible to specify the skin color area in the face area.

  In next step S303, the face detection unit 113 specifies an area obtained by removing the organ area from the face area, that is, a skin color area of the face, and further detects a shine area in the skin color area and performs a shine correction process on the shine area. Get evaluation value to use. FIG. 4 is a diagram illustrating an example of the face area 401 detected by the face detection unit 113. In S303, the face detection unit 113 divides the human face area 401 into a plurality of blocks as shown in FIG. 4, and obtains an integrated pixel value for each block. Further, the face detection unit 113 removes each block of the skin color area by excluding each block of the organ area such as the eyes and mouth detected in S302, that is, each block of the non-skin color area, from each block of the face area 401. Identify. Then, the face detection unit 113 acquires the integrated value of each block in the skin color area as an evaluation value. As described above, in S303, the evaluation value of each block of the skin color area is obtained by excluding the non-skin color areas such as eyes and mouth in the face area 401. The evaluation value for each block of the flesh color area acquired in S303 is temporarily stored in the memory 106, for example, and then read out by the memory control unit 107 under the control of the system control unit 50, and the image processing unit 105 is read out. To the shine correction circuit 203.

  Next, as the process of S304, the shine correction circuit 203 extracts a shine area from the skin color area of the face area 401 based on each block integration value (evaluation value) of the skin color area acquired in S303. A saturation level representing the saturation state of the region is determined for each pixel. Then, the shine correction circuit 203 performs a shine correction process corresponding to the saturation level on the shine area. Hereinafter, with reference to each figure after FIG. 5, the detail of the shine correction process according to the saturation level in this embodiment is demonstrated.

FIG. 5 is a flowchart showing the flow of the shine correction process performed by the shine correction circuit 203.
In S501, the shine correction circuit 203 calculates luminance information (referred to as face luminance information) of the face area 401 based on the block integration value of the skin color area acquired in S303. That is, the shine correction circuit 203 acquires the luminance information of the skin color area in which organ areas such as eyes and mouth are excluded from the face area 401 as the face luminance information in S501.

  Further, in S502, the shine correction circuit 203 calculates the average value of the color of the skin color area based on the block integration value of the skin color area acquired in S303. In other words, the shine correction circuit 203 calculates the average value of the skin color area in which the organ area is excluded from the face area 401 (hereinafter referred to as the skin color average value) in S502.

  Next, the shine correction circuit 203 acquires luminance information for each block based on the block integration value of the skin color area acquired in S303 in S503. Further, in the next step S504, the shine correction circuit 203 determines whether there is a high-luminance block that is equal to or higher than a predetermined luminance threshold value. In other words, in S503 and S504, the shine correction circuit 203 is a high color composed of one or more blocks of high luminance whose luminance is equal to or higher than a predetermined luminance threshold in the skin color region excluding the organ region from the face region 401. It is determined whether there is a luminance region. Then, if it is determined in S504 that there is no high brightness area (No), the shine correction circuit 203 ends the shine correction process shown in the flowchart of FIG. On the other hand, if it is determined in S504 that there is a high-luminance area (Yes), the shine correction circuit 203 extracts the high-luminance area as a shine area, and the process proceeds to S505.

  In step S505, the shine correction circuit 203 determines the saturation state of the R, G, and B color signals for each pixel in the high luminance area (shine area) extracted in S504, and determines each color of R, G, and B. Determine a saturation level representing the saturation of the signal. In this embodiment, the saturation level is determined for each pixel. However, the saturation level may be determined for each block, or the saturation level may be determined for each high luminance region. In the present embodiment, the shine correction circuit 203 is a saturation level that represents the saturation state of the R, G, and B color signals, and all the R, G, and B color signals are saturated at a predetermined threshold value or more. It is divided into a saturation level LC and a partial saturation level in which some color signals are saturated at a threshold value or more. Further, the shine correction circuit 203 sets the partial saturation level to the first partial saturation level L1 in which the R color signal is equal to or greater than the threshold and the G and B color signals are less than the threshold, and the R and G color signals are equal to or greater than the threshold. This is divided into the second partial saturation level L2 at which the B color signal is less than the threshold. Note that the saturation level may be different for each color, and the predetermined threshold value for determining the saturation level may be different for each color.

  In step S505, the shine correction circuit 203 determines whether or not the saturation level LC is reached for each pixel in the high luminance area. If it is determined in S505 that the saturation level LC is the full saturation level LC (Yes), the shine correction circuit 203 advances the process to S506. In step S506, the shine correction circuit 203 performs subtraction processing such as subtracting the complementary color of the flesh color average value calculated from the flesh color area of the face area 401 in S502 from the pixels determined to be the full saturation level LC in S505. Perform shine correction processing. After S506, the shine correction circuit 203 ends the shine correction process shown in the flowchart of FIG.

  On the other hand, if the shine correction circuit 203 determines in S505 that it is not the full saturation level LC (No), that is, if it is determined that it is the first partial saturation level L1 or the second partial saturation level L2, the process proceeds to S507. . In step S507, the shine correction circuit 203 performs a shine correction process by performing a gain reduction process for reducing the gains of the R, G, and B color signals for the pixel. After this S507, the shine correction circuit 203 ends the shine correction process shown in the flowchart of FIG.

  As described above, in the present embodiment, when the pixels in the high luminance region are at the full saturation level LC, the shine correction process for subtracting the complementary color of the skin color average value is performed, while the first partial saturation level L1 or the second In the case of the partial saturation level L2, the shine correction process by the gain reduction process is performed. Hereinafter, the reason and effect of switching the shine correction process according to the saturation level will be described with reference to FIGS.

  FIG. 6 shows a high luminance region composed of pixels of all saturation levels LC in which all color signals of R, G, and B are saturated, and part of color signals of R, G, and B are saturated. It is the figure which showed the example of the face image in which the high-intensity area | region which consists of each pixel of 1 and 2nd partial saturation levels L1 and L2 exists. In the case of the high luminance region of the full saturation level LC, the color information is lost because all the R, G, and B color signals of the pixel are saturated, and the originally skin-colored region is in a state where it is white. . On the other hand, in the case of the high luminance regions of the first and second partial saturation levels L1 and L2, a color shift (so-called color curve) occurs because some color signals of R, G, and B of the pixel are saturated. It is in a state. For example, in the case of the high luminance region of the second partial saturation level L2 in which the two color signals R and G are saturated, the region that should be the skin color is bright yellow. Such color bending is caused by saturation of the analog signals of the R and B pixels at the time of exposure of the image sensor of the imaging unit 103, and thereafter, the color of the pixel combining the R, G, and B color signals is shifted. It happens.

  FIG. 7 shows an output from the image sensor in response to input (intensity of incident light) of R, G, and B color components of incident light from a skin-colored subject such as a face in the image sensor of the imaging unit 103. It is a figure which shows the relationship with the color signal (output signal) for 1 pixel of R, G, B. As shown in FIG. 7, as the incident light from the skin-colored object becomes stronger, the value of the output signal of each color of R, G, B increases accordingly. When the intensity of the incident light from the skin-colored subject is less than the threshold value, the output signals of R, G, and B are at non-saturated non-saturation levels LN. , B output signals gradually increase in value. On the other hand, when the intensity of the incident light from the skin-colored subject reaches a certain threshold value, the output signals of the R, G, and B colors are saturated so that the values do not increase any more. For example, when the output signal value is represented by 0 to 255 and the threshold value is 250, the incident light gradually increases, and when the output signal value of each color of R, G, and B gradually increases to the threshold value of 250, The output signal for that color is saturated. Here, when the subject is a human face, the amounts of R, G, and B color components of the incident light, that is, skin-colored incident light, generally have a relationship of R> G> B. Therefore, as the incident light gradually increases, as shown in FIG. 7, the output signal of R is first saturated (first partial saturation level L1), and then the output signal of B is saturated (second Finally, the output signal of B is saturated (total saturation level LC).

  FIG. 8 is a diagram illustrating a relationship between an input signal to the shine correction circuit 203 and an output signal after the shine correction process is performed by the shine correction circuit 203. The input signal to the shine correction circuit 203 corresponds to the output signal of the image sensor described in FIG. That is, the input signal to the shine correction circuit 203 is such that the R signal is saturated at the first partial saturation level L1, the R and G signals are saturated at the second partial saturation level, and R, G, B at the full saturation level LC. The signal becomes a saturated signal. Further, at the non-saturation level LN, the R, G, and B signals are input signals that are not saturated. As described above, the input signal to the shine correction circuit 203 is saturated with the R signal at the boundary M1 where the non-saturation level LN switches to the first partial saturation level L1. Also, the input signal to the shine correction circuit 203 is saturated at the boundary M2 where the first partial saturation level L1 is switched to the second partial saturation level L2, and further from the second partial saturation level L2 to the full saturation level LC. The B signal saturates at the switching boundary M3. Then, as described above, the shine correction circuit 203 according to the present embodiment performs a process of subtracting the complementary color of the skin color average value when the pixels in the high luminance region are at the full saturation level LC, and the first partial saturation level. In the case of L1 or the second partial saturation level L2, gain reduction processing is performed.

  Here, at the boundary where the saturation level of the signals of the respective colors R, G, and B is switched, a pseudo contour or a color step may be generated due to a sudden color change. In particular, at the boundary M1 where the non-saturation level LN is switched to the first partial saturation level L1, the color changes abruptly, and the pseudo contour and the color step are easily noticeable. For this reason, the shine correction circuit 203 according to the present embodiment sets the following predetermined signal value range, corrects the output signal value by linear interpolation, and reduces the color change at each boundary where the saturation level is switched. Set the correction target value.

  The shine correction circuit 203 sets a predetermined signal value range E1 including the boundary M1 in the vicinity of the boundary M1 where the non-saturation level LN switches to the first partial saturation level L1. Then, when the input signal value is within the signal value range E1, the shine correction circuit 203 corresponds the output signal value corresponding to the input signal value to both ends V1 and V2 of the predetermined signal value range E1. A signal value on a straight line obtained by linear interpolation between the two signal values V1R and V2R is calculated. One end V1 of the predetermined signal value range E1 is set as a signal value at a location away from the boundary M1 by a predetermined width in a direction in which the signal value decreases from the boundary M1. The other end V2 of the predetermined signal value range E1 is in a direction in which the signal value increases from the boundary M1 and reaches the boundary M2 where the first partial saturation level L1 switches to the second partial saturation level L2. It is set as the signal value of the location.

  For this reason, for example, in the case of an R signal whose saturation level is switched at the boundary M1, the output signal value corresponding to the R input signal value is two signal values V1R corresponding to both ends V1 and V2 of the predetermined signal value range E1. , V2R is calculated as a signal value on a straight line obtained by linear interpolation. That is, the R output signal within the predetermined signal value range E1 becomes a signal represented by a straight line connecting the two signal values V1R and V2R, and the output signal changes so as to be bent sharply at the boundary M1. Disappear. Similarly, for G and B where the saturation level does not switch at the boundary M1, the output signal values corresponding to the input signal values of G and B are the two signal values at both ends V1 and V2 of the predetermined signal value range E1. It is calculated as a signal value on a straight line obtained by linear interpolation between them. Note that these G and B output signals are not originally signals whose saturation level switches at the boundary M1, and therefore the input / output relationship does not substantially change within the signal value range E1. As a result, within the predetermined signal value range E1, the signals of the R, G, and B colors do not change abruptly, and therefore the color changes abruptly and pseudo contours or color steps are generated. Disappear.

  Further, the shine correction circuit 203 applies to the boundary M2 that switches from the first partial saturation level L1 to the second partial saturation level L2 and the boundary M3 that switches from the second partial saturation level L2 to the full saturation level LC. A predetermined signal value range E2 including M2 and M3 is set. Then, when the input signal value is within the signal value range E2, the shine correction circuit 203 linearly interpolates the output signal value corresponding to the input signal value between both ends V2 and V3 of the signal value range E2. It is calculated as a signal on the straight line. Here, one end V2 of the signal value range E2 is a value corresponding to the end V2 of the signal value range E2. On the other hand, the other end V3 of the signal value range E2 is based on the values of the R, G, and B skin color average values of the skin color area of the face area 401 in the direction in which the signal value increases from the boundary M3. Is set. In the case of the present embodiment, the shine correction circuit 203 sets correction target values V3R, V3G, and V3B based on the skin color average values R, G, and B of the skin color area of the face area 401, and corrects them. Corresponding to the target values V3R, V3G, and V3B, the end V3 of the signal value range E2 is set.

  Thereby, the output signal value corresponding to the input signal value of R is calculated as a signal value on a straight line obtained by linear interpolation between the two signal values V2R and V3R at both ends V2 and V3 of the predetermined signal value range E2. The Similarly, the output signal value of G is calculated as a signal value on a straight line obtained by linear interpolation between the signal values V2G and V3G of the signal value range E2, and the output signal value of B is the signal values V2B and V3B of the signal value range E2. As a signal value on a straight line obtained by linear interpolation. For this reason, within the predetermined signal value range E2, the signals of the R, G, and B colors do not change abruptly, and therefore the color changes abruptly, resulting in a pseudo contour or a color step. Disappear.

  9A to 9C show the input signals (input signals before the shine correction) input to the shine correction circuit 203 and the shine correction by the shine correction circuit 203 for each of R, G, and B colors. It is a figure explaining the correspondence of the output signal of. FIG. 9A shows the case of the total saturation level, FIG. 9B shows the case of the second partial saturation level, and FIG. 9C shows the case of the first partial saturation level. Also, the lengths in the vertical direction of the signals of R, G, and B in FIGS. 9A to 9C represent the magnitudes of the signal values. Further, each threshold value in FIGS. 9A to 9C indicates an upper limit value of the output signal after the shine correction with respect to the input signal before the shine correction.

  In the case of the full saturation level shown in FIG. 9A, the input signals of each color of R, G, B are all saturated, there is no difference between the colors, and the color information is lost. In the shine correction process of the present embodiment, in the case of the full saturation level, as described above, the shine correction process by the process of subtracting the complementary color of the skin color average value of the skin color region is performed. Further, the threshold value of the output signal is an upper limit value. For this reason, in the case of the full saturation level, among the input signals of R, G, and B colors, the R output signal is corrected to a signal having a threshold value as an upper limit value, and the G and B output signals are respectively skin tone average values. The signal is corrected to a signal obtained by subtracting the complementary color. As a result, the image by the R, G, B output signals after the shine correction is an image in which the skin color is recovered.

  In the case of the second partial saturation level shown in FIG. 9B, the R and G input signals are saturated, and the B input signal is not saturated. In the shine correction process of the present embodiment, in the case of the second partial saturation level, the R output signal is corrected to a signal having the threshold value as the upper limit value by the gain reduction process, and the G and B output signals are also corrected by the gain reduction process. It is corrected. As a result, the image based on the R, G, and B output signals after the shine correction is a natural skin color image in which no color misregistration (color distortion) occurs.

  In the case of the first partial saturation level shown in FIG. 9C, the R input signal is saturated and the G and B input signals are not saturated. In the shine correction process of the present embodiment, in the case of the first partial saturation level, the R output signal is corrected to a signal having a threshold value as an upper limit value by the gain reduction process, and the G and B output signals are also corrected by the gain reduction process. It is corrected. As a result, the image based on the R, G, and B output signals after the shine correction is a natural skin color image in which no color misregistration (color distortion) occurs.

<Other embodiments>
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.
In the above-described embodiment, an example in which the shine correction process is performed on the skin color area of the face image has been described. However, the present invention is not limited to the shine correction process on the skin color area of the face image. As an example, it can also be applied to shine correction for a high brightness area such as the body of an automobile. In this case, the high brightness area is detected from the body color area in the image and evaluated from the color signal of the high brightness area. The value is obtained, and the shine correction is performed based on the evaluation value.
In the above-described embodiment, the skin color having the relationship of R>G> B is taken as an example. Therefore, the partial saturation level is the first partial saturation when the R signal is saturated and the G and B signals are not saturated. The second partial saturation level L2 is reached when the level L1, R and G signals are saturated and the B signal is not saturated. On the other hand, in the case of a color other than skin color, for example, a color having a relationship of B>G> R, the partial saturation level is the first partial saturation level when the B signal is saturated and the G and R signals are not saturated. When the L1, B, and G signals are saturated and the R signal is not saturated, the second partial saturation level L2 is reached. That is, the first partial saturation level L1 is a saturation level when one of the three color signals R, G, and B is saturated at a predetermined threshold value or more, and the second partial saturation level L2 is This is the saturation level when two of the three color signals are saturated above the threshold.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  50: System control unit, 100: Digital camera, 103: Image capturing unit, 105: Image processing unit, 106: Memory, 107: Memory control unit, 113: Face detection unit, 121: Non-volatile memory, 203: Deflate correction circuit

Claims (12)

Detecting means for detecting an image area of a predetermined subject from the input image;
Extracting means for extracting a high brightness area from the image area of the subject;
Determining means for determining a saturation level of the high luminance region based on values of a plurality of color signals determining the color of the high luminance region;
Correction means for performing correction in accordance with the determined saturation level for the high-luminance region in which at least one color signal of the plurality of color signals is saturated;
An image processing apparatus comprising:   The determination means determines the saturation level of the high luminance region based on a determination as to whether or not each value of the plurality of color signals in the high luminance region is equal to or greater than a predetermined threshold value. An image processing apparatus according to 1. The plurality of color signals for determining the color are three color signals of three primary colors,
The determining means determines a saturation level of the high luminance region,
A total saturation level at which all of the three color signals in the high luminance region are equal to or higher than the predetermined threshold;
A first partial saturation level at which one of the three color signals in the high-luminance region is equal to or higher than the predetermined threshold;
A second partial saturation level at which two color signals out of the three color signals in the high luminance region are equal to or higher than the predetermined threshold;
The image processing apparatus according to claim 2, wherein all of the three color signals in the high luminance region are determined to be any one of the non-saturation levels that are less than the predetermined threshold. The correction means includes
When the high luminance region is the first partial saturation level or the second partial saturation level, the correction is performed by reducing the gain of each color signal in the high luminance region,
When the high brightness area is at the total saturation level, the correction for subtracting the complementary color of the color average value calculated based on each color signal of the image area of the predetermined subject from each color signal of the high brightness area. The image processing apparatus according to claim 3, wherein the image processing apparatus performs the processing. The correction means includes
A predetermined signal value range is set with respect to a boundary at which the saturation level of the high luminance region is switched;
When the value of the input color signal is within the signal value range, the output signal value corresponding to the input color signal is linearly interpolated between the signal values at both ends of the signal value range. The image processing apparatus according to claim 3, wherein the image processing apparatus calculates the signal value on a straight line. The correction means includes
For a first boundary where the non-saturation level switches to the first partial saturation level, a first signal value range including the first boundary is set, and an output signal value is calculated based on the linear interpolation. And
For the second boundary that switches from the first partial saturation level to the second partial saturation level and the third boundary that switches from the second partial saturation level to the full saturation level, the second boundary The image processing apparatus according to claim 5, wherein an output signal value is calculated based on the linear interpolation by setting a second signal value range including a third boundary.   The correction means is a value corresponding to a color average value calculated based on each color signal of the image area of the predetermined subject, with the signal value at the end corresponding to the total saturation level in the second signal value range. The image processing apparatus according to claim 6.   The extraction unit divides the image area of the subject into a plurality of blocks, calculates an integrated value of pixel values of each block, calculates luminance information from the integrated value of each block, and calculates luminance information of each block. The image processing apparatus according to claim 1, wherein the high luminance region is extracted based on the image.   The image processing apparatus according to claim 1, wherein the detection unit detects an image area of a human face as the image area of the predetermined subject.   The image processing apparatus according to claim 9, wherein the detection unit detects an image area obtained by removing a facial organ area from the face image area. A detection step of detecting an image area of a predetermined subject from the input image;
An extraction step of extracting a high brightness area from the image area of the subject;
A determination step of determining a saturation level of the high luminance region based on a plurality of color signal values determining the color of the high luminance region;
A correction step of performing correction according to the determined saturation level for the high luminance region in which at least one color signal of the plurality of color signals is saturated;
An image processing method for an image processing apparatus, comprising:   The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 to 10.

Images