JP2009038582A - Image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem wherein flesh color is corrected by an improper correction coefficient by preventing a skin portion from being detected, or by detecting an incorrect place. <P>SOLUTION: In an image processing unit for performing plurality of image processing means to an input image, at least one of the image processing means is first image processing means having processing coefficient determination means for determining coefficients used for the execution of processing, the processing coefficient determination means is processing coefficient determination means for performing determination by utilizing other second image processing means, and a processing coefficient determined by the processing coefficient determination means is used to execute the first image processing means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method for performing a plurality of image processing.

  The performance of personal computers over the last few years has made it easier and faster to handle even large files. More and more people are handling photographic-like natural images, which are generally considered to have a large capacity. In addition, input devices represented by digital cameras and scanners have become widespread, and various images have been digitized.

  There are various types of photographic images, from high-quality images taken by professionals to images with poor quality. An image with poor quality referred to here is an image that is overexposed or underexposed, an image that causes a tinted color cast on the entire image, and the like. In general, the possible causes of bad images are due to inadequate light intensity during shooting, shooting conditions such as shooting under fluorescent light, and images generated during digitization such as noise and characteristics of digital cameras. Deterioration, etc. These poor quality images can be improved by digitizing the image and taking it into a personal computer or the like and then performing correction.

  Such correction processing is generally attached as a function of an application for retouching an image or a driver of a printing apparatus represented by a printer.

  In recent years, with the development of PictBridge (registered trademark) standards in which a digital camera and a printer are directly connected to perform recording processing, and the appearance of printers with card slots, the usage of recording images without going through a PC has become widespread. . Under such circumstances, even in printing in a non-PC environment without using a PC, it is currently required to realize correction processing similar to that of a PC application or printer driver.

  In general, printers are significantly inferior to system environments such as PC CPU capacity, memory capacity, and bus speed. In order to prevent performance degradation due to this, Japanese Patent Application Laid-Open No. 2004-151867 takes measures by using a processor dedicated to image processing in addition to a CPU for printer operation. In addition, it is possible to shorten the image processing time by adopting a memory that can be accessed at high speed, increasing the bus speed, and mounting a dedicated ASIC. However, these speeding-ups are costly and ultimately burden the user. In other words, speeding up and cost increase are in a trade-off relationship.

  On the other hand, various proposals have been made for image correction. Generally, in these correction processes, a correction amount determination unit that calculates a correction amount for performing correction, and correction based on the obtained correction amount. It is divided into correction execution units to be executed.

  Among them, the correction amount determination unit can be roughly classified into two types according to the processing contents. One is a method of generating accumulated data of an image and determining a correction amount based on the generated data, and a process of extracting a feature amount of a specific part in the image and performing correction based on the value.

  JP-A 2000-13624 and JP-A 2000-13616 can be cited as correction methods using the former accumulated data. In this process, color cast and level correction are performed on an image for which white balance setting is insufficient or exposure is not optimal. As the accumulation process for obtaining the correction amount, the luminance value for each pixel is accumulated to create a histogram, and based on this, the necessity of correction is determined and the correction amount is determined. Correction is executed by executing a luminance conversion process for correcting the generated luminance histogram as shown in FIG. 1A as shown in FIG. Appropriate gray balance is realized by calculating an average hue of luminance values corresponding to the HL and SD points in FIG. 1A, obtaining conversion coefficients so that they become achromatic colors, and applying them. .

  Further, contrast correction and saturation correction that make the image more suitable are also possible. From the acquired luminance histogram distribution, a luminance conversion amount such as an S-shaped curve is determined and contrast correction is performed, or a correction coefficient amount is calculated from the average saturation of the image, and the color difference component of the image is multiplied by the correction coefficient. Thus, each correction is performed.

  On the other hand, the latter process of extracting a specific part is generally a detection process that is mainly a main part of an image such as human skin. Japanese Patent No. 3203946 is a correction of a video signal such as a television, but detects a skin color average luminance level from a color difference signal and a luminance signal, and adds a control signal to a gamma correction circuit in relation to the luminance level of all images. It is.

  In the process involving extraction of the specific part, the detection rate for detecting the part is very important. If the detection rate is low, not only the correction is not applied, but an incorrect correction coefficient is set, and the image may be deteriorated. In the skin color detection of special registration 3203946, a skin color portion is detected by using a color difference signal among signal values decomposed into luminance and color difference components.

  Moreover, when detecting a human skin part as another means, the determination by a color is represented by following formula (4).

により行うものもある。ここで、Ｒ、Ｇ、Ｂは画像の画素信号のレッド、グリーン、ブルー信号を表す。色味による検出のほか、ＤＣＴ変換後の周波数特性を利用して、高精度に人肌部分を検出処理が行うことも可能である。こうした処理により、主被写体である人物が逆光で暗くなった場合やフラッシュにより色飛びを起こした場合に、適切な露出に補正することや、人肌部分にローパスフィルタを適用することにより、肌補正を実現している。

There are also things to do. Here, R, G, and B represent red, green, and blue signals of image pixel signals. In addition to detection by color, it is also possible to detect human skin portions with high accuracy using frequency characteristics after DCT conversion. With this process, when the person who is the main subject becomes dark due to backlighting or when a color jump occurs due to flash, the skin is corrected by applying an appropriate exposure or applying a low-pass filter to the human skin. Is realized.

  As an example of the low-pass filter, a smoothing filter such as a general average value filter or a median filter can be given. The filter size may be a fixed size such as 3 × 3 or 5 × 5, or may be a method of changing dynamically according to the size of the detected human skin area.

  Further, by applying a smoothing filter to the entire image and changing the smoothing strength in the human skin part, it is possible to correct the noise by removing the entire noise and further smoothing the human skin. A method of changing the smoothing intensity can be realized by switching the coefficient of the filter.

  As described above, when printing with NonPC, as well as performing printing, it reflects the benefits of digital technology such as image correction and processing such as color correction, skin correction such as backlight correction, and noise reduction. It is supposed to be.

In addition, as a conventional technique, an example of NonPC, which has been a recent characteristic trend, has been given. Needless to say, the same processing can be performed on a PC for image correction.
JP 2000-13624 A JP 2000-13616 A Special Registration No. 3203946 JP2003-200261A

  In the conventional technology described above, the specific part is detected and processed. However, if the original image is colored or has a lot of noise, the specific part Misdetection is likely to occur during detection. In particular, in the detection process in which judgment is performed by color as in the special registration 3203946 or the above formula (4), the color covering greatly shifts the judgment result, and the skin part is not detected or an erroneous part is detected. To do. As a result, an incorrect correction coefficient is calculated, the correction for the skin color cannot be made appropriate, and the image may be deteriorated.

The present invention has been made to solve the above problems,
In an image processing apparatus that applies a plurality of image processing means to an input image,
At least one of the image processing means is a first image processing means having a processing coefficient determining means for determining a coefficient to be used when executing the processing,
The processing coefficient determination means is a processing coefficient determination means that determines using another second image processing means,
The first image processing means is executed using the processing coefficient determined by the processing coefficient determination means.

  As described above, according to the present invention, according to the present invention, when there are a plurality of image processes, and one of them is a process for detecting a specific portion from an image, the data used for the detection process in advance is used. By performing correction processing, detection processing with higher detection accuracy can be realized.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Embodiment 1]
Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present embodiment, not only a printer capable of printing from a normal PC but also an image taken with a digital camera having a PC card slot and recorded on the PC card can be printed. A recording apparatus (photo direct printer apparatus) that employs an inkjet method as a recording method will be described as an example.

  Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 2 is a schematic perspective view of the photo direct printer 1000 according to the embodiment of the present invention. This photo direct printer receives and prints data from a host computer (PC), functions as a normal PC printer, and directly reads and prints image data stored in a storage medium such as a memory card, Alternatively, it has a function of directly receiving and printing image data from a digital camera.

  In FIG. 2, the main body that forms the outer shell of the photo direct printer 1000 according to the present embodiment includes a lower case 1001, an upper case 1002, an access cover 1003, and an exterior member for a discharge tray 1004. The lower case 1001 forms a substantially lower half of the apparatus 1000, and the upper case 1002 forms a substantially upper half of the main body. A combination of both cases provides a storage space for storing each mechanism described below. A hollow body structure is formed, and openings are formed on the upper surface portion and the front surface portion, respectively. Further, one end of the discharge tray 1004 is rotatably held by the lower case 1001, and the opening formed in the front surface of the lower case 1001 can be opened and closed by the rotation. For this reason, when executing the recording operation, the discharge tray 1004 is rotated to the front side to open the opening, so that the recording sheets can be discharged from here, and the discharged recording sheets are sequentially discharged. It can be loaded. The paper discharge tray 1004 stores two auxiliary trays 1004a and 1004b. By pulling out the respective trays to the front as needed, the sheet support area can be expanded and reduced in three stages. It has become.

  One end of the access cover 1003 is rotatably held by the upper case 1002 so that an opening formed on the upper surface can be opened and closed. By opening the access cover 1003, the access cover 1003 is accommodated inside the main body. The recording head cartridge (not shown), the ink tank (not shown), or the like can be replaced. Although not particularly shown here, when the access cover 1003 is opened and closed, the protrusion formed on the back surface rotates the cover opening and closing lever, and the rotation position of the lever is detected by a micro switch or the like. Thus, the open / closed state of the access cover can be detected.

  A power key 1005 is provided on the upper surface of the upper case 1002 so that it can be pressed. An operation panel 1010 including a liquid crystal display unit 1006 and various key switches is provided on the right side of the upper case 1002. The structure of the operation panel 1010 will be described in detail later with reference to FIG. An automatic feeding unit 1007 automatically feeds the recording sheet into the apparatus main body. Reference numeral 1008 denotes a paper interval selection lever which is used to adjust the interval between the recording head and the recording sheet. Reference numeral 1009 denotes a card slot, into which an adapter capable of mounting a memory card is inserted, through which image data stored in the memory card can be directly captured and printed. Reference numeral 1011 denotes a viewer (liquid crystal display unit) which can be attached to and detached from the main body of the apparatus. When searching for an image to be printed from images stored in a PC card, an image or index image for each frame is displayed. Used to display. Reference numeral 1012 denotes a terminal for connecting a digital camera, which will be described later, and 1013 denotes a USB bus connector for connecting a personal computer (PC).

  FIG. 3 is a schematic view of operation panel 1010 according to the present embodiment.

  In the figure, the liquid crystal display unit 1006 displays menu items for setting various data relating to items printed on the left and right. The items displayed here are the first photo number of the range to be printed, the designated frame number (start / -designate), the last photo number of the range to be printed (end), the number of copies (number of copies), and used for printing. Type of paper (recording sheet) to be printed (paper type), setting of the number of photos to be printed on one sheet (layout), specification of print quality (quality), and whether to print the date of shooting (date printing) ), Designation of whether or not to print a photograph (image correction), display of the number of sheets necessary for printing (number of sheets), and the like. These items are selected or designated using the cursor keys 2001. The correction processing described in the conventional example is selected in this image correction item. For example, by cursor key

のように表示がトグルし、ユーザーが選択出来るようになる。

The display toggles like so that the user can select.

  2002 is a mode key, and each time the key 2002 is pressed, the type of printing (index printing, full-frame printing, single-frame printing, etc.) can be switched, and the corresponding LED of the LED 2003 is lit accordingly. . Reference numeral 2004 denotes a maintenance key, which is used to enter a printer maintenance or print area designation mode such as cleaning of the recording head 1301. Reference numeral 2005 denotes a print start key which is pressed when instructing the start of printing or when establishing maintenance settings. Reference numeral 2006 denotes a print cancel key which is pressed when printing is stopped or when maintenance is instructed.

  Next, with reference to FIG. 4, the configuration of the main part relating to the control of the photo direct printer 1000 according to the present embodiment will be described. Reference numeral 3000 denotes a control unit (control board). Reference numeral 3001 denotes an ASIC (dedicated custom LSI), and 3002 a DSP (digital signal processor), which has an internal CPU and is in charge of image processing and the like to be described later. A memory 3003 has a program memory 3003a that stores a control program for the CPU of the DSP 3002, a RAM area that stores a program at the time of execution, and a memory area that functions as a work memory that stores image data and the like. Reference numeral 3004 denotes a printer engine. Here, a printer engine of an ink jet printer that prints a color image using a plurality of color inks is installed. Reference numeral 3005 denotes a USB bus connector as a port for connecting the digital camera 3012. Reference numeral 3006 denotes a connector for connecting the viewer 1011. Reference numeral 3008 denotes a USB bus hub (USB HUB). When the printer apparatus 1000 performs printing based on image data from the PC 3010, the data from the PC 3010 is directly passed through and output to the printer engine 3004 via the USB bus 3021. To do. As a result, the connected PC 3010 can directly perform printing by exchanging data and signals with the printer engine 3004 (functions as a general PC printer). Reference numeral 3009 denotes a power connector which inputs a DC voltage converted from commercial AC by a power source 3013. A PC 3010 is a general personal computer, 3011 is the memory card (PC card) described above, and 3012 is a digital camera. Note that the exchange of signals between the control unit 3000 and the printer engine 3004 is performed via the USB bus 3021 or the IEEE 1284 bus 3022 described above.

  FIG. 5 shows a block diagram of image processing in the present embodiment. It is a block from acquiring an image from a memory card to a data format that can be output by a printer. Processing contents in each block will be described.

(JPEG decoding processing unit)
The most used image format in digital cameras is the JPEG (Joint Photographic Experts Group) format. In this embodiment, it is assumed that there is a JPEG image in the memory card. Although details of the decoding method are not essential in this application, they are omitted, but the main processes are entropy code expansion (decoding) processing, inverse quantization processing, and inverse DCT transformation processing.

(Backlight correction coefficient determination unit)
If the decoded image data requires backlight correction, its coefficient is determined by the backlight correction coefficient determination unit. Although there are several backlight correction processing methods, in this embodiment, a method of automatically detecting a human skin portion and calculating a correction amount from the luminance value is given.

  FIG. 6 shows a flowchart of the backlight correction coefficient determination process. STEP 601 is a human skin determination data generation unit. Two data for color determination and frequency determination are generated. As the color determination data, image expansion is executed by the decoding process described above, and image data of luminance Y color difference CbCr component is obtained. These are sent to STEP 602 and converted into RGB data by the following equation in conformity with ITU-R BT.601.

R = Y + 1.402 (Cr-128)
G = Y−0.34414 (Cb−128) −0.71414 (Cr−128)
B = Y + 1.772 (Cb-128)
In step 603, color determination processing is executed using the obtained RGB data. As the determination condition, Formula (4) described in the related art may be used. These determination processes need not be executed for all pixels in consideration of the process in STEP 604 described later, and may be determined in units of 8 × 8. The determination may be made by calculating the respective RGB average values in the 8 × 8 block, or may be determined by a preset point, for example, four corner points. By the processing in STEP 603, it is determined whether or not the condition of human skin is satisfied as a color.

  On the other hand, in STEP 601, the human skin determination data generation unit stores frequency data relating to luminance Y stored in JPEG 8 × 8 MCU units before executing inverse DCT conversion processing after inverse quantization. FIG. 7 is a diagram showing an 8 × 8 MCU in JPEG. In addition to the DC component at the upper left, the DCT coefficient of the AC component is stored. The upper left represents a low frequency component, and the lower left is a high frequency component. In STEP 604, whether or not the frequency is suitable for the skin is determined by comparing the DCT coefficient of the AC component with a parameter prepared in advance. The parameter is set so as to determine that it is not suitable for the skin when a high-frequency component is included in a certain amount or more. The low frequency component can also be set to be determined as not being skin if it is extremely strong. In addition, since parameters for all 63 DCT coefficients of the AC component are prepared and compared, a long processing time or a large number of parameters may be required, so comparison may be made at a predetermined point. . Further, an average value in a certain section may be obtained and compared. FIG. 8 shows a case where the AC component is divided into seven parts. The average value in each section is compared with a preset value. In this case, human skin frequency is determined by seven comparison processes. By the processing of STEP 604, it is determined whether the frequency satisfies the condition of human skin.

  In STEP 605, when the result determined in STEPs 603 and 604 is received and it is determined that both satisfy the condition of human skin, the 8 × 8 block is recognized as human skin.

  In STEP 606, it is determined whether the human skin is determined for the entire image. If not, the steps from SETP 601 to STEP 606 are repeated.

  If the determination process has been executed for all the images, the process proceeds to STEP 607 and a correction coefficient is determined. In this embodiment, a brightness conversion table is created as a correction coefficient. This is, for example, a table storing correspondence indicating what value should be set for the luminance Y component. If the input is 8 bits, the entry is a table of 256. The output result may be 8 bits from 0 to 255, but in consideration of gradation loss, an output with more than 8 bits is desirable.

The determination method uses the luminance value of the block determined as human skin in STEP5. By calculating the luminance average Y _human skin of the human skin block, the brightness index of the human skin portion in the image can be acquired. The target brightness Y _TARGET that is set in advance is set, for example, the brightness is increased by 10%. Y _TARGET = Y _{human skin} × k
(K = 1.1)
A coefficient that brightens the human skin portion is obtained by generating a table such that In order to obtain a more appropriate coefficient, the increase rate to the target luminance is changed according to the luminance value of the human skin, or the target luminance is changed by considering the luminance value Y _{overall luminance} of the _entire image. Is also possible.

Y _TARGET = Y _{human skin} x k (Y _{overall brightness} )
A conversion table LUT _{Y (backlight)} as shown in FIG. 9 is created from the obtained target luminance. Examples of the creation method include a creation method that does not cause a sharp jump in gradation, such as a method of connecting with a straight line, a method of performing smoothing thereto, or a method of obtaining an approximate curve.

Y _NEW = LUT _{Y (backlight)} [Y _ORG ]
Further, as the luminance increases, the saturation decreases, and it is necessary to correct it. This varies with the rate of increase in brightness,
S _(backlight) = f (Y _NEW -Y _ORG )
Is applied to the original color difference components Cb _ORG and Cr _ORG calculated as _follows , and Cb _NEW and Cr _NEW are obtained.

Cb _NEW = Cb _ORG × S _(backlight)
Cr _NEW = Cr _ORG × S _(Backlight)
(Color cast correction coefficient determination unit)
As described in the prior art, the determination of the color fog correction coefficient is to determine the coefficient for obtaining the movement amount from the luminance histogram and realizing the movement.

  First, an image decoding process is performed to develop a JPEG image. For the developed YCbCr, only the Y component is accumulated to generate a luminance histogram. If the decoding process for the entire image is performed here, it takes time to create the histogram, and a lot of memory is consumed. Therefore, it is possible to develop the image while thinning out the MCU units. Another possible method is to use only the DC component of JPEG and omit performing inverse DCT conversion.

FIG. 1A shows an example of a luminance histogram of an image in which the color cast and the exposure are under. In this luminance histogram, the point that should be white is not white. Therefore, the luminance corresponding to several percent from the top of the total number of pixels is determined as the highlight point (Y _HL ), and the luminance corresponding to several percent from the bottom is determined as the shadow point (Y _SD ). Generally, the highlight point and the shadow point are about 0 to 5% of the total number of pixels from the top (Y = 255) and the bottom (Y = 0), respectively. Then, the conversion is performed so that the luminance _YHL is white and the luminance _YSD is black.

Y _HL = 255
Y _SD = 0
Further, when it was converted to the above equation, the luminance _{0 to Y SD,} since gradation between Y HL to 255 is likely to be lost, slightly lower than the 255 _{Y HL} as shown in the following formula value Y255 _', the _{Y SD} slightly higher value Y0 than 0' it is possible to prevent a tone loss by setting.

Y _HL = Y ₂₅₅ ′ (Y ₂₅₅ ′ ≈255)
Y _SD = Y ₀ ′ (however, Y ₀ ′ ≈0)
By performing conversion for setting the brightness of the highlight point and the shadow point on the entire image, an image with underexposed or overexposed images is improved.

Further, color fog correction is performed on the color difference signals Cb and Cr of the image. The condition for white is Cb = Cr = 0 and Y = 255
It is. Similarly, the shadow point pixel should be black, and the condition is Cb = Cr = 0 and Y = 0.
It becomes. If the average of the color difference signals of the pixels having the luminance at the highlight point and the shadow point is (Cb _HL , Cr _HL ) and (Cb _SD , Cr _SD ), respectively, Cb _HL ≠ 0 or Cr _HL ≠ 0
Cb _SD ≠ 0 or Cr _SD ≠ 0
It has become. Therefore, in order to correct the color cast, (Cb _HL , Cr _HL ) and (Cb _SD , Cr _SD ) are converted into original white and black values, respectively. As an example of the method, a color solid line passing through a highlight point and a shadow point as shown in FIG. 10 may be considered, and the line may be overlapped with the luminance axis. It can be realized by the following formula.

ここでＹ_ＮＥＷ、Ｃｂ_ＮＥＷ、Ｃｒ_ＮＥＷは、それぞれ変換後の輝度、色差を表し、Ｙ_ＯＲＧ、Ｃｂ_ＯＲＧ、Ｃｒ_ＯＲＧは、それぞれ変換前の輝度、色差を表す。色かぶり補正係数決定部では上記３ｘ３マトリクスを算出する。

Here, Y _NEW , Cb _NEW , and Cr _NEW represent luminance and color difference after conversion, respectively, and Y _ORG , Cb _ORG , and Cr _ORG represent luminance and color difference before conversion, respectively. The color cast correction coefficient determination unit calculates the 3 × 3 matrix.

In this embodiment, when color cast correction is selected, the correction is made more effective by simultaneously performing contrast correction and saturation enhancement. Therefore, when the contrast enhancement degree is obtained from the distribution of the luminance histogram and the contrast is strengthened, a conversion table LUT _{Y (contrast)} is created,
Y _NEW = LUT _{Y (contrast)} [Y _ORG ]
Perform the correction. For saturation correction, the average saturation of the entire image is calculated, the saturation enhancement coefficient S (emphasis) is obtained,
Cb _NEW = Cb _ORG × S _(emphasis)
Cr _NEW = Cr _ORG × S _(emphasis)
To achieve saturation enhancement.

(Correction processor)
In the correction processing unit, the correction coefficient LUT _{Y (backlight)} relating to the backlight correction obtained above,
Saturation correction coefficient S _(backlight) , color cast correction, contrast correction, saturation enhancement correction coefficient, 3 × 3 matrix, LUT _{Y (contrast)} , S _{(enhancement)} , etc. are applied to the image.

  At this time, if backlight correction and color cast correction (including contrast correction and saturation enhancement) are selected at the same time, it takes a lot of processing time if the respective processes are appropriately performed. Therefore, in the correction relating to luminance and saturation, it is possible to combine tables with the following colors before starting the processing.

Y _{(backlight + contrast)} = LUT _{Y (backlight)} [Y _ORG ] × LUT _{Y (contrast)} [Y _ORG ]
S _{(backlight + enhancement)} = S _(backlight) x S _(emphasis)
By _{performing the} processing only with the obtained Y _{(backlight + contrast)} and S _{(backlight + enhancement)} , it is possible to achieve the same effect as when each processing is performed.

(Scaling processing unit)
The scaling factor for enlargement / reduction is determined from the number of pixels of the selected image and the selected paper size. As the scaling method, there are several methods such as simple enlargement / reduction, bilinear method, and bicubic method. Which one to select is determined by the system environment and the required processing speed. The above-described scaling means will be omitted because they are not the essence of this proposal.

(Color conversion processor)
The color conversion processing unit converts the input signals YCbCr through RGB data into a printer device color. For example, in the case of a printer loaded with six types of ink, the signals are converted into cyan (C), magenta (M), yellow (Y), black (K), light cyan (LC), and light magenta (LM) signals. . If the input is an RGB signal value, the conversion is (R, G, B) → (C, M, Y, K, LC, LM).

  There are several conversion methods, but here, a method using a look-up table (hereinafter referred to as LUT) and interpolation processing is taken as an example.

  The LUT prepared in advance describes C, M, Y, K, LC, and LM signal values corresponding to 4913 points obtained by dividing the RGB values as shown in FIG. 11 into 16 levels each. The C, M, Y, K, LC, and LM values corresponding to the values of the inputs R, G, and B are referred to. When the values of R, G, and B are between 16 divisions, they are obtained by interpolation processing.

  An example of the interpolation method is a tetrahedral interpolation method. The tetrahedral interpolation method is linear interpolation using four grid points with a unit of three-dimensional space as a tetrahedron. As the procedure, first, as shown in FIG. Then, the tetrahedron to which the target point p belongs is determined. The four vertices of the tetrahedron are defined as p0, p1, p2, and p3, and are divided into smaller tetrahedrons as shown in FIG. Also, if the conversion values of the extension points are f (p0), f (p1), f (p2), and f (p3), respectively,

より求まる。ここで、w0、w1、w2、w3は、各頂点piと反対向位置の小四面体の体積比である。これにより、インク色Ｃ，Ｍ，Ｙ，Ｋ，ＬＣ，ＬＭへの変換が行われる。

More. Here, w0, w1, w2, and w3 are volume ratios of the small tetrahedrons at positions opposite to each vertex pi. Thereby, conversion to ink colors C, M, Y, K, LC, and LM is performed.

(Quantization processing unit)
The data converted into the ink color is subjected to quantization processing. This converts multi-bit data into the number of bits that can be recorded by a recording apparatus. In the present embodiment, a case of an inkjet printer having 1 bit / binary of recording (1) / non-recording (0) is considered. Here, the quantization method is an error diffusion method that is optimal for the quantization of photographic images. The input signal is 8 bits from 0 to 255.

FIG. 13 is a diagram showing an error distribution method in the error diffusion method. When the signal value of the target pixel is L (0 ≦ L ≦ 255), it is compared with the threshold value TH. Depending on its size,
L> TH ・ ・ ・ ・ ・ ・ 1 (Recording)
L ≦ TH ・ ・ ・ ・ ・ ・ 0 (not recorded)
It is determined. The error E (= L−TH) generated at that time is distributed to surrounding pixels according to the distribution coefficient shown in FIG. By performing this processing for all pixels and all ink color colors C, M, Y, K, LC, and LM, 1-bit image data colors C ′, M ′, Y ′, K ′, LC ′, Quantized to LM ′.

  Next, how each processing block is executed will be described using the processing flow shown in FIG.

(When no correction is selected)
First, in STEP 1401 and whether or not color fog correction or backlight correction is selected by the user. Here, when none is selected, the normal flow is performed without any correction processing. After proceeding to STEP 1402, JPEG decoding processing is performed, and then sent to STEP 1403 correction processing. The correction coefficient is a value that has not been applied in an initialization process (not shown) 3 × 3 matrix:

ＬＵＴ：Ｙ＝Ｘのリニアな直線
Ｓ：１．００
が記述されているので、ＳＴＥＰ１４０３が行われてもデータは不変である。処理負荷を減らしたいのであれば、ＳＴＥＰ１４０１の判定の結果により、ＳＴＥＰ１４０３をスキップすることも可能である。その後、ＳＴＥＰ１４０４の変倍処理が実行され、ＳＴＥＰ１４０５の色変換処理、続いてＳＴＥＰ１４０６の量子化処理が実行される。量子化後のデータ、プリンタに送信され、プリンタはこのデータに基づいて記録を行う（図示せず）。ＳＴＥＰ１４１３において、全ての画素が終了したかの判定を行い、もしそうでなければＳＴＥＰ１４０２へと戻る。これを全ての画素について行い、終了となる。

LUT: Y = X linear straight line S: 1.00
Therefore, even if STEP 1403 is performed, the data is not changed. If it is desired to reduce the processing load, STEP 1403 can be skipped based on the determination result of STEP 1401. Thereafter, the scaling process of STEP 1404 is executed, the color conversion process of STEP 1405, and the quantization process of STEP 1406 are executed. The quantized data is transmitted to the printer, and the printer performs recording based on this data (not shown). In STEP1413, it is determined whether all the pixels have been completed. If not, the process returns to STEP1402. This is performed for all the pixels, and the process ends.

(When only color cast correction is selected)
Similarly, a flow when only color cast correction is selected by panel operation will be described with reference to FIG.

  Since color cast correction is selected according to the determination in STEP 1401, the process proceeds to STEP 1407.

In STEP 1407, the decoding process is performed as described above to obtain a 3 × 3 matrix of color fog correction coefficients, a contrast correction coefficient LUT _{Y (contrast)} , and a saturation enhancement coefficient S _{(enhancement)} .

  In STEP 1408, it is determined whether or not reverse correction is selected. If this is the case, the process proceeds to STEP 1402 for JPEG decoding. From this point onward, the processing is equivalent to [when no correction is selected]. However, the coefficient obtained in STEP 1407 is used as the correction coefficient in STEP 1403. Thus, color cast correction is realized, and further contrast correction and saturation enhancement processing are performed.

(When only backlight compensation is selected)
Similarly, FIG. 14 will be used to explain the flow when only backlight correction is selected by panel operation.

  As the backlight correction is selected based on the determination in STEP1401, the process proceeds to STEP1407.

In STEP 1407, the decoding process is performed as described above to obtain a 3 × 3 matrix of color fog correction coefficients, a contrast correction coefficient LUT _{Y (contrast)} , and a saturation enhancement coefficient S _{(enhancement)} .

  In STEP 1408, it is determined whether or not reverse correction is selected. If this is the case, the process proceeds to STEP 1409. Here, color determination data and frequency determination data used for human skin determination for backlight correction are generated. For the color determination data, RGB data representing an 8 × 8 block is output on the memory. These data are subjected to YCbCr conversion by the following formula compliant with ITU-R BT.601.

Y = 0.900 × R + 0.587 × G + 0.114 × B
Cb = (− 0.299 × R−0.587 × G + 0.886 × B)
× 0.564 + 128
Cr = (0.701 × R−0.587 × G−0.114 × B)
× 0.713 + 128
The converted YCbCr data is subjected to the 3 × 3 matrix of the color fog correction coefficient generated in STEP 1407, the contrast correction coefficient LUT _{Y (contrast)} , and the saturation enhancement coefficient S _{(enhancement)} , and again,
R = Y + 1.402 (Cr-128)
G = Y−0.34414 (Cb−128) −0.71414 (Cr−128)
B = Y + 1.772 (Cb-128)
Thus, the RGB signal value is restored.

  In STEP 1410, by using the converted RGB data, color determination is performed on the image with the color cast corrected, and frequency component determination is performed at the same time. Thus, a correction coefficient for appropriately brightening is determined from the obtained human skin luminance value.

  Thereafter, in STEP 1411, color fog correction is determined, and in this case, the process proceeds to STEP 1412. In STEP 1412, a process of returning the 3 × 3 matrix to a default value (unit matrix) is performed so that the color cast correction is not actually executed.

  Thereafter, the process proceeds to STEP 1402 and the processing from here is the same as [when correction is not selected]. However, in this case, as the correction coefficient in STEP 1403, the default value is not used for the coefficient relating to the color cast correction, and the value obtained in STEP 1410 is used for the coefficient related to the backlight correction for converting the luminance. As a result, accurate backlight correction is realized for an image in which color cast or the like has occurred.

(When color cast correction and backlight correction are both selected)
Similarly, FIG. 14 will be used to explain the flow when color cast correction and backlight correction are selected by panel operation.

  Since both color fog correction and backlight correction are selected according to the determination in STEP 1401, the process proceeds to STEP 1407.

In STEP 1407, the decoding process is performed as described above to obtain a 3 × 3 matrix of color fog correction coefficients, a contrast correction coefficient LUT _{Y (contrast)} , and a saturation enhancement coefficient S _{(enhancement)} .

In STEP 1408, it is determined whether or not the retrograde correction is selected. In this case, the process proceeds to STEP 1409. Here, color determination data and frequency determination data used for human skin determination for backlight correction are generated. For the color determination data, RGB data representing an 8 × 8 block is output on the memory. These data are YCbCr converted according to the above-mentioned formula based on ITU-R BT.601 when only backlight correction is selected. The converted YCbCr data is subjected to the 3 × 3 matrix of the color fog correction coefficient generated in STEP 1407, the contrast correction coefficient LUT _{Y (contrast)} , and the saturation enhancement coefficient S _{(enhancement)} , and again the RGB signal value by the above conversion equation. Returned to

  In STEP 1410, by using the converted RGB data, color determination is performed on the image with the color cast corrected, and frequency component determination is performed at the same time. Thus, a correction coefficient for appropriately brightening is determined from the obtained human skin luminance value.

  Thereafter, in STEP 1411, it is determined whether or not color fog correction is performed. In this case, since color fog correction is selected, the process proceeds to STEP 1402.

  In STEP 1402 and subsequent steps, the process is the same as [when correction is not selected]. However, the coefficient obtained in STEP 1407 is used as the correction coefficient in STEP 1403. As a result, color cast and backlight correction are realized.

  As described above, in the first embodiment of the present invention, in the printer capable of direct printing from the PC card, the case where the user selects the color fog correction and the backlight correction by the panel is shown. By performing the means of the first embodiment, it is possible to detect human skin appropriately and to perform more appropriate correction.

[Embodiment 2]
As a second embodiment of the present invention, a case where a noise removal process is added to the first embodiment described above will be described. As in the first embodiment, the recording apparatus is an ink jet recording apparatus (photo direct printer apparatus) capable of printing an image taken with a digital camera recorded on a PC card. Therefore, detailed description of the recording apparatus is omitted here.

  The settings are made using the panel.

と表示が切り替わり、ユーザーが設定可能となっている。

The display is switched and the user can set it.

  FIG. 15 shows a block diagram of processing in the present embodiment. The difference from FIG. 5 which is the first embodiment is that a noise removal processing unit is added after the JPEG decoding processing unit. The noise removal processing unit will be described.

(Noise removal processing part)
Various methods have been proposed and implemented as a noise removal method. Here, an average value filter which is one of general noise removal methods will be described as an example. FIG. 16 is an example of the average value filter. This is a 5 × 5 window size filter, and the pixel marked with * in the center is the target pixel. When the luminance value of the target pixel is Y _TGT , this is obtained by the following equation.

これらを全画素について行うことで、平滑化が実行され、結果としてノイズが除去される。画像の端部については、５ｘ５のウィンドウサイズを確保出来ないので、その場合は確保可能な限りのサイズで行えばよい。同様に色差成分についても、平均化処理を行う。

By performing these operations for all pixels, smoothing is executed, and as a result, noise is removed. Since the 5 × 5 window size cannot be secured at the edge of the image, in that case, the size may be as large as possible. Similarly, the color difference component is averaged.

  Further, in the case of a simple average value filter, the edge portion is also smoothed, the edge becomes dull, and the image is blurred. In order to avoid this, an edge is detected, and when the edge is determined, the smoothing filter is not applied and the edge is secured. For edge detection, a differential filter represented by a Prewitt filter or a Sobel filter may be used.

  FIG. 17 shows a flowchart in the present embodiment. The difference from FIG. 14, which is the first embodiment, is that STEPs 1703, 1704, and 1713 related to noise processing are added. Take up and explain the main cases.

(When only noise reduction is selected)
The case where only the noise removal process is selected will be described with reference to FIG.

  In STEP1701, since it is only noise removal, it does not determine to STEP1710, but progresses to STEP1702. After the decoding processing of the JPEG image is performed in STEP 1702, the noise removal is determined in STEP 1703. If noise removal is selected, the above-described noise removal processing is executed in STEP 1704. Thereafter, STEPs 1705 to 1709 are the same as the flow described in the first embodiment, and the details are omitted. After the quantization processing in STEP 1708 is completed, it is determined in STEP 1709 whether processing has been completed for all pixels. If not, the processing returns to STEP 1702 and the processing is repeated again.

(When only backlight compensation is selected)
Similarly, the case where only the retrograde correction is selected will be described with reference to FIG.

  Since backlight correction is selected, the process proceeds from STEP 1701 to STEP 1710, JPEG decoding processing is performed, and parameters for color cast correction are determined from the histogram. In STEP 1711, the presence / absence of backlight correction is determined. If this is the case, the process proceeds to STEP 1712. In STEP 1712, the processing is the same as in STEP 1409 in FIG. 14 of the first embodiment, and the parameters created in STEP 1710 are applied to the human skin determination data to remove the color cast. Further, noise removal processing is executed on the human skin determination image converted in STEP 1713. In STEP 1714, the backlight correction coefficient determination process is executed using the image. As a result, the image subjected to human skin determination processing is an image having no color cast and noise removed by noise removal processing. Human skin is detected with higher accuracy, and as a result, a more optimal backlight correction coefficient is determined. Is done.

  Thereafter, since color cast correction is not selected, the color cast coefficient is returned to the default value in STEP 1715, and the process proceeds to STEP 1702.

  In STEP 1703, noise removal is determined. In this case, STEP 1704 is skipped and the process proceeds to STEP 1705. The subsequent flow is the same as the above-described flow.

(When color cast correction, backlight correction, and noise reduction are all selected)
Similarly, the case where all the correction processes are selected will be described with reference to FIG.

  If all the correction processes are selected in STEP 1701, the process proceeds to STEP 1710, JPEG decoding is performed, the color cast correction parameters are determined from the histogram, and after the determination of STEP 1711, the human skin is determined in STEP 1712. The color cast by the coefficient of STEP 1710 is removed from the image for determination, and the noise removal process of STEP 1713 is executed. In STEP 1714, the backlight correction coefficient determination process is executed using the image. This means that human skin has been detected for images that have no color cast and noise is removed by noise removal processing, and detection is realized with higher accuracy. As a result, a more optimal backlight correction coefficient is determined. Is done.

  Thereafter, the process returns from STEP 1715 to STEP 1702, and thereafter, the same processing as described above is executed.

  As described above, according to the second embodiment, it has been described that human skin detection processing with higher detection accuracy is realized when color fog correction and noise removal processing exist in addition to backlight correction. Although the filtering process is used in the noise removal process, it goes without saying that the detection accuracy can be improved by combining with a filtering process having various other effects.

[Other Embodiments]
In the embodiment of the present invention, direct printing from a card is representatively described. However, the application range is not limited to this, and printing from a normal PC is also included. Further, not only the output device but also an input device such as a scanner can be applied with the same processing, and an effect can be obtained. Furthermore, the present invention may be applied to a complex system such as a copy composed of a plurality of devices, scanners, and printers.

  Another object of the present invention is to supply a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

It is a figure which shows an example of the brightness | luminance histogram of the image which is performing the color cast which exposure is not appropriate. 1 is a schematic perspective view of a photo direct printer according to an embodiment of the present invention. FIG. 3 is an overview of an operation panel of the photo direct printer according to the embodiment of the present invention. It is a block diagram which shows the structure of the principal part which concerns on control of the photo direct printer apparatus which concerns on embodiment of this invention. It is a block diagram about the image processing which concerns on 1st embodiment of this invention. It is a flowchart showing the coefficient determination of the backlight correction process which concerns on embodiment of this invention. It is a figure which shows the structure of MCU of a JPEG image. It is a figure explaining the handling method of AC component of the frequency determination of the human skin detection which concerns on embodiment of this invention. It is a figure which shows the luminance conversion table for backlight correction which concerns on embodiment of this invention. It is a figure of YCC space rotated so that the line which connects the highlight point and shadow point which concerns on embodiment of this invention may overlap with a luminance axis. It is a figure which shows an example of the look-up table used for the color conversion process which concerns on this Embodiment. It is a figure which shows the tetrahedral interpolation process used for the color conversion process which concerns on this Embodiment. It is a figure showing the distribution method in the error diffusion method which concerns on this Embodiment. It is a flowchart about the image processing which concerns on 1st embodiment of this invention. It is a block diagram about the image processing which concerns on 2nd embodiment of this invention. It is a figure regarding the filter process which concerns on 2nd embodiment of this invention. It is a flowchart about the image processing which concerns on 2nd embodiment of this invention.

Claims (10)

In an image processing apparatus that applies a plurality of image processing means to an input image,
At least one of the image processing means is a first image processing means having a processing coefficient determining means for determining a coefficient to be used when executing the processing,
The processing coefficient determination means is a processing coefficient determination means that determines using another second image processing means,
An image processing apparatus, wherein the first image processing means is executed using the processing coefficient determined by the processing coefficient determination means.   The image processing apparatus according to claim 1, wherein the processing coefficient determination unit includes a characteristic region extraction unit that extracts a specific region, and determines a processing coefficient based on characteristics of the extracted region.   The image processing apparatus according to claim 2, wherein the feature region extraction unit performs the second image processing on data used for extraction determination.   The image processing apparatus according to claim 1, wherein the second processing unit changes a hue or luminance of the entire image.   5. The image processing apparatus according to claim 1, wherein the second processing unit is a filter process that determines a value using surrounding pixel values. In an image processing method for performing a plurality of image processing steps on an input image,
At least one of the image processing steps is a first image processing step including a processing coefficient determination step for determining a coefficient to be used when executing the process.
The processing coefficient determination step is a processing coefficient determination step that is determined using another second image processing step,
An image processing method, wherein the first image processing step is performed using the processing coefficient determined by the processing coefficient determination step.   The image processing method according to claim 6, wherein the processing coefficient determination method includes a feature area extraction method for extracting a specific area, and the processing coefficient is determined based on characteristics of the extracted area.   The image processing method according to claim 7, wherein the feature region extraction method performs the second image processing on data used for extraction determination.   The image processing method according to claim 6, wherein the second processing method changes a hue or luminance of the entire image.   The image processing apparatus according to claim 1, wherein the second processing unit is a filter process that determines a value by using a surrounding pixel value.

Images