JP2004242068A - Method, apparatus, and program for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing technology capable of forming a well-balanced image by suppressing unnaturalness easy to occur at a boundary between objects, and reproducing an important object in proper image characteristics. <P>SOLUTION: The nature of a boundary between divided regions is evaluated, and the amount of correction for a region near the boundary is determined according to the nature of the boundary. Namely, the image processing method divides an image into a plurality of regions, and determines the amount of correction for image characteristics for every region, and conducts image correction processing. The method evaluates the nature of the boundary between a plurality of regions, and determines the amount of correction for a region near the boundary according to the nature of the evaluated boundary. Consequently, as an image is conventionally divided into a plurality of regions and correction processing is conducted for image characteristics, troubles such as the need for a high technology and a long experience or taking a time and effort are solved. Accordingly, an image processing technology offering proper correction in an automatic or partial automatic way is acquired. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【００６５】
を用いて、入力信号ｆ（ｘ）に対するウェーブレット変換係数〈ｆ，ψ_ａ，_ｂ〉を
【００６６】
【数２】

【００６７】
で求める事により、入力信号を下記のようなウェーブレット関数の総和に分解する変換である。
【００６８】
【数３】

【００６９】
上式で、ａはウェーブレット関数のスケールを表し、ｂはウェーブレット関数の位置を示す。図２に例示するように、スケールａの値が大きいほどウェーブレット関数ψ_ａ，_ｂ（ｘ）の周波数は小さくなり、また位置ｂの値に従ってウェーブレット関数ψ_ａ，_ｂ（ｘ）が振動する位置が移動する。従って式（３）は、入力信号ｆ（ｘ）を種々のスケールと位置を持つウェーブレット関数ψ_ａ，_ｂ（ｘ）の総和に分解する事を意味している。
【００７０】
上記のような変換を可能にするウェーブレット関数は多くのものが知られているが、画像処理分野では計算が高速な直交ウェーブレット（ｏｒｔｈｏｇｏｎａｌ ｗａｖｅｌｅｔ）・双直交ウェーブレット（ｂｉｏｒｔｈｏｇｏｎａｌ ｗａｖｅｌｅｔ）が広く用いられている。以下、直交ウェーブレット・双直交ウェーブレットの変換計算の概要を説明する。
【００７１】
直交ウェーブレット・双直交ウェーブレットのウェーブレット関数は下記のように定義される。
【００７２】
【数４】

【００７３】
式（４）と式（１）を比べると、直交ウェーブレット・双直交ウェーブレットではスケールａの値が２のｉ乗で離散的に定義され、また位置ｂの最小移動単位が２^ｉで離散的に定義されている事が判る。このｉの値はレベルと呼ばれる。また実用的にはレベルｉを有限な上限Ｎまでに制限して、入力信号を下記のように変換することが行われる。
【００７４】
【数５】

【００７５】
式（５）の第２項は、レベル１のウェーブレット関数ψ_１，_ｊ（ｘ）の総和で表せない残差の低周波数帯域成分を、レベル１のスケーリング関数φ_１，_ｊ（ｘ）の総和で表したものである。スケーリング関数はウェーブレット関数に対応して適切なものが用いられる（前記文献を参照）。式（５）に示す１レベルのウェーブレット変換により入力信号ｆ（ｘ）≡Ｓ_０は、レベル１の高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１に信号分解された事になる。ウェーブレット関数ψ_ｉ，_ｊ（ｘ）の最小移動単位は２^ｉなので、入力信号Ｓ_０の信号量に対して高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１の信号量は各々１／２となり、Ｗ_１とＳ_１の信号量の総和は、入力信号Ｓ_０の信号量と等しくなる。レベル１の低周波数帯域成分Ｓ_１は式（６）でレベル２の高周波数帯域成分Ｗ_２と低周波数帯域成分Ｓ_２に分解され、以下同様にレベルＮ迄の変換を繰り返すことで、入力信号Ｓ_０は、式（７）に示すようにレベル１〜Ｎの高周波数帯域成分の総和とレベルＮの低周波数帯域成分の和に分解される。
【００７６】
ここで、式（６）で示す１レベルのウェーブレット変換は、図３に示すようなフィルタ処理で計算できる事が知られている（前記文献を参照）。図３においてＬＰＦはローパスフィルタ、ＨＰＦはハイパスフィルタを示している。フィルタ係数はウェーブレット関数に応じて適切に定められる（前記文献及び表１を参照）。
【００７７】
【表１】

【００７８】
また２↓は、信号を１つおきに間引くダウンサンプリングを示す。画像信号のような２次元信号における１レベルのウェーブレット変換は、図４に示すようなフィルタ処理で計算される。図４においてＬＰＦｘ，ＨＰＦｘ，２↓ｘはｘ方向の処理を示し、ＬＰＦｙ，ＨＰＦｙ，２↓ｙはｙ方向の処理を示す。この１レベルのウェーブレット変換により、低周波数帯域成分Ｓ_ｎ−１は３つの高周波数帯域成分Ｗｖ_ｎ，Ｗｈ_ｎ，Ｗｄ_ｎと１つの低周波数帯域成分Ｓ_ｎに分解される。分解で生成するＷｖ_ｎ，Ｗｈ_ｎ，Ｗｄ_ｎ，Ｓ_ｎの各々の信号量は、分解前のＳ_ｎ−１に比べて縦横ともに１／２となるので、分解後の４成分の信号量の総和は、分解前のＳ_ｎ−１の信号と等しくなる。入力信号Ｓ_０が３レベルのウェーブレット変換で信号分解される過程の模式図を図５に示す。
【００７９】
また、分解で生成したＷｖ_ｎ，Ｗｈ_ｎ，Ｗｄ_ｎ，Ｓ_ｎに図６で示すようなフィルタ処理で計算されるウェーブレット逆変換をほどこすことにより、分解前の信号Ｓ_ｎ−１を完全再構成できる事が知られている。図６においてＬＰＦ’はローパスフィルタ、ＨＰＦ’はハイパスフィルタを示している。このフィルタ係数は、直交ウェーブレットの場合にはウェーブレット変換に用いたのと同じ係数が使用されるが、双直交ウェーブレットの場合にはウェーブレット変換に用いたのと異なる係数が使用される。（前述の参考文献を参照）。また２↑は、信号に１つおきにゼロを挿入するアップサンプリングを示す。またＬＰＦ’ｘ，ＨＰＦ’ｘ，２↑ｘはｘ方向の処理を示し、ＬＰＦ’ｙ，ＨＰＦ’ｙ，２↓ｙはｙ方向の処理を示す。
【００８０】
本発明で利用する二項ウェーブレット（Ｄｙａｄｉｃ Ｗａｖｅｌｅｔ）変換については、“Ｓｉｎｇｕｌａｒｉｔｙ ｄｅｔｅｃｔｉｏｎ ａｎｄ ｐｒｏｃｅｓｓｉｎｇ ｗｉｔｈ ｗａｖｅｌｅｔｓ”ｂｙ Ｓ．Ｍａｌｌａｔ ａｎｄ Ｗ．Ｌ．Ｈｗａｎｇ，ＩＥＥＥ Ｔｒａｎｓ．Ｉｎｆｏｒｍ．Ｔｈｅｏｒｙ３８ ６１７（１９９２）や“Ｃｈａｒａｃｔｅｒｉｚａｔｉｏｎ ｏｆ ｓｉｇｎａｌｓ ｆｒｏｍ ｍｕｌｔｉｓｃａｌｅ ｅｄｇｅｓ”ｂｙ Ｓ．Ｍａｌｌａｔ ａｎｄ Ｓ．Ｚｈｏｎｇ，ＩＥＥＥ Ｔｒａｎｓ．Ｐａｔｔｅｒｎ Ａｎａｌ．Ｍａｃｈｉｎｅ Ｉｎｔｅｌ．１４ ７１０（１９９２）や“Ａ ｗａｖｅｌｅｔ ｔｏｕｒ ｏｆ ｓｉｇｎａｌ ｐｒｏｃｅｓｓｉｎｇ ２ｅｄ．”ｂｙ Ｓ．Ｍａｌｌａｔ，Ａｃａｄｅｍｉｃ Ｐｒｅｓｓに詳細な説明があるが、以下に概要を説明する。
【００８１】
二項ウェーブレットのウェーブレット関数は下記のように定義される。
【００８２】
【数６】

【００８３】
直交ウェーブレット・双直交ウェーブレットのウェーブレット関数は前述のようにレベルｉにおける位置の最小移動単位が２^ｉで離散的に定義されていたのに対し、二項ウェーブレットはレベルｉにかかわらず位置の最小移動単位が一定である。この相違により、二項ウェーブレット変換には下記の特徴が生じる。
【００８４】
特徴１：下記に示す１レベルの二項ウェーブレット変換で生成する、高周波数帯域成分Ｗ_ｉと低周波数帯域成分Ｓ_ｉの各々の信号量は、変換前の信号Ｓ_ｉ−１と同一である。
【００８５】
【数７】

【００８６】
特徴２：スケーリング関数φ_ｉ，_ｊ（ｘ）とウェーブレット関数ψ_ｉ，_ｊ（ｘ）の間に下記の関係が成立する。
【００８７】
【数８】

【００８８】
従って二項ウェーブレット変換で生成する、高周波数帯域成分Ｗ_ｉは、低周波数帯域成分Ｓ_ｉの一次微分（勾配）を表す。
【００８９】
特徴３：ウェーブレット変換のレベルｉに応じて定められた表２に示される係数γ_ｉ（前出の二項ウェーブレットに関する参考文献参照）を高周波数帯域成分に乗じたＷ_ｉ・γ_ｉ（以下、これを補正済高周波数帯域成分と呼ぶ）について、入力信号の信号変化の特異性（ｓｉｎｇｕｌａｒｉｔｙ）に応じて、前記変換後の補正済高周波数帯域成分Ｗ_ｉ・γ_ｉの信号強度のレベル間の関係が一定の法則に従う。すなわち図７の１や４に示すなだらかな（微分可能な）信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉはレベル数ｉが増大するほど信号強度が増大するのに対して、図７の２に示すステップ状の信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉはレベル数ｉに関わらず信号強度が一定となり、図７の３に示すδ関数状の信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉはレベル数ｉが増大するほど信号強度が減少する。
【００９０】
【表２】

【００９１】
特徴４：画像信号のような２次元信号における１レベルの二項ウェーブレット変換の方法は、前述の直交ウェーブレット・双直交ウェーブレットの方法と異なり、図８のように行われる。この１レベルのウェーブレット変換により、低周波数帯域成分Ｓ_ｎ−１は２つの高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎと１つの低周波数帯域成分Ｓ_ｎに分解される。２つの高周波数帯域成分は低周波数帯域成分Ｓ_ｎの２次元における変化ベクトルＶ_ｎのｘ成分とｙ成分に相当する。変化ベクトルＶ_ｎの大きさＭ_ｎと偏角Ａ_ｎは下式で与えられる。
【００９２】
【数９】

【００９３】
また二項ウェーブレット変換で得られた２つの高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎと１つの低周波数帯域成分Ｓ_ｎに図８に示す二項ウェーブレット逆変換をほどこす事で、変換前のＳ_ｎ−１を再構成できる事が知られている。
【００９４】
また、入力信号Ｓ_０に対してＮレベルの二項ウェーブレット変換を行う場合の概念を図１０に示す。入力信号Ｓ_０に対してＮレベルの二項ウェーブレット変換を行い、得られた高周波数成分に対して、必要に応じ、操作１の作業を行った後に、Ｎレベルの二項ウェーブレット逆変換を行い、また、低周波成分について、前述の二項ウェーブレット逆変換の操作の各ステップにおいて、操作２の作業を行う。なお、本実施の形態の一例においては、操作１とはエッジ検出、パターン検出等の操作であり、操作２とはマスキング処理にあたる。
【００９５】
図１０においてＬＰＦは変換用ローパスフィルタ、ＨＰＦは変換用ハイパスフィルタを示し、ＬＰＦ’は逆変換用ローパスフィルタ、ＨＰＦ’は逆変換用ハイパスフィルタを示している。これらのフィルタ係数はウェーブレット関数に応じて適切に定められる（前述の参考文献及び表３を参照）。
【００９６】
【表３】

【００９７】
また、ＬＰＦｘ，ＨＰＦｘ，ＬＰＦ’ｘ，ＨＰＦ’ｘ，はｘ方向の処理を示し、ＬＰＦｙ，ＨＰＦｙ，ＬＰＦ’ｙ，ＨＰＦ’ｙはｙ方向の処理を示す。また二項ウェーブレットにおいては、レベル毎にフィルタ係数が異なり、レベルｎのフィルタ係数は、レベル１のフィルタの各係数の間に２^ｎ−１−１個のゼロを挿入したものが用いられる（前述の参考文献及び表３を参照）。
【００９８】
前述の二項ウェーブレット変換の特徴１で、変換後の分解画像サイズが、変換前の画像と同一である旨説明したが、このことにより、特徴３に示されるような画像構造の解析において高い位置精度をもって評価を行うことができるという副次的な特徴も得られる。
【００９９】
次に、多重解像度変換の手法を用いた被写体パターンの抽出について、図１１〜１３の例により説明する。
【０１００】
前記に説明した二項ウェーブレット変換を用いて画像を多重解像度変換し、多重解像度変換の各レベルに現れたエッジを検出、領域分割を行う。
【０１０１】
そして、抽出するパターンに応じて、パターン抽出に利用する解像度レベルを設定する。
【０１０２】
ここでいうパターン、特に被写体パターンとして一般に認知されるものは、その輪郭のみではなく、種種の固有の部分要素をもっているものがほとんどである。
【０１０３】
たとえば人物の頭部であれば、その輪郭そのもののほか、目（さらに瞳、虹彩、まつげ、白目の血管）、鼻、口、ほほの起伏、えくぼ、眉、などがある。
【０１０４】
これらの内、抽出するパターンを識別するのに有効な部分要素をその「構成要素」と位置付け、それぞれに対し、パターン抽出に利用する解像度レベルを設定する。
【０１０５】
たとえば、図１２に示される通り、人物の頭部の輪郭そのものは低レベルの分解画像に抽出されるエッジで、はっきりと、かつ正確に認識され、その内部に存在する、顔の構成要素の緩やかなパターン、たとえば鼻筋、唇の形状、笑顔の口唇周囲に出来る線、「えくぼ」、「ほほのふくらみ」などは、より高レベル分解画像に現れる、エッジ情報を用いることで、その特徴を的確に捉えることが出来る。
【０１０６】
次に、被写体パターンの構成要素の決定方法、及び、各々を識別する、好適解像度レベルの決定方法について、好ましい１例を説明する。
【０１０７】
まず、被写体パターンの構成要素を設定する。たとえば、一般的に「人物の顔」であれば、下記記載のような、あらかじめ記憶された、各種構成要素となる。
【０１０８】
（「人物の顔」の場合の構成要素の１例）
ａ：顔の輪郭
ｂ：瞳
ｃ：眉
ｄ：口
ｅ：髪の毛
ｆ：鼻梁
ｇ：鼻孔
ｈ：ほほの凸部
また、特定人物を被写体パターンとして登録したような場合は、これらに追加して新たな構成要素を設定しても良く、個人特定が好ましく実施できる。
【０１０９】
（「特定人物の顔」で追加される構成要素の例）
ｉ：しみ、ほくろ
ｊ：えくぼ
ｋ：髭
特定人物の場合では、ａ〜ｆの構成要素について、一般的な「人物の顔」という場合とは異なる特性を設定できるし、いくつかの構成要素は「無い」という場合もある。
【０１１０】
目的とする被写体パターンについて各々の構成要素が設定できたら、この画像を二項ウェーブレット変換を用いて、多重解像度変換し、各々の構成要素について、多重解像度変換の各レベルの分解信号における信号強度を求め、最大となるレベルを求める。前述の最大となるレベルを好適解像度として用いればよいが、実際の画像処理結果を評価して、若干のレベル変更を行ってもかまわない。
【０１１１】
なお、この場合の信号とは、各々のレベルで検出されたエッジ成分を示す信号の最大値であるが、複数のレベル間で信号強度を比較する際には、信号値として、前出二項ウェーブレットで説明した補正済み高周波帯域成分を用いる事が好ましいのは言うまでも無い事である。
【０１１２】
ところで、二項ウェーブレット変換を利用した場合、ナイフエッジパターンのような、非常に輪郭のはっきりした構成要素の場合では、エッジの信号レベルが解像度レベルによって大きく変化しない特性があるが、このような場合は、当該構成要素の輪郭形態がはっきりと認識できるレベル、または、もともとの画像解像度が十分でない場合においては、最も低レベルの分解解像度を好適解像度レベルとする。
【０１１３】
前述の構成要素には、輪郭の比較的はっきりしたものと、はっきりしないものがある。
【０１１４】
たとえばａ、ｆ、ｉなどが前者にあたり、ｆ、ｈ、ｊなどが後者にあたる。前者のような構成要件の抽出、登録は、たとえばモニタに画像を表示し、当該箇所をマウス、や接触型センサなどで指定して、近傍領域を自動的あるいは、手動的に切り抜いて行うことが出来る。
【０１１５】
後者のような場合には、当該構成要素の存在する領域を、存在しない領域と、はっきりと区別し、切り抜くことは困難であるが、そのような場合においては、その構成要素が存在する領域を大まかに指定すればよい。
【０１１６】
このような構成要件に対して設定される好適解像度は、前者の輪郭のはっきりしたものよりも高レベルとなっているのが普通である。
【０１１７】
したがって、前述のように、大まかな領域指定を行った場合に実際に後者のような構成要素の抽出を行う際には、以下のようにして、目標とする構成要素を抽出できる。
【０１１８】
構成要素を抽出する候補領域に検出されたエッジをすべて抽出し、これらについて、各解像度レベルの信号強度を比較する。
【０１１９】
好適解像度レベルより低レベルの分解画像で信号強度が強く検出されたエッジ成分は当該構成要素には含まれないものと考えられるため、候補領域から除外する。そして、残った領域を、好適解像度レベルで検査して目的とする構成要素を抽出する。
【０１２０】
以上の例では、分解前の画像をモニタに表示し、構成要素の指定を行ったが、たとえば画像処理技術に関してある程度の知識を有しているものが構成要素の指定を行う場合には、実際に解像度変換を行った分解画像をモニタに表示し、好ましくは分解前の画像と対比可能な構成で表示して表示されている解像度レベルで抽出すべき構成要素を指定できるようにすると、元画像だけでは認識し得ない、新たな特徴点の発見も簡単に行うことが出来、より、被写体パターン識別精度を向上することが出来る。
【０１２１】
図示の例では、瞳、上瞼のエッジをＡ、鼻筋、口唇周囲の線をＢ、ほほのふくらみをＣとしている。
【０１２２】
前述の通り、ＡよりＢ、ＢよりＣをより高い解像度レベルの画像で検出することで、的確に顔の特徴認識が出来る。
【０１２３】
さらに、図示されたように、抽出すべきパターンの大きさに応じて、前記構成要素の検出に用いるレベルを設定すると、たとえば抽出すべきパターンが十分に大きな場合には、パターンを構成する要素各々の特性が良く分離され、それぞれの構成要素に適した解像度レベルが設定でき、上記エッジ情報の検出に用いるレベルを設定すると、大きなパターンでは細かな情報まで用いたパターン検出、小さなパターンでは、その大きさで得られる情報までを用いて、最大限、効果的、かつ高速な検出処理が行えるという、優れた特徴を有する。
【０１２４】
上記パターンの大きさは、別途、仮のパターン検出を行って、その大きさから求めてもよく、または、シーン属性（記念写真、ポートレートなど）、画像サイズ、から仮に求めても良い。
【０１２５】
仮のパターン抽出は、たとえば次の方法により行うことができる。
顔のパターン抽出を行うような場合は、まず肌色領域を画面内から抽出し、その領域の形状評価を行って、丸い形をしていたら「顔候補」として、抽出する手法がある。
【０１２６】
制服のような、特定色を持っているものの場合は、特定色領域を抽出し、領域形状の評価条件が、丸から、長方形、三角等、他の形態に変わるだけである。
【０１２７】
その他、画像からエッジ成分を求めて、外形パターンが似ているものをすべて抽出する方法も用いることが出来、この際のエッジ成分を求める処理は、前述多重解像度変換の、所定レベルの分解画像から求めても良いし、一般的なラプラシアンフィルタ処理で抽出しても良い。
【０１２８】
ここで、パターンの大きさとは、たとえばパターンのサイズを画素数で表す事が出来るが、図示の例では、顔の大きさ「中」のサイズがあれば、Ａ、Ｂ、Ｃそれぞれに好ましい特徴抽出レベルが定められる。
【０１２９】
もともとの画像サイズ（つまりパターンのサイズ、画像解像度）が非常に大きい場合、前述の「中」の大きさに相当する画像サイズまで解像度変換を行い、パターン抽出処理を行うことで、必要な計算処理量を大きく減らすことが出来、好都合である。
【０１３０】
前処理として行う解像度変換は、たとえば周知の技術である、最近傍法、線形補間法等の手法で簡単に行うことが出来る。
【０１３１】
特開２０００−１８８６８９号や、特開２００２−２６２０９４号には、拡大、縮小の手法について詳細な記載があるので、これらに記載の手法を用いるのも良い。
【０１３２】
さらに、フィルムスキャナやフラットベッドスキャナのように、あらかじめプレスキャンを行い、画像スキャン領域、あるいは本スキャン駒を確定するような処理シーケンスを有する画像処理装置の場合、前述の仮のパターン抽出とパターンの大きさ評価をプレスキャンの段階で行い、本スキャンの読み取り解像度を、パターン抽出に適した画像解像度で読み取るようにしても良い。
【０１３３】
このようにすることで、抽出するパターンが小さい場合にも十分な解像度が確保できるし、大きい場合には本スキャンの解像度を必要十分な値に設定することで、スキャンに要する時間を軽減することが出来る。
【０１３４】
次に、画像内から抽出できる被写体パターンをすべて探す方法を例を挙げて説明する。前述のように、抽出すべき被写体パターンは決定されたシーン属性に応じて切り替える。以下にいくつかの例を示す
（例）シーン属性 → 抽出する被写体パターン（左のほうが、優先順位が高い）
修学旅行・京都 → 顔／制服を着た人物／歴史建築（和建築）
結婚披露宴 → 新婦／新郎／顔／ドレス／スポットライト
上記例の、新婦、新郎と顔、スポットライトとドレスのように、重なり合って存在するパターン要件もある
ここで、上記被写体パターンは、あらかじめ定められているものでも良いが、たとえば図１４、１５で示されるような、以下の手法で新たに設定することも出来る。
【０１３５】
画像をモニタに表示し、主要画像部分を指示する。そして、指示部分を含む輪郭領域を自動抽出し、得られたパターンを、仮に単位パターンと称する事にする。
【０１３６】
必要とするパターン全体が含まれていない場合は、上記操作を繰り返し、微小輪郭を結合していき、全体の輪郭抽出が終了したところで、登録指示を行う（登録キーを押す）。
【０１３７】
登録情報は、選択された領域に関する情報（いくつの、どんな単位パターンが、どのように連結している集合か、や、領域全体に関する各種特性値）、領域の名称（制服を着た学生、等）、優先順位情報などからなる。
【０１３８】
さらに前記単位パターンとして、「顔」や「制服」等、前出の被写体パターンに相当するやや複雑な構成のものを指定してもよく、これらの結合で、「学生」等の、より高度な被写体パターンの登録が簡単に出来る。
【０１３９】
このようにして登録された被写体パターンの一例について、図１４、１５を用いて説明する。図１４に示されるように、「学生」というカテゴリには、（ａ）男子学生（ｂ）女子学生の二つのカテゴリがあり、それぞれ、▲１▼、▲２▼及び▲３▼と、▲１▼、▲４▼及び▲５▼という固有の要素を持っており、これらを単位パターンとした結合状態で、「学生」が定義される。
【０１４０】
これを論理式を用いて表すと、
「学生」＝（▲１▼ａｎｄ▲２▼ａｎｄ▲３▼）ｏｒ（▲１▼ａｎｄ▲４▼ａｎｄ▲５▼）となる。
【０１４１】
以上の▲１▼〜▲５▼の構成要素それぞれは、各々が個別の単位パターンが結合した状態で定義されるが、その一例として、女子学生の上衣について、図１５に示すが、図示のとおり、図１５（ａ）中の構成要素は、さらに単位パターンａ〜ｆの各要素から構成されており、その結合状態を表した図１５（ｂ）で定義される。
【０１４２】
なお、写真店における写真プリントの一般的な状況として、ロールフィルムからの同時プリント、デジタルカメラで撮影時に利用した画像記憶メディア等、関連した複数の駒について、一括してプリント注文する場合が多い（以下、一連の注文と表記）。
【０１４３】
一連の注文内に複数の画像がある場合には、その中の代表的な１枚の画像で、上記抽出、登録作業を行い、この情報を元に一連の画像群内、全画像のパターン抽出作業を行うことが出来、パターン登録作業の回数を減らし、効率的な作業が出来る。
【０１４４】
また、前記登録パターンが、ある個別顧客固有のものであった場合には、パターン登録したパターンを顧客情報といっしょに保存しておき、次回のプリント注文時に顧客情報から、必要な登録パターンを呼び出すようにしておくと、より手間が省け、高度なサービスが実現できる。
【０１４５】
さらに前記のような、一連の注文処理を行うような場合、全画面から、色々な想定され得る被写体パターンを抽出し、その出現頻度や、画面内における存在位置の統計結果から、シーン属性や優先順位を類推することも出来る。
【０１４６】
このようにすれば、注文主からシーン属性に関する情報が得られない場合でも、顧客のもっとも大事にしたい被写体が推測できる為、より高い確率で、顧客にとって好ましいプリントが簡単に得られる。
【０１４７】
次に、前記の処理により抽出した被写体に優先順位をつける。シーン属性に対応して定められている優先順位情報を元につけるが、さらに、被写体パターンの大きさ（大きいものを重視、など）、位置（中央部にあるものをより重視、など）により、優先順位情報に重み付けしても良く、これにより、被写体パターンの重要さに関し、さらに好ましい情報が得られる。以下、このようにして得られた優先度に関する情報を「重要度」とする。
【０１４８】
抽出すべき被写体パターンと、それら被写体パターンの優先順位情報の決定法として、さらに、ＧＰＳ信号と、時刻、地図、地勢情報や、インターネット等の自動検索エンジンを用いた検索情報、当該自治体、観光協会、商工会等の情報、など、やこれらをリンクした情報を用い、画像撮影地点において一般的に重要な被写体パターン、ランドマーク等を、優先順位の高い情報と位置付けることも出来る。
【０１４９】
重要度の高い被写体パターンをより重視した画像処理を行う。
一例として、重要度の高い被写体パターンが、より好ましい階調に仕上がるように階調変換条件を定める画像処理を説明する。
【０１５０】
この例は、明るさについての階調補正の例である。図１６に示す、前記修学旅行・京都の例では、
「修学旅行・京都」の例
▲１▼制服を着た人物 ：優先順位１、重み付け係数５
▲２▼歴史建築（和建築） ：優先順位２、重み付け係数２
▲３▼顔 ：優先順位３、重み付け係数１
と、優先順位情報が設定されていたとする。
【０１５１】
実画像から、全要素が見つかったが、▲３▼は▲１▼の中に包含されていて（抽出要素としては▲１▼となる）、どちらもやや小さく、▲２▼が中央部に大きく存在していたとする。副優先順位情報として大きさに対応する重み付けを以下のとおりとすると、
ａ：被写体「大」 重み付け係数 １．０
ｂ：被写体「中」 重み付け係数 ０．８
ｃ：被写体「やや小」 重み付け係数 ０．３
ｄ：被写体「小」 重み付け係数 ０．１
▲１▼と▲２▼の重み付けは、
▲１▼： ５×０．３ ＝ １．５
▲２▼： ２×１．０ ＝ ２．０
となる、この画像は、歴史的建造物の前で撮影した記念写真と考えられるが、以上の処理により、人物写真であるが、建造物（旅行の目的物）に重点の置かれた写真が得られることになる。
【０１５２】
図１６の画像に対する前記の重み付けに従った階調補正について図１７、１８により説明する。
【０１５３】
上記の例に於いて、▲１▼をもっとも好ましく仕上げる階調補正量がα、▲２▼をもっとも好ましく仕上げる階調補正量がβとすると重みを考慮した階調補正量γは、たとえば下記の式で求められる
γ＝ （１．５×α＋２．０×β）／（１．５＋２．０）
なお、上記計算式（後述の計算式でも同様）の１．５、２．０の値は、前述▲１▼と▲２▼の重み付け計算で一例として求めた重み付けの値であり、一般的な画像処理では変数として扱うものである。
【０１５４】
もう一つの例としては、重要度の高い被写体パターンが、もっとも好ましい階調に仕上がるよう全体の階調変換を行い、その他の被写体パターンについてはその領域のみの階調を選択的に変える覆い焼き的な手法を用い例がある。
【０１５５】
覆い焼き的な処理を加えることで、各被写体要素、▲１▼〜▲３▼の明るさをそれぞれ適当な状態に補正することが可能である。
【０１５６】
前記の数式例で説明すれば、全体の階調補正量を▲２▼をもっとも好ましく処理するβとし、▲１▼については、その領域のみ、（α−β）に相当する階調処理を行えばよい。
【０１５７】
一方で、１枚の画像中に複数の被写体が存在している場合、ばらばらに補正することは画像の自然さを損なうこととなる。すなわち、前記の数式例で（α−β）の階調補正量が、大きすぎる場合、１枚の写真としてのバランスを欠く結果となる懸念がある。
【０１５８】
自然な階調補正が出来る補正量の上限がδ（かつ、δ＜（α−β）、δ＞０）であったとすると、たとえば以下のように階調補正すれば全体に自然な補正結果が得られる。
【０１５９】
ε ＝（α−β）−δ
▲２▼の階調補正量はβ＋ε×１．５／（１．５＋２．０）
▲１▼の階調補正量はε×１．５／（１．５＋２．０）＋δ（覆い焼き的処理分）
以上説明したように、優先順位（重み付け情報）を決め、重みの大きい物を適切な明るさに、他の構成要素を、自然な明るさバランスに揃える手法を用いることが出来る。
【０１６０】
ところで、覆い焼き的処理が自然に行える限界δについては、覆い焼き的処理の行い方、特に、パターン境界近傍領域で、どのような処理を行うかによってその値が変わってくる。以下、本処理を好ましく行う手法について、一例を説明する。
【０１６１】
図１９は、実施の形態の概要を表すブロック図である。原画像は、釣鐘型の窓があいた室内の物体を撮影した状態を表している。室内の被写体は単純化のため、星型としている。
【０１６２】
室外、斜め右方向から日光が差し込んでいる状態で、星型の被写体を含む窓枠内の画像は、右側にかげりがあり写真として見苦しい状態である。このかげりのある部分を領域Ａ、他の、窓枠内の部分を領域Ｂとする。このＡの影の部分を覆い焼き処理によって明るく再現するのが本実施例の目的である。
【０１６３】
まず、画像を多重解像度変換する。変換手法は一般的に知られている手法でかまわないが、ここでは好ましい例として、前述のウェーブレット変換、特に、二項ウェーブレット変換を用いる。
【０１６４】
該変換により、順次、低レベルから高レベルまでの分解画像が出来、残渣の低周波画像▲１▼が出来上がる。ここで、領域Ａの部分に注目すると、領域右側（窓枠エッジ部）は低レベルの分解画像からはっきり認識できるが、領域左側（窓枠エッジが、室内に射影された影の輪郭）は低レベルの分解画像からは認識されず、高レベルの分解画像ではっきりと認識される。これは、窓枠エッジと比較し、影の輪郭がはっきりとしたものではなく、あいまいなぼんやりしたものと評価できることを意味している。
【０１６５】
次に、領域Ａに対しマスキング処理を行う。これは、分解画像を逆変換によって、もとの画像にもどす過程で行われる。まず低周波画像▲１▼にマスク画像▲１▼を加算（便宜上、加算と表記したが、黒を０、白を正の大きな値と定義すれば、この図では減算。以下、同じ）し、これと、高レベル分解画像とを合成する逆変換処理を行い、より低レベル方向の、低周波画像▲２▼を得る。次に、これにマスク画像▲２▼を加算し、前述と同様の処理によって、変換済み画像を得る。
【０１６６】
ところで前述のマスク画像▲１▼は、領域Ａの左半分、マスク画像▲２▼は、領域Ａの右半分を覆うマスクとなっている。図９及び図１０で示された通り逆変換の過程で、加算されたマスク画像はローパスフィルタを通過する為にぼやけるが、マスク画像▲１▼の方が、多回数、かつ、強いローパスフィルタ処理が施される為、ＡとＢ領域境界近傍のマスキング処理量がより緩やかに変化するマスキング処理として作用する。したがって、なだらかな変化を示す、影の輪郭に良好に対応した覆い焼き処理を行うことが出来る。同様な理由で、マスク画像▲２▼は小さなボケ量のマスクとして作用するので窓枠エッジに適合した覆い焼き処理を行うことが出来る。
【０１６７】
マスキング処理をどのレベルの逆変換にかけるかは、当該の領域境界の特性がもっとも強く出た解像度レベルの逆変換時にかければ良いが、画像の特性や、実際の試行結果から、前記、当該の領域境界の特性がもっとも強く出た解像度レベルから所定量移動したレベルにマスキング処理を施してもよく、これによって主観的に好ましい画像処理チューニングが可能になる。
【０１６８】
マスクは次のようにして用意される。
階調、色調、彩度補正に関するマスクについては、あらかじめ領域が分割され、たとえば図２０のとおりに作成し利用される。領域分割については大きく分けて以下の２方式が挙げられるが、これに限定されない。
【０１６９】
（１）被写体パターン抽出した結果に基づいて、たとえば図１７（ａ）の例でいえば、被写体パターン▲１▼（人物）と被写体パターン▲２▼（寺社）を切り抜き、マスクとする。それぞれのマスクにおける画像代表（多くは平均）値を求め、それぞれの被写体に好ましい階調再現からの隔たりが階調補正量となるが、この階調補正量が、（本例のように）人物と寺社とで大きく異なっている場合に、領域ごとの補正が必要になる。このケースでは、「人物」「寺社」「その他」という３領域について、各々補正量α、β、γが計算出来、画面全体を何らかの補正量ωとすると、それぞれのマスク補正量は、
「人物」 α−ω
「寺社」 β−ω
「その他」 γ−ω となり、これらの値を当該領域に配置し、その他の領域を補正量０としたものが各々のマスクとなる。たとえばすべてのマスクを同一のレベルで作用させることになれば、３つのマスクを合成し、所定のレベルで低周波画像に加算する。
【０１７０】
（２）たとえば、同一の被写体パターンでも影が強く、うまく階調再現できない場合があり、この場合は、たとえば画面全体から画像信号値のヒストグラムを作成し、たとえば２階調化の手法などを用いて、被写体の明るさをいくつかのブロックに分解し、それぞれに所属する画素について、１と同様に補正値を与え、マスクを作成する。このマスクは画像信号によってはきれいな領域分割とはならず、ノイズによる微小領域が多数出来たりするが、これはノイズフィルタ（単純な平滑化フィルタでも可）を用いて単純化することができる。ヒストグラムを分割し、異なった補正量を与える手法については、特開平１１−２８４８６０号に詳しく記載されている。そして、この計算結果から領域境界を定め、その境界の特性を、多重解像度変換の手法を用いて評価し、マスクを作用させるレベルを決定する。（１）との違いは、パターンの区切りとは別に領域を切り分けるということで、実際の覆い焼きでは、一つの被写体が光と影で分断されているような場合が良くあり、そのような条件では、（２）が有効である。
【０１７１】
鮮鋭性、粒状性については、マスクに記載される補正値が、エッジ強調フィルタや、ノイズフィルタの強度パラメータとなる。また、このマスクを施す段階が階調、色調、彩度補正とは異なり、多重解像度変換されていない画像、または、特定解像度レベルの、分解画像となる。また、マスクの作り方そのものは階調、色調、彩度補正のケースと同一であるが、そのマスクを作用させる前に、マスクそのものに、ぼかしフィルタを作用させる必要がある。これは、階調、色調、彩度補正のケースでは、低周波画像にマスクをかけていたため、マスクの輪郭がはっきりしていても、その後の逆変換過程で、適切なローパスフィルタを通過するから、輪郭が自然にボケる為で、鮮鋭性、粒状性の処理シーケンスではこの効果が得られない為である。どの程度のぼかしフィルタをかけるかについては、前述（２）と同じ方法でエッジを評価し、実際には、前述（２）のマスク画像が受けるであろうぼかし量を与えるフィルタが妥当なところとなる。
【０１７２】
図２０〜図２２は、前述のような手法で用いることの出来るマスク形態の他の例を表したものである。
【０１７３】
図２０は、図１９のマスク部分の例で、前述のとおりかげの領域を２つの小領域、▲１▼と▲２▼に分けている。ここで丸付き数字の大きいほうが、よりはっきりしたエッジに対応するマスクである。小領域▲１▼と▲２▼の間にも、点線で図示される領域境界が存在する。ここで、領域を挟む、数字の小さい側のマスクは、この領域境界ではっきり切れていてもかまわないが、大きい側のマスクは、この領域境界で緩やかにマスキング処理量が変化する、好ましくは、境界を接する相手側のマスクが、当該マスクと合成されるまで、逆変換過程で施されるローパスフィルタの特性に適合した変化特性をもっていると、領域境界間のつながり感向上に好ましい効果を与える。
【０１７４】
図２１はそれぞれ別個の被写体パターン▲１▼「雲」、▲２▼「樹木の葉、梢の部分」、▲３▼「人物、樹木の幹の部分」に、別解像レベルのマスク処理を施す例である。
【０１７５】
図２２は、模式的に、上辺のエッジが丸められた円柱に、水平に近い、斜め上、右方から光が差し込んだ状態の図である。
【０１７６】
以上、全体の補正レベルを決定する手法、部分的なマスク（覆い焼き的）手法を説明したが、さらに上記２例を併用、あるいはシーンに応じて切り替えて使用してもかまわない。
【０１７７】
また、以上の説明では階調、明るさの例を示したが、色再現、彩度再現等の各種条件設定に応用しても良い。たとえば図１６に示される▲１▼と▲２▼、それぞれについて、以下のような望ましい処理状態の差が考えられ、これらについて、前記のような平均的な処理や、領域を分けた個別処理、これらの併用処理を行うことが出来る。
【０１７８】

さらに、シャープネス、粒状性等の処理条件設定についても、複数の被写体パターンの、優先順位情報に応じた、重み付け平均を元に画面全体に対し画像処理を行って顧客の希望に添った画像処理結果を得ることが出来、さらに後述の手法を用いれば、領域を分けた個別処理、これらの併用処理を行うことが出来る。
【０１７９】
シャープネス、粒状性についても、図１６に示される▲１▼と▲２▼、それぞれについて、以下のような望ましい処理状態の差が考えられる。
【０１８０】

図２３は、鮮鋭性（ここでは強調処理）、粒状性（ここでは粒状除去処理）に関し、領域分割の例を示したものである。
【０１８１】
例として、領域を「Ｃ：雲」、「Ｂ：青空」、「Ａ：山、木々」３つに分けることが出来たとする。図示のように、Ａ、Ｂ、Ｃそれぞれ、好ましいとされる、鮮鋭性と粒状性の組み合わせは異なっている。またそれぞれの境界領域の関係は、ＡとＢの間ははっきりとした輪郭であり、ＢとＣはぼんやりとした輪郭となっている。この領域境界の特徴は、前述、図１９で述べた多重解像度変換処理で生成される、各解像度レベルの画像を評価することによって、容易に判断できることは明らかである。
【０１８２】
その上で、たとえば鮮鋭性処理の例では、鮮鋭性強調係数を画面位置に対応して並べたマスクを作成し（図１９の例におけるマスクと同様のものである）、領域Ａ〜Ｃ、それぞれに適合する解像度レベルを、前述図１９で説明した手法などにより求め、それぞれのマスクを当該の適合解像度レベルに対応したぼかし量でぼかした、修正マスクを取得し、領域Ａ〜Ｃの合計３枚の修正マスクを合成する。
【０１８３】
合成されたマスクに記載された補正量情報に応じて、マスクと対応した位置にある画素の補正量を決めれば、Ａ〜Ｃ各領域の特性に応じた鮮鋭性強調が施され、さらに、ＡとＢの領域境界では、鮮鋭性強調の補正量がはっきり変化し、ＢとＣの領域境界では、鮮鋭性強調の補正量が緩やかに変化する、最も好ましい状態を得ることが出来る。
【０１８４】
また、たとえばカラー画像のように、複数の色次元を持っている画像情報の場合は、必要に応じて色座標変換を行い、必要な座標軸についてのみ、ここまで説明してきたような処理を行ってもかまわない。
【０１８５】
たとえば、覆い焼き的な階調補正をする上で特に重要となる明るさ補正について、ＲＧＢ３色で表された画像の場合は、いったん、輝度、色差（Ｌａｂなど）に変換し、輝度情報についてのみ処理を行うことで、画像処理品位の低下を抑え、画像処理量を大幅に抑えることが出来る。
【０１８６】
また、花、海、空など、領域で区分すべき領域、被写体が、固有の色調をもっている場合に、領域境界を定める処理、領域境界の特性を評価する処理のいずれか１方、または両方を、固有色調をもっとも抽出しやすい色座標で行い、実際の領域ごとの画像処理は、これとは別の色座標、たとえば、輝度や彩度座標に対し、行うことも出来、「ある種（たとえば真っ赤なバラ）の花」など、特定、特殊な画像に対して特化した性能チューニングも行うことが可能である。
【０１８７】
次に、本発明に係る画像処理方法を実行し、また、本発明に係る画像処理装置の画像処理手段を機能させるプログラムを実行する工程を図２４〜２７のフローチャートで説明する。
【０１８８】
図２４は基本的な工程を示す。
まず、画像情報を取得して（ステップ１）、シーン属性情報を取得する（ステップ２）。
【０１８９】
次いで、取得されたシーン属性情報から抽出すべき被写体パターンを定め（ステップ３）、各々の被写体パターンを特徴付ける構成要素を定める（ステップ４）。
【０１９０】
さらに、構成要素各々について、構成要素の抽出に好ましい好適解像度レベルを設定し（ステップ５）、画像情報を多重解像度変換する（ステップ６）。
【０１９１】
各々の構成要素を、各々の好適解像度レベルで抽出し（ステップ７）、抽出された構成要素に基づいて被写体パターンの抽出を行う（ステップ８）。
【０１９２】
最後に、抽出された被写体パターンと、被写体パターン境界領域の特性を評価した評価結果に応じて、階調や鮮鋭性について、画像全体、あるいは領域ごとに異なる補正を行う覆い焼き的な処理を施し、その他、画像切り出し等の各種画像処理を行い（ステップ９）、処理を終了する。
【０１９３】
図２５は、被写体パターンの大きさ情報に応じて、被写体パターンを特徴付ける構成要素を抽出する好適解像度レベルを設定する好ましい一例である。
【０１９４】
被写体パターンを特徴付ける構成要素を定めるステップ４までは、図２４の例と同様である。その後、被写体パターンの大きさ情報を取得し（ステップ２０１）、構成要素各々について、被写体パターンの大きさ情報に基づいて設定された構成要素の抽出に好ましい好適解像度レベルを設定する（ステップ６）。以降の処理は図２４の場合と同様である。
【０１９５】
図２６は、階調補正の一部を覆い焼き処理によって実現する、別の一例である。
【０１９６】
まず入力画像情報を取得し（ステップ１）、フィルムやメディアにシーン属性に類する情報があるかどうかチェックして（ステップ３０２）、ある場合は（ステップ３０２のＹｅｓ）取得情報を情報記録部に格納する（ステップ３０３）。一方、画像表示部に画像を表示して顧客からもシーン属性の情報を取得し情報記録部に格納する（ステック３０４）。
【０１９７】
これらを元にシーン属性を決定し（ステップ３０５）、抽出する被写体パターンを定める（ステップ３０６）。
【０１９８】
次に、定められた被写体パターンを、たとえば多重解像度変換処理を用いた手法で抽出し（ステップ３０７）、優先順位情報を重み付け係数を用いるなどして付与し（ステップ３０８）、さらに、抽出された被写体パターンの存在位置、大きさに応じて優先順位を修正する（ステップ３０９）。
【０１９９】
さらに、抽出された各被写体パターンに対応する階調補正量を、情報記憶部に記憶されている各種情報、たとえば、好ましい階調、色調再現、に関する情報を元に決定する（ステップ３１０）。
【０２００】
次に、各被写体パターンの階調補正量を、覆い焼き処理成分と残りの成分に分離し（ステップ３１１）、多重解像度変換処理を応用した、本願記載の覆い焼き手法を用いてマスキング処理を行い（ステップ３１２）、ステップ３０９で求めた各被写体パターンの重み付け係数を用いて、ステップ３１１で求めた各被写体パターンの階調補正量の、残り成分の重み付け平均値を算出し（ステップ３１３）、重み付け平均値に対応する分量の階調補正を画像に施して（ステップ３１４）、処理を終了する。
【０２０１】
図２７は、覆い焼き的処理の鮮鋭性の補正、ここでは強調処理に関し適用した本発明のさらに別の一例である。
【０２０２】
入力画像情報を取得し、シーン属性お決定し、抽出する被写体パターンを定め、定められた被写体パターンを抽出するまでの処理（ステップ１〜ステップ３０７）は、前例と同一である。
【０２０３】
次に、抽出された各被写体パターンに応じて、好ましい鮮鋭性強調係数を設定する（ステップ４０８）。
【０２０４】
さらに、設定された鮮鋭性強調係数を、被写体パターン各々の存在する領域に２次元配列的に並べたマスクを作成し（ステップ４０９）、被写体パターン各々の境界領域の特性を、二項ウェーブレットの分解画像に現れる信号強度を比較する事で評価する（ステップ４１０）。
【０２０５】
ステップ４０９で作成したマスクを、ステップ４１０の評価結果に基づいてぼかし処理し（ステップ４１１）、作成された被写体パターンごとのマスクを合成する（ステップ４１２）。
【０２０６】
合成されたマスクに作成された、各画素位置に対応した鮮鋭性補正量を、対応する各画素に適用して画像処理を行い（ステップ４１３）、処理を終了する。
【０２０７】
【発明の効果】
請求項１〜２４のいずれかの発明により、画像の領域ごとに異なる画像補正量を適用した場合に、領域境界に発生する補正結果の不自然さを軽減でき、主要被写体を適正な画像特性で再現出来るとともに、全体としてバランスのとれた画像特性の画像を得ることが出来る。
【０２０８】
請求項３、１１または１９の発明により、領域境界の特性を確実に判断できるので、境界の性質に対応した確度の高い領域分割を行う事が出来る。
【０２０９】
請求項４、１２または２０の発明により、マスキング処理を行うレベルを切り替えることで、簡単にマスクのボケ量を切り替えることが出来るので、領域境界の評価結果に応じた処理が簡単に実現できる。
【０２１０】
請求項５、７、１３、１５、２１または２３の発明により、境界領域の位置を確実、かつ正確に特定でき、高精度な画像処理結果が得られる。さらに、二項ウェーブレット変換の各ステップに於いて、鮮鋭性補正、粒状性補正を好ましく実施することが可能となる。
【０２１１】
請求項８、１４、１６、２２または２４の発明により、特性評価、画像補正処理を、それぞれに適した色座標で実施できる為、高確度、かつ、高速度の画像処理が実現できる。
【図面の簡単な説明】
【図１】本発明の実施の形態に係る画像処理装置を備えたデジタルミニラボの基本的な構成を示すブロック図である。
【図２】ウェーブレット関数の示す図である。
【図３】ウェーブレット変換の概念図である。
【図４】ウェーブレット変換の概念図である。
【図５】ウェーブレット変換で信号分解する過程の概念図である。
【図６】ウェーブレット変換の概念図である。
【図７】画像信号の例を示す図である。
【図８】逆ウェーブレット変換の概念図である。
【図９】ウェーブレット変換の概念図である。
【図１０】ウェーブレット変換の概念図である。
【図１１】被写体パターン及び構成要素の例を示す図である。
【図１２】解像度れべると検出される構成要素の関係を示す図である。
【図１３】パターンの大きさと検出される構成要素との関係を示す図である。
【図１４】被写体パターン及び構成要素の例を示す図である。
【図１５】複数の構成要素を結合する論理を説明する図である。
【図１６】被写体パターンの抽出を説明する図である。
【図１７】複数の被写体パターンに対する階調補正を説明する図である。
【図１８】複数の被写体パターンに対する階調補正を説明する図である。
【図１９】覆い焼き的な処理を示すブロック図である。
【図２０】覆い焼き的な処理において用いられるマスクの例を示す図である。
【図２１】覆い焼き的な処理の例を示す図である。
【図２２】覆い焼き的な処理の例を示す図である。
【図２３】鮮鋭性や粒状性に関連した領域分割による処理の例を示す図である。
【図２４】本発明の実施の形態に係る画像処理方法を実行し、また、本発明の実施の形態に係る画像処理装置の画像処理手段を機能させるプログラムのフローチャートの例である。
【図２５】本発明の実施の形態に係る画像処理方法を実行し、また、本発明の実施の形態に係る画像処理装置の画像処理手段を機能させるプログラムのフローチャートの例である。
【図２６】本発明の実施の形態に係る画像処理方法を実行し、また、本発明の実施の形態に係る画像処理装置の画像処理手段を機能させるプログラムのフローチャートの例である。
【図２７】本発明の実施の形態に係る画像処理方法を実行し、また、本発明の実施の形態に係る画像処理装置の画像処理手段を機能させるプログラムのフローチャートの例である。
【符号の説明】
１ デジタルカメラ
２ 画像記録メディア
３ カメラ
４ フィルム
５ メディアドライバ
６ フィルムスキャナ
７ 画像入力部
８ 画像処理部
９ 銀塩露光プリンタ
１０ インクジェットプリンタ
１１ 画像記録メディア
１２ 指示入力部
１３ キーボード
１４ マウス
１５ 接触センサ
１６ 画像表示部
１７ 情報記憶部[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method and apparatus for performing image processing based on input image information obtained from an image input unit and obtaining output image information, and an image processing program for controlling its operation.
[0002]
[Prior art]
Conventionally, a system that takes a picture using a conventional camera using a silver halide film or a digital still camera that has become popular in recent years, and reproduces the image by displaying the obtained image on a hard copy or a display device such as a CRT. Used from.
[0003]
In these image reproduction systems, in order to preferably reproduce the captured image, it is general to adjust the brightness, contrast, etc., by modifying the original image and reproduce the image as a viewing image.
[0004]
For example, in the case of a conventional silver halide negative / positive system, a method of changing the exposure time and the light amount of a printing light source when printing and exposing a photographic paper from a film has been used for a long time.
[0005]
When performing various kinds of adjustments as described above, it is essential to make preferable adjustments in accordance with a captured image. It is often difficult to perform this adjustment manually, requiring a high level of skill and experience, or requiring too many steps, and an image processing method for performing these operations automatically or semi-automatically has been disclosed. As an example, Patent Literature 1 states that a preferable photograph can be obtained by extracting face information from image information and finishing it with a preferable gradation.
[0006]
However, a single photographic image includes subjects of various brightnesses, and if a photographic print is intended to have a contrast that is visually perceived, it tends to be a photograph in which the gradations of bright and dark parts are lost. Was.
[0007]
Patent Document 2 describes a method of dividing an image at a luminance level using a histogram obtained from an original image, creating a mask, and performing a dodging process.
[0008]
According to this method, it is possible to reproduce an image while maintaining the necessary contrast while maintaining the gradation of the bright part and the dark part.
[0009]
However, when the above-described partial gradation correction is performed very large, an unnatural contour may be generated in the vicinity of the image edge existing near the boundary of the mask, and a sufficient correction result may not be obtained. Not always.
[0010]
[Patent Document 1]
JP 2001-84274 A
[0011]
[Patent Document 2] Patent Document
JP-A-11-284860
[0012]
[Problems to be solved by the invention]
The present invention has been made in view of such a conventional technique, and reproduces a main subject with appropriate image characteristics, suppresses unnaturalness that is likely to occur at a boundary between subjects, and provides a balanced image. It is an object of the present invention to provide an image processing technique capable of forming the image.
[0013]
[Means for achieving the object]
The object of the present invention is achieved by the following invention.
[0014]
1. In an image processing method of dividing an image into a plurality of regions, determining a correction amount of an image characteristic value for each region, and performing an image correction process,
An image processing method comprising: evaluating a property of a boundary between the plurality of regions; and determining a correction amount for a region near the boundary according to the evaluated property of the boundary.
[0015]
2. The method according to claim 1, wherein the image correction processing includes at least one of gradation correction of an image signal value, color tone correction of a color image, saturation correction, sharpness correction, and graininess correction. Image processing method.
[0016]
3. 3. The image processing method according to the above item 1 or 2, wherein the property of the boundary is evaluated based on a result of multi-resolution conversion processing of the input image information.
[0017]
4. The image correction process is a process including at least one of the following: tone correction of an image signal value, color tone correction of a color image, and saturation correction, and performs multi-resolution conversion processing on input image information. The image processing method according to any one of claims 1 to 3, wherein the image processing method is performed on a low-frequency image generated at each level at which this is inversely transformed.
[0018]
5. 5. The image processing method according to the above item 3 or 4, wherein the multi-resolution conversion process is performed by a Dyadic Wavelet conversion process.
[0019]
6. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed in accordance with the content of the image correction processing. And at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, and the property evaluation 6. The image processing method according to any one of 1 to 5, wherein the information is information on luminance, saturation, or hue.
[0020]
7. The image correction process is a process including at least one of a sharpness correction and a graininess correction of an image signal value, and the multi-resolution conversion process is based on a binomial wavelet (Dyadic Wavelet) conversion process. 5. The image processing method according to the above 3 or 4, wherein
[0021]
8. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed on a color space determined in accordance with the content of the image correction processing. Performed based on the input image information of at least one dimension, and at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, 8. The image processing method according to the item 7, wherein the characteristic evaluation is information on luminance.
[0022]
9. In an image processing apparatus having an image processing unit that divides an image into a plurality of regions, determines a correction amount of an image characteristic value for each region, and performs an image correction process,
The image processing apparatus, wherein the image processing unit evaluates a property of a boundary between the plurality of regions, and determines a correction amount for a region near the boundary in accordance with the evaluated property of the boundary.
[0023]
10. The image processing means performs the image correction processing including at least one of a tone correction of an image signal value, a color tone correction of a color image, a saturation correction, a sharpness correction, and a graininess correction. 10. The image processing apparatus according to the item 9, wherein
[0024]
11. The image processing apparatus according to claim 9 or 10, wherein the image processing means performs the evaluation of the property of the boundary based on a result of multi-resolution conversion processing of input image information.
[0025]
12. The image processing means performs the image correction processing including at least one of gradation correction of image signal values, color tone correction of color images, and saturation correction, and performs multi-resolution conversion processing on input image information. 12. The image processing apparatus according to any one of 9 to 11, wherein the image correction processing is performed on a low-frequency image generated at each level for performing inverse conversion of the image processing.
[0026]
13. 13. The image processing apparatus according to 11 or 12, wherein the multi-resolution conversion processing is performed by a Dyadic Wavelet conversion processing.
[0027]
14． The input image information is a color image composed of a three-dimensional color space, and the image processing means determines the property evaluation of the region boundary and / or determines the image correction processing according to the content of the image correction processing. Performed based on image information of at least one dimension on the color space, and at least one dimension on the color space is information on luminance or saturation of a color image with respect to the image correction processing. The image processing apparatus according to any one of Items 9 to 13, wherein the property evaluation is information on luminance, saturation, or hue.
[0028]
15. The image correction process is a process including at least one of a sharpness correction and a graininess correction of an image signal value, and the multi-resolution conversion process is based on a binomial wavelet (Dyadic Wavelet) conversion process. The image processing apparatus according to the above item 10 or 11, wherein
[0029]
16. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed on a color space determined in accordance with the content of the image correction processing. Performed based on the input image information of at least one dimension, and at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, 16. The image processing apparatus according to the item 15, wherein the characteristic evaluation is information on luminance.
[0030]
17. An image processing unit that divides an image into a plurality of regions, determines a correction amount of an image characteristic value for each region, and performs an image correction process.
A computer-readable storage medium storing an image processing program for evaluating a property of a boundary between a plurality of regions and determining a correction amount for a region near the boundary according to the evaluated property of the boundary.
[0031]
18. 18. The image processing apparatus according to claim 17, wherein the image correction processing includes at least one of a tone correction of an image signal value, a color tone correction of a color image, a saturation correction, a sharpness correction, and a graininess correction. Image processing program.
[0032]
19. 19. The image processing program according to the item 17 or 18, wherein the property of the boundary is evaluated based on a result of multi-resolution conversion processing of the input image information.
[0033]
20. The image correction process is a process including at least one of tone correction of image signal values, color tone correction of a color image, and saturation correction, and performs multi-resolution conversion of input image information. 20. The image processing program according to any one of claims 17 to 19, wherein the image processing program is applied to a low-frequency image generated at each level for inversely transforming.
[0034]
21. 21. The image processing program according to claim 19 or 20, wherein the multi-resolution conversion processing is based on a Dyadic Wavelet conversion processing.
[0035]
22. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed in accordance with the content of the image correction processing. And at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, and the property evaluation 22. The image processing program according to any one of 17 to 21, wherein the information is information on luminance, saturation, or hue.
[0036]
23. The image correction process is a process including at least one of a sharpness correction and a graininess correction of an image signal value, and the multi-resolution conversion process is based on a binomial wavelet (Dyadic Wavelet) conversion process. 21. The image processing program according to the item 19 or 20, wherein
[0037]
24. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed on a color space determined in accordance with the content of the image correction processing. Performed based on the input image information of at least one dimension, and at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, 24. The image processing program according to the item 23, wherein the characteristic evaluation is information on luminance.
[0038]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the present invention will be described by taking as an example a digital minilab that provides an image writing service to a recording medium such as a print or a CDR in accordance with an order from a customer that has become widespread in a photo store in recent years.
[0039]
FIG. 1 is a block diagram showing a basic configuration of a digital minilab provided with an image processing device according to an embodiment of the present invention.
[0040]
Images photographed by the digital camera 1 (hereinafter, referred to as DSC) are stored in various image recording media 2 such as smart media and compact flash (R), and are brought to the store.
[0041]
The image photographed by the conventional camera 3 is subjected to a development process and recorded on the film 4 as a negative image or a positive image.
[0042]
The image from the DSC 1 is read as an image signal by the corresponding media driver 5 in the image input unit 7, and the image on the film 4 is converted into an image signal by the film scanner 6.
[0043]
In addition, in the case of a reflection original, an image input from the image input unit 7 such as inputting an image using a reflection scanner (not shown) such as a flatbed scanner or inputting image information through a LAN or an Internet line. The species is not necessarily from DSC1, but is not shown here. Of course, these images can be subjected to image processing described later.
[0044]
The input image information acquired by the image input unit 7 is sent to the image processing unit 8, where various processing including image processing of the present invention is performed.
[0045]
Output image information that has been subjected to various types of processing is output to various output devices. Image output devices include a silver halide exposure printer 9, an ink jet printer 10, and the like. The image output information may be recorded on various image recording media 11.
[0046]
A function of inputting and registering scene attributes is connected to the image processing unit 8. That is, for example, an instruction input unit 12 including a keyboard 13, a mouse 14, and a contact sensor 15 capable of instructing position information by directly touching the screen while viewing an image displayed on the image display unit 16; , An information storage unit 17 for storing input and registration information, and an image processing unit 8. The information stored in the information storage unit 17 is taken into the image processing unit 8 and processed by the image processing unit 8. The image of the image information is displayed on the image display unit 16 and monitored.
[0047]
The instruction input unit 12 can input or select a scene attribute. Here, the scene attribute is a keyword that characterizes a subject recorded in a photograph, such as a type of photograph, a motive of photographing, a photographing place, and the like, and includes, for example: travel photograph, outdoor photograph, event photograph, nature photograph, portrait, and the like. .
[0048]
Further, it is preferable that the film scanner 6 and the media driver 5 have a function of reading the information from a film or a medium photographed by a camera having a function of storing scene attributes and similar information. This makes it possible to reliably acquire scene attribute information.
[0049]
The information read by the film scanner 6 and the media driver 5 includes, for example, various information recorded on a magnetic layer applied to a film in an APS (Advanced Photo System) of a silver halide camera. Examples include PQI information set for improving print quality, message information set at the time of shooting and displayed on a print, and the like. As information read by the media driver 5, for example, various types of information defined by the type of image recording format such as Exif, information described in the above-described silver halide film example, and various other types of information are recorded. These can be read and used effectively.
[0050]
If there is information obtained from these media, the scene attribute may be obtained from the information, or by analogy, the trouble of confirming the scene attribute at the time of each order reception may be omitted.
[0051]
Further, for example, customer information may be managed in a photo shop or the like, and a scene attribute may be separately set for each customer, or customer information may be used as the scene attribute itself. This makes it possible to easily extract the customer's preference once set at the time of setting the priority order described later, which is preferable in terms of improving work efficiency and improving customer satisfaction.
[0052]
These pieces of information and various kinds of information described later are stored in the information storage unit 17 and used appropriately.
[0053]
An image processing unit 8 serving as an image processing unit, which is a main part of the image processing apparatus, includes a CPU 8a for performing arithmetic processing, a memory 8b for storing programs for various processing described later, a memory 8c as a work memory, and an image for performing image processing arithmetic. It has a processing circuit 8d.
[0054]
Hereinafter, the processing performed mainly by the image processing unit 8 will be described.
When the scene attributes are determined by the various methods described above, the subject pattern to be extracted is determined correspondingly.
[0055]
The subject pattern referred to here is a recognizable separate and specific subject existing in the image as shown below, and information on the subject pattern includes subject pattern priority information (described later). Ranking, or represented by a value represented by a weighting coefficient). Further, information regarding a preferable gradation and color tone reproduction of the subject, and the position and size of the subject pattern, average gradation, gradation Information such as range and color tone
Examples of the subject pattern include a person, a person wearing specific clothing (uniform, sports uniform, etc.), a building (Japanese, Western, modern, historical, religious architecture, etc.), and further, a cloud, a blue sky There is something like the sea.
[0056]
Depending on the order status of the customer, the classification status of the subject pattern may be different. For example, taking a person as an example, if it is simply a "person", it can be treated as information about one pattern regardless of the number of people, but it can be classified as "student" or "general person" (or "male" or "female"). Is meaningful to the orderer, the person will be two types of subject patterns.
[0057]
In addition, the customer and the other party, such as “the bride”, “the groom”, “the other attendees” at the wedding reception, or “Ms. A” and “Ms. B” are also identifiable individuals as the ordering party. It can be a subject pattern.
[0058]
Now, a method of extracting a subject pattern is generally known, and various pattern extraction methods may be used, or a new extraction method may be set.
[0059]
As a preferable example, a method that can be used to extract a pattern with high accuracy by using a multi-resolution conversion process based on a binomial wavelet (Dyadic Wavelet) newly discovered by the present inventors will be described.
[0060]
The multi-resolution conversion is a process of obtaining a plurality of separated images separated from image information at different resolution levels. Preferably, the multi-resolution conversion is performed using a binomial wavelet (Diadic Wavelet) conversion. For example, an orthogonal wavelet transform and a biorthogonal wavelet transform can be used.
[0061]
Next, the Wavelet transform will be briefly described.
2. Description of the Related Art As an efficient method of dividing a frequency band for each local region of an image and performing suppression / emphasis for each frequency band, a technique using a wavelet transform is known.
[0062]
For details of the wavelet transform, see, for example, “Wavelet and Filter Banks” by G.W. Strong & T. Nguyen, Wellesley-Cambridge Press (Japanese translation "Wavelet Analysis and Filter Bank", G. Strang and T. Nguyen, Baifukan) and "A wavelet tour of signal processing 2ed." Although described in Mallat, Academic Press, an outline will be described here.
[0063]
Wavelet transform is a wavelet function that oscillates in a finite range as illustrated in FIG.
[0064]
(Equation 1)

[0065]
And the wavelet transform coefficient <f, ψ for the input signal f (x). _a , _b 〉
[0066]
(Equation 2)

[0067]
Is a transformation for decomposing the input signal into a sum of the following wavelet functions.
[0068]
[Equation 3]

[0069]
In the above equation, a represents the scale of the wavelet function, and b represents the position of the wavelet function. As illustrated in FIG. 2, as the value of the scale a increases, the wavelet function ψ _a , _b The frequency of (x) becomes smaller, and the wavelet function ψ _a , _b The position where (x) vibrates moves. Thus, equation (3) describes the input signal f (x) as a wavelet function ψ _a , _b (X) is decomposed into the sum total.
[0070]
Many wavelet functions that enable the above-described conversion are known, but in the field of image processing, orthogonal wavelets and biorthogonal wavelets, which are fast to calculate, are widely used. . The outline of the orthogonal wavelet / biorthogonal wavelet transform calculation will be described below.
[0071]
The wavelet function of the orthogonal wavelet / biorthogonal wavelet is defined as follows.
[0072]
(Equation 4)

[0073]
Comparing Equations (4) and (1), in the orthogonal wavelet / biorthogonal wavelet, the value of the scale a is discretely defined by 2 to the power of i, and the minimum movement unit of the position b is 2 ⁱ It can be seen that is defined discretely by. This value of i is called a level. In practice, the level i is limited to a finite upper limit N, and the input signal is converted as follows.
[0074]
(Equation 5)

[0075]
The second term in equation (5) is the level 1 wavelet function ψ ₁ , _j The low frequency band component of the residual that cannot be expressed by the sum of (x) is converted to a level 1 scaling function φ. ₁ , _j (X). An appropriate scaling function is used corresponding to the wavelet function (see the above-mentioned document). The input signal f (x) ≡S is obtained by the one-level wavelet transform shown in Expression (5). ₀ Is the level 1 high frequency band component W ₁ And low frequency band component S ₁ This means that the signal has been decomposed. Wavelet function ψ _i , _j The minimum movement unit of (x) is 2 ⁱ Therefore, the input signal S ₀ High frequency band component W ₁ And low frequency band component S ₁ Are 各 々 each, and W ₁ And S ₁ Is the sum of the input signals S ₀ Signal amount. Level 1 low frequency band component S ₁ Is the high frequency band component W of level 2 in equation (6). ₂ And low frequency band component S ₂ , And by repeating the conversion up to the level N in the same manner, the input signal S ₀ Is decomposed into the sum of the high frequency band components of levels 1 to N and the low frequency band component of level N as shown in equation (7).
[0076]
Here, it is known that the one-level wavelet transform represented by the equation (6) can be calculated by a filter process as shown in FIG. 3 (see the above-mentioned document). In FIG. 3, LPF indicates a low-pass filter, and HPF indicates a high-pass filter. The filter coefficient is appropriately determined according to the wavelet function (see the literature and Table 1).
[0077]
[Table 1]

[0078]
2 ↓ indicates downsampling for thinning out every other signal. One-level wavelet transform of a two-dimensional signal such as an image signal is calculated by a filter process as shown in FIG. In FIG. 4, LPFx, HPFx, 2 ↓ x indicates processing in the x direction, and LPFy, HPFy, 2 ↓ y indicates processing in the y direction. By this one-level wavelet transform, the low frequency band component S _n-1 Are the three high frequency band components Wv _n , Wh _n , Wd _n And one low frequency band component S _n Is decomposed into Wv generated by decomposition _n , Wh _n , Wd _n , S _n Are the signal amounts before decomposition. _n-1 , The sum of the signal amounts of the four components after decomposition is S _n-1 Signal. Input signal S ₀ FIG. 5 is a schematic diagram showing a process in which the signal is decomposed by the three-level wavelet transform.
[0079]
Also, Wv generated by decomposition _n , Wh _n , Wd _n , S _n Is subjected to the inverse wavelet transform calculated by the filter processing as shown in FIG. _n-1 It is known that can be completely reconstructed. In FIG. 6, LPF 'indicates a low-pass filter, and HPF' indicates a high-pass filter. As the filter coefficient, in the case of an orthogonal wavelet, the same coefficient as that used for the wavelet transform is used, but in the case of the bi-orthogonal wavelet, a different coefficient from that used for the wavelet transform is used. (See references above). Also, 2 ↑ indicates upsampling in which zeros are inserted every other signal. LPF′x, HPF′x, 2 ↑ x indicate processing in the x direction, and LPF′y, HPF′y, 2 ↓ y indicate processing in the y direction.
[0080]
The Dyadic Wavelet transform used in the present invention is described in "Singularity detection and processing with wavelengths" by S.D. Mallat and W.M. L. Hwang, IEEE Trans. Inform. Theory 38 617 (1992) and "Characterization of signals from multiscale edges" by S.E. Mallat and S.M. Zhong, IEEE Trans. Pattern Anal. Machine Intel. 14 710 (1992) or "A wavelet tour of signal processing 2ed." Mallat, Academic Press has a detailed description, but the outline is described below.
[0081]
The wavelet function of the binomial wavelet is defined as follows.
[0082]
(Equation 6)

[0083]
The wavelet function of the orthogonal wavelet / biorthogonal wavelet is, as described above, the minimum movement unit of the position at level i is 2 ⁱ In the binomial wavelet, the minimum movement unit of the position is constant regardless of the level i. This difference results in the following features of the binomial wavelet transform:
[0084]
Feature 1: High-frequency band component W generated by the following one-level binomial wavelet transform _i And low frequency band component S _i Is the signal S before conversion. _i-1 Is the same as
[0085]
(Equation 7)

[0086]
Feature 2: Scaling function φ _i , _j (X) and wavelet function ψ _i , _j The following relationship is established between (x).
[0087]
(Equation 8)

[0088]
Therefore, the high frequency band component W generated by the binomial wavelet transform _i Is the low frequency band component S _i Represents the first derivative (gradient) of
[0089]
Feature 3: Coefficient γ shown in Table 2 determined according to wavelet transform level i _i (See the reference for binomial wavelets above) multiplied by the high frequency band component _i ・ Γ _i (Hereinafter, this is referred to as a corrected high-frequency band component.) According to the singularity of the signal change of the input signal, the converted high-frequency band component W _i ・ Γ _i The relationship between the signal strength levels follows a certain law. That is, the corrected high frequency band component W corresponding to the gentle (differentiable) signal change shown in 1 and 4 in FIG. _i ・ Γ _i The signal intensity increases as the number of levels i increases, whereas the corrected high frequency band component W corresponding to the step-like signal change shown in FIG. _i ・ Γ _i The signal intensity becomes constant regardless of the number of levels i, and the corrected high frequency band component W _i ・ Γ _i The signal intensity decreases as the number of levels i increases.
[0090]
[Table 2]

[0091]
Feature 4: A one-level binomial wavelet transform method for a two-dimensional signal such as an image signal is performed as shown in FIG. 8 unlike the above-described orthogonal wavelet / biorthogonal wavelet method. By this one-level wavelet transform, the low frequency band component S _n-1 Are two high frequency band components Wx _n , Wy _n And one low frequency band component S _n Is decomposed into The two high frequency band components are low frequency band components S _n Change vector V in two dimensions _n X component and y component. Change vector V _n Size M _n And declination A _n Is given by:
[0092]
(Equation 9)

[0093]
Also, two high frequency band components Wx obtained by the binomial wavelet transform _n , Wy _n And one low frequency band component S _n The inverse binomial wavelet transform shown in FIG. _n-1 It is known that can be reconstructed.
[0094]
Also, the input signal S ₀ FIG. 10 shows a concept in the case of performing an N-level binomial wavelet transform on. Input signal S ₀ , An N-level binomial wavelet transform is performed on the obtained high-frequency components, and if necessary, the operation of the operation 1 is performed, and then an N-level binomial wavelet inverse transform is performed. With respect to the frequency components, the operation 2 is performed in each step of the above-described inverse binomial wavelet transform operation. In one example of the present embodiment, operation 1 is an operation such as edge detection and pattern detection, and operation 2 is a masking process.
[0095]
In FIG. 10, LPF indicates a low-pass filter for conversion, HPF indicates a high-pass filter for conversion, LPF ′ indicates a low-pass filter for inverse conversion, and HPF ′ indicates a high-pass filter for inverse conversion. These filter coefficients are appropriately determined according to the wavelet function (see the above-mentioned reference and Table 3).
[0096]
[Table 3]

[0097]
Further, LPFx, HPFx, LPF'x, and HPF'x indicate processing in the x direction, and LPFy, HPFy, LPF'y, and HPF'y indicate processing in the y direction. In the binomial wavelet, the filter coefficient differs for each level, and the filter coefficient of level n is 2 between each coefficient of the filter of level 1. ^n-1 One with one zero inserted is used (see references and Table 3 above).
[0098]
In the feature 1 of the above-described binomial wavelet transform, it has been described that the size of the decomposed image after the conversion is the same as the size of the image before the conversion. There is also a secondary feature that evaluation can be performed with accuracy.
[0099]
Next, the extraction of a subject pattern using the multi-resolution conversion method will be described with reference to the examples of FIGS.
[0100]
The image is subjected to multi-resolution conversion using the binomial wavelet transform described above, edges appearing at each level of the multi-resolution conversion are detected, and region division is performed.
[0101]
Then, a resolution level used for pattern extraction is set according to the pattern to be extracted.
[0102]
Most of the patterns generally recognized as a subject pattern here have not only the outline but also various kinds of unique partial elements.
[0103]
For example, in the case of the head of a person, in addition to the outline itself, there are eyes (further, pupils, irises, eyelashes, white blood vessels), nose, mouth, cheeks, dimples, eyebrows, and the like.
[0104]
Among these, the partial elements effective for identifying the pattern to be extracted are positioned as “components”, and the resolution level used for pattern extraction is set for each of them.
[0105]
For example, as shown in FIG. 12, the outline of a person's head itself is an edge extracted in a low-level decomposed image, is clearly and accurately recognized, and the gentleness of the face components existing inside it is reduced. Patterns such as the nasal muscles, lip shape, lines around the lips of a smile, `` Ekubo '' and `` Honey swelling '' can be accurately characterized using edge information that appears in higher-level decomposition images. Can be caught.
[0106]
Next, a preferred example of a method for determining the components of the subject pattern and a method for determining a suitable resolution level for identifying each component will be described.
[0107]
First, the components of the subject pattern are set. For example, in general, if it is a "person's face", it will be various components stored in advance as described below.
[0108]
(One example of the component in the case of "person's face")
a: Face outline
b: pupil
c: eyebrows
d: Mouth
e: Hair
f: nose bridge
g: Nostril
h: Slight convex
In the case where a specific person is registered as a subject pattern, a new component may be set in addition to these, and personal identification can be preferably performed.
[0109]
(Examples of components added by "specific person's face")
i: Stain, mole
j: dimple
k: beard
In the case of a specific person, characteristics different from those of a general “person's face” can be set for the components a to f, and some components may be “absent”.
[0110]
When each component is set for the target subject pattern, this image is subjected to multi-resolution conversion using binomial wavelet transform, and for each component, the signal strength in the decomposition signal at each level of the multi-resolution conversion is calculated. Find the maximum level. The above-described maximum level may be used as a suitable resolution, but the actual image processing result may be evaluated and a slight level change may be performed.
[0111]
Note that the signal in this case is the maximum value of the signal indicating the edge component detected at each level, but when comparing the signal intensities between a plurality of levels, the signal values are expressed as the above two terms. It is needless to say that it is preferable to use the corrected high-frequency band component described in the wavelet.
[0112]
By the way, in the case of using the binomial wavelet transform, in the case of a component having a very sharp contour such as a knife edge pattern, there is a characteristic that the signal level of the edge does not greatly change with the resolution level. Sets the preferred resolution level to the level at which the contour form of the component can be clearly recognized, or the lowest resolution level when the original image resolution is not sufficient.
[0113]
Some of the aforementioned components are relatively sharp in outline and others are not.
[0114]
For example, a, f, i, etc. correspond to the former, and f, h, j, etc. correspond to the latter. The extraction and registration of the configuration requirements as in the former case can be performed, for example, by displaying an image on a monitor, designating the relevant location with a mouse, a contact type sensor, or the like, and automatically or manually cutting out a nearby area. I can do it.
[0115]
In the latter case, it is difficult to clearly distinguish and clip the area where the component exists from the non-existing area. You can specify it roughly.
[0116]
The preferred resolution set for such a component is usually higher than the former with a sharp outline.
[0117]
Therefore, as described above, when the rough component is actually specified and the latter component is actually extracted, the target component can be extracted as follows.
[0118]
All the edges detected in the candidate area for extracting the constituent elements are extracted, and the signal intensities at the respective resolution levels are compared for these edges.
[0119]
An edge component whose signal intensity is strongly detected in a decomposed image at a lower level than the preferred resolution level is considered to be not included in the component, and is excluded from the candidate region. Then, the remaining area is inspected at a suitable resolution level to extract a target component.
[0120]
In the above example, the image before disassembly is displayed on the monitor and the components are specified. For example, when a person who has some knowledge about the image processing technology specifies the components, the actual When a resolution-converted disassembled image is displayed on a monitor, and preferably displayed in a configuration that can be compared with the image before disassembly, a component to be extracted can be specified at the displayed resolution level. It is possible to easily find a new feature point that cannot be recognized only by itself, and it is possible to further improve the object pattern identification accuracy.
[0121]
In the illustrated example, the edge of the pupil and upper eyelid is A, the line around the nose and lips is B, and the bulge is C.
[0122]
As described above, by detecting B from B and C from B in an image with a higher resolution level, facial feature recognition can be performed accurately.
[0123]
Further, as shown in the figure, when the level used for detecting the component is set according to the size of the pattern to be extracted, for example, when the pattern to be extracted is sufficiently large, Characteristics are well separated, resolution levels suitable for each component can be set, and when the level used for edge information detection is set, pattern detection using fine information for large patterns, It has an excellent feature that a maximum, effective, and high-speed detection process can be performed using information obtained by the above.
[0124]
The size of the pattern may be obtained from the size by separately performing a temporary pattern detection, or may be temporarily obtained from a scene attribute (commemorative photograph, portrait, etc.) and an image size.
[0125]
The provisional pattern extraction can be performed, for example, by the following method.
In the case of extracting a face pattern, there is a method of first extracting a skin color region from the screen, evaluating the shape of the region, and extracting the region as a “face candidate” if the region is round.
[0126]
In the case of a specific color such as a uniform, a specific color region is extracted, and the evaluation condition of the region shape is changed from a circle, a rectangle, a triangle, or another form.
[0127]
In addition, it is also possible to use a method of obtaining an edge component from an image and extracting all similar external patterns. It may be obtained, or may be extracted by general Laplacian filter processing.
[0128]
Here, the pattern size can be represented by, for example, the size of the pattern in terms of the number of pixels. In the illustrated example, if there is a “medium” face size, it is preferable for each of A, B, and C to be a characteristic. An extraction level is determined.
[0129]
If the original image size (that is, the pattern size and image resolution) is very large, the necessary conversion processing is performed by performing resolution conversion up to the image size corresponding to the above-mentioned "medium" size and performing pattern extraction processing. The amount can be greatly reduced, which is convenient.
[0130]
The resolution conversion performed as the preprocessing can be easily performed by a known technique such as a nearest neighbor method or a linear interpolation method.
[0131]
JP-A-2000-188689 and JP-A-2002-262094 have detailed descriptions of enlargement and reduction methods, and the methods described therein may be used.
[0132]
Further, in the case of an image processing apparatus such as a film scanner or a flatbed scanner, which has a processing sequence in which a pre-scan is performed in advance and an image scan area or a main scan frame is determined, the provisional pattern extraction and pattern The size may be evaluated at the pre-scanning stage, and the reading resolution of the main scan may be read at an image resolution suitable for pattern extraction.
[0133]
In this way, sufficient resolution can be secured even when the pattern to be extracted is small, and when it is large, the time required for scanning can be reduced by setting the resolution of the main scan to a necessary and sufficient value. Can be done.
[0134]
Next, a method of searching for all the subject patterns that can be extracted from the image will be described with reference to an example. As described above, the subject pattern to be extracted is switched according to the determined scene attribute. Here are some examples
(Example) Scene attribute → subject pattern to be extracted (left side has higher priority)
School trip / Kyoto → Face / Person in uniform / History architecture (Japanese architecture)
Wedding reception → bride / groom / face / dress / spotlight
There are also overlapping pattern requirements, such as the bride, groom and face, spotlight and dress in the above example.
Here, the subject pattern may be predetermined, but may be newly set by the following method, for example, as shown in FIGS.
[0135]
The image is displayed on the monitor, and the main image portion is specified. Then, the contour area including the designated portion is automatically extracted, and the obtained pattern is temporarily referred to as a unit pattern.
[0136]
If the entire required pattern is not included, the above operation is repeated to combine the minute outlines, and when the extraction of the entire outline is completed, a registration instruction is performed (a registration key is pressed).
[0137]
The registration information includes information on the selected area (how many and what unit patterns are connected and how they are connected, and various characteristic values regarding the entire area), name of the area (uniform students, etc.) ), And priority information.
[0138]
Further, as the unit pattern, a slightly complicated configuration corresponding to the above-described subject pattern such as "face" or "uniform" may be designated, and by combining these, a more advanced Registration of a subject pattern can be easily performed.
[0139]
An example of the subject pattern registered in this manner will be described with reference to FIGS. As shown in FIG. 14, the category of "students" includes two categories: (a) male students and (b) female students. The categories are (1), (2) and (3) and (1), respectively. "Student" is defined by having unique elements ▼, ▲ 4 ▼ and ５5 ▼, and combining them with unit patterns.
[0140]
If this is expressed using a logical expression,
“Student” = (1) and (2) and (3)) or (1) and (4) and (5).
[0141]
Each of the above components (1) to (5) is defined in a state where individual unit patterns are combined. As an example, the upper clothing of a female student is shown in FIG. The components in FIG. 15A are further composed of the respective elements of the unit patterns a to f, and are defined in FIG.
[0142]
As a general situation of photo prints in photo shops, it is often the case that print orders are made for a plurality of related frames at once, such as simultaneous printing from a roll film, image storage media used at the time of shooting with a digital camera, and the like ( Hereafter, it is described as a series of orders).
[0143]
When there are a plurality of images in a series of orders, the above-mentioned extraction and registration work is performed with a representative one of the images, and pattern extraction of all images in a series of images is performed based on this information. Work can be performed, the number of pattern registration work can be reduced, and efficient work can be performed.
[0144]
If the registration pattern is unique to a particular customer, the registered pattern is stored together with the customer information, and the necessary registration pattern is called from the customer information at the next print order. By doing so, it is possible to save trouble and achieve advanced services.
[0145]
Further, in the case where a series of order processing is performed as described above, various possible subject patterns are extracted from the entire screen, and scene attributes and priority are determined based on the appearance frequency and the statistical result of the location in the screen. The ranking can be inferred.
[0146]
In this way, even if the orderer cannot obtain the information on the scene attribute, it is possible to guess the subject of the customer who is most important, so that a print preferable for the customer can be easily obtained with a higher probability.
[0147]
Next, priorities are assigned to the subjects extracted by the above processing. Based on the priority information defined corresponding to the scene attribute, the size of the subject pattern (larger ones, etc.) and the position (more importantly the central part, etc.) The priority information may be weighted, so that more preferable information regarding the importance of the subject pattern can be obtained. Hereinafter, the information on the priority obtained in this manner is referred to as “importance”.
[0148]
As a method of determining the subject pattern to be extracted and the priority information of the subject pattern, further, a GPS signal, time, map, terrain information, search information using an automatic search engine such as the Internet, the local government, and the tourist association By using information such as the information of a commercial and industrial association, and the like, and information linked to them, it is also possible to position a generally important subject pattern, landmark, or the like at the image capturing point as information with a high priority.
[0149]
Image processing is performed with more emphasis on subject patterns with high importance.
As an example, a description will be given of image processing for determining a gradation conversion condition so that a subject pattern with a high degree of importance is finished to a more preferable gradation.
[0150]
This example is an example of gradation correction for brightness. In the example of the school excursion and Kyoto shown in FIG.
Example of "School Trip / Kyoto"
(1) Person in uniform: priority 1, weighting coefficient 5
(2) Historical architecture (Japanese architecture): priority 2, weighting factor 2
(3) Face: priority 3, weighting factor 1
It is assumed that priority information has been set.
[0151]
All elements were found from the actual image, but (3) is included in (1) (the extracted element is (1)), both are slightly smaller, and (2) is larger in the center. Suppose that it existed. Assuming that the weights corresponding to the sizes are as follows as the sub-priority information,
a: subject "large" weighting factor 1.0
b: Subject "medium" Weighting coefficient 0.8
c: Subject "slightly small" Weighting coefficient 0.3
d: subject “small” weighting coefficient 0.1
The weighting of (1) and (2)
(1): 5 × 0.3 = 1.5
{Circle around (2)}: 2 × 1.0 = 2.0
This image is considered to be a commemorative photo taken in front of a historic building. By the above processing, it is a portrait photo, but a photo that emphasizes the building (travel object) Will be obtained.
[0152]
The gradation correction according to the above-mentioned weighting for the image of FIG. 16 will be described with reference to FIGS.
[0153]
In the above example, assuming that the gradation correction amount for finishing (1) most preferably is α and the gradation correction amount for finishing (2) most preferably is β, the gradation correction amount γ in consideration of the weight is, for example, Calculated by the formula
γ = (1.5 × α + 2.0 × β) / (1.5 + 2.0)
Note that the values of 1.5 and 2.0 in the above calculation formulas (the same applies to the calculation formulas described later) are weighting values obtained as an example in the weighting calculations of the above (1) and (2). In image processing, it is handled as a variable.
[0154]
Another example is dodging, in which the overall tone conversion is performed so that the subject pattern with high importance is finished to the most desirable tone, and the other subject patterns are selectively changed in tone only in that area. There is an example using a simple method.
[0155]
By adding a dodging process, it is possible to correct the brightness of each subject element, (1) to (3) to an appropriate state.
[0156]
Explaining with the above formula example, the total gradation correction amount is set to β which is most preferably processed in (2), and in (1), gradation processing corresponding to (α−β) is performed only in that region. Just do it.
[0157]
On the other hand, when there are a plurality of subjects in one image, performing the correction separately degrades the naturalness of the image. That is, if the gradation correction amount of (α-β) in the above formula example is too large, there is a concern that the balance of a single photograph may be lost.
[0158]
Assuming that the upper limit of the correction amount at which the natural gradation correction can be performed is δ (and δ <(α−β), δ> 0), for example, if the gradation correction is performed as follows, a natural correction result is obtained as a whole. can get.
[0159]
ε = (α−β) −δ
The gradation correction amount of (2) is β + ε × 1.5 / (1.5 + 2.0)
The gradation correction amount in (1) is ε × 1.5 / (1.5 + 2.0) + δ (for dodging).
As described above, it is possible to use a method of determining the priority order (weighting information), and adjusting the weight of an object having a large weight to an appropriate brightness, and adjusting the other components to a natural brightness balance.
[0160]
By the way, the value of the limit δ at which the dodging process can be naturally performed changes depending on how the dodging process is performed, particularly, what kind of processing is performed in the region near the pattern boundary. Hereinafter, an example of a method of preferably performing this processing will be described.
[0161]
FIG. 19 is a block diagram illustrating an outline of the embodiment. The original image shows a state where an object in a room with a bell-shaped window is photographed. The subject in the room has a star shape for simplicity.
[0162]
The image inside the window frame including the star-shaped subject is shaded on the right side and is unsightly as a photograph in a state where sunlight enters from outside and diagonally right. The shaded portion is referred to as a region A, and the other portion within the window frame is referred to as a region B. The purpose of this embodiment is to reproduce the shadow portion of A brightly by dodging processing.
[0163]
First, the image is subjected to multi-resolution conversion. The transformation method may be a generally known method, but as the preferred example, the above-described wavelet transform, in particular, the binomial wavelet transform is used.
[0164]
By this conversion, a decomposed image from a low level to a high level is sequentially formed, and a low-frequency image (1) of the residue is completed. Here, paying attention to the region A, the right side of the region (window frame edge) can be clearly recognized from the low-level decomposed image, but the left side of the region (window frame edge is the outline of the shadow projected into the room) is low. It is not recognized from the high-level decomposed image, but is clearly recognized from the high-level decomposed image. This means that the outline of the shadow is not clear as compared with the window frame edge, and can be evaluated as being vague and vague.
[0165]
Next, a masking process is performed on the region A. This is performed in the process of returning the decomposed image to the original image by the inverse transformation. First, the mask image {circle around (1)} is added to the low-frequency image {circle around (1)} (for convenience, black is defined as 0, and white is defined as a large positive value. An inverse transformation process for combining this with the high-level decomposition image is performed to obtain a low-frequency image (2) in the lower-level direction. Next, the mask image {circle around (2)} is added thereto, and a converted image is obtained by the same processing as described above.
[0166]
Incidentally, the mask image (1) is a mask that covers the left half of the area A, and the mask image (2) is a mask that covers the right half of the area A. As shown in FIGS. 9 and 10, in the process of the inverse transformation, the added mask image is blurred because it passes through the low-pass filter. Is performed, the masking processing amount in the vicinity of the boundary between the A and B regions acts as a masking processing that changes more gradually. Therefore, it is possible to perform the dodging process that smoothly corresponds to the outline of the shadow and shows a gentle change. For the same reason, the mask image {circle around (2)} acts as a mask having a small blur amount, so that a dodging process suitable for the window frame edge can be performed.
[0167]
The level of the masking process to be applied to the inverse conversion may be determined at the time of the inverse conversion at the resolution level at which the characteristic of the region boundary appears most strongly, but from the characteristics of the image and the actual trial results, A masking process may be applied to a level shifted by a predetermined amount from the resolution level at which the characteristic of the region boundary appears most strongly, thereby enabling subjectively favorable image processing tuning.
[0168]
The mask is prepared as follows.
As for the mask relating to the gradation, color tone, and saturation correction, the area is divided in advance and created and used as shown in FIG. 20, for example. The area division is roughly divided into the following two methods, but is not limited thereto.
[0169]
(1) Based on the result of subject pattern extraction, for example, in the example of FIG. 17A, a subject pattern (1) (person) and a subject pattern (2) (temple) are cut out and used as a mask. The image representative (mostly the average) value of each mask is calculated, and the distance from the preferable gradation reproduction for each subject is the gradation correction amount. The gradation correction amount is determined by the person (as in this example). When there is a great difference between a temple and a shrine, a correction for each area is required. In this case, the correction amounts α, β, and γ can be calculated for the three regions “person”, “shrine”, and “other”, and if the entire screen is some correction amount ω, the respective mask correction amounts are
"Person" α-ω
“Shrines” β-ω
“Other” γ−ω, these values are arranged in the area, and the other areas with the correction amount of 0 are the respective masks. For example, if all the masks are to operate at the same level, three masks are combined and added to the low-frequency image at a predetermined level.
[0170]
(2) For example, there is a case where even the same subject pattern has a strong shadow and gradation cannot be reproduced well. In this case, for example, a histogram of image signal values is created from the entire screen, and a method such as two-gradation is used, for example. Then, the brightness of the object is decomposed into several blocks, correction values are given to the pixels belonging to each of the blocks in the same way as 1, and a mask is created. This mask is not divided into fine areas depending on the image signal, and many small areas due to noise can be formed. However, this can be simplified by using a noise filter (a simple smoothing filter is also possible). A method of dividing the histogram and providing different correction amounts is described in detail in JP-A-11-284860. Then, a region boundary is determined from the calculation result, and the characteristics of the boundary are evaluated by using a multi-resolution conversion technique to determine a level at which a mask is applied. The difference from (1) is that the area is cut apart from the pattern break. In actual dodging, one subject is often separated by light and shadow. Then, (2) is effective.
[0171]
As for the sharpness and the graininess, the correction values described in the mask serve as the intensity parameters of the edge enhancement filter and the noise filter. Also, unlike the gradation, color tone, and saturation correction, the step of applying the mask is an image that has not been subjected to multi-resolution conversion or a decomposed image at a specific resolution level. The method of making the mask itself is the same as in the case of gradation, color tone, and saturation correction, but it is necessary to apply a blur filter to the mask itself before applying the mask. This is because, in the case of gradation, color tone, and saturation correction, the low-frequency image is masked, so even if the outline of the mask is clear, it passes through an appropriate low-pass filter in the subsequent inverse conversion process. This is because the contour is naturally blurred, and this effect cannot be obtained in the sharpness and granularity processing sequence. Regarding how much blur filter is applied, the edge is evaluated in the same manner as in the above (2), and in practice, the filter which gives the amount of blur that the mask image of the above (2) will receive is appropriate. Become.
[0172]
FIG. 20 to FIG. 22 show other examples of mask forms that can be used in the above-described method.
[0173]
FIG. 20 shows an example of the mask portion of FIG. 19, and the shaded area is divided into two small areas, (1) and (2), as described above. Here, the larger one with a circled number is a mask corresponding to a clearer edge. An area boundary indicated by a dotted line also exists between the small areas (1) and (2). Here, the mask on the smaller number side sandwiching the region may be sharply cut off at the boundary of this region, but the mask on the larger side gradually changes the masking processing amount at this boundary of the region, preferably, If the mask on the other side in contact with the boundary has a change characteristic that matches the characteristic of the low-pass filter applied in the inverse transformation process until the mask is combined with the mask, a favorable effect is provided for improving the sense of connection between the region boundaries.
[0174]
FIG. 21 shows a case where mask processing of a different resolution level is applied to separate subject patterns (1) "clouds", (2) "tree leaves and treetops", and (3) "persons and tree trunks". It is an example.
[0175]
FIG. 22 is a diagram schematically showing a state in which light is nearly horizontally, diagonally above, and rightwardly inserted into a column having a rounded upper edge.
[0176]
The method for determining the overall correction level and the partial mask (dodging) method have been described above. However, the above two examples may be used in combination or switched according to the scene.
[0177]
In the above description, examples of gradation and brightness have been described, but the invention may be applied to setting of various conditions such as color reproduction and saturation reproduction. For example, for each of (1) and (2) shown in FIG. 16, the following desirable processing state differences can be considered. For these, the above-described average processing, individual processing by dividing the area, These combined treatments can be performed.
[0178]

Furthermore, regarding processing condition settings such as sharpness and graininess, image processing is performed on the entire screen based on a weighted average according to priority information of a plurality of subject patterns, and image processing results according to customer's wishes are obtained. Can be obtained, and further, by using a method described later, individual processing in which an area is divided, or a combination thereof can be performed.
[0179]
Regarding sharpness and granularity, the following desirable processing state differences are considered for each of (1) and (2) shown in FIG.
[0180]

FIG. 23 shows an example of area division regarding sharpness (here, emphasis processing) and graininess (here, grain removal processing).
[0181]
As an example, it is assumed that the area can be divided into three: “C: clouds”, “B: blue sky”, and “A: mountains, trees”. As shown, each of A, B, and C has a different combination of sharpness and granularity that is preferable. In addition, the relationship between the respective boundary areas is such that a clear outline exists between A and B, and a blurry outline exists between B and C. It is clear that the characteristics of the area boundaries can be easily determined by evaluating the images of each resolution level generated by the multi-resolution conversion processing described above with reference to FIG.
[0182]
Then, in the example of the sharpness processing, for example, a mask in which the sharpness enhancement coefficients are arranged corresponding to the screen position is created (similar to the mask in the example of FIG. 19), and the regions A to C are respectively Is obtained by the method described above with reference to FIG. 19 and the like, and a correction mask is obtained by blurring each mask with a blur amount corresponding to the relevant resolution level. A total of three areas A to C are obtained. Is synthesized.
[0183]
If the correction amount of the pixel at the position corresponding to the mask is determined in accordance with the correction amount information described in the synthesized mask, sharpness enhancement according to the characteristics of each of the areas A to C is performed. At the boundary between the areas B and B, the correction amount of the sharpness enhancement changes clearly, and at the boundary between the areas B and C, the correction amount of the sharpness enhancement changes gently.
[0184]
In the case of image information having a plurality of color dimensions, such as a color image, color coordinate conversion is performed as necessary, and only the necessary coordinate axes are subjected to the processing described above. It doesn't matter.
[0185]
For example, regarding brightness correction, which is particularly important for dodging gradation correction, in the case of an image represented by three colors of RGB, the image is first converted into luminance and color difference (Lab or the like), and only luminance information is converted. By performing the processing, it is possible to suppress a decrease in image processing quality and to significantly reduce the amount of image processing.
[0186]
In addition, when a region to be classified by a region such as a flower, the sea, or the sky, or a subject has a unique color tone, one or both of a process of determining a region boundary and a process of evaluating characteristics of the region boundary are performed. In addition, it is also possible to perform the unique color tone using the color coordinates that are most easily extracted, and to perform the actual image processing for each area on another color coordinate, for example, the luminance or saturation coordinate. It is also possible to perform specialized performance tuning for specific and special images such as "red roses".
[0187]
Next, steps for executing the image processing method according to the present invention and executing a program for causing the image processing means of the image processing apparatus to function according to the present invention will be described with reference to the flowcharts of FIGS.
[0188]
FIG. 24 shows the basic steps.
First, image information is obtained (step 1), and scene attribute information is obtained (step 2).
[0189]
Next, a subject pattern to be extracted is determined from the acquired scene attribute information (step 3), and components characterizing each subject pattern are determined (step 4).
[0190]
Further, for each component, a preferable resolution level suitable for component extraction is set (step 5), and the image information is subjected to multi-resolution conversion (step 6).
[0191]
Each component is extracted at each suitable resolution level (step 7), and a subject pattern is extracted based on the extracted component (step 8).
[0192]
Finally, dodging-like processing is performed to perform different corrections for the entire image or for each area in terms of gradation and sharpness according to the evaluation result obtained by evaluating the characteristics of the extracted subject pattern and the subject pattern boundary area. In addition, various image processing such as image clipping is performed (step 9), and the processing ends.
[0193]
FIG. 25 is a preferred example of setting a suitable resolution level for extracting a component characterizing a subject pattern according to size information of the subject pattern.
[0194]
The steps up to step 4 for determining the components that characterize the subject pattern are the same as in the example of FIG. Thereafter, size information of the subject pattern is obtained (step 201), and for each component, a preferable resolution level suitable for extracting the component set based on the size information of the subject pattern is set (step 6). The subsequent processing is the same as in the case of FIG.
[0195]
FIG. 26 is another example in which part of the gradation correction is realized by dodging processing.
[0196]
First, input image information is obtained (step 1), and it is checked whether or not there is information similar to a scene attribute in a film or a medium (step 302). If there is (YES in step 302), the obtained information is stored in the information recording unit. (Step 303). On the other hand, the image is displayed on the image display unit, the information of the scene attribute is also obtained from the customer, and stored in the information recording unit (stick 304).
[0197]
Based on these, scene attributes are determined (step 305), and a subject pattern to be extracted is determined (step 306).
[0198]
Next, the determined subject pattern is extracted by, for example, a method using a multi-resolution conversion process (step 307), and priority information is given by using a weighting coefficient or the like (step 308), and further extracted. The priority is corrected according to the position and size of the subject pattern (step 309).
[0199]
Further, a gradation correction amount corresponding to each of the extracted subject patterns is determined based on various information stored in the information storage unit, for example, information on preferable gradation and color tone reproduction (step 310).
[0200]
Next, the gradation correction amount of each subject pattern is separated into a dodging process component and the remaining components (step 311), and a masking process is performed using a dodging method described in the present application to which a multi-resolution conversion process is applied. (Step 312) The weighted average value of the remaining components of the gradation correction amount of each subject pattern obtained in Step 311 is calculated using the weighting coefficient of each subject pattern obtained in Step 309 (Step 313). The image is subjected to gradation correction by an amount corresponding to the average value (step 314), and the process is terminated.
[0201]
FIG. 27 shows still another example of the present invention applied to the correction of sharpness of dodging processing, in this case, enhancement processing.
[0202]
The processing (steps 1 to 307) of acquiring input image information, determining scene attributes, determining a subject pattern to be extracted, and extracting the determined subject pattern is the same as the previous example.
[0203]
Next, a preferable sharpness enhancement coefficient is set according to each of the extracted subject patterns (step 408).
[0204]
Further, a mask is created in which the set sharpness enhancement coefficients are arranged in a two-dimensional array in the area where each object pattern exists (step 409), and the characteristics of the boundary area of each object pattern are determined by decomposing the binomial wavelet. Evaluation is made by comparing the signal intensities appearing in the image (step 410).
[0205]
The mask created in step 409 is blurred based on the evaluation result in step 410 (step 411), and the created mask for each subject pattern is synthesized (step 412).
[0206]
The image processing is performed by applying the sharpness correction amount corresponding to each pixel position created in the combined mask to each corresponding pixel (step 413), and the process ends.
[0207]
【The invention's effect】
According to the invention of any one of claims 1 to 24, when a different image correction amount is applied to each region of an image, the unnaturalness of the correction result generated at the region boundary can be reduced, and the main subject can be displayed with appropriate image characteristics. It is possible to reproduce and obtain an image with balanced image characteristics as a whole.
[0208]
According to the third, eleventh, or nineteenth aspect, the characteristics of the region boundary can be determined with certainty, so that highly accurate region division corresponding to the characteristic of the boundary can be performed.
[0209]
According to the fourth, twelfth or twentieth aspect of the present invention, the mask blur amount can be easily switched by switching the level at which the masking process is performed, so that the process according to the evaluation result of the region boundary can be easily realized.
[0210]
According to the fifth, seventh, thirteenth, fifteenth, twenty-first, and twenty-third aspects of the present invention, the position of the boundary region can be specified reliably and accurately, and a highly accurate image processing result can be obtained. Further, in each step of the binomial wavelet transform, it is possible to preferably perform the sharpness correction and the graininess correction.
[0211]
According to the invention of claims 8, 14, 16, 22, or 24, since the characteristic evaluation and the image correction processing can be performed with the color coordinates suitable for each, high-accuracy and high-speed image processing can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a basic configuration of a digital minilab provided with an image processing device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a wavelet function.
FIG. 3 is a conceptual diagram of a wavelet transform.
FIG. 4 is a conceptual diagram of a wavelet transform.
FIG. 5 is a conceptual diagram of a process of decomposing a signal by wavelet transform.
FIG. 6 is a conceptual diagram of a wavelet transform.
FIG. 7 is a diagram illustrating an example of an image signal.
FIG. 8 is a conceptual diagram of an inverse wavelet transform.
FIG. 9 is a conceptual diagram of a wavelet transform.
FIG. 10 is a conceptual diagram of a wavelet transform.
FIG. 11 is a diagram illustrating an example of a subject pattern and components.
FIG. 12 is a diagram showing a relationship between components detected when the resolution is reduced.
FIG. 13 is a diagram illustrating a relationship between a pattern size and a detected component.
FIG. 14 is a diagram illustrating an example of a subject pattern and components.
FIG. 15 is a diagram illustrating logic for combining a plurality of components.
FIG. 16 is a diagram illustrating extraction of a subject pattern.
FIG. 17 is a diagram illustrating gradation correction for a plurality of subject patterns.
FIG. 18 is a diagram illustrating gradation correction for a plurality of subject patterns.
FIG. 19 is a block diagram showing dodging processing.
FIG. 20 is a diagram illustrating an example of a mask used in a dodging process.
FIG. 21 is a diagram illustrating an example of dodging processing.
FIG. 22 is a diagram illustrating an example of dodging processing.
FIG. 23 is a diagram illustrating an example of processing by area division related to sharpness and granularity.
FIG. 24 is an example of a flowchart of a program for executing the image processing method according to the embodiment of the present invention and causing the image processing means of the image processing apparatus according to the embodiment of the present invention to function;
FIG. 25 is an example of a flowchart of a program for executing the image processing method according to the embodiment of the present invention and causing the image processing means of the image processing apparatus according to the embodiment of the present invention to function;
FIG. 26 is an example of a flowchart of a program for executing the image processing method according to the embodiment of the present invention and causing the image processing means of the image processing apparatus according to the embodiment of the present invention to function;
FIG. 27 is an example of a flowchart of a program for executing the image processing method according to the embodiment of the present invention and causing the image processing means of the image processing apparatus according to the embodiment of the present invention to function;
[Explanation of symbols]
1 Digital camera
2 Image recording media
3 Camera
4 Film
5 Media driver
6 Film scanner
7 Image input section
8 Image processing unit
9 Silver halide exposure printer
10 Inkjet printer
11 Image recording media
12 Instruction input unit
13 Keyboard
14 mouse
15 Contact sensor
16 Image display section
17 Information storage unit

Claims (24)

In an image processing method of dividing an image into a plurality of regions, determining a correction amount of an image characteristic value for each region, and performing an image correction process,
An image processing method comprising: evaluating a property of a boundary between the plurality of regions; and determining a correction amount for a region near the boundary according to the evaluated property of the boundary. 2. The image correction method according to claim 1, wherein the image correction processing includes at least one of a gradation correction of an image signal value, a color tone correction of a color image, a saturation correction, a sharpness correction, and a graininess correction. The image processing method described in the above. 3. The image processing method according to claim 1, wherein the property of the boundary is evaluated based on a result of multi-resolution conversion processing of the input image information. The image correction process is a process including at least one of the following: tone correction of an image signal value, color tone correction of a color image, and saturation correction, and performs multi-resolution conversion processing on input image information. The image processing method according to any one of claims 1 to 3, wherein the image processing is performed on a low-frequency image generated at each level at which this is inversely transformed. 5. The image processing method according to claim 3, wherein the multi-resolution conversion processing is based on a Dyadic Wavelet conversion processing. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed in accordance with the content of the image correction processing. And at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, and the property evaluation The image processing method according to claim 1, wherein the information is information on luminance, saturation, or hue. The image correction process is a process including at least one of a sharpness correction and a graininess correction of an image signal value, and the multi-resolution conversion process is a process based on a binomial wavelet (Dyadic Wavelet) conversion process. The image processing method according to claim 3, wherein: The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed on a color space determined in accordance with the content of the image correction processing. Performed based on the input image information of at least one dimension, and at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, The image processing method according to claim 7, wherein the characteristic evaluation is information on luminance. In an image processing apparatus having an image processing unit that divides an image into a plurality of regions, determines a correction amount of an image characteristic value for each region, and performs an image correction process,
The image processing apparatus, wherein the image processing unit evaluates a property of a boundary between the plurality of regions, and determines a correction amount for a region near the boundary in accordance with the evaluated property of the boundary. The image processing means performs the image correction processing including at least one of a tone correction of an image signal value, a color tone correction of a color image, a saturation correction, a sharpness correction, and a graininess correction. The image processing device according to claim 9. The image processing apparatus according to claim 9, wherein the image processing unit performs the evaluation of the property of the boundary based on a result of performing a multi-resolution conversion process on input image information. The image processing means performs the image correction processing including at least one of gradation correction of image signal values, color tone correction of color images, and saturation correction, and performs multi-resolution conversion processing on input image information. The image processing apparatus according to any one of claims 9 to 11, wherein the image correction processing is performed on a low-frequency image generated at each level for inversely transforming the low-frequency image. . The image processing apparatus according to claim 11, wherein the multi-resolution conversion processing is based on a Dyadic Wavelet conversion processing. The input image information is a color image composed of a three-dimensional color space, and the image processing means determines the property evaluation of the region boundary and / or determines the image correction processing according to the content of the image correction processing. Performed based on image information of at least one dimension on the color space, and at least one dimension on the color space is information on luminance or saturation of a color image with respect to the image correction processing. The image processing apparatus according to any one of claims 9 to 13, wherein the property evaluation is information relating to luminance, saturation, or hue. The image correction process is a process including at least one of a sharpness correction and a graininess correction of an image signal value, and the multi-resolution conversion process is based on a binomial wavelet (Dyadic Wavelet) conversion process. The image processing apparatus according to claim 10, wherein: The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed on a color space determined in accordance with the content of the image correction processing. Performed based on the input image information of at least one dimension, and at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, The image processing apparatus according to claim 15, wherein the characteristic evaluation is information on luminance. An image processing unit that divides an image into a plurality of regions, determines a correction amount of an image characteristic value for each region, and performs an image correction process.
A computer-readable storage medium storing an image processing program for evaluating a property of a boundary between a plurality of regions and determining a correction amount for a region near the boundary according to the evaluated property of the boundary. 18. The image processing method according to claim 17, wherein the image correction processing includes at least one of a gradation correction of an image signal value, a color tone correction of a color image, a saturation correction, a sharpness correction, and a graininess correction. Image processing program as described. 19. The image processing program according to claim 17, wherein the evaluation of the property of the boundary is performed based on a result obtained by performing a multi-resolution conversion process on the input image information. The image correction process is a process including at least one of tone correction of image signal values, color tone correction of a color image, and saturation correction, and performs multi-resolution conversion of input image information. The image processing program according to any one of claims 17 to 19, wherein the image processing program is applied to a low-frequency image generated at each level for inversely transforming. 21. The image processing program according to claim 19, wherein the multi-resolution conversion processing is based on a Dyadic Wavelet conversion processing. The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed in accordance with the content of the image correction processing. And at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, and the property evaluation 22. The image processing program according to claim 17, wherein the information is information on luminance, saturation, or hue. The image correction process is a process including at least one of a sharpness correction and a graininess correction of an image signal value, and the multi-resolution conversion process is based on a binomial wavelet (Dyadic Wavelet) conversion process. The image processing program according to claim 19, wherein: The input image information is a color image composed of a three-dimensional color space, and the property evaluation of the area boundary and / or the image correction processing is performed on a color space determined in accordance with the content of the image correction processing. Performed based on the input image information of at least one dimension, and at least one dimension in the color space is information on luminance or saturation of a color image with respect to the image correction processing, The image processing program according to claim 23, wherein the characteristic evaluation is information on luminance.

Images