JP2004282318A - Scene change detection method and apparatus - Google Patents

Abstract

When a scene change between two images is detected using a histogram, accurate scene change detection is possible even for a faded image.
A histogram generation circuit for generating a histogram includes a one-block / component summation circuit, a level determination circuit, a subdivision addition amount determination circuit portion, and a level counter portion. Each subdivision addition amount determination circuit 203 _i is a first histogram count corresponding to that level in the allocation ratio to obtain the second histogram count level adjacent to that level, the first corresponding to that level respectively outputs a second level counter adjacent levels level counter 204 _i and its level. These level counters calculate the histogram by accumulating the histogram count values, respectively. The histogram difference calculation circuit 105 calculates the difference between the histograms of the previous and current screen images, calculates the sum of the differences, and the comparison judgment circuit 106 compares the sum of the differences with a threshold to detect a scene change. .
[Selection] Fig. 2

Description

【００４４】
ここで、段階ｊがｍ／２未満の整数のときはレベル（ｉ−１）とレベルｉのヒストグラムを加算し、段階ｊがｍ／２以上の整数のときはレベルｉとレベル（ｉ＋１）のヒストグラムを加算する。
隣り合うヒストグラムへの加算量は任意に設定できるが、それぞれに加算する値の合計は一定の値ａｄｄになるようにする。たとえば、ａｄｄ＝８とする。
基本細分割加算パラメータａ，ｂ，ｃ，ｄは、０〜ａｄｄまでの間の任意の整数値で、たとえば、ａｄｄ＝８のとき、ａ＝４、ｂ＝２、ｃ＝１、ｄ＝０とすれば、表１の値は表２に示す値になる。これが、レベルｉの細分割加算量（ヒストグラム計数値）を示している。
【００４５】
【表２】

【００４６】
なお基本オーバーラップパラメータａ，ｂ，ｃ，ｄは、０からａｄｄの範囲で自由に設定できる。そのため、入力されるビデオ信号の性質、たとえば、フェードするビデオ信号、あるいは、通常に変化するビデオ信号などの性質に応じて、基本オーバーラップパラメータａ，ｂ，ｃ，ｄの設定を任意に使い分けることが可能である。
【００４７】
細分割加算量処理を行なう意義について述べる。
よりレベル境界に近い場合には、その境界をまたぐヒストグラムのレベルに対してなるべく均等に振り分け、また、レベルのほぼ中心に近い場合には振り分けをしないという設定になる。
たとえば、細分割加算量判定回路２０３_ｉ のｊ＝０ の分配処理部２０３ｂ_０は、レベルカウンタ２０４_ｉ−１ に「４」の整数値（ヒストグラム計数値）を出力し、レベルカウンタ２０４_ｉ に「４」の整数値を出力し、レベルカウンタ２０４_ｉ＋１ に「０」の整数値を出力する。
細分割加算量判定回路２０３_ｉ のｊ＝１ の分配処理部２０３ｂ_１ は、レベルカウンタ２０４_ｉ−１ に「２」の整数値（ヒストグラム計数値）を出力し、レベルカウンタ２０４_ｉ に「６」の整数値を出力し、レベルカウンタ２０４_ｉ＋１ に「０」の整数値を出力する。
細分割加算量判定回路２０３_ｉ のｊ＝２ の分配処理部２０３ｂ_２ は、レベルカウンタ２０４_ｉ−１ に「１」の整数値（ヒストグラム計数値）を出力し、レベルカウンタ２０４_ｉ に「７」の整数値を出力し、レベルカウンタ２０４_ｉ＋１ に「０」の整数値を出力する。 以下、同様である。
【００４８】
図４に図解した例では、レベルカウンタ２０４_ｉ−１ にａｄｄ_ｉ−１ を、レベルカウンタ２０４_ｉ にａｄｄ_ｉ を加えて（出力して）いる例を示しているが（なお、表１、表２ではレベルカウンタ２０４_ｉ＋１ には「０」が出力されるから実質的に細分割加算量は加算されない）、ａｄｄ＝ａｄｄ_ｉ−１ ＋ａｄｄ_ｉ ＋ａｄｄ_ｉ−１ を満たさなければならない。
【００４９】
なお、レベルの両端（ｉ ＝ ０， ｎ−１）に関しては、隣のヒストグラムが存在しない場合があるので異なる処理を行う必要がある。
左端のレベル０について表３に例示する。
【００５０】
【表３】

【００５１】
右端のレベル（ｍ−１）についても、レベル０と同様である。
このように、図３および図４に図解した細分割加算量判定回路部分２０３およびレベルカウンタ部分２０４は、単純に、ヒストグラム・レベル判定回路２０２の判定結果を該当するレベルのカウンタで累積してヒストグラムを算出するのではなく、１ブロック分・成分合計回路２０１で算出された和ｓｕｍがあるレベルｉに属すると判定された場合、そのレベルｉのレベルカウンタ２０４_ｉ はもとより、隣接するレベル（ｉ−１）、（ｉ＋１）のレベルカウンタ２０４_ｉ−１ 、２０４_ｉ＋１ にもヒストグラムを算出する整数値のデータを出力する。
すなわち、細分割加算量判定回路２０３_ｉ は、自己のレベルを中心として、それに隣接するレベルについてのレベルカウンタ２０４_ｉ−１ 、２０４_ｉ＋１ にも、表２に例示したような配分比率で、ヒストグラム算出のデータを与える。
【００５２】
細分割加算量判定回路２０３_ｉ からヒストグラム算出用の整数値（ヒストグラム計数値）を入力された、レベルカウンタ２０４_ｉ 、２０４_ｉ−１ 、２０４_ｉ＋１ はそれぞれ、入力された整数値（ヒストグラム計数値）のデータを累積して、ヒストグラムを算出する。
【００５３】
同様に、（ｉ＋１）番目の細分割加算量判定回路２０３_ｉ＋１ と、その真下に位置するレベルカウンタ２０４_ｉ＋１ と、このレベルカウンタ２０４_ｉ＋１ に隣接する２個のレベルカウンタ２０４_ｉ 、２０４_ｉ＋２ の接続関係は、図４に図解した接続関係と同様になり、上述したと同様の処理を行なう。
【００５４】
以上のとおり、ヒストグラム生成回路１０３のヒストグラム・レベル判定回路２０２およびレベルカウンタ部分２０４は、各ブロックの和ｓｕｍに対して、隣り合うヒストグラムに振り分けて加算するオーバーラップ加算を行う。
【００５５】
規格化回路
１画像の処理が終了したら、図２に図解した規格化回路２０５で各レベルのヒストグラムの合計値ΣＨを一定の値ａｄｄで割り算して（ΣＨ／ａｄｄ）、ヒストグラムの合計値ΣＨの規格化を行う。
規格化回路２０５で行なう規格化処理は本発明において本質的ではないが、上述したように、一定の値ａｄｄを用いて、ヒストグラム計数値を変化させているので、それを規格するため、便宜的に用いている。ただし、規格化することにより、後述する比較判断回路における判断処理が、絶対値ではなく相対値で画一的に判断できるという利点がある。
【００５６】
ヒストグラム差分計算回路
以上のように作成された現（今回の）画像のヒストグラムは、ヒストグラム差分計算回路１０５で前（前回の）画像のヒストグラムと各レベルの差分の和を計算する。
現画像のヒストグラム結果は、次の画像との比較のため、ヒストグラム保存レジスタ１０４に結果が保持される。
【００５７】
比較判断回路
ヒストグラム差分計算回路１０５で計算した結果は、比較判断回路１０６で外部から設定可能なしきい値（または閾値）ｔｈｒｅｓと比較し、ヒストグラム差分計算回路１０５の結果がしきい値ｔｈｒｅｓを越えていたら、比較判断回路１０６はシーンチェンジと判断してシーンチェンジ検出フラグをオン（たとえば、「１」）にする。
【００５８】
以上述べた本発明の第１実施の形態によれば、図７（Ａ）、（Ｂ）を参照して述べたように、フェードする画像についても正確にシーンチェンジ検出を行なうことができる。
もちろん、本発明の第１実施の形態によれば、通常の画像変化についても正確にシーンチェンジ検出ができる。
したがって、本発明の第１実施の形態のシーンチェンジ検出方法およびシーンチェンジ検出装置を、図１を参照して述べた装置、あるいは、ビデオ信号編集装置などの各種のビデオ信号処理装置におけるシーンチェンジ検出に適用できる。
【００５９】
変形態様
本発明の実施の形態の変形態様として、ヒストグラムを細分化する際、ここでは段数ｍ＝８の例を示したが、さらに細かく分割すること、たとえば、ｍ＝２０とすることなども可能である。このときは、もちろん、基本加算パラメータ（オーバーラップパラメータ）の数、および、値も上述した例示とは異なる。
基本的に、ヒストグラムのレベル数ｎや、細分化する段数ｍ、オーバーラップパラメータの範囲（一定の値ａｄｄ）は大きくすることができるが、本発明を実施するハードウエア（回路）の構成をある程度の大きさに制限するため、適切な値にしている。
たとえば、シーンチェンジ検出能力を高める方法としては、レベルの数を非常に大きくする方法が想定されるが、そうすると、ハードウエア（回路）の規模が大きくなる。これに対して、本発明においては、ハードウェアの規模をあまり大きくせずに、より正確なシーンチェンジ検出を実現できることが大きな利点である。
【００６０】
さらに、たとえば、本発明の他の実施の形態として、連続して到来する過去４画面分の画像分程度のヒストグラムの情報を保持しておき、これら４画面の画像についてのシーンチェンジを判断し、たとえ、今回の画面の画像と直前の前回の画像とのヒストグラムの差分が大きくても、２〜３画像前との差分が小さければフラッシュ（一時的な（瞬間的な）画像変化）と見なし、シーンチェンジを無効にする（正規のシーンチェンジとはしない）といった処理も可能である。
このように本発明のシーンチェンジ検出装置を構成すると、より高精度なシーンチェンジの検出が可能である。
【００６１】
また、オーバーラップパラメータの値を、動きが早い画像か、動きが多い画像かなど、画像の内容、簡単な画像が複雑な画像かなどの画像の性質などに応じて、適切に変更することも可能である。
【００６２】
また、１個の細分割加算量判定回路２０３_ｉ から出力されるレベルカウンタ部分の数を非常に多くしておき、ヒストグラム計数を行なわないレベルカウンタには分配処理部２０３ｂ_０ 〜２０３ｂ_ｍ−１ からヒストグラム計数値を「０」として出力することもできる。すなわち、本発明においては、オーバーラップパラメータの設定だけで、ある細分割加算量判定回路２０３_ｉ と関連するレベルカウンタ２０４との接続関係をヒストグラム計数値の設定で実質的に制御することができる。
特に、このような方法は、シーンチェンジ検出装置をＤＳＰ（ディジタル信号プロセッサ）などで処理する場合には都合がよい。たとえば、ＤＳＰにおいて、細分割加算量判定回路２０３_ｉ と全てのレベルカウンタ２０４との接続関係をつけておき、そのとき接続関係のないレベルカウンタ２０４には「０」のヒストグラム計数値を出力すれば、実質的に接続関係がないことになり、接続関係のあるレベルカウンタ２０４に必要なヒストグラム計数値を出力すればよい。
【００６３】
本発明のシーンチェンジ検出装置としては、図２〜図４を参照して述べたハードウエア回路で実現するだけでなく、ＤＳＰあるいは、コンピュータを用いて実現することもできる。
【００６４】
【発明の効果】
本発明によれば、フェードイン／フェードアウトする画像であっても、しかも、通常の画像変化であっても、正確に画像の切り替わり、すなわち、シーンチェンジを検出可能できる。
また本発明によれば、シーンチェンジの検出を極力簡単な方法で、かつ、迅速に行なうことができる。
さらに本発明によれば、簡単な構成、かつ、容易にシーンチェンジ検出が可能である。
また本発明のシーンチェンジ検出方法およびシーンチェンジ検出装置は種々のビデオ信号処理装置に適用できる。
【図面の簡単な説明】
【図１】図１は本発明のシーンチェンジ検出方法を適用した１例としての画像情報処理シーンの構成図である。
【図２】図２は本発明のシーンチェンジ検出方法および装置の第１実施の形態のブロック図である。
【図３】図３は図１に図解したヒストグラム生成回路の構成図である。
【図４】図４は図３に図解した細分割加算量判定回路とそれに関連するレベルカウンタとの接続関係を示す構成図である。
【図５】図５は画像のブロック分割を説明する図である。
【図６】図６はヒストグラムの作成例を示すグラフである。
【図７】図７はヒストグラムの差分を求める方法を説明するグラフである。
【図８】図８はフェード時のヒストグラムの変化を示すグラフである。
【符号の説明】
１００・・シーンチェンジ検出装置
１０１・・画像データ分離回路
１０２・・画像データブロック分割回路
１０３・・ヒストグラム生成回路
２０１・・１ブロック分・成分合計回路
２０２・・ヒストグラム・レベル判定回路
２０３・・細分割加算量判定回路部分
２０３ａ・・細分割加算量判定部
２０３ｂ・・分配処理部
２０４・・レベルカウンタ部分
２０５・・規格化回路
１０４・・ヒストグラム保存レジスタ
１０５・・ヒストグラム差分計算回路
１０６・・比較判断回路
３００・・シーンチェンジ検出・処理情報処理装置
３０１・・ビデオ信号再生装置
３０２・・プリ映像信号処理装置
３０３・・符号化装置
３０４・・大容量記録装置
３０５・・サムネール画像生成装置
３０６・・ホストコンピュータ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a scene change detection method and apparatus for detecting a change in a scene (scene or scene) of a continuous image.
In particular, the present invention relates to a scene change detection method and apparatus capable of accurately detecting a scene change even in an image that fades in and out.
[0002]
[Prior art]
When performing image processing on continuous images, for example, when encoding a video signal of a moving image using the MPEG method, the user often encounters a case where it is desired to detect a large change in a scene (image switching). As a technique for detecting such a scene change, a scene change detection method is used, and various scene change detection methods have been proposed so far.
The basic method of such a scene change detection method is to detect a change in the image between the image of the previous screen (frame or field) and the image of the current screen and change the image to some extent between the two images. Is determined as a scene change.
[0003]
In this specification, the term “screen” refers to a set of images collected as one screen such as one field or one frame.
[0004]
Until now, various improved methods have been proposed in order to detect scene changes more accurately in accordance with various image types and situations.
[0005]
Japanese Patent Application Laid-Open No. H10-294923 (Japanese Patent No. 3175632) discloses, as illustrated in FIG. 2 of this publication, (1) a frame image is divided into blocks, and (2) a color histogram of each block is calculated. (3) calculating the forward similarity, the backward similarity, and the front-back similarity using the color histogram; (4) determining the threshold to determine the effective block position; and (5) calculating the correlation value of the effective block position. An evaluation value is obtained by averaging, and (6) a scene change is detected from the magnitude relationship between the obtained evaluation value and the threshold value.
[0006]
However, the method disclosed in Japanese Patent Application Laid-Open No. H10-294923 not only obtains a histogram, but also obtains a forward similarity, a backward similarity, a front-back similarity, and further calculates and evaluates a correlation value. However, there is a disadvantage that the processing is complicated and detection takes time, the circuit (device) configuration for realizing the method is complicated, and the circuit scale is increased.
[0007]
Japanese Unexamined Patent Publication No. Hei 10-276435 discloses that, in the configuration illustrated in FIG. 1 of this publication, (1) the level determination circuit determines the color value of the image data of the current screen, and (2) the level is determined by the counter. A histogram is obtained by counting for each determined level, (3) a local maximum / minimum value of the histogram obtained by the local maximum / minimum detection circuit is detected, and (4) a local maximum value or local minimum value is detected by the representative point selection circuit. Find a representative point. The above processing is also performed for the previous image, and the representative point of the previous image is stored in the register. (5) The representative point comparison circuit compares the representative point of the previous screen with the representative point of the current screen, and determines whether or not a scene change has occurred, based on whether or not both take close values.
[0008]
The method disclosed in Japanese Patent Application Laid-Open No. 10-276435 is intended to solve the problem of erroneously detecting a continuous scene having a low degree of correlation as a scene change. The difference between the representative point of the previous image and the representative point of the current image is used as a criterion. In other words, the method disclosed in Japanese Patent Application Laid-Open No. H10-276435 is a technique for detecting a scene change from a scene having a low degree of correlation, so as to detect the magnitude of the change sharply. However, the technique disclosed in Japanese Patent Application Laid-Open No. H10-276435 has too high sensitivity and may frequently cause a scene change detection state.
[0009]
Another scene change detection method that improves the above-described scene change detection method will be described with reference to FIGS.
(1) As illustrated in FIG. 5, a digitized video signal (video signal) of the current screen is converted into a one-component video signal, for example, in the case of a component signal, an R, G, B component, or a composite signal. , The luminance signal Y, the first chrominance signal Cb, and the second chrominance signal Cr are separated, and the separated one-component video signal is divided into an effective image area for each image of one screen (frame or field). Is divided into a plurality of blocks each having (x pixels × y lines), and a video signal of one component, for example, a sum sum of R components (or luminance components) is calculated for each block.
(2) As illustrated in FIG. 6, an n-level histogram is created according to the sum value of one component video signal of each block. In the example illustrated in FIG. 6, an example where n = 16 is shown, and a 16-level histogram is shown.
(3) As illustrated in FIG. 7, the absolute value of the difference between the histogram of the image of the previous screen and the histogram of the image of the current screen, which is held, is calculated for each level. Sum of difference of sum_diffIs calculated.
(4) Sum sum of differences for all levels_diffIs greater than a preset threshold (or threshold) thres (sum_diff≧ thres), a scene change is determined on the assumption that a scene change corresponding to scene detection has occurred between the image of the previous screen and the image of the current screen.
[0010]
The above processes (1) to (4) are also performed on video signals of other components, for example, the G component and the B component, or the first dye component Cb and the second dye component Cr, and integrate them. By doing so, the scene change of the entire image is detected.
However, in the following description, for simplicity, the description is limited to one component video signal, for example, an R component or a luminance component.
[0011]
[Patent Document 1]
JP-A-10-294923 (Japanese Patent No. 3175632)
[Patent Document 2]
JP-A-10-276435
[0012]
[Problems to be solved by the invention]
The scene change detection method described with reference to FIGS. 5 to 7 encounters the following problem.
When an image that fades in or fades out, that is, an image that changes little by little (for example, an image in which the luminance increases or an image in which the luminance decreases) is input, the image components gradually change. As illustrated in FIG. 8A, when the sum sum is concentrated on a certain level boundary of the histogram in the histogram creation method in the processing of the above step (2), the image of the next screen is as illustrated in FIG. Then, the histogram moves from the state shown by the broken line to the state shown by the solid line, all to adjacent levels. As a result, the sum of the histogram differences sum_diffIs greater than the threshold (or threshold) thres, that is, (sum_diff.Gtoreq.thres). As a result, even when the scene is not recognized by the human naked eye as a scene change, it may be determined as a scene change when processed by the image signal processing circuit.
[0013]
Such scene changes occur periodically.
That is, a scene change is continuously and periodically detected for a fade-in / fade-out image, particularly for a relatively long fade image.
Of course, as long as a person views such a fade-in / fade-out image, it is not recognized as a scene change and is not an original scene change. That is, erroneous detection of a scene change in the image signal processing circuit.
[0014]
SUMMARY OF THE INVENTION An object of the present invention is to provide a method and apparatus capable of accurately detecting image switching, that is, a scene change, even if the image is a fade-in / fade-out image and a normal image change. is there.
Another object of the present invention is to provide a scene change detection method and apparatus capable of detecting the scene change as quickly and simply as possible.
It is a further object of the present invention to provide a scene change detection device which has a simple configuration and can easily realize the above-described scene change detection method.
[0015]
Another object of the present invention is to provide a scene change detection method and apparatus which can be applied to various uses.
[0016]
[Means for Solving the Problems]
According to the first aspect of the present invention, for images of two screens of a continuous field or frame, histograms of components constituting the image are created, and the total value of the differences is calculated and set. In the scene change detection method for detecting that the scene of the image has changed when the threshold value is larger than the threshold value, a certain number is assigned to the corresponding level and the adjacent levels at the time of creating the histogram and added ( A new histogram result is calculated by normalizing, and then a normalization is performed, and a result of a new histogram is calculated, and a change in an image scene is detected using the calculated histogram. Provided.
[0017]
According to a second aspect of the present invention, there is provided a scene change detection method for detecting a scene change between images of a first screen and a second screen, wherein (a) the level of a component signal of the image of the first screen is determined. (B) For each of the determined levels, for each of the determined level and the level adjacent to the determined level, calculate a count value for each level at which a distribution ratio is calculated at a distribution ratio that specifies a distribution ratio. (C) counting, as a histogram, count values distributed at the distribution ratios for the determined level and for levels adjacent to the determined level, and (d) holding the histogram, e) The same processing as in (a) to (d) above is performed on the image of the second screen, and the obtained histogram and the histogram of the held image of the first screen are obtained. (G) calculating for each of the above levels and summing the histogram differences for all levels; (h) when the sum of the histogram differences is greater than a predetermined threshold, A scene change detecting method for determining that a scene change has occurred between the image of the second screen and the image of the second screen.
[0018]
According to a third aspect of the present invention, there is provided a scene change detection device for detecting a scene change between images on a first screen and a second screen, comprising: a histogram generation unit; a histogram storage unit; , Comparing means
The histogram generating means calculates the sum of the image data of each component in one block in order to generate a histogram for the image data of each component of each block obtained by dividing the image video signal of one screen into a plurality of blocks. A block / component summing unit; a level judgment unit for judging which of a plurality of levels the sum calculated by the one block / component summing unit belongs to; and a plurality of subdivisions of the number of the level judged. A subdivision addition amount determination unit having an addition amount determination unit, and a level counter unit having the same number of level counter units as the number of the plurality of subdivision addition amount determination units,
Each of the subdivision addition amount determination units of the subdivision addition amount determination unit includes a first histogram count value for calculating a histogram corresponding to the level at a certain distribution ratio and a calculation of a histogram at a level adjacent to the first histogram count value. And outputs the obtained first and second histogram counts to a first level counter corresponding to the level and a second level counter adjacent to the level, respectively,
The first and second level counters respectively accumulate the histogram count values input from the subdivision addition amount determination means to calculate a histogram,
The histogram storage unit stores the histogram generated by the histogram generation unit,
The histogram difference calculation unit calculates a difference between the histogram of each level generated by the histogram generation unit and the histogram of each level of the previous screen image stored in the histogram storage unit, and calculates the sum of these differences. Calculate,
When the sum of the differences between the histograms is greater than a predetermined threshold value, the comparing and determining means determines that a scene change has occurred between the image on the first screen and the image on the second screen. An apparatus is provided.
[0019]
The calculation of the histogram by the histogram generation means in the present invention is not simply a histogram calculation for each level, but a distribution ratio that defines a distribution ratio for each level and the level adjacent to the level. A count value for each level for which the distribution ratio has been calculated is calculated, and the count value is counted (accumulated) by a level counter unit corresponding to the level and an adjacent level counter unit to calculate a histogram.
The calculated histogram is stored in the histogram storage unit, and is used for calculating the difference between the histograms of the next image of the second screen.
The histogram difference calculation means calculates a difference between the histogram of the current second screen and the histogram of the previous image of the first screen stored in the histogram storage means, and further calculates all histogram differences. Total for levels.
The comparison determining means determines that a scene change has occurred between the image on the first screen and the image on the second screen when the sum of the differences between the histograms is greater than a predetermined threshold.
[0020]
BEST MODE FOR CARRYING OUT THE INVENTION
Application Example of Scene Change Detection Method and Apparatus of the Present Invention
Before describing a preferred embodiment of the scene change detection method and apparatus of the present invention, an example to which the scene change detection method and apparatus of the present invention are applied will be described.
FIG. 1 is a configuration diagram of an application example to which the scene change detection method and apparatus of the present invention are applied.
The device illustrated in FIG. 1 is a device that receives a terrestrial wave or a BS broadcast wave, compresses and encodes the terrestrial wave or the BS broadcast wave, and records the coded data on a recording medium, for example, a disk recording medium such as a hard disk (HD) or (DVD + RW) disk. , A video signal reproducing device 301, a pre-video signal processing device 302, an encoding device 303, a recording device 304, a thumbnail (thumb-nail) image generating device 305, and a host computer 306.
[0021]
An example of the operation of the device illustrated in FIG. 1 will be described.
The video signal reproducing device 301 is, for example, an NTSC video signal decoding device, receives and decodes a terrestrial or BS broadcast wave, reproduces a video signal, and converts it into a digital video signal.
The pre-video signal processing device 302 performs pre- (pre) video signal processing, such as dividing one screen into a plurality of blocks, on the digital video signal converted by the video signal reproducing device 301, and performs scene change for each block. Is detected, and scene change information is passed to the host computer 306 when a scene change is detected.
The host computer 306 controls the encoding device 303, the recording device 304, and the thumbnail image generation device 305 based on the scene change information detected by the pre-video signal processing device 302.
[0022]
After the pre-video signal processing device 302 completes the pre-video processing, for example, the encoding device 303, which is an MPEG encoder, performs an encoding process on the digital video signal. Is recorded together with the additional information.
Index information is given to a disk recording medium to be controlled by the encoding device 303 and recorded by the recording device 304, and a thumbnail image is created by the thumbnail image generating device 305 by the thumbnail image specified by the host computer 306, and the thumbnail image is recorded by the recording device 304. It is recorded together with index information on a recording medium.
By doing so, it is possible to improve the usage efficiency of the user by using a menu with thumbnails and an index delimited for each scene.
[0023]
In particular, when the encoding device 303 employs a high-efficiency encoding method such as MPEG, if a scene change can be appropriately detected in a video signal received and decoded by the video signal reproducing device 301, a scene change The coding procedure can be effectively changed using the video signal of the frame in which is detected as an I (intra) picture, and more accurate and efficient coding can be performed in the coding device 303. As a result, the video signal can be encoded with high image quality at a reduced compression rate, and can be recorded on a disk medium by the recording device 304.
That is, when a digitized video input image is encoded (encoded) and recorded on a disk recording medium, it is possible to record with higher image quality by obtaining accurate scene change information. . In addition, index information for quickly searching for the scene that the user wants to view from the recorded video, scene change information can be used to create thumbnail images, and it can be used as additional information to detect commercial video parts .
[0024]
In this way, it is possible to reduce false detection of a scene change at a switching point of an image using a fade which is frequently used in a movie, a drama, and a home video, and as a result, a video signal is processed. For this purpose, necessary information is carefully selected, and effective creation of a thumbnail image and detection of a commercial video portion at an appropriate position can be performed.
For that purpose, it is necessary to accurately detect scene changes in the pre-video signal processing device 302.
That is, the scene change detection method and the scene change detection device of the present invention are introduced into the pre-video signal processing device 302 and the like in the device illustrated in FIG.
[0025]
If the scene change detection in the pre-video signal processing device 302 is appropriate, index information for quickly searching for a scene desired by the user from the encoded video signal recorded on the disc and scene change information for creating a thumbnail image There is an advantage that it can be used as additional information for detecting a commercial video portion.
[0026]
Of course, the scene change detection method and the scene change detection device of the present invention are not limited to the application to the device described with reference to FIG. 1, but can be applied to various video processing devices such as a video signal editing system.
[0027]
Preferred embodiments of the scene change detection method and the scene change detection device of the present invention used for such an application will be described below.
[0028]
First embodiment
FIG. 2 is a configuration diagram of a first embodiment of a scene change detection device according to the present invention.
2 includes an image data separation circuit 101, an image data block division circuit 102, a histogram generation circuit 103, a histogram storage register 104, a histogram difference calculation circuit 105, and a comparison determination circuit 106. And
[0029]
The scene change detection device 1 illustrated in FIG. 2 is incorporated in, for example, the pre-video signal processing device 302 illustrated in FIG.
In this case, it is assumed that a digital video signal provided by a video signal processing device such as the video signal reproducing device 301 is input to the image data separation circuit 101, for example.
Of course, the scene change detection device 1 illustrated in FIG. 1 is not limited to the device illustrated in FIG. 1 and can be used for scene change detection in, for example, a video signal editing device. The video signal recorded on the recording medium is reproduced by a digital video signal reproducing device, and the reproduced digital video signal is input to the image data separation circuit 101.
As described above, in the present embodiment, a case where a digital video signal in units of one frame or one field is input to the image data separation circuit 101, for example, as a bit stream will be described.
Note that one frame image or one field image is referred to as one screen image for convenience.
FIG. 5 illustrates an image configuration of one screen.
[0030]
Image data separation circuit
The image data separation circuit 101 separates a digitized video signal into component components or components of a composite signal, for example, R, G, B, or a luminance signal, and first and second color difference signals.
However, in the following description, only signal processing for one component video signal, for example, an R signal or a luminance signal will be described. Other component video signals, G and B, or first and second color difference signals are processed in the same manner.
The reason why the image data separation circuit 101 separates such a component video signal is that the following process is performed for each component for the sake of simplicity of processing and appropriateness of scene change detection. is there. Therefore, the image data separation circuit 101 is not an essential requirement for implementing the present invention.
[0031]
Image data block division circuit
The image data block dividing circuit 102 divides a detection area in an effective image from a video signal of one screen illustrated in FIG. 5 into a plurality of blocks. When the size of one block is x pixels × y lines, for example, 16 pixels × 16 lines, there are 256 video signals in one block.
As described above, the image data block dividing circuit 102 divides an image (video signal) of one screen into a plurality of blocks by appropriately setting a scene change detection capability (detection accuracy) and improving signal processing. This is because easiness is taken into consideration, and is not essential when the present invention is implemented.
[0032]
If the size (size) of the block is too small, the determination of a scene change becomes more strict (easy), but the number of blocks in one screen increases, so that the result of the memory such as a histogram is held. The capacity increases. On the other hand, if the size of the block is too large, the capacity of the memory for storing the results such as the histogram becomes small, but the sensitivity of the scene change becomes slow. Therefore, the size of one block is appropriately determined in consideration of the degree of image change, the sensitivity of scene change detection, and the like.
[0033]
Histogram generation circuit
The histogram generation circuit 103 counts the number of blocks at each level for a plurality of levels of the video signal of each block divided by the image data block division circuit 102, and calculates the count result as a histogram. .
However, the configuration and processing content of the histogram generation circuit 103 are different from the above-described conventional simple histogram calculation method.
Although the processing described below is performed for each block, only the processing for one block will be described below for convenience.
[0034]
FIG. 3 is a circuit diagram of the histogram generation circuit 103 illustrated in FIG.
The histogram generation circuit 103 includes a one-block / component summation circuit 201, a histogram / level determination circuit 202, a subdivision addition amount determination circuit portion 203, a level counter portion 204, and a normalization circuit 205.
The subdivision addition amount determination circuit portion 203 includes a plurality of, and n, subdivision addition amount determination circuits according to the level division provided in parallel.
The level counter section 204 is also configured by providing a plurality (n) of level counters in parallel corresponding to the number of subdivision addition amount determination circuits of the subdivision addition amount determination circuit section 203.
[0035]
The one-block / component summation circuit 201 calculates the sum, that is, the sum sum of the one-block component video signal, for example, the R component. The sum sum calculated by one block / component summation circuit 201 indicates the magnitude of one video signal component of the block, for example, the R component.
[0036]
The histogram / level determination circuit 202 calculates the sum sum into a plurality (n) of level determination thresholds A₀  ~ A_n-1  (A₀  <A₁  <... <A_n-1  ) To determine which range, that is, which level the sum sum belongs to.
When it is determined that the sum sum belongs to, for example, the i-th level, the histogram / level determination circuit 202 determines whether the i-th subdivision addition amount determination circuit 203 of the subdivision addition amount determination circuit portion 203 is present._i  In addition, of the sum sum calculated by the one-block / component summation circuit 201, data necessary for counting (counting up) the histogram, for example, upper several bits of the sum value, and an "enable signal", for example, " "1" and outputs an "invalidation signal", for example, "0" to the other subdivision addition amount determination circuits.
[0037]
In the conventional scene change detection method, the “validation signal” from the histogram / level determination circuit 202 is counted by a corresponding counter as it is. However, the histogram generation circuit 103 illustrated in FIG. 3 is provided with a subdivision addition amount determination circuit portion 203, and does not simply count the result of the histogram level determination circuit 202 with a counter as in the related art.
The configuration and processing of the subdivision addition amount determination circuit portion 203 and the level counter portion 204 will be described with reference to FIG.
[0038]
FIG. 4 shows an i-th subdivision addition amount determination circuit 203 out of n parallelly arranged in the subdivision addition amount determination circuit portion 203 and the level counter portion 204 illustrated in FIG._i  And the level counter 204 located immediately below_i  And this level counter 204_i  Two level counters 204 adjacent to_i-1  , 204_{i + 1}  FIG. 5 is a diagram showing a connection relationship of the present invention.
[0039]
In the configuration illustrated in FIG. 4, "adjacent" level counters are two level counters 204_i-1  , 204_{i + 1}  However, the subdivision addition amount determination circuit 203 described later_i  The number of “adjacent” level counters is not limited to two in accordance with the number m of stages in_i  Level counters 204 on both sides of_i-2  , 204_i-1  , 204_{i + 1}  204_{i + 2}  It can also be. Therefore, the meaning of “adjacent” in the present specification does not simply mean both sides, but also extends to the side, and further to the side.
However, in this embodiment, the level counter 204_i  Two level counters 204 on both sides of_i-1  , 204_{i + 1}  Will be exemplified.
[0040]
Subdivision addition amount determination circuit 203_i  Is a subdivision / addition amount determining unit 203a that determines the subdivision / addition amount in m stages for the level i divided by the histogram / level determination circuit 202, and m distribution processing units 203b provided in parallel₀  ~ 203b_m-1  Consists of m distribution processing units 203b₀  ~ 203b_m-1  Is referred to as a distribution processing unit 203b.
[0041]
Subdivision addition amount determination circuit 203_i  The data necessary for counting (counting up) the histogram, for example, the upper few bits of the sum value, from the sum sum calculated by the one-block / component summing circuit 201 from the histogram / level determination circuit 202 and “ When the "validation signal" of "1" is output, m parallel distribution processing units 203b are provided according to the bit value of the sum value, respectively.₀  ~ 203b_m-1  And outputs a signal having a value corresponding to the subdivision addition amount (histogram count value).
[0042]
Subdivision addition parameter
An example of the subdivision addition parameter (or the overlap parameter or the histogram count value) will be described with reference to Table 1.
Table 1 shows the subdivision addition parameters (overlap parameters) when the stage m is set to 8, and a, b, c, and d are the basic subdivision addition parameters (basic overlap parameters or basic distribution ratio coefficients). Good, let the sum of the subdivision addition parameters (overlap parameter) be add.
[0043]
[Table 1]

[0044]
Here, when the stage j is an integer less than m / 2, the histogram of the level (i-1) and the level i are added, and when the stage j is an integer of m / 2 or more, the level i and the level (i + 1) are added. Add the histogram.
The amount of addition to adjacent histograms can be set arbitrarily, but the sum of the values added to each histogram is set to a constant value add. For example, add = 8.
The basic subdivision addition parameters a, b, c, and d are arbitrary integer values from 0 to add. For example, when add = 8, a = 4, b = 2, c = 1, d = 0. Then, the values in Table 1 become the values shown in Table 2. This indicates the subdivision addition amount (histogram count value) at level i.
[0045]
[Table 2]

[0046]
The basic overlap parameters a, b, c, and d can be freely set in a range from 0 to add. Therefore, the setting of the basic overlap parameters a, b, c, and d may be arbitrarily used depending on the characteristics of the input video signal, for example, the characteristics of a faded video signal or a normally changing video signal. Is possible.
[0047]
The significance of performing the subdivision addition amount processing will be described.
When the level is closer to the level boundary, distribution is performed as evenly as possible with respect to the level of the histogram straddling the boundary, and when the level is near the center of the level, the distribution is not performed.
For example, the subdivision addition amount determination circuit 203_i  J = 0 distribution processing unit 203b₀Is the level counter 204_i-1  Outputs an integer value of “4” (histogram count value) to the level counter 204._i  Outputs an integer value of “4” to the level counter 204._{i + 1}  Outputs an integer value of "0".
Subdivision addition amount determination circuit 203_i  J = 1 distribution processing unit 203b₁  Is the level counter 204_i-1  Outputs an integer value (histogram count value) of "2" to the level counter 204._i  Outputs an integer value of “6” to the level counter 204._{i + 1}  Outputs an integer value of "0".
Subdivision addition amount determination circuit 203_i  J = 2 distribution processing unit 203b₂  Is the level counter 204_i-1  Outputs an integer value (histogram count value) of "1" to the level counter 204._i  Outputs an integer value of “7” to the level counter 204._{i + 1}  Outputs an integer value of "0". Hereinafter, the same applies.
[0048]
In the example illustrated in FIG._i-1  Add_i-1  To the level counter 204_i  Add_i  Are added (output). (Tables 1 and 2 show the level counter 204_{i + 1}  , “0” is output, so that the subdivision addition amount is not substantially added), and add = add_i-1  + Add_i  + Add_i-1  Must be satisfied.
[0049]
Note that for both ends of the level (i = 0, n-1), there is a case where there is no adjacent histogram, so different processing needs to be performed.
Table 3 shows an example of level 0 at the left end.
[0050]
[Table 3]

[0051]
The same applies to level 0 at the right end level (m-1).
As described above, the subdivision addition amount determination circuit portion 203 and the level counter portion 204 illustrated in FIGS. 3 and 4 simply accumulate the determination result of the histogram / level determination circuit 202 by the counter of the corresponding level and obtain the histogram. Is calculated, the sum sum calculated by the one block / component summation circuit 201 is determined to belong to a certain level i, and the level counter 204 of the level i is determined._i  Needless to say, the level counters 204 of the adjacent levels (i-1) and (i + 1)_i-1  , 204_{i + 1}  Also outputs integer value data for calculating the histogram.
That is, the subdivision addition amount determination circuit 203_i  Is the level counter 204 for the level adjacent to the level centered on the own level._i-1  , 204_{i + 1}  Also, the data of the histogram calculation is given at the distribution ratio as exemplified in Table 2.
[0052]
Subdivision addition amount determination circuit 203_i  Counter 204 to which an integer value (histogram count value) for calculating a histogram is input from_i  , 204_i-1  , 204_{i + 1}  Calculates the histogram by accumulating the input integer value (histogram count value) data.
[0053]
Similarly, the (i + 1) th subdivision addition amount determination circuit 203_{i + 1}  And the level counter 204 located immediately below_{i + 1}  And this level counter 204_{i + 1}  Two level counters 204 adjacent to_i  , 204_{i + 2}  Is similar to the connection relationship illustrated in FIG. 4, and the same processing as described above is performed.
[0054]
As described above, the histogram / level determination circuit 202 and the level counter section 204 of the histogram generation circuit 103 perform overlap addition for distributing the sum sum of each block to adjacent histograms and adding them.
[0055]
Normalization circuit
When the processing of one image is completed, the normalization circuit 205 illustrated in FIG. 2 divides the total value ΣH of the histogram of each level by a fixed value add (ΣH / add), and normalizes the total value ΣH of the histogram Do.
Although the normalization process performed by the normalization circuit 205 is not essential in the present invention, the histogram count value is changed using the constant value add as described above. Used for However, by normalizing, there is an advantage that determination processing in a comparison determination circuit described later can be performed uniformly using relative values instead of absolute values.
[0056]
Histogram difference calculation circuit
For the histogram of the current (current) image created as described above, the histogram difference calculation circuit 105 calculates the sum of the histogram of the previous (previous) image and the difference of each level.
The result of the histogram of the current image is stored in the histogram storage register 104 for comparison with the next image.
[0057]
Comparison judgment circuit
The result calculated by the histogram difference calculation circuit 105 is compared with a threshold value (or threshold value) thres which can be set from the outside by a comparison judgment circuit 106. If the result of the histogram difference calculation circuit 105 exceeds the threshold value thres, the comparison is performed. The determination circuit 106 determines that a scene change has occurred, and turns on a scene change detection flag (for example, “1”).
[0058]
According to the above-described first embodiment of the present invention, as described with reference to FIGS. 7A and 7B, a scene change can be accurately detected even for a faded image.
Of course, according to the first embodiment of the present invention, a scene change can be accurately detected even for a normal image change.
Therefore, the scene change detection method and the scene change detection device according to the first embodiment of the present invention are described with reference to FIG. 1 or scene change detection in various video signal processing devices such as a video signal editing device. Applicable to
[0059]
Modification
As a modification of the embodiment of the present invention, when the histogram is subdivided, the example in which the number of stages is m = 8 is shown here, but it is also possible to further subdivide the histogram, for example, to set m = 20. . In this case, of course, the number and values of the basic addition parameters (overlap parameters) are also different from those described above.
Basically, the number n of histogram levels, the number m of subdivision steps, and the range of the overlap parameter (constant value add) can be increased. However, the configuration of the hardware (circuit) for implementing the present invention is limited to some extent. In order to limit to the size of, it is set to an appropriate value.
For example, as a method of increasing the scene change detection ability, a method of enlarging the number of levels is supposed. However, in this case, the scale of hardware (circuit) increases. On the other hand, in the present invention, it is a great advantage that more accurate scene change detection can be realized without increasing the scale of hardware so much.
[0060]
Further, for example, as another embodiment of the present invention, information on histograms corresponding to the images of the past four screens arriving continuously is held, and a scene change is determined for these four screen images, Even if the difference between the histogram of the current screen image and the histogram of the immediately preceding image is large, if the difference between the previous image and the previous image is small, it is regarded as a flash (temporary (temporary) image change). Processing such as invalidating a scene change (not a regular scene change) is also possible.
By configuring the scene change detection device of the present invention in this way, it is possible to detect a scene change with higher accuracy.
[0061]
Also, the value of the overlap parameter may be appropriately changed depending on the content of the image, such as whether the image is fast-moving or an image with a lot of movement, the nature of the image such as whether a simple image is a complex image, or the like. It is possible.
[0062]
Also, one subdivision addition amount determination circuit 203_i  The number of the level counters output from the CPU is very large, and the distribution processing unit 203b is provided for the level counter that does not perform histogram counting.₀  ~ 203b_m-1  Can output the histogram count value as “0”. That is, in the present invention, only by setting the overlap parameter, a certain subdivision addition amount determination circuit 203_i  Can be substantially controlled by setting the histogram count value.
In particular, such a method is convenient when the scene change detection device is processed by a DSP (Digital Signal Processor) or the like. For example, in the DSP, the subdivision addition amount determination circuit 203_i  Is connected to all the level counters 204, and if the histogram counter value of "0" is output to the level counter 204 having no connection at that time, there is substantially no connection. A necessary histogram count value may be output to the relevant level counter 204.
[0063]
The scene change detection device of the present invention can be realized not only by the hardware circuits described with reference to FIGS. 2 to 4 but also by using a DSP or a computer.
[0064]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, even if it is an image which fades in / fade out, and even if it is a normal image change, image switching, ie, a scene change, can be detected correctly.
Further, according to the present invention, it is possible to detect a scene change by a simple method as quickly as possible.
Further, according to the present invention, it is possible to easily detect a scene change with a simple configuration.
Further, the scene change detection method and scene change detection device of the present invention can be applied to various video signal processing devices.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an image information processing scene as an example to which a scene change detection method of the present invention is applied.
FIG. 2 is a block diagram of a first embodiment of a scene change detection method and device according to the present invention.
FIG. 3 is a configuration diagram of a histogram generation circuit illustrated in FIG. 1;
FIG. 4 is a configuration diagram illustrating a connection relationship between a subdivision addition amount determination circuit illustrated in FIG. 3 and a level counter related thereto;
FIG. 5 is a diagram illustrating block division of an image.
FIG. 6 is a graph showing an example of creating a histogram.
FIG. 7 is a graph illustrating a method for obtaining a difference between histograms.
FIG. 8 is a graph showing a change in a histogram at the time of fading.
[Explanation of symbols]
100 ・ ・ Scene change detection device
101 Image data separation circuit
102 .. Image data block dividing circuit
103 Histogram generation circuit
201 ··· 1 block · total component circuit
202 ·· Histogram / level judgment circuit
203 ·· Subdivision addition amount determination circuit
203a... Subdivision addition amount determination unit
203b ··· Distribution processing unit
204 Level counter part
205 ・ ・ Standardized circuit
104 Histogram save register
105 Histogram difference calculation circuit
106 ··· Comparison judgment circuit
300 Scene change detection / processing information processing device
301 Video signal reproducing apparatus
... Pre-video signal processing device
303 ·· Encoding device
304 large-capacity recording device
305 ・ ・ Thumbnail image generation device
306-Host computer

Claims (12)

Histograms of the components that compose the image are created for two consecutive screen images of one field or one frame, and the sum of the differences is calculated. In a scene change detection method for detecting that a scene has changed,
When creating the histogram, a certain number is assigned to the corresponding level and adjacent levels on both sides thereof and added (overlapping),
After that, the result of the new histogram is calculated by normalization,
Detecting that the scene of the image of every two screens has changed using the calculated histogram.
Scene change detection method. The corresponding level of the histogram is further subdivided into m stages, the direction of selecting which side of the histogram is to be selected on both sides with respect to the level of the histogram according to the place to which the subdivision belongs, and how much is added to the selected histogram. To distribute the amount of addition for the calculation of the histogram by changing
The scene change detection method according to claim 1. The value to be added to the selected one is changed according to the content, property, or user's instruction of the image,
3. The scene change detection method according to claim 2. Histograms of the components that compose the image are created for two consecutive screen images of one field or one frame, and the sum of the differences is calculated. In a scene change detection device that detects that a scene has changed,
A histogram generating means for sorting and adding (overlapping) a certain number to the corresponding level and adjacent levels on both sides of the histogram when creating the histogram;
A histogram calculation unit that calculates a new histogram result by normalizing the histogram;
Scene change detecting means for detecting that a scene of an image on every two screens has changed using the result of the histogram calculating means.
Scene change detection device. A scene change detection method for detecting a scene change between images of a first screen and a second screen,
(A) determining the level of the component signal of the image of the first screen;
(B) For each of the determined levels, for the determined level and the level adjacent to the determined level, calculate a count value for each level at which a distribution ratio is calculated at a distribution ratio that specifies a distribution ratio;
(C) for the determined level and for the level adjacent to the determined level, count the count value distributed at the distribution ratio as a histogram,
(D) holding the histogram obtained above,
(E) The same processing as in (a) to (d) above is performed on the image of the second screen, and the difference between the obtained histogram and the stored normalized histogram of the image of the first screen is calculated. And
(F) calculating for each of the above levels and summing the histogram differences for all levels;
(G) determining that a scene change has occurred between the image of the first screen and the image of the second screen when the sum of the differences between the histograms is larger than a predetermined threshold value;
Scene change detection method. The distribution ratio can be defined according to the type and / or content of the image.
The scene change detection method according to claim 5. The adjacent range can vary depending on the type and / or content of the image,
The scene change detection method according to claim 5. The image of the first screen and the image of the second screen are images of one screen for one frame or one field that are continuously input,
The component signal of the image is a composite component signal or a component component signal of an image signal of a portion obtained by dividing the image of one screen into a plurality of blocks.
The scene change detection method according to claim 5. A scene change detection device that detects a scene change between images of a first screen and a second screen,
Histogram generation means;
Histogram storing means;
Histogram difference calculating means;
Comparison means,
The histogram generating means generates a histogram for image data of each component of each block obtained by dividing the image video signal of one screen into a plurality of blocks,
One block / component summing means for calculating the sum of the image data of each component in one block;
Level determining means for determining which of a plurality of levels the sum calculated by the one block / component summing means belongs to;
A subdivision addition amount determination unit having a plurality of subdivision addition amount determination units of the number of the levels determined,
A level counter unit having the same number of level counter units as the number of the plurality of subdivision addition amount determination units,
Each of the subdivision addition amount determination units of the subdivision addition amount determination unit is configured to calculate a first histogram count value for calculating a histogram corresponding to the level at a certain distribution ratio and a histogram of a level adjacent to the first histogram count value. And outputs the obtained first and second histogram count values to the first level counter section corresponding to the level and the second level counter section adjacent to the level. ,
The first and second level counters respectively accumulate the histogram count values input from the subdivision addition amount determination means to calculate a histogram,
The histogram storage unit stores the histogram generated by the histogram generation unit,
The histogram difference calculation means obtains a difference between the histogram of each level generated by the histogram generation means and the histogram of each level of the previous screen image stored in the histogram storage means, and sums the differences. Calculate,
The comparing and judging means judges that a scene change has occurred between the image of the first screen and the image of the second screen when the sum of the differences between the histograms is larger than a predetermined threshold value.
Scene change detection device. The distribution ratio can be defined according to the type and / or content of the image.
The scene change detection device according to claim 9. The range adjacent to the level counter unit to which the addition amount is input from the subdivision addition amount determination unit can be changed according to the type and / or content of the image.
The scene change detection device according to claim 9. The image of the first screen and the image of the second screen are images of one screen for one frame or one field that are continuously input,
The component signal of the image is a composite component signal or a component component signal of an image signal of a portion obtained by dividing the image of one screen into a plurality of blocks.
The scene change detection device according to claim 9.

Images