JP4607394B2 - Person detection system and person detection program - Google Patents

Description

この式（１）をテイラー展開し、Δｔを０に限りなく近づけると次式が得られる。
【数２】

ここで、（ｕ，ｖ）は見かけ速度ベクトルであり、Ｐｘ，Ｐｙ，Ｐｔはそれぞれｘ，ｙ，ｔに対する偏微分である。
【００３６】
もう一つの仮定は、「見かけ速度は滑らかに変化する」ことである。速度の滑らかさは、速度ベクトルのラプラシアンの和で評価され、次式のように表すことができる。
【数３】

しかし、実際の計算においては、（２）式のＰｘ，Ｐｙ，Ｐｔは有限差分で近似するため誤差が発生する。また、（３）式も離散化する必要があるので、実際の計算においては、速度の滑らかさを次式で表す。
【数４】

ここで、上部にバーを付したｕ，ｖは、近傍速度ベクトルの平均値であり、次式で表すことができる。
【数５】

【００３７】
さらに、滑らかさの重み係数α²を導入して、（４）式と（５）式を用いて次式のような誤差評価式が得られる。
【数６】

（８）式を用いることによって、誤差Ｅが最小値を取る条件で速度分布の計算式が得られる。そして、次式を反復して収束させることによって速度分布を求める。
【数７】

但し、ｋは反復回数とする。
【００３８】
ベクトル計算部１３では、このような方法で連続する２枚のＹ画像の計測空間範囲内の全画素についての推定速度ベクトル（ｕ，ｖ）を求める。
【００３９】
なお、オプティカルフロー推定は、移動物体の完全剛性体と、移動に伴う輝度不変性で理想解とするが、通常の移動物体はあまり成立しない。また、該人物検出は、歩行者（移動する人物）の実速度の正確さを追求するのではなく、速度場の発生分布から人物の存在を判断する目的のため、厳密に速度ベクトルの信頼性解釈を必要としない。
【００４０】
また、前方に歩行する人物の実態で発生する局所速度ベクトルは、頭部、手足、体を見ても複雑な動きとなる。更に、歩行中の手足の反動で生じる進行方向と明らかに異なる（反する）速度ベクトルも存在しており、小領域に偏在した速度ベクトルを削除する方式も可能ではあるが、移動人物全体で発生する局所（画素）速度ベクトルを全て精査すると、明らかに移動方向の速度ベクトル成分が圧倒的に多く存在することがわかる。
そこで、具体的には、オプティカルフロー推定を収束させるための反復計算の回数を３０〜４０回と少なくし、演算コストの低減を計り、動き成分のローパスフィルタ的効果となる滑らかさの重み係数を８０〜１５０とする。更に、オプティカルフロー推定の速度ベクトルで小さい成分、すなわち低速度成分は削除する。
【００４１】
次に、ベクトル計算部１３において求められた推定速度ベクトルは、分布状況判別部１４に入力され、分布状況判別部１４は、推定速度ベクトルを持つ画素の数を、ベクトルのＸ方向及びＹ方向の２象限の方向へそれぞれ計数することによりヒストグラム化し、前記速度ベクトルによって時系列連続動画像中の計測空間範囲に形成される速度場の分布状況を求める（ステップＳ４）。
【００４２】
図７は、分布状況判別部１４による速度場の分布状況の求め方を示した図であって、図７（ａ）に示すように、まずベクトル計算部１３において計測空間範囲の画素速度演算点の推定速度ベクトルが求められているとする。このとき、分布状況判別部１４は、図７（ｂ）に示すように計測空間範囲に設定された計測断面に対して、推定速度ベクトルが右運動と左運動のどちらに属するベクトルかを判別する。例えばここでは、推定速度ベクトルの角度がθ１からθ２の範囲に含まれる場合は右運動、θ３からθ４の範囲に含まれる場合は左運動とする。
このように推定ベクトルを右運動と左運動に分類したら、それぞれ右運動と左運動のベクトル毎に、そのベクトルを持つ画素の数を計数して速度場の分布状況を求める。例えば、図７（ａ）は右運動の推定速度ベクトルを持つ画素の数を、それぞれＸ方向及びＹ方向へ計数して、右運動Ｘ軸投射面と右運動Ｙ軸投射面にそれぞれヒストグラムを求めた様子を示す。
【００４３】
次に、移動人物検出部１５は、分布状況判別部１４により求められた速度場の分布状況が示す方向別の速度ベクトルを持つ画素の範囲に基づいて、人物を示すと思われる画像に対する外接四角形を推定し、時系列連続動画像中を移動する人物形状画像とその移動方向を検出する（ステップＳ４）。
図８（ａ）は、分布状況判別部１４により求められた速度場の分布状況を示す図であって、移動している人物１０１、１０２、１０３による画素の輝度値変化によって、左運動Ｘ軸投射面に幅Ｗａ（画素）のヒストグラム、左運動Ｙ軸投射面に幅Ｗｂ（画素）のヒストグラム、右運動Ｘ軸投射面に幅Ｗｃ（画素）のヒストグラム、右運動Ｙ軸投射面に幅Ｗｄ（画素）のヒストグラムが形成されている。
【００４４】
従って、図８（ｂ）に示すように、それぞれのヒストグラムの幅を延長した閉領域を求め、これが予め記憶している人物１人あたりの領域の大きさ以上であれば、これを計測空間範囲を移動する人物の外接四角形を示す人物形状画像１０４、１０５として検出する。また、人物形状画像１０４は右運動の速度ベクトルを持つ画素のヒストグラムから求められたので右方向へ移動する人物の画像であって、人物形状画像１０５は左運動の速度ベクトルを持つ画素のヒストグラムから求められたので左方向へ移動する人物の画像であることがわかる。
【００４５】
なお、上述したように、画質調整部１１と、撮影画像記憶部１２と、ベクトル計算部１３と、分布状況判別部１４と、移動人物検出部１５は、第１の実施形態の人物検出システムの場合は入力される１つのカメラによる時系列連続動画像に対してのみ処理を行えば良いが、第２の実施形態の人物検出システムの場合は入力される複数のカメラによる時系列連続動画像の数だけ、同じ処理を各画像毎に繰り返し行う。
【００４６】
移動人物検出部１５において人物形状画像が検出されたら、次に第２の実施の形態の人物検出システムのみ、重複人物排除部１９が、複数の時系列連続動画像毎に検出した人物形状画像の検出時刻と空間座標を比較して、重複する人物形状画像があるか否かを判定する（ステップＳ６）。
ステップＳ６において、重複する人物形状画像がある場合は（ステップＳ６のＹＥＳ）、重複する人物形状画像の内の１つを残して他はすべて排除する（ステップＳ７）。
ステップＳ６において、重複する人物形状画像がない場合は（ステップＳ６のＮＯ）、何もせずに次のステップＳ８へ進む。
また、第１の実施の形態の人物検出システムの場合は、ステップＳ５において人物形状画像が検出されたら、何もせずに次のステップＳ８へ進む。
【００４７】
ステップＳ８では、人物分離部１６が、計測空間範囲における人物形状画像の空間占有率を求め、空間占有率が１人以上の人物形状画像があるか否かを判定する（ステップＳ８）。
ステップＳ８において、空間占有率が１人以上の人物形状画像がある場合（ステップＳ８のＹＥＳ）、該空間占有率と予め記憶している人物１人あたりの空間占有率との比較から該人物形状画像に含まれる人数を推定する（ステップＳ９）。例えば図８（ｂ）に示す例では、人物形状画像１０４は空間占有率が大きく、人物１０１、１０２の２人が含まれていることを推定できる。
ステップＳ８において、空間占有率が１人以上の人物形状画像がない場合（ステップＳ８のＮＯ）、何もせずに次のステップＳ１０へ進む。
【００４８】
ステップＳ１０では、人数計数部１７が、計測空間範囲に設定された計測断面を通過する人物形状画像の数を計数して人数を求める（ステップＳ１０）。
図９は、人数計数部１７における計数の方法を説明する図であって、人数計数部１７は、人物１０６や人物１０７のように計測断面に交わる方向へ移動する人物形状画像が計測断面を通過する際に計数を行い、人物１０８のように計測断面と交わらない方向（計測断面と平行）に移動する人物形状画像については計数を行わない。
なお、人物分離部１６において、含まれる人物の人数が複数と推定されている人物形状画像が計数断面を通過する際は、その推定されている人数分だけ人数の計数を行う。
そして、人数計数部１７において計数された人数は、表示処理部１８を介して操作表示端末３または／及び表示装置４へ表示される。
【００４９】
なお、上述の第１、第２の実施の形態において、人物分離部１６が、計測空間範囲における人物形状画像の空間占有率を求め、空間占有率が１人以上の人物形状画像があるか否かを判定する場合に、撮像手段であるカメラ２の設置場所や装着されているレンズの焦点距離によって、予め記憶している人物１人あたりの空間占有率を補正を必要とする場合がある。何故ならば、撮像手段であるカメラ２の設置場所や装着されているレンズの焦点距離によって撮影される画角、すなわち撮影面積が決定する。このことは、移動人物１名あたりの基準画像面積が、カメラ２の設置場所や装着されているレンズの焦点距離によって撮影される画角によって変化することにつながり、従って、混雑状況下において空間占有率から人数を推定するために必要な比較値である人物１人あたりの空間占有率を、設置場所や画角により補正する必要が生じる。
【００５０】
そこで、人物分離部１６に、計測前に学習校正を行うモードを設け、１名の人物が計測断面を何度か繰り返して通過し、その際に検出された人物形状画像の面積を平均し、その値を人物１人あたりの空間占有率とすると共に、計測中は、１名の人物と判断した人物形状画像の面積を移動平均演算し、随時比較値とする人物１人あたりの空間占有率を学習、更新する学習更新手段を設けても良い。
このようにすることで、前述の撮像手段であるカメラ２の設置場所や装着されているレンズの焦点距離による人物１人あたりの空間占有率の補正の他、屋外における日照角度の推移による移動人物の影の面積変化による空間占有率の変動にも追随できるようになる。
【００５１】
（第３の実施の形態）
次に、本発明の第３の実施の形態について説明する。
本発明の第３の実施の形態は、第１、第２の人物検出システムに対して、更に人物の移動する方向毎に人数を計数する場合の構成例である。本発明の第３の実施の形態による人物検出システムの構成を示す概要図は、第１の実施の形態の概要図と第２の実施の形態の概要図のどちらでも良く、処理装置１または処理装置６が、以下に示す処理装置７に変更されれば良い。従って、概要図の説明はここでは省略する。
【００５２】
次に、処理装置７の構成について、図面を用いて詳細に説明する。図１０は、人物の移動する方向毎に人数を計数する処理装置７の細部構成を示すブロック図であって、ここでは第２の実施の形態のように複数のカメラにより撮影された時系列連続動画像から人数を計数する場合を一例として示す。
図１０において、処理装置７は、画質調整部１１と、撮影画像記憶部１２と、ベクトル計算部１３と、分布状況判別部１４と、移動人物検出部１５と、人物分離部１６と、人数計数部１７と、重複人物排除部１９と、移動方向分類部２０と、通過時刻測定部２１と、警報発生部２２とを備えている。その構成は、人数計数部１７において人数の計数が行われた人物形状画像の移動方向を、所定の角度毎に分類する移動方向分類部２０と、人物形状画像が計測空間範囲に設けられた計測断面を通過する際の通過時刻を測定する通過時刻測定部２１と、移動方向分類部２０において分類された人物形状画像の移動方向を判定して、所定の移動方向へ進む人物形状画像を発見した場合に、操作表示端末３または／及び表示装置４へ警報を発する警報発生部２２とを除いて、他は第２の実施の形態で説明した処理装置６と同一である。
【００５３】
次に、図１１に示すフローチャートを用いて、第３の実施の形態の人物検出システムの処理装置７の動作を説明する。
処理装置７の動作の中でステップＳ１からステップＳ１０までは、上述の第２の実施の形態の処理装置６と同一であるので、ここでは説明を省略する。
処理装置７では、ステップＳ１０において、人数計数部１７が、計測空間範囲に設定された計測断面を通過する人物形状画像の数を計数して人数を求めたら、通過時刻測定部２１が、計測空間範囲に設定された計測断面を人物形状画像が通過する時刻を測定する（ステップＳ１１）。
一方、移動方向分類部２０は、人数計数部１７が計数した人物形状画像の移動方向を、予め処理装置７に設定された所定の角度毎に分類する（ステップＳ１２）。
【００５４】
移動方向分類部２０において人物形状画像の移動方向の分類ができたら、警報発生部２２は、分類された人物形状画像の移動方向を判定して、所定の移動方向へ進む人物形状画像があるか否か、現在の時刻はその方向の移動を許可しているか否か等を判定し、警報を発する必要があるか否かを決定する（ステップＳ１３）。
ステップＳ１３において、警報を発する必要がある場合（ステップＳ１３のＹＥＳ）、警報発生部２２は、警報を発するとともに、警報の対象となった人物形状画像の人数または／及び画像を操作表示端末３または／及び表示装置４へ出力する（ステップＳ１４）。
【００５５】
第３の実施の形態の人物検出システムは、以上の構成において、例えば移動方向分類部２０における人物形状画像の移動方向の分類角度を、計数断面に対して０度から１８０度（例えば出入り口における入場を想定）と１８０度から３６０度（例えば出入り口における退場を想定）に設定し、どちらかの角度へ移動する人物形状画像を発見したときに警報を発するようにした場合、一方通行のみの出入り口や通路における人物の流れを監視して、流れに逆らう人物を発見することができるようになる。
また、大規模店舗や工場などの退出時間帯の従業員通路において、退出する従業員の流れは感知せずに、施設内に侵入しようとする人物のみを検知して警報を発するようにすることもできる。
【００５６】
なお、警報とともに警報の対象となった人数を報告する必要がなければ、人数計数部１７は具備しなくても良い。
また、人物検出対象の出入り口や通路において、時間によって通行の可否や通行を許可する方向を変更することがなければ、通過時刻測定部２１は具備しなくても良い。
更に、人数の計数結果に基づいて周囲に所定の情報を報知する情報報知手段を設けても良い。この場合、例えば人物検出対象の出入り口や通路毎に本実施の形態の人物検出システムを設置し、出入り口や通路毎の人数計測を行い、空いている出入り口や通路へ人を誘導することができるようになる。なお、空いている出入り口や通路への誘導は、出入り口や通路毎に設置される人物検出システムに予め出入り口や通路で一度に許容できる人数を設定し、計数された人数がそれ以下の場合には、この出入り口や通路は空いているという情報を周囲に報知しても良いし、すべての出入り口や通路に設置された人物計数システムの情報を集中管理して空いている出入り口や通路の情報を周囲へ報知しても良い。このような人物検出システムを利用すると、避難誘導等において非常口に対する効率的な人の誘導を行うことができる。
【００５７】
（第４の実施の形態）
次に、本発明の第４の実施の形態について説明する。
本発明の第４の実施の形態は、第１、第２の人物検出システムに対して、更に人物の移動する方向毎に人数を計数し、その方向や方向の先にある場所の人数分布を推定する場合の構成例である。本発明の第４の実施の形態による人物検出システムの構成を示す概要図は、第１の実施の形態の概要図と第２の実施の形態の概要図のどちらでも良く、処理装置１または処理装置６が、以下に示す処理装置８に変更されれば良い。従って、概要図の説明はここでは省略する。
【００５８】
次に、処理装置８の構成について、図面を用いて詳細に説明する。図１２は、人物の移動する方向毎に人数を計数し、その方向や方向の先にある場所の人数分布を推定する処理装置８の細部構成を示すブロック図であって、ここでは第２の実施の形態のように複数のカメラにより撮影された時系列連続動画像から人数を計数する場合を一例として示す。
図１２において、処理装置８は、画質調整部１１と、撮影画像記憶部１２と、ベクトル計算部１３と、分布状況判別部１４と、移動人物検出部１５と、人物分離部１６と、人数計数部１７と、重複人物排除部１９と、移動方向分類部２０と、通過時刻測定部２１と、人物分布推定部２３と、情報報知部２４とを備えている。その構成は、第３の実施の形態の処理装置７における警報発生部２２が、移動方向分類部２０によって分類された人物形状画像の移動方向と通過時刻測定部２１によって測定された人物形状画像の計測断面の通過時刻、更にはその人数とから該人物形状画像の挙動を推定し、所定の場所または方向に対する人数分布を求める人物分布推定部２３と、人数、移動方向、通過時刻、人数分布のいずれかの情報に基づいて、所定の情報を操作表示端末３または／及び表示装置４へ報知する情報報知部２４とに置き換わっているだけで、他は第３の実施の形態で説明した処理装置７と同一である。
【００５９】
次に、図１３に示すフローチャートを用いて、第４の実施の形態の人物検出システムの処理装置８の動作を説明する。
処理装置８の動作の中でステップＳ１からステップＳ１２までは、上述の第２、第３の実施の形態の処理装置６、７と同一であるので、ここでは説明を省略する。
処理装置８では、ステップＳ１２において、移動方向分類部２０が、計測空間範囲に設定された計測断面を通過する人物形状画像の移動方向を所定の角度毎に分類したら、人物分布推定部２３が、人数と通過時刻、及び移動方向から該人物形状画像の挙動を推定し、所定の場所または方向に対する人数分布を求める（ステップＳ１５）。
【００６０】
図１４は、人物分布推定部２３の動作を説明するための、人物検出システムを利用する応用例を示す図である。図１４（ａ）に示すように、例えば店舗の出入り口に本実施の形態の人物検出システムを設置し、移動方向分類部２０における人物形状画像の移動方向の分類角度を、計数断面に対して直進方向（直角方向）左手方向、右手方向の３種類に分類したときに、人物分布推定部２３は、図１４（ｂ）に示すように、各分類された方向毎の人数を集計して人数分布を作成する。これにより、例えば図１４（ａ）に示すように店舗の各売場Ａ、Ｂ、Ｃ毎の集客状況を把握することができるようになる。また、通過時刻測定部２１によって測定された人物形状画像の計測断面の通過時刻を併用することで、時間毎の人数分布を求めることもできる。従って、店舗の売場の集客情報等、比較的長い時間における人物分布の他に、避難誘導における避難状況（非常口に対する人の集中具合など）等、短時間における人物分布を求めることで、例えば非常口に対する避難誘導を効率的に行うことができるようになる。
【００６１】
そして、人物分布推定部２３により求められた人物分布に従って、情報報知部２４は、所定の情報を操作表示端末３または／及び表示装置４を介して報知する（ステップＳ１６）。なお、ここで報知する内容は、例えば店舗の各売場の集客情報であったり、集客情報に基づいて利用客を誘導するアナウンスであったり、避難誘導における警告や誘導、指示等の情報であり、それぞれ人物検出システムの利用方法によって異なる。
【００６２】
また、上述の第４の実施の形態の人物検出システムに、例えば横断歩道の歩行者のための交通制御用信号を接続し、処理装置８に人物分布推定部２３の推定する道路上の人物分布に基づいて、交通制御用信号を制御する交通制御手段を設けても良い。この場合、交通制御手段は、例えば横断歩道を渡ろうとする、あるいは渡っている歩行者の人物分布に従って、例えば歩行者が多い場合は信号の「青：横断可」の時間を長く、歩行者が少ない場合は信号の「青：横断可」の時間を短くするように制御する。
【００６３】
また、上述の第３の実施の形態の警報発生部２２と、第４の実施の形態の人物分布推定部２３とを両方備え、この２つの機能を例えば時間で切り替えて作動させる切り替え手段を設け、昼は店舗の売場毎の集客情報を収集し、夜は店舗への侵入者に対する侵入センサとして利用するようにしても良い。
【００６４】
（第５の実施の形態）
次に、本発明の第５の実施の形態を説明する。本発明の第５の実施の形態は、通行を許可された人物のみが出入りするために、条件が満たされた場合に出入り口の鍵を開錠する入退場制御装置であって、例えば非接触ＩＤカードによるＩＤ番号の照合などの所定の動作と人物の移動方向によって、その人物の入退場の可否を判断する入退場制御装置の構成例である。
上述の第４の実施の形態では、図１４（ａ）に示すように、計測空間範囲を（店舗）出入り口の内側に設定する場合を説明したが、第５の実施の形態では、計測空間範囲を出入り口の外側に設定し、図１２に示す処理装置８において、人物分布推定部２３の代わりに、例えば非接触ＩＤカードによるＩＤ番号の照合などの所定の動作と人物の移動方向によって、その人物の入退場の可否を判断する入退場制御部（図示せず）を設ける。
【００６５】
第５の実施の形態の人物検出システムは、以上の構成において、例えば移動方向分類部２０における人物形状画像の移動方向の分類角度を、図９に示したように、人物１０６や人物１０７のように計測断面に交わる方向と、人物１０８のように計測断面と交わらない方向（計測断面と平行）に移動する方向とに分類して設定する。
この場合、入退場制御部は、計測断面に交わる方向へ進む人物形状画像を検出し、かつ所定の時間内に非接触ＩＤカードによるＩＤ番号の照合などの所定の動作が行われた場合は、その人物の入退場を許可し出入り口の鍵を開錠する。また、計測断面に交わらない方向へ進む人物形状画像を検出し、かつ所定の時間内に非接触ＩＤカードによるＩＤ番号の照合などの所定の動作が行われた場合は、その人物に出入り口を通行する意志は無いもの（入退場制御装置の前を横切っただけ）として、出入り口の鍵は開錠しない。
なお、情報報知部２４は、人物の出入りを監視して、出入りした人数などの情報を操作表示端末３へ報告する。また、報告の必要がなければ、情報報知部２４は設けなくても良い。
【００６６】
従って、例えば非接触方式のＩＤカードを所持した人物が出入り口の前を横切っただけでＩＤ照合が実行され、入退場制御装置が入退場を許可する（入退場制御装置が出入り口の鍵を開錠してしまう）というような誤動作を防止することができるという効果が得られる。
【００６７】
（第６の実施の形態）
次に、本発明の第６の実施の形態を説明する。本発明の第６の実施の形態は、本発明の第３の実施の形態が、第１、第２の人物検出システムに対して、更に人物の移動する方向毎に人数を計数し、この結果を元に警報発生部２２から警報を発生させていたのに対し、人物の移動する方向毎に計数された人数を元に、周囲に対する警告や誘導、指示等の所定の情報を報知する場合の構成例である。
上述の第３の実施の形態では、図１０に示すように、警報発生部２２を備えた処理装置７を説明したが、第６の実施の形態では、処理装置７において、警報発生部２２の代わりに、第４の実施の形態で説明した人数、移動方向、通過時刻、のいずれかの情報に基づいて（第６の実施の形態では人数分布は用いない）、所定の情報を操作表示端末３または／及び表示装置４へ報知する情報報知部２４を設ける。
【００６８】
第６の実施の形態の人物検出システムは、以上の構成において、例えば大規模店舗の入り口において、移動方向分類部２０によって人物の進行方向を判断することで、入店する客と退店する客とをそれぞれ検知し、情報報知部２４はそれに応じた情報を表示装置４へ出力することで、表示装置４は入店する客には「いらっしゃいませ。」、退店する客には「ありがとうございました。」などのアナウンスを出力することができる。
更に、入店後の客に対しては、客の進行方向をそれぞれ検知して、進行方向に応じた情報を表示装置４へ出力することで、表示装置４は「そちらは被服売場です。」「そちらは玩具売場です。」等々、進行方向の先にある売場アナウンスを出力することもできる。
また、通過時刻測定部２１によって通行者の通過時刻を判断して、所定の情報を表示装置４へ出力することで、「おはようございます、いってらっしゃいませ。」「こんばんは。」等、時間の変化に応じたアナウンスを出力することができる。
【００６９】
なお、第３の実施の形態と同様に、情報報知とともに報知の対象となった人数を報告する必要がなければ、人数計数部１７は具備しなくても良い。また、人物検出対象の出入り口や通路において、時間によって報知する情報の内容を変更することがなければ、通過時刻測定部２１は具備しなくても良い。
【００７０】
なお、上述の第１から第６までの実施の形態では、処理装置１、６、７、８と操作表示端末３は直接接続されているように説明したが、例えば処理装置１を例にして図１５を用いて説明するように、処理装置１、６、７、８に通信装置５０を接続して、通信網５１を介して計数した人数、警報やその内容、更には報知する情報を操作表示端末３と送受信するようにしても良い。ここで、通信網５１は、例えばＷＡＰ（Wireless Application Protocol ）による無線通信、ＷＷＷ（World Wide Web）を利用したインターネット、あるいはＰＳＴＮ（Public Switch Telephone Network）やＩＳＤＮ（Integrated Services Digital Network）等の公衆回線網を介した有線通信により情報の送受信を行う通信網を含むものとする。また、１台の操作表示端末３に対して処理装置を複数台接続し、複数の処理装置を１台の操作表示端末３により制御するようにしても良い。
【００７１】
なお、上述の第１から第６の実施の形態における処理装置１、６、７、８の機能を実現するためのプログラムをコンピュータ読み取り可能な記録媒体に記録して、この記録媒体に記録されたプログラムをコンピュータシステムに読み込ませ、実行することにより人物検出を行ってもよい。なお、ここでいう「コンピュータシステム」とは、ＯＳや周辺機器等のハードウェアを含むものとする。また、「コンピュータシステム」は、ＷＷＷシステムを利用している場合であれば、ホームページ提供環境（あるいは表示環境）も含むものとする。また、「コンピュータ読み取り可能な記録媒体」とは、フレキシブルディスク、光磁気ディスク、ＲＯＭ、ＣＤ−ＲＯＭ等の可搬媒体、コンピュータシステムに内蔵されるハードディスク等の記憶装置のことをいう。さらに「コンピュータ読み取り可能な記録媒体」とは、インターネット等のネットワークや電話回線等の通信回線を介してプログラムが送信された場合のサーバやクライアントとなるコンピュータシステム内部の揮発性メモリ（ＲＡＭ）のように、一定時間プログラムを保持しているものも含むものとする。
【００７２】
また、上記プログラムは、このプログラムを記憶装置等に格納したコンピュータシステムから、伝送媒体を介して、あるいは、伝送媒体中の伝送波により他のコンピュータシステムに伝送されてもよい。ここで、プログラムを伝送する「伝送媒体」は、インターネット等のネットワーク（通信網）や電話回線等の通信回線（通信線）のように情報を伝送する機能を有する媒体のことをいう。また、上記プログラムは、前述した機能の一部を実現するためのものであっても良い。さらに、前述した機能をコンピュータシステムにすでに記録されているプログラムとの組み合わせで実現できるもの、いわゆる差分ファイル（差分プログラム）であっても良い。
【００７３】
【発明の効果】
以上の如く、請求項１に記載の発明によれば、撮像手段により撮影された時系列連続動画像の計測空間範囲を物体が移動することによって発生する画素の速度ベクトルを求め、方向及び速度において所定の範囲の速度ベクトルを持つ画素の存在個数を、該速度ベクトルが示す所定の方向毎にヒストグラム化して速度場の分布状況とする。そして、この速度場の分布状況が示す方向別の速度ベクトルを持つ画素の範囲から、この方向へ移動する人物と推定できる画像範囲を人物形状画像として識別する。
従って、ヒストグラムによって表される画素の範囲から、計測空間範囲内の人物の存在範囲を移動方向別に正確に推定し、人物の識別精度を向上することができるという効果が得られる。
【００７４】
請求項２に記載の発明によれば、複数の撮像手段により撮影された各時系列連続動画像毎に、人物と推定できる画像範囲を人物形状画像として識別する。そして、各時系列連続動画像毎に識別された人物形状画像の識別時刻と空間座標を比較して、同じ識別時刻と空間座標を持つ人物形状画像を重複している人物形状画像と判断し、１つを残して他は排除する。
従って、複数の撮像手段を用いることにより、１つの撮像手段では得られなかったシームレスで広範囲な計測空間範囲（パノラマ画像）を得ることができるという効果が得られる。
【００７５】
請求項３に記載の発明によれば、速度場の分布状況が示す方向別の速度ベクトルを持つ画素の範囲の大きさの違いによって、検出された人物形状画像の空間占有率を判別する。そして、該空間占有率と予め見積もっている１人あたりの空間占有率との比較から、該人物形状画像に含まれる人数を推定する。
従って、従来、通行者によって混み合う交通状況下や通行者が交錯する交通状況下において、複数の人物が重なって移動していても、一人の人物が移動していると判断してしまうという問題を回避し、計測空間範囲を移動する人数の推定精度を向上することができるという効果が得られる。
【００７６】
請求項４に記載の発明によれば、撮像手段の撮影する画像の画角や撮影される人物の見え方が変化したとしても、いつでも人物形状画像の空間占有率の比較対象とする人物１名あたりに必要な空間占有率を正確に保持していることができる。
従って、撮像手段であるカメラ２の設置場所や装着されているレンズの焦点距離による人物１人あたりの空間占有率の補正の他、屋外における日照角度の推移による移動人物の影の面積変化による空間占有率の変動にも追随できるようになる。
請求項５に記載の発明によれば、計測空間範囲内に所定の計測断面を設定し、この計測断面を通過する人物形状画像の数を計数することで、所定の場所を通過した人物の数を求める。
従って、例えば建物等に出入りする人物を検出して、その人数の計測を正確に行うことができるという効果が得られる。
【００７７】
請求項６に記載の発明によれば、通信網に接続された操作表示手段に対して計数した人数を出力する。
従って、例えば遠方の建物等に出入りする人物を検出して、その人数を手元の操作表示手段において正確に把握することができるという効果が得られる。
請求項７に記載の発明によれば、識別された人物形状画像がどの方向へ移動しているかを分類し、所定の方向へ進む人物が存在する場合、警報を発する。
従って、例えば入り口専用や出口専用等、一方通行の通路を逆行して進もうとする人物や進入禁止場所への侵入者等を発見した場合に、これを検出して知らせることができるという効果が得られる。
【００７８】
請求項８に記載の発明によれば、識別された人物形状画像がどの方向へ移動しているかを分類し、所定の方向へ進む人物が存在する場合、その時間帯によって人物の進む方向が正しいか否かを判断し、正しくない場合に警報を発する。
従って、例えば時間別に通行方向の指定が変化する一方通行の通路を、指定に逆らって逆行して進もうとする人物等を発見した場合に、これを検出して知らせることができるという効果が得られる。
請求項９に記載の発明によれば、識別された人物形状画像がどの方向へ、いつ、何人移動したかを分類し、人物形状画像の挙動を推定して所定の場所または方向に対する人数分布を求める。
従って、例えば店舗における売り場毎の集客状況を集計したり、避難誘導において避難状況を把握したりすることができるという効果が得られる。
【００７９】
請求項１０に記載の発明によれば、識別された人物形状画像がどの方向へ、いつ、何人移動したかを分類し、人物形状画像の挙動を推定して所定の場所または方向に対する人数分布を求める機能と、進入禁止場所への侵入者を発見した場合に、これを検出して知らせる機能とを切り替えて利用する。
従って、例えば店舗の営業時間中は売り場の集客状況を集計し、店舗の営業時間外は店舗への侵入者を検出する侵入者センサとして利用することができるという効果が得られる。
請求項１１に記載の発明によれば、警報を発するとともに、所定の移動方向へ進む人物形状画像の人数または／及び画像を出力する。
従って、警報とともに、その警報の原因となった状況も同時に知らせることができるという効果が得られる。
【００８０】
請求項１２に記載の発明によれば、求められた人数、移動方向、通過時刻、人数分布のいずれかの情報に基づいて、周囲に対する警告や誘導、指示等の所定の情報を報知する。
従って、例えば避難誘導などにおいて、出入り口に殺到する人数を正確に把握し、適切な指示を発することができるという効果が得られる。
請求項１３に記載の発明によれば、求められた人数分布に基づいて、交通制御用信号を制御する。
従って、例えば交差点を横断する人数を正確に把握し、交通制御信号の横断許可時間や横断許可回数を制御して、円滑な交通制御を行うことができるという効果が得られる。
【００８１】
請求項１４に記載の発明によれば、例えば非接触ＩＤカードによるＩＤ番号の照合などの所定の動作と人物の移動方向によって、その人物の入退場の可否を判断する。
従って、例えば非接触方式のＩＤカードを利用してＩＤ番号を照合することで入退場の可否を判断する入退場制御装置の前を、ＩＤカードを所持した人物が横切っただけでＩＤ照合が実行され、入退場制御装置が入退場を許可するような誤動作を防止することができるという効果が得られる。
【００８２】
請求項１５に記載の発明によれば、識別された人物形状画像がどの方向へ移動しているかを分類し、所定の方向へ進む人物が存在する場合、周囲に対する警告や誘導、指示等の所定の情報を報知する。
従って、例えば通路の人物の移動方向に基づいて、適切な警告や誘導、指示等の所定の情報報知を行い、通路の人物の動作を制御することができるようなるという効果が得られる。
【図面の簡単な説明】
【図１】 本発明の第１の実施の形態の人物検出システムの構成を示す概要図である。
【図２】 同実施の形態の人物検出システムの処理装置の細部構成を示すブロック図である。
【図３】 本発明の第２の実施の形態の人物検出システムの構成を示す概要図である。
【図４】 同実施の形態の人物検出システムの非同期映像合成装置の出力信号を説明する図である。
【図５】 同実施の形態の人物検出システムの処理装置の細部構成を示すブロック図である。
【図６】 第１及び第２の実施の形態の人物検出システムの処理装置の動作を示すフローチャートである。
【図７】 処理装置１または処理装置６の分布状況判別部の動作を説明するための図である。
【図８】 処理装置１または処理装置６の移動人物検出部の動作を説明するための図である。
【図９】 処理装置１または処理装置６の人数計数部の動作を説明するための図である。
【図１０】 本発明の第３の実施の形態の人物検出システムの処理装置の細部構成を示すブロック図である。
【図１１】 同実施の形態の人物検出システムの処理装置の動作を示すフローチャートである。
【図１２】 本発明の第４の実施の形態の人物検出システムの処理装置の細部構成を示すブロック図である。
【図１３】 同実施の形態の人物検出システムの処理装置の動作を示すフローチャートである。
【図１４】 同実施の形態の人物検出システムを利用する応用例を示す図である。
【図１５】 通信網を介して本発明の人物検出システムの遠隔操作を行う場合の構成を示す概要図である。
【符号の説明】
１ 処理装置
２、２ａ、２ｂ、２ｃ、２ｄ カメラ
３ 操作表示端末
４ 表示装置
５ 非同期映像合成装置
６ 処理装置
７ 処理装置
８ 処理装置
１１ 画質調整部
１２ 撮影画像記憶部
１３ ベクトル計算部
１４ 分布状況判別部
１５ 移動人物検出部
１６ 人物分離部
１７ 人数計数部
１８ 表示処理部
１９ 重複人物排除部
２０ 移動方向分類部
２１ 通過時刻測定部
２２ 警報発生部
２３ 人物分布推定部
２４ 情報報知部
５０ 通信装置
５１ 通信網[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a person detection system and a person detection program for detecting a moving direction of a person from an image taken by a camera.
[0002]
[Prior art]
Conventionally, it is very important to detect people entering and exiting a building and measure the number of people, etc., for marketing research, building management, and analysis of trends at customer collection facilities. It is currently being executed by.
Therefore, in order to count a person without human intervention, there are some proposals for detecting and counting a person from an image taken by a camera.
In these technologies, at the basic information detection stage of image processing, the movement of the person is detected by detecting the movement of the person by using the time or background difference result, or by measuring the change in the distance to the person obtained from the parallax information. To detect.
[0003]
[Problems to be solved by the invention]
However, although the above-mentioned technique can detect a person under limited conditions, a plurality of persons move in a crowded traffic situation or a traffic situation where a passerby crosses. However, there is a problem that it is impossible to perform accurate counting because the detection accuracy of the person does not improve, such as determining that one person is moving.
[0004]
The present invention has been made in view of the above problems, and a person detection system and a person detection program that can detect a person with high accuracy by differentiating information detection in an image processing stage and estimate the moving direction thereof. The purpose is to provide.
[0005]
[Means for Solving the Problems]
In order to solve the above-described problems, the present invention is a human detection system, in which a plurality of imaging units (for example, the cameras 2a and 2b according to the embodiments) that respectively capture different ranges while continuing a part of the range to be captured. 2c, 2d), from the change of the luminance value of the pixel of the time-series continuous moving image generated by the movement of the object in the time-series continuous moving image, the optical flow estimation operation is used to calculate the pixel of the pixel. Vector calculation means for obtaining a velocity vector for each time-series continuous moving image (for example, the vector calculation unit 13 in the embodiment) and the number of pixels having the velocity vector are counted for each predetermined direction indicated by the velocity vector. The distribution for determining the distribution of the velocity field for each time-series continuous moving image formed in the measurement space range in each time-series continuous moving image by the velocity vector A person who is a pixel region identified based on a speed vector from a range of pixels having a speed vector for each direction indicated by the speed field distribution status (for example, the distribution status determination unit 14 of the embodiment) A person shape image that is a shape image and moves in the time-series continuous moving image, a moving person detection means for detecting the moving direction (for example, the moving person detection unit 15 of the embodiment), and a detection time of the person shape image And a spatial coordinate, and a duplicate person exclusion means (for example, a duplicate person exclusion unit 19 in the embodiment) that eliminates a person shape image that is detected redundantly between the time-series continuous moving images. And With the above configuration, an image range that can be estimated as a person is identified as a person shape image for each time-series continuous moving image captured by a plurality of imaging means. Then, the detection time of the person shape image identified for each time series continuous moving image and the spatial coordinates are compared, and the person shape images having the same detection time and space coordinates are determined as overlapping person shape images, Leave one and exclude the other. Thereby, a panoramic image is generated by the plurality of imaging means.
[0006]
According to the present invention, in the above-described human detection system, the distribution state determination unit counts the number of pixels having the velocity vector in the X direction and the Y direction of the velocity vector, respectively, thereby determining the histogram in the X direction and the Y direction. The moving person detecting means obtains a pixel area of a closed area determined based on the widths of the histogram in the X direction and the histogram in the Y direction as a person shape image.
[0007]
  BookThe inventionAboveIn the person detection system, a person separation unit (for example, the person separation unit 16 of the embodiment) that obtains the space occupancy rate of the person shape image in the measurement space range and estimates the number of persons included in the person shape image from the space occupancy rate. ). With the above configuration, when the detected person shape image is actually a result of detecting an overlap of a plurality of persons, the space area occupied by the person shape image in the measurement space range is one person. Since it becomes larger than the case, the space occupancy rate of the detected human shape image is determined based on the difference in the size of the range of the pixels having the velocity vector for each direction indicated by the distribution state of the velocity field. Then, the number of persons included in the person shape image is estimated from a comparison between the space occupancy and the space occupancy per person estimated in advance.
[0008]
  BookThe inventionAboveIn the person detection system, the person separating unit compares the space occupancy of the person shape image when estimating the number of persons included in the person shape image from the space occupancy of the person shape image in the measurement space range. And learning update means for learning and updating a necessary space occupancy per person. With the above configuration, even if the angle of view of the image captured by the imaging unit or the appearance of the captured person changes, the space occupancy required for each person to be compared with the space occupancy rate of the human shape image at any time The rate can be held accurately.
[0009]
  BookThe inventionAboveThe person detection system includes a person counting means (for example, a person counting unit 17 in the embodiment) that calculates the number of persons by counting the number of the person shape images passing through a predetermined measurement section set in the measurement space range. It is characterized by that. With the above configuration, a predetermined measurement section is set in the measurement space range, and the number of persons passing through the predetermined place is obtained by counting the number of person shape images passing through the measurement section.
[0010]
  BookThe inventionAboveCommunication means (for example, the communication device 50 according to the embodiment) for transmitting / receiving the number of persons via a communication network to / from a remote operation display means (for example, the operation display terminal 3 according to the embodiment). ). With the above configuration, the counted number of persons is output to the operation display means connected to the communication network.
[0011]
  BookThe inventionAboveIn the human detection system, a movement direction classification unit (for example, a movement direction classification unit 20 in the embodiment) that classifies the movement direction of the person shape image, and a movement direction of the person shape image classified by the movement direction classification unit. An alarm means (for example, alarm generation unit 22 in the embodiment) that issues an alarm when a person shape image that is determined and proceeds in a predetermined movement direction is found is provided. With the above configuration, the direction in which the identified person shape image is moving is classified, and if there is a person who proceeds in a predetermined direction, an alarm is issued.
[0012]
  BookThe inventionAboveIn the human detection system of the present invention, a passage time measuring unit (for example, a passage time measuring unit 21 in the embodiment) for measuring the passage time of the measurement cross section of the person shape image is provided, and the alarm unit is configured to be a predetermined time at a predetermined time. A warning is issued when a person shape image that moves in the moving direction is found. With the above configuration, the direction in which the identified person shape image is moving is classified, and when there is a person who moves in a predetermined direction, it is determined whether the person's moving direction is correct according to the time zone. Raise an alarm if not correct.
[0013]
  BookThe inventionAboveIn the human detection system, the moving direction classifying unit (for example, the moving direction classifying unit 20 of the embodiment) for classifying the moving direction of the person shape image, and the passing time for measuring the passing time of the person shape image through the measurement cross section. A person distribution for obtaining a person distribution for a predetermined place or direction by estimating the behavior of the person shape image from the measuring means (for example, the passage time measuring unit 21 of the embodiment), the number of persons, the passage time, and the moving direction. An estimation unit (for example, the person distribution estimation unit 23 of the embodiment) is provided. With the above configuration, it is classified in which direction, when and how many people the identified person shape image has moved, the behavior of the person shape image is estimated, and the number distribution for a predetermined place or direction is obtained.
[0014]
  BookThe inventionAboveIn the human detection system, the moving direction classifying unit (for example, the moving direction classifying unit 20 of the embodiment) for classifying the moving direction of the person shape image, and the passing time for measuring the passing time of the person shape image through the measurement cross section. A person distribution for obtaining a person distribution for a predetermined place or direction by estimating the behavior of the person shape image from the measuring means (for example, the passage time measuring unit 21 of the embodiment), the number of persons, the passage time, and the moving direction. When the movement direction of the person shape image classified by the estimation means (for example, the person distribution estimation unit 23 of the embodiment) and the movement direction classification means is determined, and a person shape image moving in a predetermined movement direction is found. Alarm means for issuing an alarm (for example, the alarm generation unit 22 of the embodiment), and switching means for switching and operating the person distribution estimating means and the alarm means Characterized in that it obtain. With the above configuration, it is possible to classify in which direction, when, and how many people the identified person shape image has moved, to estimate the behavior of the person shape image, and to determine the number distribution for a given location or direction, and to prohibit entry When an intruder is detected, the function of detecting and informing the intruder is switched and used.
[0015]
  BookThe inventionAboveIn the person detection system, the alarm means outputs an alarm and outputs the number of persons and / or images of the person-shaped image that is the object of the alarm. With the above-described configuration, an alarm is issued and the number of people shape images and / or images of a person shape image traveling in a predetermined movement direction is output.
[0016]
  BookThe inventionAboveIn the person detection system, information notifying means for notifying predetermined information based on any one or more of the number of people, the moving direction, the passing time, and the number of people distribution (for example, the information notifying unit of the embodiment) 24). With the above configuration, predetermined information such as a warning, guidance, or instruction to the surroundings is notified based on any one or more of the obtained number of people, moving direction, passage time, and number distribution.
[0017]
  BookThe inventionAboveThe person detection system according to claim 1, further comprising traffic control means for controlling a traffic control signal based on the number distribution. With the above configuration, the traffic control signal is controlled based on the obtained distribution of people.
[0018]
  BookThe inventionAboveIn the human detection system, the movement direction classification means for classifying the movement direction of the person shape image, a predetermined operation is executed, and the movement direction of the person shape image determined by the movement direction classification means is a predetermined direction. If there is, an entrance / exit control means (for example, an entrance / exit control unit in the embodiment) that permits entry / exit of the person is provided. With the above configuration, for example, whether or not a person can enter and exit is determined based on a predetermined operation such as ID number verification using a non-contact ID card and the movement direction of the person.
[0019]
  BookThe inventionAboveIn the human detection system, a movement direction classification unit (for example, a movement direction classification unit 20 in the embodiment) that classifies the movement direction of the person shape image, and a movement direction of the person shape image classified by the movement direction classification unit. It is characterized by comprising information notifying means (for example, the information notifying unit 24 of the embodiment) for notifying predetermined information when a person shape image that is determined and proceeds in a predetermined moving direction is found. With the above configuration, the direction in which the identified person shape image is moving is classified. When there is a person who moves in a predetermined direction, predetermined information such as a warning, guidance, or instruction to the surroundings is notified.
[0021]
  BookThe present invention provides optical flow estimation based on a change in luminance value of a pixel of the time-series continuous moving image generated by the movement of an object in a time-series continuous moving image captured by a plurality of imaging units that capture different ranges. A vector calculation process for calculating the velocity vector of the pixel for each time-series continuous moving image using an operation, and counting the number of pixels having the velocity vector for each predetermined direction indicated by the velocity vector, A distribution state determination process for obtaining a distribution state of a velocity field for each time-series continuous moving image formed in a measurement space range in each time-series continuous moving image by a velocity vector, and a direction indicated by the distribution state of the velocity field Range of pixels with different velocity vectorsA person shape image that is a pixel region identified based on the velocity vector,A person shape image that moves in the time series continuous moving image and a moving person detection process that detects the moving direction of the person shape image are compared between the detection time and spatial coordinates of the person shape image, It is a person detection program for causing a computer to execute duplicate person elimination processing for eliminating duplicated person shape images.
[0022]
  BookAccording to the invention, in the above-described person detection program, the computer executes a person separation process for obtaining a space occupancy rate of the person shape image in the measurement space range and estimating the number of persons included in the person shape image from the space occupancy rate. It is characterized by that.
[0023]
  BookThe present invention is the person detection program described above, wherein the person separation process is performed when the number of persons included in the person shape image is estimated from the space occupation ratio of the person shape image in the measurement space range. A learning update process for learning and updating a necessary space occupancy per person to be compared for occupancy is executed by a computer.
[0024]
  BookThe invention is characterized in that, in the above-described person detection program, the computer executes a person counting process for calculating the number of persons by counting the number of the person shape images passing through the predetermined measurement section set in the measurement space range. And
[0025]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a schematic diagram showing the configuration of the person detection system according to the first embodiment of the present invention. The first embodiment of the present invention is a basic configuration example in which a person is detected and counted from an image photographed by one imaging means.
In FIG. 1, the human detection system according to the present embodiment supplies a processing device 1 that performs human detection processing from an image captured by an imaging unit, and supplies time-series continuous moving images in a measurement space range to the processing device 1 as an imaging unit. The operation display terminal 3 for operating the processing device 1 and displaying the alarm and information output from the processing device 1 and the display device 4 for displaying the alarm and information output from the processing device 1 Has been.
[0026]
Here, the camera 2 is a dome-type camera that can be attached to the ceiling in order to photograph a measurement space in which a person is coming and going from above and to determine the distribution of velocity field vectors in a planar space. The size of the captured image captured by the camera 2 is, for example, an 8-bit (256 gradation) RGB image composed of 640 pixels (lines) in the horizontal direction and 480 pixels (dots) in the vertical direction. Details of how to obtain the distribution state of the velocity field vector in the plane space will be described later.
In addition, the operation display terminal 3 displays an interface with the processing device 1, an input operation unit that can input operation commands and setting data to the processing device 1, and alarms and information output from the processing device 1 by sound and images. Any terminal may be used as long as the terminal is provided with a display means.
Furthermore, the display device 4 displays alarms and information output from the processing device 1 with sounds and images. However, the display device 4 is not necessarily provided.
[0027]
Next, the configuration of the processing apparatus 1 will be described in detail with reference to the drawings. FIG. 2 is a block diagram illustrating a detailed configuration of the processing device 1 that performs person detection processing from time-series continuous moving images captured by the camera 2.
In FIG. 2, the processing device 1 includes an image quality adjustment unit 11 that adjusts image quality such as luminance and contrast of a time-series continuous moving image captured by the camera 2, a captured image storage unit 12 that temporarily stores an image, A vector for obtaining a velocity vector of a pixel using an optical flow estimation calculation from a change in luminance value of a pixel of a time-series continuous moving image generated by moving an object in a time-series continuous moving image captured by the camera 2 And a calculator 13.
[0028]
In addition, by counting the number of pixels having the obtained velocity vector for each predetermined direction indicated by the velocity vector, the velocity field formed in the measurement space range in the time-series continuous moving image by the velocity vector. Based on a distribution status determination unit 14 for obtaining a distribution status and a range of pixels having velocity vectors for each direction indicated by the distribution status of the velocity field, a human shape image moving in a time-series continuous moving image and its moving direction are detected. A moving person detection unit 15 that determines the space occupancy rate of the person shape image in the measurement space range, and estimates the number of persons included in the person shape image from the space occupancy rate, and is set within the measurement space range The number of person shape images passing through the predetermined measurement cross section is obtained to obtain the number of people, and the number of people counted in the number of people counting unit 17 is displayed on the operation display terminal 3 and / or the table. And a display processing unit 18 to be displayed on the device 4.
[0029]
The above is the configuration of the person detection system according to the first embodiment of the present invention that detects and counts a person from an image photographed by one imaging unit. The person detection system according to the first embodiment The detailed operation will be described later together with the following description of the operation of the human detection system according to the second embodiment of the present invention.
[0030]
(Second Embodiment)
Next, a second embodiment of the present invention will be described.
FIG. 3 is a schematic diagram showing a configuration of a person detection system according to the second embodiment of the present invention. The second embodiment of the present invention is configured to expand a measurement cross section with images captured by a plurality of imaging units that capture different ranges, and detect and count a person from a wide measurement space range (panoramic image). It is an example.
In FIG. 3, the human detection system according to the present embodiment includes a plurality of (four in the present embodiment) cameras 2a, 2b, 2c, and 2d that shoot different ranges while overlapping each other's angle of view. An asynchronous video synthesizing device 5 that synthesizes time-series continuous moving images taken by 2a, 2b, 2c, and 2d into one image, a processing device 6 that performs person detection processing from images output by the asynchronous video synthesizing device 5, and a first The operation display terminal 3 and the display device 4 are the same as those in the first embodiment.
As shown in FIG. 4, the asynchronous video synthesizing device 5, for example, in order to efficiently transmit the time-series continuous moving images captured by the four cameras 2 a, 2 b, 2 c, and 2 d to the processing device 6, for example, Four RGB 8-bit (256 gradations) images composed of 640 pixels (lines) in the horizontal direction and 480 pixels (dots) in the vertical direction are 640 pixels (lines) in the horizontal direction and 480 pixels (in the vertical direction). Dot) to compose one RGB image.
[0031]
Next, the configuration of the processing device 6 will be described in detail with reference to the drawings. FIG. 5 is a block diagram illustrating a detailed configuration of the processing device 6 that performs person detection processing from a plurality of time-series continuous moving images photographed by the cameras 2a, 2b, 2c, and 2d.
In FIG. 5, the processing device 6 includes an image quality adjustment unit 11, a captured image storage unit 12, a vector calculation unit 13, a distribution state determination unit 14, a moving person detection unit 15, a person separation unit 16, and a person counting. Unit 17, display processing unit 18, and duplicate person elimination unit 19, except for the duplicate person elimination unit 19, and the other configurations are the same as those of the processing apparatus 1 described in the first embodiment. is there. A different part in the operation is that the image quality adjustment unit 11, the captured image storage unit 12, the vector calculation unit 13, the distribution state determination unit 14, and the moving person detection unit 15 are time-series continuous by a plurality of input cameras. The same process is repeated for each image by the number of moving images.
The duplicate person exclusion unit 19 compares the detection time of the person shape image detected by the moving person detection unit 15 with the spatial coordinates, and eliminates the person shape image detected redundantly between the time-series continuous moving images. To do.
[0032]
Next, operations of the processing device 1 and the processing device 6 of the person detection system according to the first and second embodiments will be described with reference to the flowchart shown in FIG.
That is, a time-series continuous moving image in the measurement space range for detecting a person who moves first is photographed (step S1).
The captured image captured by the camera 2 or the cameras 2a, 2b, 2c, and 2d is an RGB image in which one pixel has three pieces of information of a red component, a blue component, and a green component. The RGB image is then subjected to color coordinate conversion by the image quality adjustment unit 11 to create an image consisting only of luminance information (hereinafter referred to as “Y image”). A Y image can be obtained by mixing the red component at 30 percent, the blue component at 11 percent, and the green component at 59 percent.
[0033]
Further, the average brightness, the width of contrast, etc. are obtained in the measurement space range and compared with a predetermined standard image, so that the image is always converted into an image having the same image quality (step S2). This measurement space range is stored in the processing program stored in the processing device 1 or the processing device 6 by the user by setting the position and size in advance using the operation display terminal 3 before operating the person detection system. I will remember it.
In this way, by using Y images having the same image quality regardless of the conditions of the measurement location, it is possible to perform processing for person detection under the same conditions.
[0034]
The Y image whose image quality has been adjusted by the above-described procedure is temporarily stored in the captured image storage unit 12.
Next, the vector calculation unit 13 obtains the velocity vector of the pixel using the optical flow estimation calculation from the change in the luminance value during the minute time Δt of the stored Y image (step S3). That is, the change in the luminance value during the minute time Δt of the Y image is caused by the movement of the object in the time-series continuous moving image. The change in the luminance value of the pixel is used as follows. A velocity vector for each pixel is estimated.
[0035]
The optical flow estimation is to obtain an apparent velocity distribution of a substance on the screen that is felt by the observer, and a basic formula for determining the distribution is derived from two assumptions. The first is that “the luminance value of the target physical point is kept constant during movement”. The luminance value of the point (x, y) on the screen at time t is P (x, y, t), and the luminance value when the physical point moves to the point (x + Δx, y + Δy) after a minute time Δt. Is P (x + Δx, y + Δy, t + Δt). At this time, the following equation holds.
[Expression 1]

When this equation (1) is Taylor-expanded and Δt is made as close as possible to 0, the following equation is obtained.
[Expression 2]

Here, (u, v) is an apparent velocity vector, and Px, Py, Pt are partial differentials with respect to x, y, t, respectively.
[0036]
Another assumption is that “the apparent speed changes smoothly”. The smoothness of speed is evaluated by the sum of Laplacian of speed vectors, and can be expressed as the following equation.
[Equation 3]

However, in actual calculations, an error occurs because Px, Py, and Pt in the equation (2) are approximated by a finite difference. Further, since it is necessary to discretize the expression (3), the smoothness of the speed is expressed by the following expression in the actual calculation.
[Expression 4]

Here, u and v with a bar at the top are the average values of the neighboring velocity vectors and can be expressed by the following equations.
[Equation 5]

[0037]
In addition, smoothness weight coefficient α²Then, an error evaluation formula such as the following formula is obtained using formulas (4) and (5).
[Formula 6]

By using the equation (8), the equation for calculating the velocity distribution can be obtained under the condition that the error E takes the minimum value. Then, the velocity distribution is obtained by repeating the following equation to converge.
[Expression 7]

However, k is the number of iterations.
[0038]
The vector calculation unit 13 obtains an estimated velocity vector (u, v) for all the pixels in the measurement space range of two consecutive Y images by such a method.
[0039]
The optical flow estimation is an ideal solution because of the completely rigid body of the moving object and the luminance invariance accompanying the movement, but a normal moving object is not so established. In addition, the person detection does not pursue the accuracy of the actual speed of the pedestrian (moving person), but strictly determines the presence of the person from the generation distribution of the speed field, and strictly determines the reliability of the speed vector. Interpretation is not required.
[0040]
Further, the local velocity vector generated in the actual situation of the person walking forward becomes a complicated movement even when looking at the head, limbs, and body. Furthermore, there is also a velocity vector that is clearly different from (averse to) the direction of travel caused by the reaction of limbs during walking, and a method of deleting the velocity vector that is unevenly distributed in a small area is possible, but it occurs in the entire moving person If all the local (pixel) velocity vectors are scrutinized, it can be clearly seen that there are overwhelmingly many velocity vector components in the moving direction.
Therefore, specifically, the number of iterative calculations for converging the optical flow estimation is reduced to 30 to 40 times, the calculation cost is reduced, and a smoothing weighting coefficient that becomes a low-pass filter effect of the motion component is obtained. 80-150. Further, a small component, that is, a low velocity component is deleted from the velocity vector of the optical flow estimation.
[0041]
Next, the estimated velocity vector obtained by the vector calculation unit 13 is input to the distribution state determination unit 14, and the distribution state determination unit 14 determines the number of pixels having the estimated velocity vector in the X direction and the Y direction of the vector. A histogram is obtained by counting in the direction of two quadrants, and the distribution state of the velocity field formed in the measurement space range in the time-series continuous moving image is obtained by the velocity vector (step S4).
[0042]
FIG. 7 is a diagram showing how the distribution state discriminating unit 14 obtains the distribution state of the velocity field. As shown in FIG. 7A, the vector calculation unit 13 first calculates the pixel velocity calculation point in the measurement space range. Assume that an estimated velocity vector is obtained. At this time, the distribution status determination unit 14 determines whether the estimated velocity vector belongs to the right motion or the left motion with respect to the measurement cross section set in the measurement space range as shown in FIG. 7B. . For example, here, when the angle of the estimated velocity vector is included in the range of θ1 to θ2, it is assumed to be right motion, and when it is included in the range of θ3 to θ4, it is assumed to be left motion.
When the estimated vectors are classified into right motion and left motion in this way, the number of pixels having the vector is counted for each of the right motion and the left motion, and the distribution state of the velocity field is obtained. For example, FIG. 7 (a) counts the number of pixels having an estimated velocity vector of right motion in the X direction and Y direction, respectively, and obtains histograms for the right motion X axis projection surface and the right motion Y axis projection surface, respectively. Shows how it was done.
[0043]
Next, the moving person detection unit 15 is a circumscribed rectangle for an image that is supposed to indicate a person based on the range of pixels having velocity vectors for each direction indicated by the distribution state of the velocity field obtained by the distribution state determination unit 14. And a human shape image moving in the time-series continuous moving image and its moving direction are detected (step S4).
FIG. 8A is a diagram showing the distribution state of the velocity field obtained by the distribution state determination unit 14, and the left motion X axis according to the change in the luminance value of the pixel by the moving persons 101, 102, 103. Width Wa (pixel) histogram on projection surface, left motion Y-axis projection surface with width Wb (pixel) histogram, right motion X-axis projection surface with width Wc (pixel) histogram, right motion Y-axis projection surface with width Wd A (pixel) histogram is formed.
[0044]
Accordingly, as shown in FIG. 8B, a closed region in which the width of each histogram is extended is obtained, and if this is greater than the size of the region per person stored in advance, this is measured in the measurement space range. Are detected as person shape images 104 and 105 showing circumscribed rectangles of the moving person. Since the human shape image 104 is obtained from a histogram of pixels having a right motion velocity vector, the human shape image 105 is a human image moving in the right direction. The human shape image 105 is obtained from a histogram of pixels having a left motion velocity vector. Since it is obtained, it can be seen that the image is a person moving leftward.
[0045]
As described above, the image quality adjustment unit 11, the captured image storage unit 12, the vector calculation unit 13, the distribution state determination unit 14, and the moving person detection unit 15 are the same as those in the person detection system according to the first embodiment. In this case, it is only necessary to perform processing on a time-series continuous moving image input from one camera. In the case of the human detection system according to the second embodiment, time-series continuous moving images input from a plurality of cameras are input. The same processing is repeated for each image by the number.
[0046]
When the human figure image is detected by the moving person detection unit 15, only the person detection system of the second embodiment, the duplicate person exclusion unit 19 detects the person shape image detected for each of a plurality of time-series continuous moving images. The detection time is compared with the spatial coordinates to determine whether there are overlapping person shape images (step S6).
If there is an overlapping person shape image in step S6 (YES in step S6), one of the overlapping person shape images is left and all others are excluded (step S7).
In step S6, if there is no overlapping person shape image (NO in step S6), the process proceeds to the next step S8 without doing anything.
In the case of the person detection system according to the first embodiment, if a person shape image is detected in step S5, the process proceeds to the next step S8 without doing anything.
[0047]
In step S8, the person separation unit 16 obtains the space occupancy rate of the person shape image in the measurement space range, and determines whether there is a person shape image having a space occupancy rate of one or more (step S8).
In step S8, when there is a person shape image having a space occupancy rate of one or more (YES in step S8), the person shape is determined by comparing the space occupancy rate with the preliminarily stored space occupancy rate per person. The number of people included in the image is estimated (step S9). For example, in the example shown in FIG. 8B, the person shape image 104 has a large space occupancy rate, and it can be estimated that two persons 101 and 102 are included.
In step S8, if there is no person shape image having a space occupancy of 1 or more (NO in step S8), the process proceeds to the next step S10 without doing anything.
[0048]
In step S10, the person counting unit 17 obtains the number of persons by counting the number of person shape images that pass through the measurement section set in the measurement space range (step S10).
FIG. 9 is a diagram for explaining a counting method in the number counting unit 17, and the number counting unit 17 is such that a person shape image that moves in a direction crossing the measurement section like the person 106 and the person 107 passes through the measurement section. Counting is performed, and a person shape image that moves in a direction that does not intersect the measurement cross section (parallel to the measurement cross section) like the person 108 is not counted.
In addition, when the person shape image in which the number of persons included is estimated to be plural passes through the counting section, the person separation unit 16 counts the number of persons by the estimated number of persons.
The number of people counted by the number of people counting unit 17 is displayed on the operation display terminal 3 and / or the display device 4 via the display processing unit 18.
[0049]
In the first and second embodiments described above, the person separation unit 16 obtains the space occupancy rate of the person shape image in the measurement space range, and whether there is a person shape image having a space occupancy rate of one or more. In this case, it may be necessary to correct the space occupancy per person stored in advance depending on the installation location of the camera 2 as the image pickup means and the focal length of the attached lens. This is because the angle of view, that is, the shooting area, is determined by the installation location of the camera 2 that is the imaging means and the focal length of the attached lens. This leads to the fact that the reference image area per moving person changes depending on the angle of view taken depending on the installation location of the camera 2 and the focal length of the lens that is mounted, and therefore space occupies under crowded conditions. It is necessary to correct the space occupancy per person, which is a comparison value necessary for estimating the number of people from the rate, based on the installation location and the angle of view.
[0050]
Therefore, the person separation unit 16 is provided with a mode for performing learning calibration before measurement, and one person passes through the measurement cross section several times, and the area of the person shape image detected at that time is averaged. The value is used as the space occupancy per person, and during measurement, the area of the person shape image determined to be one person is subjected to a moving average calculation, and the space occupancy per person as a comparison value at any time Learning updating means for learning and updating the above may be provided.
In this way, in addition to the correction of the space occupancy rate per person based on the installation location of the camera 2 as the above-mentioned imaging means and the focal length of the lens mounted, the moving person by the change in the sunlight angle outdoors It becomes possible to follow the change of the space occupancy rate due to the change of the shadow area.
[0051]
(Third embodiment)
Next, a third embodiment of the present invention will be described.
The third embodiment of the present invention is a configuration example in the case of counting the number of people for each direction in which a person moves with respect to the first and second person detection systems. The schematic diagram showing the configuration of the person detection system according to the third embodiment of the present invention may be either the schematic diagram of the first embodiment or the schematic diagram of the second embodiment. The apparatus 6 should just be changed into the processing apparatus 7 shown below. Therefore, the description of the schematic diagram is omitted here.
[0052]
Next, the configuration of the processing apparatus 7 will be described in detail with reference to the drawings. FIG. 10 is a block diagram showing a detailed configuration of the processing device 7 that counts the number of persons for each direction in which a person moves. Here, time-series continuous images taken by a plurality of cameras as in the second embodiment are shown. An example of counting the number of people from a moving image is shown as an example.
In FIG. 10, the processing device 7 includes an image quality adjustment unit 11, a captured image storage unit 12, a vector calculation unit 13, a distribution state determination unit 14, a moving person detection unit 15, a person separation unit 16, and a person counting. Unit 17, duplicate person exclusion unit 19, movement direction classification unit 20, passage time measurement unit 21, and alarm generation unit 22. The configuration includes a moving direction classifying unit 20 that classifies the moving direction of the person shape image for which the number of people is counted in the number counting unit 17 for each predetermined angle, and a measurement in which the person shape image is provided in the measurement space range. The movement time of the passage time measurement unit 21 that measures the passage time when passing through the cross section and the movement direction of the person shape image classified by the movement direction classification unit 20 are determined, and a person shape image that proceeds in a predetermined movement direction is found. In this case, except for the alarm generation unit 22 that issues an alarm to the operation display terminal 3 and / or the display device 4, the other processing is the same as the processing device 6 described in the second embodiment.
[0053]
Next, the operation of the processing device 7 of the person detection system according to the third embodiment will be described with reference to the flowchart shown in FIG.
Steps S1 to S10 in the operation of the processing device 7 are the same as those of the processing device 6 of the second embodiment described above, and thus description thereof is omitted here.
In the processing device 7, when the number counting unit 17 counts the number of person shape images passing through the measurement cross section set in the measurement space range and obtains the number of people in step S <b> 10, the passage time measuring unit 21 measures the measurement space. The time when the person shape image passes through the measurement cross section set in the range is measured (step S11).
On the other hand, the movement direction classification unit 20 classifies the movement direction of the person shape image counted by the number counting unit 17 for each predetermined angle set in the processing device 7 in advance (step S12).
[0054]
When the movement direction classification unit 20 can classify the movement direction of the person shape image, the alarm generation unit 22 determines the movement direction of the classified person shape image and determines whether there is a person shape image that proceeds in the predetermined movement direction. It is determined whether or not the current time permits movement in that direction, and it is determined whether or not an alarm needs to be issued (step S13).
In step S13, when it is necessary to issue an alarm (YES in step S13), the alarm generation unit 22 issues an alarm and displays the number or / and images of the person shape image that is the object of the alarm on the operation display terminal 3 or / And output to the display device 4 (step S14).
[0055]
In the human detection system according to the third embodiment, in the above configuration, for example, the classification angle in the movement direction of the human shape image in the movement direction classification unit 20 is 0 to 180 degrees with respect to the counting section (for example, entrance at the entrance / exit). ) And 180 to 360 degrees (for example, assuming exit at the doorway), and an alarm is triggered when a person-shaped image that moves to either angle is found, The person's flow in the passage can be monitored, and a person who is against the flow can be found.
Also, in employee passages during leaving hours such as large-scale stores and factories, it is not possible to detect the flow of leaving employees, but to detect only those who are about to enter the facility and issue an alarm. You can also.
[0056]
In addition, if it is not necessary to report the number of persons who are the target of the alarm together with the alarm, the number counting unit 17 may not be provided.
In addition, the passage time measuring unit 21 may not be provided in the entrance / exit of the person detection target and the passage unless the passage permission and the direction in which the passage is permitted are changed depending on the time.
Furthermore, information notifying means for notifying the surroundings of predetermined information based on the counting result of the number of people may be provided. In this case, for example, the person detection system of the present embodiment is installed for each entrance / exit of the person detection target, and the number of people for each entrance / exit / passage is measured, so that a person can be guided to an open entrance / exit. become. For guidance to vacant doorways and passages, set the number of people allowed at the doorway and passage in advance in the person detection system installed in each doorway and passage, and if the number of people counted is less than that , You may inform the surroundings that this doorway / passage is vacant, or centrally manage the information of the person counting system installed in all the doorways / passageways and vacant doorway / passage information May be notified. By using such a person detection system, it is possible to efficiently guide a person to an emergency exit in evacuation guidance or the like.
[0057]
(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.
In the fourth embodiment of the present invention, the number of people is counted for each direction in which the person moves in the first and second person detection systems, and the distribution of the number of people ahead of the direction and direction is calculated. It is an example of a structure in the case of estimating. The schematic diagram showing the configuration of the person detection system according to the fourth embodiment of the present invention may be either the schematic diagram of the first embodiment or the schematic diagram of the second embodiment. The apparatus 6 should just be changed into the processing apparatus 8 shown below. Therefore, the description of the schematic diagram is omitted here.
[0058]
Next, the configuration of the processing device 8 will be described in detail with reference to the drawings. FIG. 12 is a block diagram showing a detailed configuration of the processing device 8 that counts the number of people for each direction in which a person moves and estimates the number distribution of people ahead of that direction and direction. A case where the number of persons is counted from time-series continuous moving images taken by a plurality of cameras as in the embodiment will be described as an example.
In FIG. 12, the processing device 8 includes an image quality adjustment unit 11, a captured image storage unit 12, a vector calculation unit 13, a distribution state determination unit 14, a moving person detection unit 15, a person separation unit 16, and a person counting. Unit 17, duplicate person exclusion unit 19, movement direction classification unit 20, passage time measurement unit 21, person distribution estimation unit 23, and information notification unit 24. The configuration is such that the alarm generation unit 22 in the processing device 7 of the third embodiment uses the movement direction of the person shape image classified by the movement direction classification unit 20 and the person shape image measured by the passage time measurement unit 21. A person distribution estimation unit 23 for estimating the behavior of the person shape image from the passage time of the measurement cross section and the number of persons, and obtaining the number distribution for a predetermined place or direction, and the number of persons, moving direction, passage time, and number distribution The processing apparatus described in the third embodiment except that the information notifying unit 24 that notifies predetermined information to the operation display terminal 3 and / or the display device 4 is replaced based on any information. 7 is the same.
[0059]
Next, the operation of the processing device 8 of the person detection system according to the fourth embodiment will be described with reference to the flowchart shown in FIG.
Steps S1 to S12 in the operation of the processing device 8 are the same as those of the processing devices 6 and 7 in the second and third embodiments described above, and thus description thereof is omitted here.
In the processing device 8, when the movement direction classification unit 20 classifies the movement direction of the person shape image passing through the measurement cross section set in the measurement space range for each predetermined angle in step S12, the person distribution estimation unit 23 The behavior of the person shape image is estimated from the number of people, passage time, and moving direction, and the number of people distribution for a predetermined place or direction is obtained (step S15).
[0060]
FIG. 14 is a diagram illustrating an application example using the person detection system for explaining the operation of the person distribution estimation unit 23. As shown in FIG. 14 (a), for example, the person detection system of the present embodiment is installed at the entrance of a store, and the classification angle in the movement direction of the person shape image in the movement direction classification unit 20 goes straight with respect to the counting section. When classified into three types of direction (right-angle direction) left-hand direction and right-hand direction, the person distribution estimation unit 23 counts the number of persons for each classified direction as shown in FIG. Create As a result, for example, as shown in FIG. 14A, it is possible to grasp the customer attracting situation for each sales floor A, B, C of the store. Further, by using together the passage time of the measurement cross section of the person shape image measured by the passage time measurement unit 21, the number of people distribution for each hour can be obtained. Therefore, in addition to the distribution of people over a relatively long time, such as information on attracting customers at store sales floors, the distribution of people over a short period of time, such as the evacuation status (such as the concentration of people with respect to an emergency exit) during evacuation guidance, can be obtained. Evacuation guidance can be performed efficiently.
[0061]
And according to the person distribution calculated | required by the person distribution estimation part 23, the information alerting | reporting part 24 alert | reports predetermined information via the operation display terminal 3 or / and the display apparatus 4 (step S16). The contents to be notified here are, for example, information on attracting customers at each store in the store, announcements for guiding customers based on the information on attracting customers, information on warnings, guidance, instructions, etc. in evacuation guidance, Each differs depending on how the person detection system is used.
[0062]
Further, for example, a traffic control signal for a pedestrian crossing pedestrian is connected to the person detection system of the fourth embodiment described above, and the person distribution on the road estimated by the person distribution estimation unit 23 is processed in the processing device 8. Based on the above, a traffic control means for controlling the traffic control signal may be provided. In this case, the traffic control means, for example, according to the distribution of pedestrians who are crossing or crossing the pedestrian crossing, for example, if there are many pedestrians, the signal “blue: crossing is allowed” is lengthened. When the number is low, control is performed so as to shorten the “blue: crossing possible” time of the signal.
[0063]
Further, the alarm generation unit 22 according to the third embodiment described above and the person distribution estimation unit 23 according to the fourth embodiment are both provided, and switching means for switching these two functions by time, for example, is provided. In the daytime, customer information for each store in the store may be collected, and at night, it may be used as an intrusion sensor for intruders in the store.
[0064]
(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. The fifth embodiment of the present invention is an entrance / exit control device that unlocks an entrance / exit key when a condition is satisfied so that only a person permitted to pass can enter and exit. It is a configuration example of an entrance / exit control device that determines whether or not a person can enter and exit based on a predetermined operation such as verification of an ID number using a card and the movement direction of the person.
In the above-described fourth embodiment, as shown in FIG. 14A, the case where the measurement space range is set inside the (store) entrance / exit is described. However, in the fifth embodiment, the measurement space range is set. 12 in the processing device 8 shown in FIG. 12, instead of the person distribution estimation unit 23, for example, according to a predetermined operation such as ID number collation using a non-contact ID card and the movement direction of the person. An entrance / exit control unit (not shown) is provided for determining whether entry / exit is possible.
[0065]
In the person detection system according to the fifth embodiment, in the above configuration, for example, the movement direction classification angle of the human shape image in the movement direction classification unit 20 is as shown in FIG. Are classified into a direction crossing the measurement section and a direction moving in a direction not parallel to the measurement section (parallel to the measurement section) like the person 108.
In this case, the entrance / exit control unit detects a person shape image that proceeds in a direction crossing the measurement section, and when a predetermined operation such as ID number collation by a non-contact ID card is performed within a predetermined time, Allow the person to enter and exit, and unlock the doorway. In addition, when a person-shaped image that goes in a direction that does not cross the measurement section is detected and a predetermined operation such as ID number collation using a non-contact ID card is performed within a predetermined time, the person passes through the doorway The doorway key is not unlocked as if there is no intention to do so (just across the entrance / exit control device).
Note that the information notification unit 24 monitors the entry / exit of a person and reports information such as the number of persons entering / exiting to the operation display terminal 3. Further, if there is no need for reporting, the information notification unit 24 may not be provided.
[0066]
Therefore, for example, ID verification is performed only when a person who has a contactless ID card crosses the entrance and exit, and the entrance / exit control device permits entry / exit (the entrance / exit control device unlocks the entrance / exit key. The effect that it is possible to prevent such a malfunction as described above is obtained.
[0067]
(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described. In the sixth embodiment of the present invention, the third embodiment of the present invention further counts the number of persons in each direction in which the person moves with respect to the first and second person detection systems. In the case where the alarm is generated from the alarm generation unit 22 based on the above, the predetermined information such as warning, guidance, instruction, etc. to the surroundings is notified based on the number of people counted for each direction in which the person moves. It is a structural example.
In the above-described third embodiment, as shown in FIG. 10, the processing device 7 including the alarm generation unit 22 has been described. However, in the sixth embodiment, the processing device 7 includes the alarm generation unit 22. Instead, based on any of the information on the number of persons, the moving direction, and the passage time described in the fourth embodiment (the number distribution is not used in the sixth embodiment), predetermined information is displayed on the operation display terminal. 3 and / or an information notifying unit 24 for notifying the display device 4 is provided.
[0068]
In the person detection system according to the sixth embodiment, in the above configuration, for example, at the entrance of a large-scale store, the direction of movement of the person is determined by the movement direction classifying unit 20, so that the customer who enters the store and the customer who leaves the store are displayed. And the information notification unit 24 outputs information corresponding to the information to the display device 4, so that the display device 4 is “welcome to” customers who enter the store and “Thank you very much”. Announcement such as “It was over there” can be output.
Furthermore, for customers after entering the store, the direction of each customer is detected, and information corresponding to the direction of travel is output to the display device 4, so that the display device 4 "is a clothing department." It is also possible to output a sales floor announcement ahead of the direction of travel, such as "That is a toy section."
Further, the passage time measuring unit 21 determines the passage time of the passerby and outputs predetermined information to the display device 4, so that “Good morning, please say.” “Good evening”, etc. Announcements according to changes can be output.
[0069]
Similar to the third embodiment, if it is not necessary to report the number of people who have been notified along with the information notification, the number counting unit 17 may not be provided. In addition, the passage time measuring unit 21 may not be provided if the content of the information to be notified according to time is not changed at the entrance and the passage of the person detection target.
[0070]
In the first to sixth embodiments described above, the processing apparatuses 1, 6, 7, and 8 and the operation display terminal 3 are described as being directly connected. However, for example, the processing apparatus 1 is taken as an example. As will be described with reference to FIG. 15, the communication device 50 is connected to the processing devices 1, 6, 7, and 8, and the number of persons counted via the communication network 51, alarms and their contents, and information to be notified are operated. You may make it transmit / receive with the display terminal 3. FIG. Here, the communication network 51 is, for example, wireless communication using WAP (Wireless Application Protocol), the Internet using WWW (World Wide Web), or a public line such as PSTN (Public Switch Telephone Network) or ISDN (Integrated Services Digital Network). A communication network that transmits and receives information by wired communication via the network is included. Alternatively, a plurality of processing devices may be connected to one operation display terminal 3, and the plurality of processing devices may be controlled by one operation display terminal 3.
[0071]
The program for realizing the functions of the processing devices 1, 6, 7, and 8 in the first to sixth embodiments described above was recorded on a computer-readable recording medium and recorded on this recording medium. The person may be detected by reading the program into a computer system and executing it. Here, the “computer system” includes an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.
[0072]
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.
[0073]
【The invention's effect】
As described above, according to the first aspect of the present invention, the velocity vector of the pixel generated when the object moves in the measurement space range of the time-series continuous moving image captured by the imaging unit is obtained, and the direction and the velocity are determined. The number of pixels having a velocity vector in a predetermined range is plotted in a histogram for each predetermined direction indicated by the velocity vector to obtain a velocity field distribution state. Then, an image range that can be estimated as a person moving in this direction is identified as a person shape image from the range of pixels having velocity vectors for each direction indicated by the distribution state of the velocity field.
Therefore, it is possible to accurately estimate the existence range of the person in the measurement space range for each moving direction from the pixel range represented by the histogram, thereby improving the person identification accuracy.
[0074]
According to the second aspect of the present invention, an image range that can be estimated as a person is identified as a person shape image for each time-series continuous moving image captured by a plurality of imaging means. Then, comparing the identification time and spatial coordinates of the human shape image identified for each time-series continuous moving image, it is determined that the human shape images having the same identification time and spatial coordinates are overlapping human shape images, Leave one and exclude the others.
Therefore, by using a plurality of imaging means, an effect is obtained that a seamless and wide measurement space range (panoramic image) that cannot be obtained by one imaging means can be obtained.
[0075]
According to the third aspect of the present invention, the space occupancy rate of the detected human shape image is determined based on the difference in the size of the pixel range having the velocity vector for each direction indicated by the distribution state of the velocity field. Then, the number of persons included in the person shape image is estimated from a comparison between the space occupancy and the space occupancy per person estimated in advance.
Therefore, conventionally, in a traffic situation that is crowded with passers-by or a traffic situation in which passers-by are crossed, even if multiple people move overlapping, it is determined that one person is moving Thus, it is possible to improve the estimation accuracy of the number of persons who move in the measurement space range.
[0076]
According to the fourth aspect of the present invention, even if the angle of view of the image captured by the imaging means or the appearance of the captured person changes, one person who is the target of comparison of the space occupancy rate of the person shape image at any time It is possible to accurately maintain the space occupancy necessary for the area.
Therefore, in addition to the correction of the space occupancy rate per person based on the installation location of the camera 2 that is the imaging means and the focal length of the mounted lens, the space due to the change in the shadow area of the moving person due to the change in the sunshine angle in the outdoors It becomes possible to follow fluctuations in the occupation ratio.
According to the invention described in claim 5, the number of persons who have passed through the predetermined place is determined by setting a predetermined measurement section within the measurement space range and counting the number of person shape images passing through the measurement section. Ask for.
Therefore, for example, it is possible to detect a person entering / exiting a building or the like and accurately measure the number of persons.
[0077]
According to the sixth aspect of the invention, the counted number of persons is output to the operation display means connected to the communication network.
Therefore, for example, it is possible to detect a person who goes in and out of a distant building or the like and accurately grasp the number of persons on the operation display means at hand.
According to the seventh aspect of the present invention, the direction in which the identified person shape image is moving is classified, and if there is a person who proceeds in a predetermined direction, an alarm is issued.
Therefore, for example, when a person who tries to move backward in a one-way passage or an intruder to a place where entry is prohibited is detected, for example, only for an entrance or only for an exit, this can be detected and notified. can get.
[0078]
According to the eighth aspect of the present invention, the direction in which the identified person shape image is moving is classified, and when there is a person who moves in a predetermined direction, the person's moving direction is correct according to the time zone. If it is not correct, an alarm is issued.
Therefore, for example, when a person or the like who tries to move backward in the one-way passage where the designation of the direction of travel changes according to time is found against the designation, it can be detected and notified. It is done.
According to the ninth aspect of the present invention, the identified person shape image is classified in which direction, when and how many people have moved, the behavior of the person shape image is estimated, and the number distribution for a predetermined place or direction is obtained. Ask.
Therefore, for example, it is possible to obtain an effect that it is possible to totalize the situation of attracting customers for each sales floor in the store or to grasp the evacuation situation in the evacuation guidance.
[0079]
According to the tenth aspect of the present invention, it is classified into which direction, when and how many people the identified person shape image has moved, the behavior of the person shape image is estimated, and the number distribution for a predetermined place or direction is obtained. A function to be requested and a function to detect and notify when an intruder has been detected in a place where entry is prohibited are switched and used.
Therefore, for example, it is possible to obtain an effect that the sales situation of the sales floor can be totaled during the business hours of the store and used as an intruder sensor for detecting an intruder into the store outside the business hours of the store.
According to the eleventh aspect of the present invention, an alarm is issued and the number of people-shaped images and / or images of the person-shaped images that advance in the predetermined movement direction are output.
Therefore, it is possible to obtain an effect that, together with the alarm, the situation causing the alarm can be notified at the same time.
[0080]
According to the twelfth aspect of the present invention, predetermined information such as a warning, guidance, or instruction to the surroundings is notified based on any information of the obtained number of persons, moving direction, passage time, and number distribution.
Accordingly, for example, in evacuation guidance, the number of people rushing to the doorway can be accurately grasped and an appropriate instruction can be issued.
According to the invention described in claim 13, the traffic control signal is controlled based on the obtained distribution of the number of people.
Therefore, for example, it is possible to accurately grasp the number of people crossing the intersection and control the traffic permission signal crossing permission time and the number of times of crossing permission to perform smooth traffic control.
[0081]
According to the fourteenth aspect of the present invention, whether or not a person can enter and leave is determined based on a predetermined operation such as ID number collation using a non-contact ID card and the movement direction of the person.
Therefore, for example, ID verification is performed just by the person holding the ID card crossing in front of the entrance / exit control device that determines whether entry / exit is possible by checking the ID number using a non-contact type ID card. Thus, it is possible to prevent the malfunction that the entrance / exit control device permits entry / exit.
[0082]
According to the invention described in claim 15, when there is a person who classifies in which direction the identified person shape image is moving and proceeds in a predetermined direction, a predetermined warning, guidance, instruction, etc. The information of is notified.
Therefore, for example, based on the moving direction of the person in the passage, predetermined information notification such as an appropriate warning, guidance, or instruction can be performed, and the operation of the person in the passage can be controlled.
[Brief description of the drawings]
FIG. 1 is a schematic diagram illustrating a configuration of a person detection system according to a first embodiment of this invention.
FIG. 2 is a block diagram showing a detailed configuration of a processing device of the person detection system according to the embodiment;
FIG. 3 is a schematic diagram showing a configuration of a person detection system according to a second embodiment of the present invention.
FIG. 4 is a diagram for explaining an output signal of the asynchronous video composition device of the person detection system according to the embodiment;
FIG. 5 is a block diagram showing a detailed configuration of a processing device of the person detection system according to the embodiment;
FIG. 6 is a flowchart showing the operation of the processing device of the person detection system according to the first and second embodiments.
FIG. 7 is a diagram for explaining an operation of a distribution status determination unit of the processing device 1 or the processing device 6;
FIG. 8 is a diagram for explaining an operation of a moving person detection unit of the processing device 1 or the processing device 6;
FIG. 9 is a diagram for explaining the operation of the number counting unit of the processing apparatus 1 or the processing apparatus 6;
FIG. 10 is a block diagram illustrating a detailed configuration of a processing device of a person detection system according to a third embodiment of this invention.
FIG. 11 is a flowchart showing the operation of the processing device of the person detection system according to the embodiment;
FIG. 12 is a block diagram illustrating a detailed configuration of a processing device of a person detection system according to a fourth embodiment of this invention.
FIG. 13 is a flowchart showing the operation of the processing device of the person detection system according to the embodiment;
FIG. 14 is a diagram showing an application example using the person detection system of the embodiment.
FIG. 15 is a schematic diagram showing a configuration for performing a remote operation of the human detection system of the present invention via a communication network.
[Explanation of symbols]
1 Processing device
2, 2a, 2b, 2c, 2d camera
3 Operation display terminal
4 display devices
5 Asynchronous video composition device
6 processing equipment
7 Processing equipment
8 processing equipment
11 Image quality adjustment section
12 Captured image storage
13 Vector calculator
14 Distribution status discriminator
15 Moving person detection unit
16 Person separation part
17 Number counting section
18 Display processing section
19 Duplicate person exclusion section
20 Movement direction classification part
21 Passing time measurement unit
22 Alarm generator
23 Person Distribution Estimator
24 Information reporting unit
50 Communication device
51 communication network

Claims (19)

Luminance of pixels of the time-series continuous moving image generated by the movement of an object in a time-series continuous moving image captured by a plurality of imaging units that respectively capture different ranges while continuing a part of the range to be captured A vector calculation means for obtaining a velocity vector of the pixel for each time-series continuous moving image from a change in value using an optical flow estimation calculation;
By counting the number of pixels having the velocity vector for each predetermined direction indicated by the velocity vector, each time-series continuous formed by the velocity vector in the measurement space range in each time-series continuous moving image. A distribution status determination means for obtaining a distribution status of the velocity field for each moving image;
A human shape image that is a pixel region identified based on a speed vector from a range of pixels having direction-specific speed vectors indicated by the speed field distribution state, and that moves in the time-series continuous moving image A moving person detecting means for detecting an image and its moving direction;
A person detection system comprising: a duplicate person exclusion unit that compares a detection time of the person shape image with a spatial coordinate and eliminates a person shape image that is duplicated between the time-series continuous moving images. . The distribution situation determination means obtains a histogram in the X direction and a histogram in the Y direction by counting the number of pixels having the velocity vector in the X direction and the Y direction of the velocity vector,
The human detection system, wherein the moving person detection means obtains a pixel area of a closed area determined based on the width of each of the histogram in the X direction and the histogram in the Y direction as a person shape image .   3. A person separation unit that obtains a space occupancy rate of the person shape image in the measurement space range and estimates the number of persons included in the person shape image from the space occupancy rate. The person detection system described in 1. The person separating means is
When estimating the number of persons included in the person shape image from the space occupancy rate of the person shape image in the measurement space range, the space occupancy required per person to be compared with the space occupancy rate of the person shape image The person detection system according to claim 3, further comprising learning update means for learning and updating the rate.   5. The apparatus according to claim 1, further comprising a person counting unit that counts the number of the person shape images passing through a predetermined measurement section set in the measurement space range to obtain the number of persons. The person detection system described.   6. The person detection system according to claim 5, further comprising communication means for transmitting / receiving the number of persons to / from a remote operation display means capable of displaying the number of persons via a communication network. Movement direction classification means for classifying the movement direction of the person shape image;
2. An alarming unit for determining a moving direction of the person shape image classified by the moving direction classifying unit and issuing an alarm when a person shape image moving in a predetermined moving direction is found. The person detection system according to claim 6. Passage time measuring means for measuring the passage time of the measurement cross section of the person shape image,
8. The person detection system according to claim 7, wherein the warning means issues a warning when a person shape image that advances in a predetermined moving direction is found at a predetermined time. Movement direction classification means for classifying the movement direction of the person shape image;
Passage time measuring means for measuring the passage time of the measurement cross section of the person shape image;
2. A person distribution estimation unit that estimates a behavior of the person shape image from the number of persons, the passage time, and the moving direction, and obtains a number distribution for a predetermined place or direction. 7. The person detection system according to any one of 6. Movement direction classification means for classifying the movement direction of the person shape image;
Passage time measuring means for measuring the passage time of the measurement cross section of the person shape image;
A person distribution estimating means for estimating the behavior of the person shape image from the number of persons and the passing time and the moving direction, and obtaining a number distribution for a predetermined place or direction;
A warning means for determining a movement direction of the person shape image classified by the movement direction classification means, and for issuing a warning when a person shape image traveling in a predetermined movement direction is found;
The person detection system according to claim 1, further comprising a switching unit that switches between the person distribution estimation unit and the warning unit to operate. The alarm means includes
The person detection system according to any one of claims 7, 8, and 10, wherein an alarm is issued and the number of people shape images and / or images of an alarm target are output.   12. The information notifying means for notifying predetermined information based on any one or more information of the number of people, the moving direction, the passing time, and the number of people distribution. The person detection system according to any one of the above.   The person detection system according to claim 9, further comprising a traffic control unit that controls a traffic control signal based on the number distribution. Movement direction classification means for classifying the movement direction of the person shape image;
Entrance / exit control means for permitting entry / exit of the person when a predetermined operation is executed and the movement direction of the person shape image determined by the movement direction classification means is a predetermined direction. The person detection system according to any one of claims 1 to 6, characterized in that: Movement direction classification means for classifying the movement direction of the person shape image;
An information notifying unit for determining a moving direction of the person shape image classified by the moving direction classifying unit and notifying predetermined information when a person shape image moving in the predetermined moving direction is found. The person detection system according to any one of claims 1 to 6. Using the optical flow estimation calculation from the change in the luminance value of the pixel of the time-series continuous moving image generated by the movement of the object in the time-series continuous moving image captured by a plurality of imaging means for capturing different ranges. A vector calculation process for obtaining the velocity vector of the pixel for each time-series continuous moving image;
By counting the number of pixels having the velocity vector for each predetermined direction indicated by the velocity vector, each time-series continuous formed by the velocity vector in the measurement space range in each time-series continuous moving image. Distribution status determination processing for obtaining the distribution status of the velocity field for each moving image,
A human shape image that is a pixel region identified based on a speed vector from a range of pixels having direction-specific speed vectors indicated by the speed field distribution state, and that moves in the time-series continuous moving image A moving person detection process for detecting an image and its moving direction;
A person detection program for causing a computer to execute a duplicate person elimination process for comparing a detection time of the person shape image and a spatial coordinate and eliminating a person shape image detected redundantly between the time-series continuous moving images. . The person according to claim 16, for obtaining a space occupancy ratio of the person shape image in the measurement space range and causing a computer to execute a person separation process for estimating the number of persons included in the person shape image from the space occupancy ratio. Detection program. The person separation process is
When estimating the number of persons included in the person shape image from the space occupancy rate of the person shape image in the measurement space range, the space occupancy required per person to be compared with the space occupancy rate of the person shape image The person detection program according to claim 17 for causing a computer to execute a learning update process for learning and updating a rate. To claim 18 claim 16 for executing the measurement passes a predetermined measurement section set in the spatial range by counting the number of the person shape image people counting process for obtaining the number to the computer The person detection program described.

Images