JP2001045359A - Hand shake correcting method for solid-state image pickup device, and solid-state image pickup device - Google Patents
Hand shake correcting method for solid-state image pickup device, and solid-state image pickup deviceInfo
- Publication number
- JP2001045359A JP2001045359A JP11213719A JP21371999A JP2001045359A JP 2001045359 A JP2001045359 A JP 2001045359A JP 11213719 A JP11213719 A JP 11213719A JP 21371999 A JP21371999 A JP 21371999A JP 2001045359 A JP2001045359 A JP 2001045359A
- Authority
- JP
- Japan
- Prior art keywords
- image
- imaging device
- dimensional
- motion
- solid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 24
- 238000003384 imaging method Methods 0.000 claims description 93
- 230000002194 synthesizing effect Effects 0.000 claims description 46
- 238000006243 chemical reaction Methods 0.000 claims description 35
- 238000001514 detection method Methods 0.000 claims description 29
- 238000007796 conventional method Methods 0.000 abstract description 2
- 230000006866 deterioration Effects 0.000 abstract description 2
- 230000007547 defect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 14
- 230000015572 biosynthetic process Effects 0.000 description 13
- 238000003786 synthesis reaction Methods 0.000 description 11
- 238000001454 recorded image Methods 0.000 description 4
Landscapes
- Studio Devices (AREA)
Abstract
Description
【0001】[0001]
【発明の属する技術分野】本発明は固体撮像装置に関す
る。[0001] The present invention relates to a solid-state imaging device.
【0002】[0002]
【従来の技術】撮像装置は被写体からの光をレンズを使
用して撮像素子上に結像し、撮像素子のフォトダイオー
ドなどの光電変換素子で入射光を電気信号に変換する。
この光電変換が行われる時間周期は例えば標準テレビジ
ョンの場合には1フィールド期間に相当する1/60秒
である。この光電変換が行われている間に撮像装置が手
ぶれによって被写体に対して相対的に動いた場合、撮像
素子上には1/60秒間隔で移動しながら被写体画像が
結像される。この手ぶれによって引き起こされた動きを
補正するため、従来、撮像された映像信号を前記手ぶれ
による移動相当だけ水平と垂直の位置をずらして使用す
る方法が周知である。2. Description of the Related Art An image pickup apparatus forms an image of light from a subject on an image pickup device using a lens, and converts incident light into an electric signal by a photoelectric conversion element such as a photodiode of the image pickup device.
The time period in which the photoelectric conversion is performed is, for example, 1/60 second corresponding to one field period in the case of a standard television. If the imaging apparatus moves relative to the subject due to camera shake while the photoelectric conversion is being performed, a subject image is formed on the imaging device while moving at 1/60 second intervals. In order to correct the movement caused by the camera shake, a method of using a captured video signal by shifting the horizontal and vertical positions by the amount corresponding to the movement caused by the camera shake is conventionally known.
【0003】図8に手ぶれ補正方法による従来の撮像装
置の構成図を示す。FIG. 8 shows a configuration diagram of a conventional image pickup apparatus using a camera shake correction method.
【0004】撮像素子1から得られた出力信号は映像信
号処理回路2で輝度信号や色信号を形成するなどの信号
処理を行ってカラー映像信号が形成された後、このカラ
ー映像信号は画像メモリ3に書き込まれる。動き検出部
4は画像メモリ3に書き込まれたカラー映像信号から、
あるいは、動き検出素子5からの動き信号から手ぶれに
よって引き起こされた動きを検出する。動き検出部4か
ら得られた動き信号22は読み出し制御部23を制御
し、読み出し制御部23はこの動き信号を元に画像メモ
リ3に書き込まれているカラー映像信号の読み出し位置
を水平と垂直方向に手ぶれによって移動した後の位置か
ら読み出し、手ぶれを無くす。An output signal obtained from the image sensor 1 is subjected to signal processing such as formation of a luminance signal and a color signal in a video signal processing circuit 2 to form a color video signal. 3 is written. The motion detecting section 4 calculates the color video signal written in the image memory 3
Alternatively, a motion caused by camera shake is detected from a motion signal from the motion detection element 5. The motion signal 22 obtained from the motion detection unit 4 controls the read control unit 23, and the read control unit 23 sets the read position of the color video signal written in the image memory 3 based on the motion signal in the horizontal and vertical directions. To read from the position after the movement due to camera shake to eliminate camera shake.
【0005】図9はこの読み出し制御部23が前記動き
信号を元に画像メモリ3に書き込まれているカラー映像
信号を読み出す方法を示す図である。撮像素子の撮像可
能領域24はカメラの標準撮像領域25より広く取られ
ている。図8の画像メモリ3には図9の撮像素子の撮像
可能領域24に写っているすべての被写体画像のカラー
画像信号が書き込まれる。手ぶれのない静止状態では標
準撮像領域25に符号Aで示す被写体が撮像されている
が、この静止状態から手ぶれによって被写体画像が符号
Bで示す位置に移動して撮像された場合、読み出し制御
部23は動き信号22によって図9に手ぶれ補正時の撮
像領域26で示す範囲のカラー映像信号を画像メモリ3
から読み出す。FIG. 9 is a diagram showing a method in which the read control unit 23 reads a color video signal written in the image memory 3 based on the motion signal. The image-capturable area 24 of the image sensor is wider than the standard image-capturing area 25 of the camera. In the image memory 3 of FIG. 8, the color image signals of all the subject images appearing in the imageable area 24 of the image sensor of FIG. 9 are written. In the stationary state without camera shake, the subject indicated by the reference numeral A is imaged in the standard imaging area 25. When the subject image is moved from the stationary state to the position indicated by the reference sign B due to camera shake, the reading control unit 23 A color video signal in a range indicated by the imaging area 26 at the time of camera shake correction in FIG.
Read from
【0006】手ぶれ補正時の撮像領域26は、その撮像
領域内における符号Bで示す被写体画像の水平と垂直位
置が標準撮像領域25における符号Aで示す被写体画像
の水平と垂直位置と同じになるようになされている。In the image pickup area 26 at the time of camera shake correction, the horizontal and vertical positions of the subject image indicated by reference numeral B in the image pickup area are the same as the horizontal and vertical positions of the object image indicated by reference sign A in the standard image pickup area 25. Has been made.
【0007】この読み出しの結果、手ぶれ補正時の撮像
領域26内における符号Bで示す被写体画像の位置が、
標準撮像領域25に符号Aで示す被写体画像の位置と同
じ位置になり、手ぶれによる被写体の移動が補正されて
手ぶれによる動きを無くしている。As a result of this reading, the position of the subject image indicated by the symbol B in the imaging area 26 at the time of camera shake correction is
The position is the same as the position of the subject image indicated by the symbol A in the standard imaging area 25, and the movement of the subject due to camera shake is corrected, and the movement due to camera shake is eliminated.
【0008】なお、図8において、映像信号処理回路2
は読み出し制御部23の後にあっても同様である。ま
た、図10に示すように、画像メモリを用いないで、撮
像素子1の駆動信号の水平と垂直の開始位置を動きに対
応してずらすように駆動回路27を動き検出部4で制御
する撮像装置も周知である。In FIG. 8, the video signal processing circuit 2
Is the same even after the read control unit 23. Further, as shown in FIG. 10, without using an image memory, the motion detection unit 4 controls the drive circuit 27 so that the horizontal and vertical start positions of the drive signals of the image sensor 1 are shifted according to the motion. Devices are also well known.
【0009】[0009]
【発明が解決しようとする課題】上記従来の技術におい
ては、所定の周期で繰り返して撮像された画像の第N番
目のフィールドの画像に対して第N+1番目のフィール
ドの画像が手ぶれによって移動して撮像された場合に第
N+1番目の画像の読み出し位置を第N番目のフィール
ドの画像に水平と垂直の位置を合わせることで目的とす
る被写体画像が画面内で一定位置に止まるようにしてい
る。In the above-mentioned prior art, the image of the (N + 1) th field is moved by the camera shake with respect to the image of the Nth field of the image repeatedly picked up at a predetermined cycle. When the image is picked up, the read position of the (N + 1) th image is aligned horizontally and vertically with the image of the Nth field so that the target subject image remains at a fixed position in the screen.
【0010】しかしながら前記のとおり、被写体からの
入射光は撮像素子のフォトダイオードなどの光電変換素
子で光電変換しており、この光電変換が行われる時間周
期は例えば標準テレビジョンの場合には1フィールド期
間に相当する1/60秒である。手ぶれが発生している
場合にはこの光電変換が行われている期間に撮像素子上
に結像された被写体画像が動いているから、図9におい
て示した、符号Aで示す被写体画像が符号Bで示す被写
体画像位置に連続的に移動しつつ光電変換が行われ、符
号Bで示す被写体画像には画像の移動方向に“ぼけ”が
発生している。However, as described above, incident light from a subject is photoelectrically converted by a photoelectric conversion element such as a photodiode of an image pickup element, and a time period in which this photoelectric conversion is performed is, for example, one field in the case of a standard television. It is 1/60 second corresponding to the period. When camera shake occurs, the subject image formed on the image sensor moves while the photoelectric conversion is being performed. Therefore, the subject image indicated by reference symbol A shown in FIG. The photoelectric conversion is performed while continuously moving to the position of the subject image indicated by, and “blur” occurs in the moving direction of the image in the subject image indicated by the symbol B.
【0011】このため、目的とする被写体画像が画面内
で一定位置に止まるようにするだけの従来技術において
は、この“ぼけ”が発生した画像を各フィールドごとに
位置を合わせるだけであるから、1画面内の“ぼけ”を
補正することができず画質が劣化する欠点がある。For this reason, in the prior art in which the target subject image is merely stopped at a fixed position in the screen, the position of the image in which this "blur" has occurred is merely adjusted for each field. There is a disadvantage that "blur" in one screen cannot be corrected and image quality deteriorates.
【0012】また、固体撮像素子を使用した、被写体画
像を1画面のみ撮像する電子スチルカメラでは複数の画
像を利用して手ぶれを補正する従来技術は適用すること
が不可能である。したがって、同様に1画面内の“ぼ
け”を補正することができず画質劣化を補正することが
できない欠点がある。Further, in an electronic still camera that uses a solid-state image pickup device and captures only one image of a subject image, it is impossible to apply a conventional technique of correcting camera shake using a plurality of images. Accordingly, similarly, there is a disadvantage that “blur” in one screen cannot be corrected and image quality deterioration cannot be corrected.
【0013】本発明の目的は、手ぶれの動きによる画像
の“ぼけ”を無くす、固体撮像装置の手ぶれ補正方法お
よび固体撮像装置を提供することにある。An object of the present invention is to provide a method for correcting a camera shake of a solid-state image pickup device and a solid-state image pickup device, which eliminate "blurring" of an image due to the movement of a camera shake.
【0014】[0014]
【課題を解決するための手段】本発明の第1の、固体撮
像装置の手ぶれ補正方法は、複数の光電変換素子が水平
と垂直方法に所定の繰り返し周期で配列され、被写体か
らの入射光を所定の期間光電変換するように構成された
二次元固体撮像素子と、該二次元固体撮像素子の出力信
号から二次元の画像信号を形成する映像信号処理手段
と、該映像信号処理手段から得られる二次元画像信号を
記憶する画像メモリと、撮像装置の動きを検出する動き
検出手段と、該動き検出手段から得られる動き信号に基
づいて前記画像メモリに記憶された二次元画像信号を読
み出すように制御する読み出し制御手段と、この読み出
された二次元画像信号を複数回重ね合わせて合成する画
像重ね合わせ合成手段とを有する固体撮像装置の手ぶれ
補正方法であって、前記読み出し制御手段が前記二次元
固体撮像素子に結像された被写体画像が入射光を光電変
換する前記所定の期間において水平と垂直方向に前記光
電変換素子の複数個にわたって移動しつつ光電変換され
た移動範囲に相当する範囲の二次元画像信号を前記画像
メモリから読み出す段階と、前記画像重ね合わせ合成手
段が前記読み出された移動範囲に相当する範囲の二次元
画像信号を前記結像された被写体画像の移動方向と逆方
向となるように水平と垂直方向に1画素ずつずらして複
数回重ね合わせて加算合成して動きを打ち消した1画面
の画像を合成する段階を有する。According to a first aspect of the present invention, a plurality of photoelectric conversion elements are arranged in a horizontal and vertical manner at a predetermined repetition period, and the incident light from a subject is detected. A two-dimensional solid-state imaging device configured to perform photoelectric conversion for a predetermined period; a video signal processing unit that forms a two-dimensional image signal from an output signal of the two-dimensional solid-state imaging device; and a video signal processing unit. An image memory for storing a two-dimensional image signal, a motion detecting means for detecting a motion of the imaging device, and a two-dimensional image signal stored in the image memory based on a motion signal obtained from the motion detecting means. A camera-shake correction method for a solid-state imaging device, comprising: a reading control unit for controlling; and an image superimposing and synthesizing unit for superimposing and synthesizing the read two-dimensional image signal a plurality of times, The subject image formed on the two-dimensional solid-state imaging device is photoelectrically converted while moving across a plurality of the photoelectric conversion devices in the horizontal and vertical directions during the predetermined period in which the incident light is photoelectrically converted. Reading a two-dimensional image signal in a range corresponding to a moving range from the image memory, and the image superimposing / synthesizing unit forming a subject in which the two-dimensional image signal in a range corresponding to the read moving range is imaged; The method includes a step of synthesizing an image of one screen in which motion is canceled by superimposing a plurality of times by shifting one pixel in the horizontal and vertical directions so as to be in a direction opposite to the moving direction of the image and superimposing a plurality of times.
【0015】上述のように1画面の画像を合成すると、
被写体からの入射光が、手ぶれによって移動しながら固
体撮像素子の複数の光電変換素子にまたがって撮像され
て得られた画像信号が、手ぶれの動きと逆方向に引き戻
されながら1ヵ所に合成されるので、手ぶれによって発
生した“ぼけ”を補正することができる。すなわち、画
像メモリに記憶された1画面の画像を手ぶれの動きと逆
方向に引き戻しながら合成することで手ぶれを補正する
ので、電子スチルカメラのように1画面の画像しか撮像
しない撮像装置においても手ぶれを補正することができ
る。その結果、従来のTVカメラのような、目的とする
被写体画像を画面内で一定位置に止めるだけの従来技術
の問題点である1画面内の“ぼけ”を補正することがで
きないため画質劣化が起こる欠点が解決できる。When the image of one screen is synthesized as described above,
An image signal obtained by capturing an image from a plurality of photoelectric conversion elements of a solid-state imaging device while incident light from a subject moves due to camera shake is synthesized in one place while being pulled back in a direction opposite to the movement of camera shake. Therefore, it is possible to correct "blur" caused by camera shake. That is, since the camera shake is corrected by synthesizing the image of one screen stored in the image memory while pulling it back in the direction opposite to the motion of the camera shake, the camera shake may be reduced even in an image pickup apparatus such as an electronic still camera that only captures an image of one screen. Can be corrected. As a result, it is not possible to correct "blur" in one screen, which is a problem of the related art in which a target subject image is merely stopped at a fixed position in the screen, as in a conventional TV camera, so that image quality is deteriorated. The disadvantages that occur can be solved.
【0016】本発明の第2の、固体撮像装置の手ぶれ補
正方法は、複数の光電変換素子が水平と垂直方向に所定
の繰り返し周期で配列され、被写体からの入射光を所定
の期間光電変換するように構成された二次元固体撮像素
子と、該二次元固体撮像素子から得られる出力信号を記
憶する画像メモリと、撮像装置の動きを検出する動き検
出手段と、該動き検出手段から得られる動き信号に基づ
いて前記画像メモリに記憶された二次元画像信号を読み
出すように制御する読み出し制御手段と、前記読み出さ
れた二次元画像信号を複数回重ね合わせて合成する画像
重ね合わせ合成手段と、該画像重ね合わせ合成手段の出
力信号から二次元の画像信号を形成する映像信号処理手
段とを有する固体撮像装置の手ぶれ補正方法であって、
前記読み出し制御手段が前記二次元固体撮像素子に結像
された被写体画像が入射光を光電変換する前記所定の期
間において水平と垂直方向に前記光電変換素子の複数個
にわたって移動しつつ光電変換された移動範囲に相当す
る範囲の二次元画像信号を前記画像メモリから読み出す
段階と、前記画像重ね合わせ合成手段が前記読み出され
た移動範囲に相当する範囲の二次元画像信号を前記結像
された被写体画像の移動方向と逆方向となるように水平
と垂直方向に1画素ずつずらして複数回重ね合わせて加
算合成して動きを打ち消した1画面の画像を合成する段
階を有する。According to a second method of correcting a camera shake of a solid-state imaging device, a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition cycle, and photoelectrically convert incident light from a subject for a predetermined period. Two-dimensional solid-state imaging device configured as described above, an image memory for storing an output signal obtained from the two-dimensional solid-state imaging device, a motion detection unit for detecting a motion of the imaging device, and a motion obtained from the motion detection unit Reading control means for controlling to read a two-dimensional image signal stored in the image memory based on the signal, image superimposing and synthesizing means for superimposing and synthesizing the read two-dimensional image signal a plurality of times, A video signal processing means for forming a two-dimensional image signal from an output signal of the image superimposing / synthesizing means.
The subject image formed on the two-dimensional solid-state imaging device is photoelectrically converted while moving across a plurality of the photoelectric conversion devices in the horizontal and vertical directions in the predetermined period in which the readout control unit photoelectrically converts incident light. Reading a two-dimensional image signal in a range corresponding to a moving range from the image memory, and the image superimposing / synthesizing unit forming a subject in which the two-dimensional image signal in a range corresponding to the read moving range is imaged; The method includes a step of synthesizing an image of one screen in which motion is canceled by superimposing a plurality of times by shifting one pixel in the horizontal and vertical directions so as to be in a direction opposite to the moving direction of the image and superimposing a plurality of times.
【0017】本発明の第3の、固体撮像装置の手ぶれ補
正方法は、複数の光電変換素子が水平と垂直方向に所定
の繰り返し周期で配列され、被写体からの入射光を所定
の期間光電変換するように構成された二次元固体撮像素
子と、該二次元固体撮像素子から得られる出力信号を記
憶する画像メモリと、撮像装置の動きを検出する動き検
出手段と、該動き検出手段から得られる動き信号に基づ
いて前記画像メモリに記憶された二次元画像信号を読み
出すように制御する読み出し制御手段と、該読み出され
た出力信号から二次元画像信号を形成する映像信号処理
手段と、該映像信号処理手段から得られる二次元画像信
号を複数回重ね合わせて合成する画像重ね合わせ合成手
段とを有する固体撮装置の手ぶれ補正方法であって、前
記読み出し制御手段が前記二次元固体撮像素子に結像さ
れた被写体画像が入射光を光電変換する前記所定の期間
において水平と垂直方向に前記光電変換素子の複数個に
わたって移動しつつ光電変換された移動範囲に相当する
範囲の二次元画像信号を前記画像メモリから読み出す段
階と、前記画像重ね合わせ合成手段が前記読み出された
移動範囲に相当する範囲の二次元画像信号を前記結像さ
れた被写体画像の移動方向と逆方向となるように水平と
垂直方向に1画素ずつずらして複数回重ね合わせて加算
合成して動きを打ち消した1画面の画像を合成する段階
を有する。According to a third method of correcting camera shake of a solid-state imaging device of the present invention, a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and photoelectrically convert incident light from a subject for a predetermined period. Two-dimensional solid-state imaging device configured as described above, an image memory for storing an output signal obtained from the two-dimensional solid-state imaging device, a motion detection unit for detecting a motion of the imaging device, and a motion obtained from the motion detection unit Reading control means for controlling to read a two-dimensional image signal stored in the image memory based on the signal, video signal processing means for forming a two-dimensional image signal from the read output signal, and the video signal An image superimposing and synthesizing means for superimposing and synthesizing a two-dimensional image signal obtained from the processing means a plurality of times, wherein Corresponds to a moving range where the subject image formed on the two-dimensional solid-state imaging device is photoelectrically converted while moving over a plurality of the photoelectric conversion devices in the horizontal and vertical directions in the predetermined period in which incident light is photoelectrically converted. Reading a two-dimensional image signal in a range to be moved from the image memory, and moving the two-dimensional image signal in a range corresponding to the read moving range by the image superimposing / synthesizing unit in the moving direction of the formed subject image. And superimposing a plurality of times by shifting one pixel in the horizontal and vertical directions so as to be in the opposite direction, and adding and synthesizing them to synthesize an image of one screen in which motion is canceled.
【0018】本発明の第4の、固体撮像装置の手ぶれ補
正方法は、複数の光電変換素子が水平と垂直方向に所定
の繰り返し周期で配列され、被写体からの入射光を所定
の期間光電変換するように構成された二次元固体撮像素
子と、該二次元固体撮像素子を駆動する駆動手段と、前
記二次元固体撮像素子の出力信号から二次元の画像信号
を形成する映像信号処理手段と、該映像信号処理手段か
ら得られた二次元画像信号を記憶するNラインメモリ
(Nは1以上の整数)と、前記Nラインメモリから読み
出された二次元画像を複数回重ね合わせて合成する画像
重ね合わせ合成手段と、撮像装置の動きを検出し、該動
きに基づいて前記駆動手段および前記読み出し制御手段
を制御する動き検出手段を有する固体撮像装置の手ぶれ
補正方法であって、前記駆動手段が前記二次元固体撮像
素子に結像された被写体画像が入射光を光電変換する前
記所定の期間において水平と垂直方向に前記光電変換素
子の複数個にわたって移動しつつ光電変換された移動範
囲に相当する範囲の二次元画像信号を前記二次元固体撮
像素子から読み出すように該二次元固体撮像素子に印加
する垂直と水平の駆動信号の駆動開始時刻を可変にする
段階と、前記読み出し制御手段が前記Nラインメモリの
読み出し開始時刻を同様に可変して前記二次元画像信号
を読み出す段階と、前記画像重ね合わせ合成手段が前記
読み出された移動範囲に相当する範囲の二次元画像信号
を前記結像された被写体画像の移動方向と逆方向となる
ように水平と垂直方向に1画素ずつずらして複数回重ね
合わせて加算合成して動きを打ち消した1画素の画像を
合成する段階を有する。In a fourth aspect of the present invention, a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and photoelectrically convert incident light from a subject for a predetermined period. A two-dimensional solid-state image sensor configured as described above, driving means for driving the two-dimensional solid-state image sensor, video signal processing means for forming a two-dimensional image signal from an output signal of the two-dimensional solid-state image sensor, An N-line memory (N is an integer of 1 or more) for storing a two-dimensional image signal obtained from the video signal processing means, and an image overlay for combining a plurality of two-dimensional images read from the N-line memory for synthesis A camera-shake correction method for a solid-state imaging device having a combination synthesis unit and a movement detection unit that detects a movement of the imaging device and controls the driving unit and the readout control unit based on the movement. The movement in which the driving means moves across a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period in which the subject image formed on the two-dimensional solid-state imaging element photoelectrically converts incident light. Varying the drive start times of vertical and horizontal drive signals applied to the two-dimensional solid-state imaging device so that two-dimensional image signals in a range corresponding to the range are read from the two-dimensional solid-state imaging device; and Means for reading out the two-dimensional image signal by similarly changing the reading start time of the N-line memory; and means for combining the two-dimensional image signal in the range corresponding to the read-out moving range by the image superposition / synthesis means. The movement is canceled by adding and combining a plurality of times by shifting one pixel at a time in the horizontal and vertical directions so as to be in a direction opposite to the moving direction of the imaged subject image. Comprising the step of synthesizing the image of one pixel.
【0019】[0019]
【発明の実施の形態】次に、本発明の実施の形態につい
て図面を参照して説明する。Next, embodiments of the present invention will be described with reference to the drawings.
【0020】図1を参照すると、本発明の第1の実施形
態の固体撮像装置は撮像素子1と映像信号処理回路2と
画像メモリ3と動き検出部4と動き検出素子5と読み出
し制御部6と画像重ね合せ合成部7で構成されている。Referring to FIG. 1, a solid-state imaging device according to a first embodiment of the present invention includes an imaging device 1, a video signal processing circuit 2, an image memory 3, a motion detection unit 4, a motion detection device 5, and a read control unit 6. And an image superposition / synthesis unit 7.
【0021】二次元CCD固体撮像素子などの撮像素子
1から得られた出力信号に対して映像信号処理回路2で
白黒映像信号を形成する信号処理を行って白黒映像信号
が形成され、あるいは、輝度信号や色信号を形成するな
どの信号処理を行ってカラー映像信号が形成された後、
これらの映像信号は画像メモリ3に書き込まれる。動き
検出部4は映像信号処理回路2の出力や画像メモリ3に
書き込まれた映像信号や動き検出素子5の1個か2個以
上の組み合わせから得られる動き信号から手ぶれによっ
て引き起こされた動きを検出する。動き検出部4から得
られた動き信号は読み出し制御部6を制御し、読み出し
制御部6はこの動き信号を元に画像メモリ3に書き込ま
れている映像信号を水平と垂直方向に手ぶれによって移
動した後の位置から読み出す。この時、手ぶれによる移
動が複数画素にわたる場合には前記複数画素にわたっ
て、移動した水平と垂直方向の手ぶれ方向に1画素ごと
の複数枚の画面を読み出す。A video signal processing circuit 2 performs signal processing for forming a black-and-white video signal on an output signal obtained from an image pickup device 1 such as a two-dimensional CCD solid-state image pickup device to form a black-and-white video signal. After a color image signal is formed by performing signal processing such as forming signals and color signals,
These video signals are written to the image memory 3. The motion detection unit 4 detects a motion caused by camera shake from an output of the video signal processing circuit 2, a video signal written in the image memory 3, or a motion signal obtained from one or more combinations of the motion detection elements 5. I do. The motion signal obtained from the motion detection unit 4 controls the read control unit 6, and the read control unit 6 moves the video signal written in the image memory 3 in the horizontal and vertical directions by shaking based on the motion signal. Read from the later position. At this time, if the movement due to camera shake extends over a plurality of pixels, a plurality of screens for each pixel are read out in the horizontal and vertical camera shake directions across the plurality of pixels.
【0022】図2と図3はこの読み出し制御部6が前記
動き信号を元に画像メモリ3に書き込まれている映像信
号を読み出す方法を示す図である。図2において、元の
被写体画像8がカメラ9のレンズ10によって撮像素子
の撮像領域11に結像している。次に、カメラ9が手ぶ
れによって記号Aで示す方向に動いた場合、撮像領域1
1に結像した画像は、図において振れがある時の撮像画
像12で示すとおり、手ぶれによって移動しながら撮像
される。移動距離lは、l=v×t(vは撮像領域上で
の実効的な移動速度)となっている。FIGS. 2 and 3 are diagrams showing a method in which the read control unit 6 reads a video signal written in the image memory 3 based on the motion signal. In FIG. 2, an original subject image 8 is formed on an imaging area 11 of an imaging element by a lens 10 of a camera 9. Next, when the camera 9 moves in the direction indicated by the symbol A due to camera shake, the imaging region 1
The image formed in 1 is captured while moving due to camera shake, as shown in a captured image 12 when there is a shake in the figure. The moving distance 1 is 1 = v × t (v is an effective moving speed on the imaging area).
【0023】次に、図3は図2の手ぶれを起こした状態
で撮像された画像が画像メモリ3に書き込まれた状態を
示す。図3において、画像メモリ3の水平アドレス1,
2,3,…,n,mと垂直アドレスA,B,C,…,
N,Mは撮像素子の1画素に対応している。手ぶれを起
こして撮像された記録画像14は、被写体画像の矢印a
で示す点が矢印bで示す点まで移動しながら撮像され、
記録されている。読み出し制御部6はこの画像メモリ3
に書き込まれている1画面の画像を複数回読み出し、画
像重ね合せ合成部7で重ね合せて合成する。図4はこの
重ね合せを示す図である。図3と図4を参照すると、図
3において、矢印aで示す点の信号を読み出し制御部6
が水平アドレスn、垂直アドレスNから読み出し、同様
に、矢印bで示す点の信号を水平アドレスm、垂直アド
レスMから読み出す。読み出された信号は次に画像重ね
合せ合成部7で矢印aで示す点に矢印bで示す点を一致
させるように加算して合成する。Next, FIG. 3 shows a state in which the image captured in the state where the camera shake has occurred in FIG. In FIG. 3, horizontal addresses 1 and 2 of the image memory 3 are shown.
, N, m and vertical addresses A, B, C,.
N and M correspond to one pixel of the image sensor. Recorded image 14 captured with camera shake is indicated by arrow a of the subject image.
Is taken while moving to the point indicated by arrow b,
Has been recorded. The read control unit 6 controls the image memory 3
Are read a plurality of times, and are superimposed and synthesized by the image superposition / combination unit 7. FIG. 4 is a diagram showing this superposition. Referring to FIG. 3 and FIG. 4, the signal at the point indicated by the arrow a in FIG.
Read from the horizontal address n and the vertical address N, and similarly read the signal at the point indicated by the arrow b from the horizontal address m and the vertical address M. Next, the read signals are added and combined by the image superimposing / combining unit 7 so that the point indicated by the arrow a coincides with the point indicated by the arrow b.
【0024】図4(a)にこの重ね合せを示す。図4
(a)において、画像メモリ記録画像15は矢印aで示
す点を基準にした座標から読み出された被写体画像A、
重ね合せる画像16が矢印bで示す点を基準にした座標
から読み出された被写体画像Bを示す。前記のとおり、
この二つの読み出された被写体画像AとBの信号を重ね
合せ合成部7で矢印aで示す点に矢印bで示す点を一致
させるように加算して合成すると、図4(b)に示すよ
うに元の被写体の同一の点が重ね合わされた画像17が
得られ、手ぶれによる被写体の移動が補正されて手ぶれ
の動きによる画像の“ぼけ”を無くすことができる。FIG. 4A shows this superposition. FIG.
In (a), an image memory recorded image 15 is a subject image A read from coordinates based on a point indicated by an arrow a,
The image 16 to be superimposed shows the subject image B read from the coordinates based on the point indicated by the arrow b. As mentioned above,
When the two read out signals of the subject images A and B are added and combined by the superposition combining unit 7 so that the point indicated by the arrow a and the point indicated by the arrow b coincide with each other, the result is shown in FIG. As described above, the image 17 in which the same point of the original subject is superimposed is obtained, and the movement of the subject due to camera shake is corrected, so that "blur" of the image due to the movement of camera shake can be eliminated.
【0025】図5を参照すると、本発明の第2の実施形
態の固体撮像装置は撮像素子1と映像信号処理回路2と
動き検出部4と動き検出素子5と駆動回路18とNライ
ンメモリ19と読み出し制御部20と画像重ね合せ合成
部21で構成されている。Referring to FIG. 5, a solid-state imaging device according to a second embodiment of the present invention includes an image sensor 1, a video signal processing circuit 2, a motion detector 4, a motion detector 5, a drive circuit 18, and an N-line memory 19. And a read control unit 20 and an image superimposing / synthesizing unit 21.
【0026】二次元CCD固体撮像素子などの撮像素子
1から得られた出力信号に対して映像信号処理回路2で
白黒映像信号を形成する信号処理を行って白黒映像信号
が形成された後か、あるいは、輝度信号や色信号を形成
するなどの信号処理を行ってカラー映像信号が形成され
た後、これらの映像信号はNラインメモリ19に書き込
まれる。動き検出部4は映像信号処理回路2の出力やN
ラインメモリ19に書き込まれた映像信号や動き検出素
子5の1個か2個以上の組み合わせから得られる動き信
号から手ぶれによって引き起こされた動きを検出する。
動き検出部4から得られた動き信号は撮像素子1を駆動
する駆動回路18を制御し、手ぶれによる被写体画像の
移動を補正するように撮像素子1から映像信号を読み出
すための水平と垂直の駆動信号の開始時刻を進め、ある
いは遅らせる。これと同時に動き検出部4は読み出し制
御部20を制御し、読み出し制御部20はこの動き信号
を元にNラインメモリ19に書き込まれている映像信号
の読み出し位置を水平と垂直方向に手ぶれによって移動
した後の位置から読み出す。以後第1の実施形態と同様
に、読み出された映像信号は画像重ね合せ合成部21で
元の被写体の同一の点が重ね合わされるように合成し、
手ぶれによる被写体の移動が補正して手ぶれの動きによ
る画像の“ぼけ”を無くす。After the output signal obtained from the image pickup device 1 such as a two-dimensional CCD solid-state image pickup device is subjected to signal processing for forming a black-and-white video signal by the video signal processing circuit 2, or Alternatively, after a color video signal is formed by performing signal processing such as forming a luminance signal and a color signal, these video signals are written to the N-line memory 19. The motion detection unit 4 outputs the output of the video signal processing circuit 2 or N
A motion caused by camera shake is detected from a video signal written in the line memory 19 or a motion signal obtained from a combination of one or more of the motion detection elements 5.
The motion signal obtained from the motion detection unit 4 controls a driving circuit 18 for driving the image sensor 1 and performs horizontal and vertical driving for reading a video signal from the image sensor 1 so as to correct movement of a subject image due to camera shake. Advance or delay the start time of the signal. At the same time, the motion detection unit 4 controls the read control unit 20, and the read control unit 20 moves the read position of the video signal written in the N-line memory 19 based on the motion signal in the horizontal and vertical directions due to camera shake. Read from the position after performing. Thereafter, in the same manner as in the first embodiment, the read video signals are synthesized by the image superposition / combination unit 21 so that the same point of the original subject is superimposed.
The movement of the subject due to the camera shake is corrected to eliminate the “blur” of the image due to the camera shake.
【0027】図6を参照すると、本発明の第3の実施形
態の固体撮像装置は撮像素子1と画像メモリ3と動き検
出部4と動き検出素子5と読み出し制御部6と画像重ね
合せ合成部7と映像信号処理回路2で構成されている。Referring to FIG. 6, a solid-state image pickup device according to a third embodiment of the present invention comprises an image pickup device 1, an image memory 3, a motion detection unit 4, a motion detection device 5, a read control unit 6, and an image superposition / synthesis unit. 7 and the video signal processing circuit 2.
【0028】二次元CCD固体撮像素子などの撮像素子
1から得られた出力信号は画像メモリ3に書き込まれ
る。動き検出部4は画像メモリ3に書き込まれた映像信
号や動き検出素子5の1個か2個以上の組み合わせから
得られる動き信号から手ぶれによって引き起こされた動
きを検出する。動き検出部4から得られた動き信号は読
み出し制御部6を制御し、読み出し制御部6はこの動き
信号を元に画像メモリ3に書き込まれている映像信号の
読み出し位置を水平と垂直方向に手ぶれによって移動し
た後の位置から読み出す。この時、手ぶれによる移動が
複数画素にわたる場合には前記複数画素にわたって移動
した水平と垂直方向の手ぶれ方向に1画素ごとの複数枚
の画面を読み出す。読み出された信号は画像重ね合せ合
成部7で重ね合わせて合成する。この読み出しと重ね合
せ合成は第1の実施形態と同様である。An output signal obtained from an image pickup device 1 such as a two-dimensional CCD solid-state image pickup device is written into an image memory 3. The motion detection unit 4 detects a motion caused by camera shake from a video signal written in the image memory 3 or a motion signal obtained from one or more combinations of the motion detection elements 5. The motion signal obtained from the motion detector 4 controls the read controller 6, and the read controller 6 moves the read position of the video signal written in the image memory 3 in the horizontal and vertical directions based on the motion signal. Read from the position after moving. At this time, if the movement due to camera shake extends over a plurality of pixels, a plurality of screens for each pixel are read out in the horizontal and vertical camera shake directions that have moved over the plurality of pixels. The read signals are superimposed and synthesized by the image superimposition synthesis unit 7. The reading and the superimposing synthesis are the same as in the first embodiment.
【0029】次に、画像重ね合せ合成部7で重ね合せて
合成された信号から映像信号処理回路2で白黒映像信号
を形成する信号処理や、あるいは輝度信号や色信号を形
成するなどの信号処理を行って、白黒映像信号やカラー
映像信号を形成する。Next, a signal processing for forming a black-and-white video signal in the video signal processing circuit 2 from the signal synthesized and superimposed by the image superimposing / synthesizing section 7 or a signal processing for forming a luminance signal and a color signal, for example. To form a monochrome video signal or a color video signal.
【0030】形成された白黒映像信号やカラー映像信号
は第1の実施形態と同様に元の被写体の同一の点が重ね
合わされた画像が得られ、手ぶれによる被写体の移動が
補正されて手ぶれの動きによる画像の“ぼけ”を無くす
ことができる。As in the first embodiment, an image in which the same point of the original subject is superimposed is obtained from the formed black-and-white video signal and color video signal, and the movement of the subject due to camera shake is corrected and the movement of camera shake is corrected. "Blurring" of the image due to the above can be eliminated.
【0031】図7を参照すると、本発明の第4の実施形
態の固体撮像装置は撮像素子1と画像メモリ3と映像信
号処理回路2と動き検出部4と動き検出素子5と読み出
し制御部6と画像重ね合せ合成部7で構成されている。Referring to FIG. 7, a solid-state image pickup device according to a fourth embodiment of the present invention comprises an image pickup device 1, an image memory 3, a video signal processing circuit 2, a motion detector 4, a motion detector 5, and a read controller 6. And an image superposition / synthesis unit 7.
【0032】二次元CCD固体撮像素子などの撮像素子
1から得られた出力信号は画像メモリ3に書き込まれ
る。動き検出部4は画像メモリ3に書き込まれた映像信
号や動き検出素子5の1個か2個以上の組み合わせから
得られる動き信号から手ぶれによって引き起こされた動
きを検出する。動き検出部4から得られた動き信号は読
み出し制御部6を制御し、読み出し制御部6はこの動き
信号を元に画像メモリ3に書き込まれている映像信号の
読み出し位置を水平と垂直方向に手ぶれによって移動し
た後の位置から読み出す。この時、手ぶれによる移動が
複数画素にわたる場合には前記複数画素にわたって移動
した水平と垂直方向の手ぶれ方向に1画素ごとの複数枚
の画面を読み出す。An output signal obtained from an image pickup device 1 such as a two-dimensional CCD solid-state image pickup device is written into an image memory 3. The motion detection unit 4 detects a motion caused by camera shake from a video signal written in the image memory 3 or a motion signal obtained from one or more combinations of the motion detection elements 5. The motion signal obtained from the motion detector 4 controls the read controller 6, and the read controller 6 moves the read position of the video signal written in the image memory 3 in the horizontal and vertical directions based on the motion signal. Read from the position after moving. At this time, if the movement due to camera shake extends over a plurality of pixels, a plurality of screens for each pixel are read out in the horizontal and vertical camera shake directions that have moved over the plurality of pixels.
【0033】次に、読み出された信号から映像信号処理
回路2で白黒映像信号を形成する信号処理や、輝度信号
や色信号を形成するなどの信号処理を行って、白黒映像
信号やカラー映像信号を形成する。この白黒映像信号や
カラー映像信号は次に画像重ね合せ合成部7で重ね合せ
て合成される。画像メモリ3からの読み出しと画像重ね
合せ合成部7での重ね合せ合成は第1の実施形態と同様
である。Next, the video signal processing circuit 2 performs signal processing such as forming a black-and-white video signal and signal processing such as formation of a luminance signal and a color signal from the read signal to obtain a black-and-white video signal and a color video signal. Form a signal. The black-and-white video signal and the color video signal are then superimposed and synthesized by the image superimposing and synthesizing unit 7. The reading from the image memory 3 and the superimposition synthesis by the image superposition / synthesis unit 7 are the same as in the first embodiment.
【0034】重ね合せ合成された白黒映像信号やカラー
映像信号から第1の実施形態と同様に元の被写体の同一
の点が重ね合わされた画像が得られ、手ぶれによる被写
体の移動が補正されて手ぶれの動きによる画像の“ぼ
け”を無くすことができる。As in the first embodiment, an image in which the same points of the original subject are superimposed is obtained from the superimposed black and white video signal and the color video signal, and the movement of the subject due to camera shake is corrected. "Blur" of the image due to the movement of the image can be eliminated.
【0035】[0035]
【発明の効果】以上説明したように本発明によれば、手
ぶれによって移動しながら撮像された画像の中の元の被
写体の同一点の信号を重ね合せ合成手段で一致させるよ
うに加算して合成するため、元の被写体の同一の点が重
ね合わされた画像が得られ、手ぶれによる被写体の移動
が引き戻されるように補正されて手ぶれの動きによる画
像の“ぼけ”を無くすことができる。As described above, according to the present invention, signals of the same point of the original subject in an image taken while moving due to camera shake are added and combined by the superimposing means so as to match. Therefore, an image in which the same point of the original subject is superimposed is obtained, and the movement of the subject due to the camera shake is corrected so as to be pulled back, so that "blur" of the image due to the movement of the camera shake can be eliminated.
【0036】さらに、従来技術では補正することが不可
能であった1画面だけの画像を撮像する電子スチルカメ
ラにおいても同様に手ぶれが補正でき、画質が改善され
た電子スチルカメラが実現できる。Further, even in an electronic still camera that captures an image of only one screen, which cannot be corrected by the prior art, camera shake can be similarly corrected, and an electronic still camera with improved image quality can be realized.
【図1】本発明の第1の実施形態による固体撮像装置の
構成図である。FIG. 1 is a configuration diagram of a solid-state imaging device according to a first embodiment of the present invention.
【図2】手ぶれ時の撮像を示す図である。FIG. 2 is a diagram illustrating imaging when camera shake occurs.
【図3】読み出し制御部6が画像メモリ3に書き込まれ
ている映像信号を読み出す方法を示す図である。FIG. 3 is a diagram illustrating a method in which a read control unit reads a video signal written in an image memory.
【図4】画像重ね合せ合成部7の重ね合せ動作を示す図
である。FIG. 4 is a diagram illustrating a superposition operation of an image superposition / synthesis unit 7;
【図5】本発明の第2の実施形態による固体撮像装置の
構成図である。FIG. 5 is a configuration diagram of a solid-state imaging device according to a second embodiment of the present invention.
【図6】本発明の第3の実施形態による固体撮像装置の
構成図である。FIG. 6 is a configuration diagram of a solid-state imaging device according to a third embodiment of the present invention.
【図7】本発明の第4の実施形態による固体撮像装置の
構成図である。FIG. 7 is a configuration diagram of a solid-state imaging device according to a fourth embodiment of the present invention.
【図8】従来の撮像装置の構成図である。FIG. 8 is a configuration diagram of a conventional imaging device.
【図9】図8に示す従来の撮像装置の動作を示す図であ
る。FIG. 9 is a diagram showing the operation of the conventional imaging device shown in FIG.
【図10】従来の撮像装置の他の例の構成図である。FIG. 10 is a configuration diagram of another example of a conventional imaging device.
1 撮像素子 2 映像信号処理回路 3 画像メモリ 4 動き検出部 5 動き検出素子 6 読み出し制御部 7 画像重ね合せ部 8 元の被写体画像 9 カメラ 10 レンズ 11 撮像領域 12 振れがある時の撮像画像 13 画像メモリ 14 記録画像 15 画像メモリ記録画像 16 重ね合せる画像位置 17 重ね合せた画像 18 駆動回路 19 Nラインメモリ 20 読み出し制御部 21 画像重ね合せ部 22 動き信号 23 読み出し制御部 24 撮像素子の撮像可能領域 25 カメラの標準撮像領域 26 手ぶれ補正時の撮像領域 27 駆動回路 REFERENCE SIGNS LIST 1 imaging device 2 video signal processing circuit 3 image memory 4 motion detection unit 5 motion detection device 6 readout control unit 7 image superposition unit 8 original subject image 9 camera 10 lens 11 imaging region 12 captured image with shake 13 image Memory 14 Recorded image 15 Image memory recorded image 16 Image position to be superimposed 17 Image superimposed 18 Drive circuit 19 N-line memory 20 Read control unit 21 Image superimposition unit 22 Motion signal 23 Read control unit 24 Image sensing area of image sensor 25 Standard imaging area of camera 26 Imaging area for camera shake correction 27 Drive circuit
Claims (14)
所定の繰り返し周期で配列され、被写体からの入射光を
所定の期間光電変換するように構成された二次元固体撮
像素子と、該二次元固体撮像素子の出力信号から二次元
の画像信号を形成する映像信号処理手段と、該映像信号
処理手段から得られる二次元画像信号を記憶する画像メ
モリと、撮像装置の動きを検出する動き検出手段と、該
動き検出手段から得られる動き信号に基づいて前記画像
メモリに記憶された二次元画像信号を読み出すように制
御する読み出し制御手段と、この読み出された二次元画
像信号を複数回重ね合わせて合成する画像重ね合わせ合
成手段とを有する固体撮像装置の手ぶれ補正方法であっ
て、 前記読み出し制御手段が前記二次元固体撮像素子に結像
された被写体画像が入射光を光電変換する前記所定の期
間において水平と垂直方向に前記光電変換素子の複数個
にわたって移動しつつ光電変換された移動範囲に相当す
る範囲の二次元画像信号を前記画像メモリから読み出す
段階と、前記画像重ね合わせ合成手段が前記読み出され
た移動範囲に相当する範囲の二次元画像信号を前記結像
された被写体画像の移動方向と逆方向となるように水平
と垂直方向に1画素ずつずらして複数回重ね合わせて加
算合成して動きを打ち消した1画面の画像を合成する段
階を有する、固体撮像装置の手ぶれ補正方法。1. A two-dimensional solid-state imaging device, comprising: a plurality of photoelectric conversion elements arranged in a horizontal and vertical direction at a predetermined repetition period; and a photoelectric conversion element configured to photoelectrically convert incident light from a subject for a predetermined period; Video signal processing means for forming a two-dimensional image signal from an output signal of a two-dimensional solid-state image sensor, an image memory for storing a two-dimensional image signal obtained from the video signal processing means, and motion detection for detecting a motion of the imaging device Means, read control means for controlling to read a two-dimensional image signal stored in the image memory based on a motion signal obtained from the motion detecting means, and superimposing the read two-dimensional image signal a plurality of times. A camera-shake correction method for a solid-state imaging device, comprising: an image superimposing / synthesizing unit configured to combine the two-dimensional solid-state imaging device; The image is read from the image memory in a range corresponding to a photoelectrically converted moving range while moving over a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period in which the incident light is photoelectrically converted. And a step in which the image superimposing and synthesizing means shifts the two-dimensional image signal in a range corresponding to the read moving range by 1 in the horizontal and vertical directions so as to be in the opposite direction to the moving direction of the formed subject image. A camera-shake correction method for a solid-state imaging device, comprising: synthesizing an image of one screen in which motion is canceled by superposing a plurality of times by shifting pixels and adding and synthesizing them.
所定の繰り返し周期で配列され、被写体からの入射光を
所定の期間光電変換するように構成された二次元固体撮
像素子と、該二次元固体撮像素子から得られる出力信号
を記憶する画像メモリと、撮像装置の動きを検出する動
き検出手段と、該動き検出手段から得られる動き信号に
基づいて前記画像メモリに記憶された二次元画像信号を
読み出すように制御する読み出し制御手段と、前記読み
出された二次元画像信号を複数回重ね合わせて合成する
画像重ね合わせ合成手段と、該画像重ね合わせ合成手段
の出力信号から二次元の画像信号を形成する映像信号処
理手段とを有する固体撮像装置の手ぶれ補正方法であっ
て、 前記読み出し制御手段が前記二次元固体撮像素子に結像
された被写体画像が入射光を光電変換する前記所定の期
間において水平と垂直方向に前記光電変換素子の複数個
にわたって移動しつつ光電変換された移動範囲に相当す
る範囲の二次元画像信号を前記画像メモリから読み出す
段階と、前記画像重ね合わせ合成手段が前記読み出され
た移動範囲に相当する範囲の二次元画像信号を前記結像
された被写体画像の移動方向と逆方向となるように水平
と垂直方向に1画素ずつずらして複数回重ね合わせて加
算合成して動きを打ち消した1画面の画像を合成する段
階を有する、固体撮像装置の手ぶれ補正方法。2. A two-dimensional solid-state imaging device, wherein a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and configured to photoelectrically convert incident light from a subject for a predetermined period. An image memory for storing an output signal obtained from a three-dimensional solid-state imaging device; a motion detecting means for detecting a motion of the imaging device; and a two-dimensional image stored in the image memory based on a motion signal obtained from the motion detecting means. Reading control means for controlling a signal to be read, image superimposing / combining means for superimposing and combining the read two-dimensional image signals a plurality of times, and a two-dimensional image from an output signal of the image superimposing / combining means A camera-shake correction method for a solid-state imaging device, comprising: a video signal processing unit that forms a signal; The image is read from the image memory in a range corresponding to a photoelectrically converted moving range while moving over a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period in which the incident light is photoelectrically converted. And a step in which the image superimposing and synthesizing means shifts the two-dimensional image signal in a range corresponding to the read moving range by 1 in the horizontal and vertical directions so as to be in the opposite direction to the moving direction of the formed subject image. A camera-shake correction method for a solid-state imaging device, comprising: synthesizing an image of one screen in which motion is canceled by superposing a plurality of times by shifting pixels and adding and synthesizing them.
所定の繰り返し周期で配列され、被写体からの入射光を
所定の期間光電変換するように構成された二次元固体撮
像素子と、該二次元固体撮像素子から得られる出力信号
を記憶する画像メモリと、撮像装置の動きを検出する動
き検出手段と、該動き検出手段から得られる動き信号に
基づいて前記画像メモリに記憶された二次元画像信号を
読み出すように制御する読み出し制御手段と、該読み出
された出力信号から二次元画像信号を形成する映像信号
処理手段と、該映像信号処理手段から得られる二次元画
像信号を複数回重ね合わせて合成する画像重ね合わせ合
成手段とを有する固体撮装置の手ぶれ補正方法であっ
て、 前記読み出し制御手段が前記二次元固体撮像素子に結像
された被写体画像が入射光を光電変換する前記所定の期
間において水平と垂直方向に前記光電変換素子の複数個
にわたって移動しつつ光電変換された移動範囲に相当す
る範囲の二次元画像信号を前記画像メモリから読み出す
段階と、前記画像重ね合わせ合成手段が前記読み出され
た移動範囲に相当する範囲の二次元画像信号を前記結像
された被写体画像の移動方向と逆方向となるように水平
と垂直方向に1画素ずつずらして複数回重ね合わせて加
算合成して動きを打ち消した1画面の画像を合成する手
段を有する、固体撮像装置の手ぶれ補正方法。3. A two-dimensional solid-state imaging device, wherein a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and are configured to photoelectrically convert incident light from a subject for a predetermined period. An image memory for storing an output signal obtained from a three-dimensional solid-state imaging device; a motion detecting means for detecting a motion of the imaging device; and a two-dimensional image stored in the image memory based on a motion signal obtained from the motion detecting means. Reading control means for controlling a signal to be read, video signal processing means for forming a two-dimensional image signal from the read output signal, and superimposing a two-dimensional image signal obtained from the video signal processing means a plurality of times A camera-shake correction method for a solid-state imaging device having image superimposing and synthesizing means for synthesizing the subject image. Reading a two-dimensional image signal from the image memory in a range corresponding to a photoelectrically converted movement range while moving over a plurality of the photoelectric conversion elements in the horizontal and vertical directions during the predetermined period in which incident light is photoelectrically converted. And the image superimposing and synthesizing means converts a two-dimensional image signal in a range corresponding to the read moving range by one pixel in the horizontal and vertical directions so as to be in a direction opposite to a moving direction of the formed subject image. A camera-shake correction method for a solid-state imaging device, comprising: means for combining a plurality of images shifted one by one and superimposing a plurality of times to add and combine the images to cancel one motion.
所定の繰り返し周期で配列され、被写体からの入射光を
所定の期間光電変換するように構成された二次元固体撮
像素子と、該二次元固体撮像素子を駆動する駆動手段
と、前記二次元固体撮像素子の出力信号から二次元の画
像信号を形成する映像信号処理手段と、該映像信号処理
手段から得られた二次元画像信号を記憶するNラインメ
モリ(Nは1以上の整数)と、前記Nラインメモリから
読み出された二次元画像を複数回重ね合わせて合成する
画像重ね合わせ合成手段と、撮像装置の動きを検出し、
該動きに基づいて前記駆動手段および前記読み出し制御
手段を制御する動き検出手段を有する固体撮像装置の手
ぶれ補正方法であって、 前記駆動手段が前記二次元固体撮像素子に結像された被
写体画像が入射光を光電変換する前記所定の期間におい
て水平と垂直方向に前記光電変換素子の複数個にわたっ
て移動しつつ光電変換された移動範囲に相当する範囲の
二次元画像信号を前記二次元固体撮像素子から読み出す
ように該二次元固体撮像素子に印加する垂直と水平の駆
動信号の駆動開始時刻を可変にする段階と、前記読み出
し制御手段が前記Nラインメモリの読み出し開始時刻を
同様に可変して前記二次元画像信号を読み出す段階と、
前記画像重ね合わせ合成手段が前記読み出された移動範
囲に相当する範囲の二次元画像信号を前記結像された被
写体画像の移動方向と逆方向となるように水平と垂直方
向に1画素ずつずらして複数回重ね合わせて加算合成し
て動きを打ち消した1画素の画像を合成する段階を有す
る、固体撮像装置の手ぶれ補正方法。4. A two-dimensional solid-state imaging device, wherein a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and configured to photoelectrically convert incident light from a subject for a predetermined period. Driving means for driving a two-dimensional solid-state image sensor, video signal processing means for forming a two-dimensional image signal from an output signal of the two-dimensional solid-state image sensor, and storing a two-dimensional image signal obtained from the video signal processing means An N-line memory (N is an integer of 1 or more), an image superimposing / synthesizing means for superimposing and synthesizing a two-dimensional image read from the N-line memory a plurality of times, and detecting a motion of the imaging device;
A camera-shake correction method for a solid-state imaging device having a motion detection unit that controls the driving unit and the read-out control unit based on the motion, wherein the driving unit forms a subject image formed on the two-dimensional solid-state imaging device. From the two-dimensional solid-state imaging device, a two-dimensional image signal in a range corresponding to a moving range photoelectrically converted while moving over a plurality of the photoelectric conversion devices in the horizontal and vertical directions in the predetermined period for photoelectrically converting incident light. Making the drive start times of the vertical and horizontal drive signals applied to the two-dimensional solid-state imaging device variable so as to read the data, and the read control means also changing the read start time of the N-line memory in the same manner. Reading a two-dimensional image signal;
The image superposition / combination unit shifts the two-dimensional image signal in the range corresponding to the read movement range by one pixel in the horizontal and vertical directions so as to be in the opposite direction to the movement direction of the formed subject image. A camera shake correction method for a solid-state imaging device, comprising the steps of superimposing a plurality of times and adding and synthesizing them to synthesize an image of one pixel in which motion is canceled.
を形成する請求項1から4のいずれか1項記載の方法。5. The method according to claim 1, wherein said video signal processing means forms a color image signal.
形成する請求項1から4のいずれか1項記載の方法。6. The method according to claim 1, wherein said video signal processing means forms a black and white image signal.
段から得られた二次元画像信号と、前記画像メモリに記
憶された二次元画像信号と、動き検出素子の1個か2個
以上の組み合わせを基に撮像装置の動きを検出する請求
項1記載の方法。7. A combination of one or two or more motion detection elements with a two-dimensional image signal obtained from the video signal processing means, a two-dimensional image signal stored in the image memory, The method according to claim 1, wherein the movement of the imaging device is detected based on the following.
所定の繰り返し周期で配列され、被写体からの入射光を
所定の期間光電変換するように構成された二次元固体撮
像素子と、 該二次元固体撮像素子の出力信号から二次元の画像信号
を形成する映像信号処理手段と、 該映像信号処理手段から得られた二次元画像信号を記憶
する画像メモリと、 撮像装置の動きを検出する動き検出手段と、 該動き検出手段から得られた動き信号に基づいて前記画
像メモリに記憶された二次元画像信号を読み出す読み出
し制御手段であって、前記二次元固体撮像素子に結像さ
れた被写体画像が入射光を光電変換する前記所定の期間
において水平と垂直方向に前記光電変換素子の複数個に
わたって移動しつつ光電変換された移動範囲に相当する
範囲の二次元画像信号を前記画像メモリから読み出す読
み出し制御手段と、 読み出された二次元画像信号を複数回重ね合わせて合成
する画像重ね合わせ合成手段であって、前記読み出され
た移動範囲に相当する範囲の二次元画像信号を前記結像
された被写体画像の移動方向と逆方向となるように水平
と垂直方向に1画素ずつずらして複数回重ね合わせて加
算合成して動きを打ち消した1画面の画像を合成する画
像重ね合わせ合成手段を有する固体撮像装置。8. A two-dimensional solid-state imaging device in which a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and configured to photoelectrically convert incident light from a subject for a predetermined period. Video signal processing means for forming a two-dimensional image signal from an output signal of a two-dimensional solid-state imaging device; an image memory for storing a two-dimensional image signal obtained from the video signal processing means; and a motion for detecting a motion of the imaging device Detecting means for reading a two-dimensional image signal stored in the image memory based on the motion signal obtained from the motion detecting means, wherein the subject image formed on the two-dimensional solid-state imaging device Is a two-dimensional image of a range corresponding to a moving range photoelectrically converted while moving over a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period in which incident light is photoelectrically converted. Read-out control means for reading a signal from the image memory; and image superimposing and synthesizing means for superimposing and synthesizing the read two-dimensional image signal a plurality of times, wherein A two-dimensional image signal is shifted one pixel at a time in the horizontal and vertical directions so as to be in the opposite direction to the moving direction of the formed subject image, and is superimposed a plurality of times to add and synthesize a single-screen image in which motion is canceled. A solid-state imaging device having image superimposing and synthesizing means.
所定の繰り返し周期で配列され、被写体からの入射光を
所定の期間光電変換するように構成された二次元固体撮
像素子と、 該二次元固体撮像素子から得られる出力信号を記憶する
画像メモリと、 撮像装置の動きを検出する動き検出手段と、 該動き検出手段から得られる動き信号に基づいて前記画
像メモリに記憶された二次元画像信号を読み出すように
制御する読み出し制御手段であって、前記二次元固体撮
像素子に結像された被写体画像が入射光を光電変換する
前記所定の期間において水平と垂直方向に前記光電変換
素子の複数個にわたって移動しつつ光電変換された移動
範囲に相当する範囲の二次元画像信号を前記画像メモリ
から読み出す読み出し制御手段と、 読み出された二次元画像信号を複数回重ね合わせて合成
する画像重ね合わせ合成手段であって、前記読み出され
た移動範囲に相当する範囲の二次元画像信号を前記結像
された被写体画像の移動方向と逆方向となるように水平
と垂直方向に1画素ずつずらして複数回重ね合わせて加
算合成して動きを打ち消した1画面の画像を合成する画
像重ね合わせ手段と、 該画像重ね合わせ合成手段の出力信号から二次元の画像
信号を形成する映像信号処理手段とを有する固体撮像装
置。9. A two-dimensional solid-state imaging device in which a plurality of photoelectric conversion elements are arrayed in a horizontal and vertical direction at a predetermined repetition cycle, and configured to photoelectrically convert incident light from a subject for a predetermined period. An image memory for storing an output signal obtained from a three-dimensional solid-state imaging device; a motion detecting means for detecting a motion of an imaging device; and a two-dimensional image stored in the image memory based on a motion signal obtained from the motion detecting means. Reading control means for controlling a signal to be read out, wherein the subject image formed on the two-dimensional solid-state imaging device photoelectrically converts incident light into a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period. Reading control means for reading from the image memory a two-dimensional image signal in a range corresponding to a moving range obtained by photoelectric conversion while moving over the plurality of read-out two-dimensional image signals; An image superimposing and synthesizing means for superimposing and synthesizing image signals a plurality of times, wherein a two-dimensional image signal in a range corresponding to the read-out moving range is moved in a direction opposite to a moving direction of the formed subject image. Image superimposing means for superimposing a plurality of times by shifting one pixel in the horizontal and vertical directions and adding and synthesizing them to synthesize an image of one screen in which motion has been canceled, and two output signals from the image superimposing and synthesizing means. A solid-state imaging device having video signal processing means for forming a two-dimensional image signal.
に所定の繰り返し周期で配列され、被写体からの入射光
を所定の期間光電変換するように構成された二次元固体
撮像素子と、 該二次元固体撮像素子から得られる出力信号を記憶する
画像メモリと、 撮像装置の動きを検出する動き検出手段と、 該動き検出手段から得られる動き信号に基づいて前記画
像メモリに記憶された二次元画像信号を読み出すように
制御する読み出し制御手段であって、前記二次元固体撮
像素子に結像された被写体画像が入射光を光電変換する
前記所定の期間において水平と垂直方向に前記光電変換
素子の複数個にわたって移動しつつ光電変換された移動
範囲に相当する範囲の二次元画像信号を前記画像メモリ
から読み出す読み出し制御手段と、 前記画像メモリから読み出された出力信号から二次元画
像信号を形成する映像信号処理手段と、 該映像信号処理手段から得られる二次元画像信号を複数
回重ね合わせて合成する画像重ね合わせ合成手段であっ
て、前記読み出された移動範囲に相当する範囲の二次元
画像信号を前記結像された被写体画像の移動方向と逆方
向となるように水平と垂直方向に1画素ずつずらして複
数回重ね合わせて加算合成して動きを打ち消した1画面
の画像を合成する画像重ね合わせ合成手段を有する固体
撮像装置。10. A two-dimensional solid-state imaging device in which a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition cycle, and configured to photoelectrically convert incident light from a subject for a predetermined period. An image memory for storing an output signal obtained from a three-dimensional solid-state imaging device; a motion detecting means for detecting a motion of an imaging device; and a two-dimensional image stored in the image memory based on a motion signal obtained from the motion detecting means. Reading control means for controlling a signal to be read out, wherein the subject image formed on the two-dimensional solid-state imaging device photoelectrically converts incident light into a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period. Reading control means for reading from the image memory a two-dimensional image signal in a range corresponding to a moving range obtained by photoelectrically converting while moving over the image memory; Video signal processing means for forming a two-dimensional image signal from the read output signal, and image superimposing and synthesizing means for superimposing and synthesizing a two-dimensional image signal obtained from the video signal processing means a plurality of times, The two-dimensional image signals in the range corresponding to the read moving range are overlapped a plurality of times by shifting one pixel in the horizontal and vertical directions so as to be in a direction opposite to the moving direction of the formed subject image, and added and synthesized. A solid-state imaging device having an image superimposing / synthesizing unit for synthesizing an image of one screen in which motion is canceled by performing the operation.
に所定の繰り返し周期で配列され、被写体からの入射光
を所定の期間光電変換するように構成された二次元固体
撮像素子と、 前記二次元固体撮像素子の出力信号から二次元の画像信
号を形成する映像信号処理手段と、 前記映像信号処理手段から得られた二次元画像信号を記
憶するNラインメモリ(Nは1以上の整数)と、 撮像装置の動きを検出する動き検出手段と、 該動き検出手段で得られた動き信号に基づいて前記固体
撮像素子を駆動する駆動手段であって、前記二次元固体
撮像素子に結像された被写体画像が入射光を光電変換す
る前記所定の期間において水平と垂直方向に前記光電変
換素子の複数個にわたって移動しつつ光電変換された移
動範囲に相当する範囲の二次元画像信号を前記二次元固
体撮像素子から読み出すように該二次元固体撮像素子に
印加する垂直と水平の駆動信号の駆動開始時刻を可変す
る駆動手段と、 前記Nラインメモリの読み出し開始時刻を同様に可変し
て読み出す読み出し制御手段と、 前記Nラインメモリから読み出された二次元画像信号を
複数回重ね合わせて合成する画像重ね合わせ合成手段で
あって、前記読み出された移動範囲に相当する範囲の二
次元画像信号を前記結像された被写体画像の移動方向と
逆方向となるように水平と垂直方向に1画素ずつずらし
て複数回重ね合わせて加算合成して動きを打ち消した1
画面の画像を合成する画像重ね合わせ合成手段とを有す
る固体撮像装置。11. A two-dimensional solid-state imaging device in which a plurality of photoelectric conversion elements are arranged in a horizontal and vertical direction at a predetermined repetition period, and configured to photoelectrically convert incident light from a subject for a predetermined period. Video signal processing means for forming a two-dimensional image signal from an output signal of a two-dimensional solid-state imaging device; N-line memory (N is an integer of 1 or more) for storing a two-dimensional image signal obtained from the video signal processing means; A motion detecting means for detecting a motion of the imaging device; and a driving means for driving the solid-state imaging device based on a motion signal obtained by the motion detecting device, wherein the imaging device forms an image on the two-dimensional solid-state imaging device. A two-dimensional image signal in a range corresponding to a moving range where the subject image is photoelectrically converted while moving over a plurality of the photoelectric conversion elements in the horizontal and vertical directions in the predetermined period in which incident light is photoelectrically converted. Driving means for varying the drive start time of vertical and horizontal drive signals applied to the two-dimensional solid-state image sensor so as to read from the two-dimensional solid-state image sensor; and similarly varying the read start time of the N-line memory. Reading control means for reading, and image superimposing and synthesizing means for superimposing and synthesizing a two-dimensional image signal read out from the N-line memory a plurality of times, the two-dimensional image signal having a range corresponding to the read moving range. The image signal is shifted one pixel at a time in the horizontal and vertical directions so as to be in a direction opposite to the moving direction of the formed subject image, and is superimposed a plurality of times to add and synthesize to cancel the motion.
A solid-state imaging device having an image superimposing / synthesizing unit for synthesizing a screen image.
号を形成する、請求項8から11のいずれか1項記載の
装置。12. The apparatus according to claim 8, wherein said video signal processing means forms a color image signal.
を形成する、請求項8から11のいずれか1項記載の装
置。13. The apparatus according to claim 8, wherein said video signal processing means forms a black and white image signal.
手段から得られた二次元画像信号と、前記画像メモリに
記憶された二次元画像信号と、動き検出素子の1個か2
個以上の組み合わせを基に撮像装置の動きを検出する、
請求項8記載の装置。14. The two-dimensional image signal obtained from the video signal processing means, the two-dimensional image signal stored in the image memory, and one or two of the motion detection elements.
Detecting the motion of the imaging device based on a combination of at least
An apparatus according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP21371999A JP3405404B2 (en) | 1999-07-28 | 1999-07-28 | Camera shake correction method for solid-state imaging device and solid-state imaging device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP21371999A JP3405404B2 (en) | 1999-07-28 | 1999-07-28 | Camera shake correction method for solid-state imaging device and solid-state imaging device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP2001045359A true JP2001045359A (en) | 2001-02-16 |
| JP3405404B2 JP3405404B2 (en) | 2003-05-12 |
Family
ID=16643872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP21371999A Expired - Fee Related JP3405404B2 (en) | 1999-07-28 | 1999-07-28 | Camera shake correction method for solid-state imaging device and solid-state imaging device |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP3405404B2 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100459437B1 (en) * | 2002-09-27 | 2004-12-03 | 엘지전자 주식회사 | Illusion elimination method for digital camera |
| US7409106B2 (en) | 2003-01-23 | 2008-08-05 | Seiko Epson Corporation | Image generating device, image generating method, and image generating program |
| US7643739B2 (en) | 2005-05-13 | 2010-01-05 | Casio Computer Co., Ltd. | Image pick-up apparatus having function of detecting shake direction |
| US7961233B2 (en) | 2006-02-23 | 2011-06-14 | Olympus Corporation | Electronic blurring compensation device and electronic blurring compensation method |
| US8212895B2 (en) | 2007-08-10 | 2012-07-03 | Megachips Corporation | Digital camera system with portrait effect |
-
1999
- 1999-07-28 JP JP21371999A patent/JP3405404B2/en not_active Expired - Fee Related
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100459437B1 (en) * | 2002-09-27 | 2004-12-03 | 엘지전자 주식회사 | Illusion elimination method for digital camera |
| US7409106B2 (en) | 2003-01-23 | 2008-08-05 | Seiko Epson Corporation | Image generating device, image generating method, and image generating program |
| US7643739B2 (en) | 2005-05-13 | 2010-01-05 | Casio Computer Co., Ltd. | Image pick-up apparatus having function of detecting shake direction |
| US7961233B2 (en) | 2006-02-23 | 2011-06-14 | Olympus Corporation | Electronic blurring compensation device and electronic blurring compensation method |
| US8212895B2 (en) | 2007-08-10 | 2012-07-03 | Megachips Corporation | Digital camera system with portrait effect |
Also Published As
| Publication number | Publication date |
|---|---|
| JP3405404B2 (en) | 2003-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103168463B (en) | Imageing sensor and picture pick-up device | |
| JP5574423B2 (en) | Imaging apparatus, display control method, and program | |
| JP2006157568A (en) | Imaging apparatus and program | |
| JP2002330329A (en) | Image pickup device | |
| JP2010020055A (en) | Image capturing apparatus and method of controlling the same | |
| US20140092220A1 (en) | Image capturing element capturing stereoscopic moving image and planar moving image and image capturing apparatus equipped with the same | |
| JP4689687B2 (en) | Imaging method and imaging apparatus | |
| JP2003319240A (en) | Image pickup device | |
| JP4985124B2 (en) | Image processing apparatus, image processing method, and image processing program | |
| JPH10276361A (en) | Imaging apparatus and method, imaging system, image information providing apparatus and method, image data processing apparatus and method, and transmission medium | |
| JP2005236662A (en) | Imaging method, imaging apparatus, and imaging system | |
| JP2827424B2 (en) | Image stabilization device | |
| JP3967853B2 (en) | Solid-state imaging device and signal readout method | |
| JP3405404B2 (en) | Camera shake correction method for solid-state imaging device and solid-state imaging device | |
| US20070269133A1 (en) | Image-data noise reduction apparatus and method of controlling same | |
| JP2006148550A (en) | Image processor and imaging device | |
| CN100512394C (en) | Signal processing apparatus | |
| JP2000106678A (en) | Image pickup device | |
| JP2006074693A (en) | Imaging device | |
| US20060197854A1 (en) | Image capturing apparatus and computer software product | |
| JP7118659B2 (en) | IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD AND PROGRAM | |
| JP2006135501A (en) | Imaging apparatus | |
| JP2006129418A (en) | Method for driving charge-transfer type solid-state image pick-up device and image pick-up method and apparatus | |
| JP2004235901A (en) | Imaging device | |
| JP2000299810A (en) | Imaging device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20080307 Year of fee payment: 5 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20090307 Year of fee payment: 6 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20090307 Year of fee payment: 6 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20100307 Year of fee payment: 7 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20100307 Year of fee payment: 7 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20110307 Year of fee payment: 8 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20110307 Year of fee payment: 8 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20120307 Year of fee payment: 9 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20120307 Year of fee payment: 9 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20130307 Year of fee payment: 10 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20130307 Year of fee payment: 10 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20140307 Year of fee payment: 11 |
|
| LAPS | Cancellation because of no payment of annual fees |