[go: up one dir, main page]

WO2006121365A1 - Formation d'images tridimensionnelles pour une videoconference - Google Patents

Formation d'images tridimensionnelles pour une videoconference Download PDF

Info

Publication number
WO2006121365A1
WO2006121365A1 PCT/RU2005/000258 RU2005000258W WO2006121365A1 WO 2006121365 A1 WO2006121365 A1 WO 2006121365A1 RU 2005000258 W RU2005000258 W RU 2005000258W WO 2006121365 A1 WO2006121365 A1 WO 2006121365A1
Authority
WO
WIPO (PCT)
Prior art keywords
cameras
images
dimensional image
image
pairs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/RU2005/000258
Other languages
English (en)
Russian (ru)
Inventor
Andrei Vladimirovich Klimov
Artem Leonidovich Yukhin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bioscrypt SA
Original Assignee
A4 Vision SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by A4 Vision SA filed Critical A4 Vision SA
Priority to PCT/RU2005/000258 priority Critical patent/WO2006121365A1/fr
Publication of WO2006121365A1 publication Critical patent/WO2006121365A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the invention relates to the field of information technology, to the formation of a three-dimensional image of an object, in particular, the face of a human user of a personal computer during communication sessions on the Internet in real time.
  • the known method for providing a three-dimensional (3D) image (stereo image) to a user during a video conference, i.e. in real time.
  • the disadvantage of this method is the mandatory fixed position of the user relative to the camera and the observation screen and the inability to display the actual three-dimensional image on the screen.
  • the known method US patent application N ° 2005/0046698 of 03.03.2005, providing the formation of a three-dimensional (3D) image (stereo image) of the user, providing for the transmission of information on the Internet in real time.
  • the method provides for the transmission of information about the "scene", which is observed by many pairs of cameras, and provides for the free placement of the object within the scene.
  • the specified method does not allow to form a three-dimensional image on the screen.
  • the known method US patent N ° 6028672 of 02.22.2000, forming a three-dimensional (3D) image, including obtaining images of an object using at least two cameras and subsequent reconstruction of a three-dimensional image.
  • This way provides three-dimensional image formation with high speed and can be used for video conferencing.
  • the disadvantage of this method which is based on the triangulation method, is the mandatory fixation of the position of the cameras and the object relative to them.
  • the known method European application JNk 1355274 from 10.22.2003, forming a three-dimensional (3D) image comprising obtaining images of an object using at least two video cameras and the subsequent reconstruction of a three-dimensional image.
  • the disadvantage of this method which is based on the triangulation method, is the mandatory fixation of the position of the cameras. Although the position of the object relative to the cameras is not fixed, but the large amount of calculations does not allow to reconstruct a three-dimensional image in real time for video conferencing.
  • a device for the formation of three-dimensional (3D) image for video conferencing US patent N ° 6028672 from 02.22.2000, which includes two cameras and a processor connected to them.
  • a disadvantage of the known device is mandatory fixed - a priori used in the processor relative position of the cameras, which complicates their location when organizing a video conference in various conditions and under various circumstances. Besides, a fixed location of the object - a participant in a video conference is required and its illumination is required.
  • the invention is aimed at providing a comfortable videoconference over the Internet and allows you to arbitrarily install video cameras before starting a video conference, and the user can move freely during a video conference, while remaining in the field of view of video cameras.
  • the technical result achieved in the method of forming a three-dimensional image for video conferencing is to ensure the formation of a real-time three-dimensional image of a user during video conferencing during his movements (rotations) in the field of view of the cameras and in determining the relative position of the cameras arbitrarily set before the start of the video conference.
  • the technical result achieved in both versions of the device for forming a three-dimensional (3D) image for video conferencing is to automatically determine the relative position of the cameras after their arbitrary installation before starting the video conference and to ensure the formation of a three-dimensional image of the user (object) in real time when it is arbitrarily moved in the field of view of the cameras .
  • the specified technical result is achieved using the method of forming a three-dimensional image for video conferencing, including registering images of an object using at least two video cameras and subsequent reconstruction in the processor of a three-dimensional image displayed on the screen.
  • spatial spectra of pairs of images recorded by the first and second video cameras are obtained, and a three-dimensional image is reconstructed from them, and the relative position of the cameras and the object is determined by changes in the spatial spectra of image pairs as a result of displacements and rotations of the object in the field of view of both cameras.
  • Reconstruct a three-dimensional (3D) image by constructing a tomogram from the spatial spectra of a number of pairs of images obtained by the first and second video cameras.
  • the use of frequency space is the basis of the algorithm for determining the relative position of cameras and an object and reconstruction of three-dimensional images in real time.
  • Reconstruction of a three-dimensional image of an object can be synchronized with a frame scan of video cameras. In this case, it is possible to synchronize the frame scans of the first and second video cameras.
  • An image of an object can be recorded by two or more pairs of cameras, then the spatial spectra of each pair of images obtained by the corresponding pair of cameras are obtained, and a single three-dimensional image of the object is reconstructed from them, and the relative position of all cameras and the object is determined by the changes in the spatial spectra of the pairs of images as a result of displacements and turns the object in the field of view of both cameras.
  • the specified technical result is achieved when using the device for forming a three-dimensional image for video conferencing, which implements the above method and includes at least two video cameras and a processor connected to them, as well as a multiplexer, a spectral conversion unit (coprocessor) and a memory unit through which the cameras are connected to a processor, providing: determining the relative position of the cameras and the object from changes in the spatial spectra of the image pairs as a result of the displacements and rotations of the object in the field of view of both cameras and constructing a tomogram of the object using the stored a memory unit for information on pairs of spectral (spatial - frequency) image images recorded by the first and second video cameras.
  • the block of spectral conversion is designed to obtain spatial spectra of images and can be made in the form of a block of fast Fourier transform.
  • the spectral transform block can also be made in the form of a wavelet transform block.
  • the device may additionally include one or more spectral transform blocks, while all spectral transform blocks that are coprocessors form a cluster of coprocessors.
  • the device may further include a unit for converting an analog signal from video cameras to digital before they enter the multiplexer.
  • the device may additionally include a display connected to the processor, indicating the location of the object in the field of view of the cameras and showing its three-dimensional image.
  • the specified technical result is achieved when using the device for forming a three-dimensional image for video conferencing, which implements the above method and includes at least two cameras and a processor connected to them, as well as a multiplexer, and a memory unit through which the cameras are connected to the processor, providing: spatial - frequency conversion of images entering the processor from the multiplexer; determining the relative position of the cameras and the object from the changes in the spatial spectra of the image pairs as a result of the displacements and rotations of the object in the field of view of both cameras and constructing a tomogram of the object using information stored on the pairs of spectral (spatial - frequency) image images recorded by the first and second cameras .
  • Figure l shows a diagram of a device that implements a method of forming a three-dimensional (3D) image for video conferencing.
  • a device that implements a method for forming three-dimensional images of object 1, see FIG. 1, includes video cameras (WEB cameras) 2 and 3, a multiplexer with an analog-to-digital converter 4, a fast Fourier transform unit (coprocessor) 5, a memory unit 6, a processor 7 and a display 8.
  • the fast Fourier transform unit can be supplemented by several connected to it parallel to the fast Fourier transform blocks (coprocessors), forming, forming together with the coprocessor 5 cluster 9.
  • the images of the object registered by two WEB cameras after the multiplexer 4 are sequentially fed into the fast Fourier transform unit 5, where they are converted into spatial spectra of images and sequentially fed into the memory unit 6, where they are stored in pairs and used by the processor 7 to determine the relative position of the WEB cameras 2 and 3 and tomogram recovery.
  • the operation of the processor 7 to restore the tomogram is synchronized with the cycle of registering a pair of flat projections, i.e. with a frame scan of WEB cameras 2 and 3.
  • the user Before starting a video conference, the user (object) places the WEB cameras 2 and 3 in a position convenient for themselves. At the same time, the user exercises control over his presence in the field of view of WEB cameras 2 and 3 through display 8. Through display 8, recommendations can be issued on the sequence of rotations and displacements of user 1 in the field of view of WEB cameras 2 and 3. At the same time, one can limit oneself to natural movements of the user in preparation for a video conference.
  • the processor 7 from the images of user 1, obtained in different angles, determines the Fourier space of the user's images, which serves as the basis for the restoration of the tomogram. The more accurately the Fourier space of the images is determined, the better the three-dimensional image of the user is reconstructed.
  • the operation of restoring a tomogram is carried out by iterations using the method of restoring digital Fourier holograms generated for each pair of flat projections by performing the calculation of the Fourier transform of each projection.
  • the process of "training" of a specialized program is carried out, and at the same time, an iterative process of restoring a three-dimensional image from pairs of flat projections is carried out.
  • the condition for completing the training of the program is the consent of the user to complete the learning process, expressed by the corresponding command entered by the user, if the latter is satisfied with the received three-dimensional image, which he can see on display 8.
  • the solution of the tomogram recovery problem using Fourier is refined — images recorded at a frame rate of each of the WEB cameras.
  • the registration processes of Web cameras can be considered parallel .
  • the parallelism of image processing can be considered absolute.
  • a digital Fourier hologram is a Fourier - the image of a holographic object. Therefore, the transmission of spatial frequencies in digital Fourier transforms is invariant to the position of the object (user). This fact is the reason to abandon the hard positioning of WEB cameras.
  • the changes relate to the scale determined by the linear increase along the optical axes of each Web camera, however, the composition of the spatial spectrum, i.e. the relative position of spatial frequencies remains unchanged.
  • the training program can correct the 3-dimensional image of the object (user) in the operating mode. In other words, the training-recovery cycle can continue many times.
  • the speed of formation and transmission of dynamic ZD images synthesized by the proposed method significantly increases with an increase in the number of Web cameras, while the number of these cameras must be even.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

La présente invention se rapporte à un procédé permettant de produire une image tridimensionnelle pour une vidéoconférence. Le procédé selon l'invention consiste à enregistrer des images d'un objet (1) à l'aide d'au moins deux caméras Web (2 et 3), et à reconstruire ensuite, dans un processeur (7), une image tridimensionnelle (3D), laquelle est affichée sur un écran. Ledit procédé est caractérisé en ce qu'il consiste à obtenir les spectres spatiaux des paires d'images qui sont enregistrées par les première et seconde caméras Web, et à procéder, à partir de ceux-ci, à la reconstruction d'images tridimensionnelles par reconstitution tomographique. La position relative des caméras Web et de l'objet est déterminée en fonction des modifications des spectres spatiaux des paires d'images causées par le déplacement et les rotations de l'objet dans le champ des deux caméras Web. L'invention permet de déterminer automatiquement la position relative des caméras Web (2 et 3), qui sont placées de manière aléatoire avant le début de la vidéoconférence.
PCT/RU2005/000258 2005-05-13 2005-05-13 Formation d'images tridimensionnelles pour une videoconference Ceased WO2006121365A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/RU2005/000258 WO2006121365A1 (fr) 2005-05-13 2005-05-13 Formation d'images tridimensionnelles pour une videoconference

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/RU2005/000258 WO2006121365A1 (fr) 2005-05-13 2005-05-13 Formation d'images tridimensionnelles pour une videoconference

Publications (1)

Publication Number Publication Date
WO2006121365A1 true WO2006121365A1 (fr) 2006-11-16

Family

ID=37396791

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/RU2005/000258 Ceased WO2006121365A1 (fr) 2005-05-13 2005-05-13 Formation d'images tridimensionnelles pour une videoconference

Country Status (1)

Country Link
WO (1) WO2006121365A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102256154A (zh) * 2011-07-28 2011-11-23 中国科学院自动化研究所 实现三维全景视频的定位与播放的方法及系统
US8570358B2 (en) 2009-11-06 2013-10-29 Sony Corporation Automated wireless three-dimensional (3D) video conferencing via a tunerless television device
US8687046B2 (en) 2009-11-06 2014-04-01 Sony Corporation Three-dimensional (3D) video for two-dimensional (2D) video messenger applications

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6028672A (en) * 1996-09-30 2000-02-22 Zheng J. Geng High speed three dimensional imaging method
RU2157056C2 (ru) * 1998-02-03 2000-09-27 Логутко Альберт Леонидович Способ объемного телевидения
JP2001197521A (ja) * 2000-01-06 2001-07-19 Toppan Printing Co Ltd 撮像装置、撮像方法及び撮像条件に係るデータを記録した記録媒体
WO2002073980A1 (fr) * 2001-03-14 2002-09-19 Koninklijke Philips Electronics N.V. Système de caméra stéréoscopique à réglage automatique

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6028672A (en) * 1996-09-30 2000-02-22 Zheng J. Geng High speed three dimensional imaging method
RU2157056C2 (ru) * 1998-02-03 2000-09-27 Логутко Альберт Леонидович Способ объемного телевидения
JP2001197521A (ja) * 2000-01-06 2001-07-19 Toppan Printing Co Ltd 撮像装置、撮像方法及び撮像条件に係るデータを記録した記録媒体
WO2002073980A1 (fr) * 2001-03-14 2002-09-19 Koninklijke Philips Electronics N.V. Système de caméra stéréoscopique à réglage automatique

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8570358B2 (en) 2009-11-06 2013-10-29 Sony Corporation Automated wireless three-dimensional (3D) video conferencing via a tunerless television device
US8687046B2 (en) 2009-11-06 2014-04-01 Sony Corporation Three-dimensional (3D) video for two-dimensional (2D) video messenger applications
CN102256154A (zh) * 2011-07-28 2011-11-23 中国科学院自动化研究所 实现三维全景视频的定位与播放的方法及系统

Similar Documents

Publication Publication Date Title
US8503763B2 (en) Image signatures for use in motion-based three-dimensional reconstruction
WO2020156195A1 (fr) Procédé de génération d'images et tomodensitomètre, dispositif informatique et support d'enregistrement lisible par ordinateur
Reimat et al. Cwipc-sxr: Point cloud dynamic human dataset for social xr
JP5940356B2 (ja) ３次元ｘ線ｃｔ装置、３次元ｃｔ画像再構成方法、及びプログラム
EP3274986A2 (fr) Procédés, systèmes et logiciel pour 3d virtuelle
CN111080778A (zh) 一种双目内窥镜软组织图像的在线三维重建方法
JP2015041370A (ja) 画像処理装置、方法、及びプログラム、並びに、立体画像表示装置
CN109712230A (zh) 三维模型补充方法、装置、存储介质及处理器
US20110176715A1 (en) Four-dimensional volume imaging system
WO2006121365A1 (fr) Formation d'images tridimensionnelles pour une videoconference
FR2863086A1 (fr) Procede de production d'une sequence d'images volumiques d'une zone d'un organe d'un etre vivant.
CN120128757B (zh) 一种视频插补方法、装置、电子设备及介质
JP2014071870A (ja) 仮想視点画像合成装置、仮想視点画像合成方法及び仮想視点画像合成プログラム
JP5311033B2 (ja) 対象物体の厚さの推定方法
CN108648796A (zh) 一种虚拟现实镜像治疗仪
KR100733267B1 (ko) 복수 위치의 ｃｔ 화상 생성 방법 및 ｘ선 ｃｔ 장치
WO2021039642A1 (fr) Dispositif de reconstruction tridimensionnelle, procédé et programme
KR20040083652A (ko) 3차원 입체 영상의 신호 처리 장치 및 방법
CN115100092B (zh) 冠脉ct图像的减影方法、装置、电子设备及存储介质
Deshpande et al. Real-Time Reconstruction of 3D Space using Holographic Video
CN119478200A (zh) 基于增强现实技术骨折二维图像的骨折块空间定位方法
JP5461934B2 (ja) 超音波診断装置
KR102617940B1 (ko) 멀티 mla 라이트 필드 카메라를 이용한 6-자유도 지원 360도 영상 생성 장치및 그 방법
Patro et al. Social snapshot: a system for temporally coupled social photography
Brundyn et al. Stereo Video Reconstruction Without Explicit Depth Maps for Endoscopic Surgery

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU

122 Ep: pct application non-entry in european phase

Ref document number: 05817796

Country of ref document: EP

Kind code of ref document: A1