[go: up one dir, main page]

WO2018176575A1 - Dispositif à caméra à centre optique commun, et ensemble et procédé d'assemblage panoramique continu - Google Patents

Dispositif à caméra à centre optique commun, et ensemble et procédé d'assemblage panoramique continu Download PDF

Info

Publication number
WO2018176575A1
WO2018176575A1 PCT/CN2017/083304 CN2017083304W WO2018176575A1 WO 2018176575 A1 WO2018176575 A1 WO 2018176575A1 CN 2017083304 W CN2017083304 W CN 2017083304W WO 2018176575 A1 WO2018176575 A1 WO 2018176575A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
plane mirror
mirror
panoramic
center
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/083304
Other languages
English (en)
Chinese (zh)
Inventor
王好谦
沈灿伟
季向阳
王兴政
方璐
张永兵
戴琼海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute Of Future Media Technology
Shenzhen Graduate School Tsinghua University
Original Assignee
Shenzhen Institute Of Future Media Technology
Shenzhen Graduate School Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute Of Future Media Technology, Shenzhen Graduate School Tsinghua University filed Critical Shenzhen Institute Of Future Media Technology
Publication of WO2018176575A1 publication Critical patent/WO2018176575A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
    • G03B37/04—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe with cameras or projectors providing touching or overlapping fields of view
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G03B17/02—Bodies
    • G03B17/17—Bodies with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera

Definitions

  • the present invention relates to the field of virtual reality technology and computer vision, and in particular, to a camera device, a seamless seamless splicing component and a method.
  • the virtual reality technology boom has swept the world. It can be widely used in entertainment, education, communication, scientific research, military, medical and other industries. It not only brings a large-scale technological revolution, but also changes people's thinking habits. lifestyle.
  • the world's major companies have entered the virtual reality field, and a large number of entrepreneurial teams have also flooded into the virtual reality industry, and a variety of virtual reality devices have emerged.
  • high-quality virtual reality content is still very scarce, which makes virtual reality technology empty and has no use.
  • Panoramic video is one of the main contents of virtual reality. At present, most of the cameras used to capture panoramic video do not have a common center of light, so there is parallax between the various videos taken, resulting in the mosaic of panoramic video is not completely seamless. Small parallax can cause blurred images, and large parallax can cause problems with misalignment and ghosting. In particular, these problems become more serious when moving objects pass through the stitching seams of the panoramic video, giving the user a very bad experience.
  • the present invention provides a camera device with a common center of light, a seamless seamless splicing component and a method, which can overlap the optical centers of a plurality of cameras, thereby capturing a video without parallax, and then seamlessly Panorama stitching.
  • the invention discloses a camera device capable of common light center, comprising a camera, a first plane mirror and a second a plane mirror, the mirrors of the first plane mirror and the second plane mirror are parallel to each other, and are perpendicular to a main optical axis of the camera; wherein: the first plane mirror is located at a preset distance in front of the camera, and the mirror faces the a camera; the second plane mirror is located at a lens position of the camera, the mirror faces the first plane mirror, and the second plane mirror is provided with a hole for the camera to capture an image.
  • the mirror size of the first plane mirror just completely covers the imaged field of view of the camera.
  • the mirror size of the second plane mirror completely covers the field of view captured by the camera through the first plane mirror.
  • the hole is disposed at a center of the second plane mirror, the size of the hole being greater than or equal to a size of a main lens of the camera.
  • the back surface of the second plane mirror is in close contact with the lens position of the camera, and the size of the hole is just to block the shooting field of the camera.
  • the invention also discloses a panoramic seamless splicing assembly, comprising a plurality of the above-mentioned camera devices, wherein the plurality of camera devices are arranged opposite each other to form a circle, and the optical center of the camera in each of the camera devices is The distance of the center of the circle is equal to the sum of the distance from the optical center of the corresponding camera to the first plane mirror and the distance from the first plane mirror to the second plane mirror, respectively.
  • the optical centers of the cameras in each of the camera devices are located on the same circle, and each of the cameras is equally spaced.
  • the invention additionally discloses a panoramic seamless splicing method, comprising the following steps:
  • S1 capturing a video by using the panoramic seamless splicing component, and performing feature extraction and feature matching on the first frame of the video captured by each of the camera devices in the panoramic seamless splicing component;
  • S3 projecting, according to parameters of each of the camera devices, an image captured by each camera device to a spherical surface having a center of the circle as a center of the circle;
  • Steps S3 and S4 are repeated, and each frame image is sequentially projected and fused to obtain a seamless panoramic image frame, and a seamless panoramic video is generated.
  • the step S1 specifically includes: extracting feature points of the first frame image of the video captured by each of the camera devices by using a SIFT, SURF or ORB feature extraction algorithm, and extracting feature points of the first frame image of each video Matching is performed; preferably, the multi-band fusion method is adopted by fusing the projected image in step S4.
  • the beneficial effects of the present invention are: the camera device of the present invention discloses that the first plane mirror and the second plane mirror which are parallel to each other are disposed respectively at the front of the camera and the lens position, so that the camera is The optical center is moved to the outside of the physical size of the camera by two reflections, thereby making it possible to recombine the optical centers of the plurality of cameras, and thus the panoramic seamless splicing component based on the plurality of cameras of the present invention can be formed on the basis of the present invention.
  • FIG. 1 is a schematic structural view of a camera device capable of a common light center according to a preferred embodiment of the present invention
  • FIG. 2 is a schematic diagram of an imaging optical path of a non-central region of an image sensor of the camera of FIG. 1;
  • FIG. 3 is an equivalent diagram of an imaging optical path of a non-center region of the image sensor of the camera of FIG. 1;
  • FIG. 4 is a schematic diagram of an imaging optical path of a central region of an image sensor of the camera of FIG. 1;
  • FIG. 5 is a schematic structural view of a panoramic seamless splicing assembly according to a preferred embodiment of the present invention.
  • FIG. 6 is a flow chart of a seamless seamless stitching method in accordance with a preferred embodiment of the present invention.
  • FIG. 1 it is a schematic structural diagram of a camera device capable of a common light center according to a preferred embodiment of the present invention.
  • the camera device includes a camera A1, a first plane mirror A2, a second plane mirror A3, a first plane mirror A2 and a second plane mirror A3. Parallel to each other, both perpendicular to the main optical axis of camera A1, wherein: first plane mirror A2 is located at a preset distance in front of the camera A1, and its mirror surface faces the camera A1; the second plane mirror A3 is located at the lens position of the camera A1, and its mirror surface faces the first plane mirror A2, and a hole is formed in the second plane mirror A3 for Take a picture for camera A1.
  • the mirror size of the first plane mirror A2 just completely covers the imaging field of view of the camera A1, that is, the light entering the inside of the camera A1 for imaging can only come from the light reflected by the first plane mirror A2.
  • the mirror size of the second plane mirror A3 completely covers the field of view of the camera A1 reflected by the first plane mirror A2, that is, the light entering the inside of the camera A1 for imaging can only be reflected twice by the second plane mirror A3 and the first plane mirror A2.
  • the light in the second plane mirror A3 is opened at the center thereof, and the size of the hole is smaller and smaller, so as not to obscure the shooting field of the camera A1, wherein the back surface of the second plane mirror A3 can be closely attached to the camera A1.
  • the size of the hole may be equal to or slightly larger than the size of the main lens of the camera A1.
  • the optical center of the camera A1 is located at the center of the main lens, and the optical centers of the plurality of cameras cannot be overlapped without using a special device. Moreover, since the photosensitive element of the camera has a certain size, it can only be a certain field of view. The range is imaged.
  • the camera A1 in the common-light camera device is reflected by two plane mirrors, which is equivalent to moving to the rear for shooting, so that the optical center of the camera A1 is moved backward, so that multiple The camera can simultaneously capture multiple different directions at the same point in space, making seamless splicing of multiple channels possible.
  • the light is subjected to two specular reflections to image on a non-center region of the image sensor A5 of the camera A1, wherein the light reflected by the edge of the first plane mirror A2 falls just at the edge of the image sensor A5, that is, The first plane mirror A2 just completely covers the field of view of the camera A1.
  • the convex lens A4 represents the main lens of the camera A1, the optical center of the camera A1 is located at the center of the convex lens A4, and the extension line of all incident light rays of the second plane mirror A3 just intersects at one point, which is exactly where the optical center of the camera A1 moves to the rear. That is the virtual light heart.
  • the first plane mirror A2 and the second plane mirror A3 are removed, and the convex lens A4 is moved to the position of the rear virtual optical center while maintaining the relative position of the image sensor A5 and the convex lens A4.
  • the light received by the non-center region of the image sensor A5 is exactly the same as that of FIG. 2, and the formed image is completely identical; that is, the first plane mirror A2 and the second plane mirror A3 function positively.
  • the camera A1 is moved to the position of the virtual optical center directly behind to shoot.
  • the light received in the central area of the image sensor A5 passes through more than two mirrors.
  • the surface reflection the closer to the light of the center of the image sensor A5, the more the number of reflections, the Droost effect may occur; and since the central area is not the overlapping area of adjacent cameras, it does not affect the splicing.
  • the optical center of the camera A1 is moved to the virtual optical center directly behind by two specular reflections. From the imaging principle, the distance from the optical center of the camera A1 to the virtual optical center is equal to the light of the camera A1. The sum of the distance from the heart to the first plane mirror A2 and the distance from the first plane mirror A2 to the second plane mirror A3.
  • the panoramic seamless splicing assembly of the present invention includes a plurality of the above-mentioned camera devices.
  • the plurality of camera devices are arranged to face each other in a circle, and the virtual optical centers of the respective camera devices are located at the center of the circle, that is, The distance from the optical center of the camera in each camera device to the center of the circle is equal to the sum of the distance from the optical center of the corresponding camera to the first plane mirror and the distance from the first plane mirror to the second plane mirror, respectively, so that the respective camera devices are shared Heart set.
  • the optical centers of the cameras in the respective camera devices are located on the same circle.
  • the virtual optical center of the device is located at the center of the circle (i.e., at the center of the aforementioned circle). At this time, it is equivalent to the camera moving to the center of the circle to shoot a plurality of different directions, thereby obtaining a multi-channel non-parallax video.
  • the panoramic seamless splicing assembly can include from 5 to 10 camera devices, each camera device being equally spaced.
  • the preferred embodiment of the present invention discloses a seamless seamless splicing method. As shown in FIG. 6, the method includes the following steps:
  • S1 adopting the panoramic seamless splicing component shown in FIG. 5 to capture video, and performing feature extraction and feature matching on the first frame (first frame) of the video captured by each camera device in the panoramic seamless splicing component, specifically, Using SIFT (Size Invariant Feature Transform), SURF (Acceleration Robust Feature) or ORB feature extraction algorithm, feature points of the first frame image of the video captured by each of the camera devices are extracted, and the first frame image of each video is extracted. Feature points are matched;
  • S2 performing homography alignment on the first frame (first frame) of each video. Specifically, using a global homography transformation matrix, the feature points on the matching are aligned, thereby calculating internal parameters of each camera device. And external parameters;
  • each camera device is photographed The image is projected onto the spherical surface with the virtual optical center as the center of the sphere. Since each camera device has been totally optical, the images captured by each camera device will be projected onto the same spherical surface, and there is no parallax;
  • S4 merging the image after projection to merge multiple images into a single panoramic image. Specifically, a multi-band fusion method may be adopted. Since the images are non-parallax, the merged panorama is seamless. of;
  • Steps S3 and S4 are repeated, and each frame image is sequentially projected and fused to obtain a seamless panoramic image frame, thereby generating a seamless panoramic video.
  • the image image captured by the panoramic seamless splicing component shown in FIG. 5 is completely seamless after the above steps are synthesized, wherein since there is no parallax between the plurality of images, no seam searching or the like is required, thereby It can increase the stitching speed and thus increase the frame rate of panoramic video, which is crucial for the virtual reality experience.
  • the illuminable camera device of the present invention makes the optical center of the camera back to the physical size of the camera by adding a set of plane mirrors at the front of the camera and at the lens position, thereby making it possible to recombine the optical centers of the plurality of cameras. Furthermore, a panoramic seamless splicing component with a common center of light is designed. Since the video obtained by the panoramic seamless splicing component of the common optical center has the advantage of no parallax, the splicing effect of the panoramic video can be improved, the seamless panoramic video can be obtained, and the algorithm of the video splicing can be simplified, and the speed of the video splicing can be improved. The frame rate of the panoramic video.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Studio Devices (AREA)
  • Stereoscopic And Panoramic Photography (AREA)

Abstract

La présente invention concerne un dispositif à caméra à centre optique commun, et un ensemble et un procédé d'assemblage panoramique continu. Le dispositif à caméra comprend une caméra, un premier miroir plan, et un second miroir plan. Les surfaces de miroir du premier miroir plan et du second miroir plan sont parallèles l'une à l'autre et perpendiculaires à un axe optique principal de la caméra. Le premier miroir plan est placé devant cette caméra à une distance prédéfinie, et sa surface de miroir fait face à ladite caméra. Le second miroir plan est disposé à un emplacement où se trouve un objectif de la caméra, et sa surface de miroir fait face au premier miroir plan. Un trou est pratiqué à l'emplacement du second miroir plan pour que ladite caméra capture une image. Grâce au dispositif à caméra, les centres optiques d'une pluralité de caméras coïncident, de telle sorte qu'une vidéo sans erreur de parallaxe puisse être produite, ce qui permet d'effectuer un assemblage panoramique continu.
PCT/CN2017/083304 2017-03-29 2017-05-05 Dispositif à caméra à centre optique commun, et ensemble et procédé d'assemblage panoramique continu Ceased WO2018176575A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710196808.5A CN107024828A (zh) 2017-03-29 2017-03-29 一种可共光心的摄像机装置、全景无缝拼接组件及方法
CN201710196808.5 2017-03-29

Publications (1)

Publication Number Publication Date
WO2018176575A1 true WO2018176575A1 (fr) 2018-10-04

Family

ID=59525820

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/083304 Ceased WO2018176575A1 (fr) 2017-03-29 2017-05-05 Dispositif à caméra à centre optique commun, et ensemble et procédé d'assemblage panoramique continu

Country Status (2)

Country Link
CN (1) CN107024828A (fr)
WO (1) WO2018176575A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110086987B (zh) * 2019-04-22 2021-01-05 惠州Tcl移动通信有限公司 一种摄像头视角裁剪方法、装置及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101521745A (zh) * 2009-04-14 2009-09-02 王广生 一组多镜头光心重合式全方位摄像装置及全景摄像、转播的方法
US20110074917A1 (en) * 2009-09-30 2011-03-31 Hon Hai Precision Industry Co., Ltd. Panoramic imaging lens and panoramic imaging system using the same
CN102084650A (zh) * 2009-05-12 2011-06-01 华为终端有限公司 远程呈现系统、方法及视频采集设备
CN204719321U (zh) * 2015-05-26 2015-10-21 上海臻恒光电系统有限公司 一种两片反射式全景成像装置
CN105530431A (zh) * 2015-12-16 2016-04-27 景好 一种反射式全景成像系统及方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20000017888A (ko) * 1999-12-28 2000-04-06 김기삼 거울을이용한단안렌즈스테레오비젼획득방법
US9244258B2 (en) * 2010-06-24 2016-01-26 Panasonic Corporation Omnidirectional imaging system
CN104469340B (zh) * 2014-12-01 2017-03-15 深圳凯澳斯科技有限公司 一种立体视频共光心成像系统及其成像方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101521745A (zh) * 2009-04-14 2009-09-02 王广生 一组多镜头光心重合式全方位摄像装置及全景摄像、转播的方法
CN102084650A (zh) * 2009-05-12 2011-06-01 华为终端有限公司 远程呈现系统、方法及视频采集设备
US20110074917A1 (en) * 2009-09-30 2011-03-31 Hon Hai Precision Industry Co., Ltd. Panoramic imaging lens and panoramic imaging system using the same
CN204719321U (zh) * 2015-05-26 2015-10-21 上海臻恒光电系统有限公司 一种两片反射式全景成像装置
CN105530431A (zh) * 2015-12-16 2016-04-27 景好 一种反射式全景成像系统及方法

Also Published As

Publication number Publication date
CN107024828A (zh) 2017-08-08

Similar Documents

Publication Publication Date Title
CN101673395B (zh) 图像拼接方法及装置
CN101872491B (zh) 基于光度立体的自由视角重光照方法和系统
EP3377853B1 (fr) Procédés et systèmes de détermination à grande échelle de poses de caméra rgbd
CN104835117B (zh) 基于重叠方式的球面全景图生成方法
CN103501409B (zh) 一种超高分辨率全景高速球一体机系统
US9142010B2 (en) Image enhancement based on combining images from multiple cameras
US9948869B2 (en) Image fusion method for multiple lenses and device thereof
CN105530431A (zh) 一种反射式全景成像系统及方法
CN106997579B (zh) 图像拼接的方法和装置
JPH06141237A (ja) 複眼撮像装置
CN110278366B (zh) 一种全景图像虚化方法、终端及计算机可读存储介质
WO2011095026A1 (fr) Procédé et système de photographie
CN114693569A (zh) 双摄像头视频融合的方法和电子设备
CN105472263B (zh) 影像撷取方法及使用此方法的影像撷取设备
WO2017092261A1 (fr) Module d'appareil photographique, terminal mobile, et procédé et appareil de prise d'image associés
KR102031485B1 (ko) 360도 카메라와 평면 거울을 이용한 다시점 영상 획득 장치 및 방법
CN115174805A (zh) 全景立体图像的生成方法、装置和电子设备
WO2018176575A1 (fr) Dispositif à caméra à centre optique commun, et ensemble et procédé d'assemblage panoramique continu
CN108810426A (zh) 一种红外大视场拼接方法及拼接系统
Popovic et al. Design and implementation of real-time multi-sensor vision systems
KR101936168B1 (ko) 평면 좌표계 영상데이터와 구면 좌표계 영상데이터를 이용한 영상처리 장치 및 방법
Lin et al. Single-view-point omnidirectional catadioptric cone mirror imager
CN106873301A (zh) 基于阵列相机对远距离小孔后方进行成像的系统及方法
CN109328460A (zh) 立体图像捕获
CN115988338B (zh) 一种基于复眼相机阵列的远场信号反演重建方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17903838

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17903838

Country of ref document: EP

Kind code of ref document: A1