[go: up one dir, main page]

CN115176459A - 虚拟视点合成方法、电子设备和计算机可读介质 - Google Patents

虚拟视点合成方法、电子设备和计算机可读介质 Download PDF

Info

Publication number
CN115176459A
CN115176459A CN202180000061.6A CN202180000061A CN115176459A CN 115176459 A CN115176459 A CN 115176459A CN 202180000061 A CN202180000061 A CN 202180000061A CN 115176459 A CN115176459 A CN 115176459A
Authority
CN
China
Prior art keywords
virtual viewpoint
pixel
image
pixels
depth map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202180000061.6A
Other languages
English (en)
Other versions
CN115176459B (zh
Inventor
孙炎
楚明磊
史天阔
习艳会
姬治华
侯一凡
赵晨曦
张硕
彭项君
张小牤
孙伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Beijing BOE Optoelectronics Technology Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Beijing BOE Optoelectronics Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd, Beijing BOE Optoelectronics Technology Co Ltd filed Critical BOE Technology Group Co Ltd
Publication of CN115176459A publication Critical patent/CN115176459A/zh
Application granted granted Critical
Publication of CN115176459B publication Critical patent/CN115176459B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
    • H04N13/268—Image signal generators with monoscopic-to-stereoscopic image conversion based on depth image-based rendering [DIBR]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2213/00—Details of stereoscopic systems
    • H04N2213/003—Aspects relating to the "2D+depth" image format

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

本公开提供了一种虚拟视点合成方法，包括：对原始图像对应的零视差时的深度图进行预处理，得到处理后的深度图(S1)；根据处理后的深度图和原始图像，生成多个虚拟视点各自对应的虚拟视点图像(S2)；对各虚拟视点图像中的空洞进行填补，生成多个填补后的虚拟视点图像(S3)。本公开还提供了一种电子设备和计算机可读介质。

Description

PCT国内申请，说明书已公开。

Claims (11)

  1. PCT国内申请，权利要求书已公开。
CN202180000061.6A 2021-01-25 2021-01-25 虚拟视点合成方法、电子设备和计算机可读介质 Active CN115176459B (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/073565 WO2022155950A1 (zh) 2021-01-25 2021-01-25 虚拟视点合成方法、电子设备和计算机可读介质

Publications (2)

Publication Number Publication Date
CN115176459A true CN115176459A (zh) 2022-10-11
CN115176459B CN115176459B (zh) 2024-09-17

Family

ID=82548398

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180000061.6A Active CN115176459B (zh) 2021-01-25 2021-01-25 虚拟视点合成方法、电子设备和计算机可读介质

Country Status (4)

Country Link
US (1) US12223621B2 (zh)
EP (1) EP4149110B1 (zh)
CN (1) CN115176459B (zh)
WO (1) WO2022155950A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119810392A (zh) * 2024-12-17 2025-04-11 东南大学 一种基于深度图片边缘检测的三维视频空洞填充方法
CN120111200A (zh) * 2025-01-15 2025-06-06 北京天马辉电子技术有限责任公司 3d显示图像生成方法及相关设备

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12524677B2 (en) * 2021-01-25 2026-01-13 International Business Machines Corporation Generating unsupervised adversarial examples for machine learning
CN117957580A (zh) * 2021-10-13 2024-04-30 索尼集团公司 信息处理设备、信息处理方法和程序
CN115022612B (zh) * 2022-05-31 2024-01-09 北京京东方技术开发有限公司 一种显示装置的驱动方法、装置及显示设备
CN115937291B (zh) * 2022-09-14 2023-12-15 北京字跳网络技术有限公司 一种双目图像的生成方法、装置、电子设备及存储介质
CN118447149A (zh) * 2023-02-06 2024-08-06 脸萌有限公司 图像渲染方法、装置、电子设备以及存储介质
CN120166308A (zh) * 2025-03-20 2025-06-17 北京航空航天大学 一种面向真实场景的光场3d片源生成方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436671A (zh) * 2011-08-16 2012-05-02 上海交通大学 一种基于深度值非线性变换的虚拟视点绘制方法
CN102592275A (zh) * 2011-12-16 2012-07-18 天津大学 虚拟视点绘制方法
KR20140037301A (ko) * 2012-09-11 2014-03-27 (주)리얼디스퀘어 3차원 영상의 스케일 변환 장치 및 그 방법
CN104780355A (zh) * 2015-03-31 2015-07-15 浙江大学 一种视点合成中基于深度的空洞修复方法
CN110062220A (zh) * 2019-04-10 2019-07-26 长春理工大学 视差层次最大化的虚拟视点图像生成方法

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2932911A1 (fr) * 2008-06-24 2009-12-25 France Telecom Procede et dispositif de remplissage des zones d'occultation d'une carte de profondeur ou de disparites estimee a partir d'au moins deux images.
KR101483659B1 (ko) * 2008-07-11 2015-01-16 삼성디스플레이 주식회사 입체영상 표시방법, 이를 수행하기 위한 표시장치
US9094660B2 (en) 2010-11-11 2015-07-28 Georgia Tech Research Corporation Hierarchical hole-filling for depth-based view synthesis in FTV and 3D video
CN102625127B (zh) * 2012-03-24 2014-07-23 山东大学 一种适于3d电视虚拟视点生成的优化方法
CN102724529B (zh) * 2012-05-28 2014-08-06 清华大学 虚拟视点视频序列的生成方法及生成装置
US9998726B2 (en) * 2012-06-20 2018-06-12 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
US20140002595A1 (en) 2012-06-29 2014-01-02 Hong Kong Applied Science And Technology Research Institute Co., Ltd. Apparatus, system and method for foreground biased depth map refinement method for dibr view synthesis
EP2775723A1 (en) * 2013-03-07 2014-09-10 imcube labs GmbH Method, apparatus and computer program for generating a multiview image-plus-depth format
CN103731657B (zh) 2014-01-26 2016-03-16 冠捷显示科技（厦门）有限公司 一种对dibr算法处理后的含有空洞影像的空洞填充处理方法
CN104378619B (zh) 2014-11-12 2016-09-28 合肥工业大学 一种基于前后景梯度过渡的快速高效的空洞填补算法
GB2534136A (en) * 2015-01-12 2016-07-20 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
CN106060512B (zh) 2016-06-28 2017-08-01 华中科技大学 一种在虚拟视点合成中选取和填补合理映射点方法
CN106060522A (zh) 2016-06-29 2016-10-26 努比亚技术有限公司 一种视频图像的处理装置及方法
JP7722818B2 (ja) * 2018-03-23 2025-08-13 インターデイジタル ヴィーシー ホールディングス インコーポレイテッド Ｄｉｂｒシステムにおいて立体的視点を生じさせるための、多焦点面ベースの方法（ｍｆｐ－ｄｉｂｒ）
US10810754B2 (en) * 2018-04-24 2020-10-20 Ford Global Technologies, Llc Simultaneous localization and mapping constraints in generative adversarial networks for monocular depth estimation
US11393113B2 (en) 2019-02-28 2022-07-19 Dolby Laboratories Licensing Corporation Hole filling for depth image based rendering
CN110660131B (zh) 2019-09-24 2022-12-27 宁波大学 一种基于深度背景建模的虚拟视点空洞填补方法
CN111311521A (zh) * 2020-03-12 2020-06-19 京东方科技集团股份有限公司 图像修复方法、装置和电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436671A (zh) * 2011-08-16 2012-05-02 上海交通大学 一种基于深度值非线性变换的虚拟视点绘制方法
CN102592275A (zh) * 2011-12-16 2012-07-18 天津大学 虚拟视点绘制方法
KR20140037301A (ko) * 2012-09-11 2014-03-27 (주)리얼디스퀘어 3차원 영상의 스케일 변환 장치 및 그 방법
CN104780355A (zh) * 2015-03-31 2015-07-15 浙江大学 一种视点合成中基于深度的空洞修复方法
CN110062220A (zh) * 2019-04-10 2019-07-26 长春理工大学 视差层次最大化的虚拟视点图像生成方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119810392A (zh) * 2024-12-17 2025-04-11 东南大学 一种基于深度图片边缘检测的三维视频空洞填充方法
CN120111200A (zh) * 2025-01-15 2025-06-06 北京天马辉电子技术有限责任公司 3d显示图像生成方法及相关设备

Also Published As

Publication number Publication date
US12223621B2 (en) 2025-02-11
US20230162338A1 (en) 2023-05-25
WO2022155950A1 (zh) 2022-07-28
EP4149110A4 (en) 2023-08-09
EP4149110A1 (en) 2023-03-15
EP4149110B1 (en) 2025-03-12
CN115176459B (zh) 2024-09-17

Similar Documents

Publication Publication Date Title
CN115176459B (zh) 虚拟视点合成方法、电子设备和计算机可读介质
US9013482B2 (en) Mesh generating apparatus, method and computer-readable medium, and image processing apparatus, method and computer-readable medium
CN106780590B (zh) 一种深度图的获取方法及系统
CN105517677B (zh) 深度图/视差图的后处理方法和装置
JP6021541B2 (ja) 画像処理装置及び方法
CN111047709B (zh) 一种双目视觉裸眼3d图像生成方法
GB2576548A (en) Method and system for reconstructing colour and depth information of a scene
CN110602474B (zh) 一种图像视差的确定方法、装置及设备
CN114697623B (zh) 投影面选取和投影图像校正方法、装置、投影仪及介质
US20140340486A1 (en) Image processing system, image processing method, and image processing program
CN108520232A (zh) 立体环景影片产生方法及装置
JP3990271B2 (ja) 簡易ステレオ画像入力装置、方法、プログラム、および記録媒体
CN114401391A (zh) 虚拟视点生成方法及装置
US20130011045A1 (en) Apparatus and method for generating three-dimensional (3d) zoom image of stereo camera
CN103369331A (zh) 图像空洞的填补方法和装置及视频图像的处理方法和装置
WO2020187339A1 (zh) 一种裸眼3d的虚拟视点图像生成方法及便携式终端
US20130321409A1 (en) Method and system for rendering a stereoscopic view
US20140205182A1 (en) Stereoscopic image processing apparatus, stereoscopic image processing method, and stereoscopic image processing program
JP5906033B2 (ja) 画像処理装置および画像処理方法、プログラム
JP5931062B2 (ja) 立体画像処理装置、立体画像処理方法、及びプログラム
CN111524075B (zh) 深度图像滤波方法、图像合成方法、装置、设备及介质
CN107610070B (zh) 基于三摄像采集的自由立体匹配方法
CN112053434B (zh) 视差图的生成方法、三维重建方法及相关装置
CN107845108A (zh) 一种光流值计算方法、装置及电子设备
Hosseinpour et al. View synthesis for FTV systems based on a minimum spatial distance and correspondence field

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant