[go: up one dir, main page]

WO2018186169A1 - Dispositif de génération vidéo, procédé de génération vidéo, et programme de génération vidéo - Google Patents

Dispositif de génération vidéo, procédé de génération vidéo, et programme de génération vidéo Download PDF

Info

Publication number
WO2018186169A1
WO2018186169A1 PCT/JP2018/011071 JP2018011071W WO2018186169A1 WO 2018186169 A1 WO2018186169 A1 WO 2018186169A1 JP 2018011071 W JP2018011071 W JP 2018011071W WO 2018186169 A1 WO2018186169 A1 WO 2018186169A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
camera
visual field
interpupillary distance
video generation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2018/011071
Other languages
English (en)
Japanese (ja)
Inventor
加藤　勝也
貴司 吉本
泰弘 浜口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sharp Corp
Original Assignee
Sharp Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Corp filed Critical Sharp Corp
Publication of WO2018186169A1 publication Critical patent/WO2018186169A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
    • H04N13/279—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals the virtual viewpoint locations being selected by the viewers or determined by tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays

Definitions

  • the present invention relates to a video generation device, a video generation method, and a video generation program.
  • the present application claims priority on Japanese Patent Application No. 2017-075213 filed in Japan on April 5, 2017, the contents of which are incorporated herein by reference.
  • Patent Documents 1 and 2 describe a display device that realizes a glasses-type terminal.
  • AR and MR are realized by displaying different images on the left-eye display and right-eye display included in the glasses-type terminal to obtain the effects of convergence and binocular parallax.
  • An aspect of the present invention has been made in view of such circumstances, and an object thereof is to provide a video generation method and a video generation program capable of realizing AR and MR while suppressing the load on the processor.
  • a video generation method and a video generation program according to an aspect of the present invention are configured as follows.
  • a video generation method is a video generation method in which an object in a 3D (Three Dimensions) pseudo space is projected by a camera in the 3D pseudo space to generate a video and displayed on a video display device.
  • a visual field setting process for setting the visual field of the camera based on visual field information input from the video display device
  • an interpupillary distance setting process for setting an interpupillary distance, the visual field, the object, and the camera.
  • the video rendering process measures the load of the processor, and when the load falls below a threshold value, the camera is located at a position away from the camera by the interpupillary distance in the 3D pseudo space. Is added, and the video is generated by performing perspective projection with each of the cameras, and the video shift process is skipped.
  • the video rendering process adds a camera at a location separated from the camera by the interpupillary distance in the 3D pseudo space. Then, the video is generated by performing perspective projection with each of the cameras, and the processing of the video shift process is skipped.
  • the video generation program causes a computer to execute the above-described video generation method.
  • An image generation apparatus is an image generation apparatus that generates an image by perspectively projecting an object in a 3D (Three) pseudo space by a camera in the 3D pseudo space, and displays the image on a video display device.
  • a visual field setting unit that sets a visual field of the camera based on visual field information input from the video display device
  • an interpupillary distance setting unit that sets an interpupillary distance, the visual field, the object, and the camera.
  • a video rendering unit that renders a reference video based on a positional relationship; and, by shifting the reference video in the horizontal direction based on the visual field, the inter-pupil distance, and the positional relationship between the object and the camera,
  • the video generation method can generate a video that can be AR displayed while reducing the load on the processor.
  • FIG. 1 is a diagram illustrating an example of a video generation apparatus 10 according to the present embodiment.
  • FIG. 1 also shows the video display unit 11.
  • the video display unit 11 is provided in a video display device such as a glasses-type terminal, a smart glass, or a head-mounted display.
  • the video generation device 10 may be provided in the video display device, or may be provided in a terminal device that can be connected to the video display device such as a smartphone. As shown in FIG.
  • a video generation device 10 in this embodiment includes a visual field setting unit (visual field setting process) 101, an interpupillary distance setting unit (interpupillary distance setting process) 102, and a video rendering unit ( A video rendering process (103) 103 and a video shift unit (video shift process) 104 are included.
  • the visual field setting unit 101 sets the size of the visual field (FoV: “Field Of View”) used in the video rendering unit 103 based on the visual field information input from the video display unit 11.
  • the field of view is, for example, an angle in the vertical direction when the camera in the 3D pseudo space performs perspective projection.
  • the visual field setting unit 101 outputs the set visual field value to the video rendering unit 103.
  • the visual field information is, for example, the visual field value itself.
  • the visual field value may be a vertical value, a diagonal value, or a horizontal value. Further, the visual field value may be one or more values among a vertical value, a diagonal value, and a horizontal value.
  • the visual field information is, for example, an ID such as a vendor ID or a product ID of the video display device provided with the video display unit 11.
  • the video generation apparatus 10 can store the ID and the field-of-view value in association with each other.
  • the video generation apparatus 10 can have a table (database) that associates IDs and field-of-view values.
  • the video generation visual field setting unit 101 can determine the visual field value based on this ID.
  • the field-of-view setting unit 101 can determine the value of the field of view by, for example, comparing this ID with its own database.
  • the field-of-view setting unit 101 can, for example, inquire a server on the Internet for this ID and receive a field-of-view value.
  • the viewing angle setting unit 101 can use the table (database) on the server.
  • the interpupillary distance setting unit 102 sets the interpupillary distance (PD) and outputs it to the video rendering unit 103 and the video shift unit 104.
  • the interpupillary distance may be a fixed value or may be manually input by the user.
  • the interpupillary distance setting unit 102 may set the interpupillary distance measured by a sensor provided in the video display unit 11 or a wearable sensor.
  • the video rendering unit 103 renders video by the camera shooting an object in the 3D pseudo space.
  • the visual field of the camera can be set to the visual field value input from the visual field setting unit 101.
  • the video shift unit 104 receives display information from the video display unit 11. This display information may be obtained from the Internet.
  • the video shift unit 104 shifts the video input from the video rendering unit 103 based on the interpupillary distance input from the interpupillary distance setting unit 102 and the display information, and converts the video into two videos.
  • the video shift unit 104 outputs the two generated videos to the video display unit 11. Note that the video before the shift can be referred to as a reference video.
  • FIG. 2 shows an example of the 3D pseudo space used by the video rendering unit 103.
  • 20 is a view of the 3D pseudo space viewed from the height direction.
  • 21 is a view of the 3D pseudo space as viewed from the side.
  • the north direction of the 3D pseudo space can be the z axis
  • the height direction can be the y axis
  • the east direction can be the x axis.
  • 20 is a diagram of the 3D pseudo space viewed from the plus direction of the y axis
  • 21 is a diagram of the 3D pseudo space viewed from the plus direction of the x axis.
  • the camera 201 can generate an image by perspective projection based on the positional relationship between itself and the object 202 in the 3D pseudo space and the field of view set in the field of view setting unit 101.
  • the object 202 is a rectangular parallelepiped.
  • This perspective projection can be performed based on the field of view 211.
  • the visual field 211 can be the visual field set in the visual field setting unit 101.
  • the vertical visual field is set, but the horizontal visual field can be set from the display size and the aspect ratio of the video display unit 11.
  • Reference numeral 203 denotes the distance between the camera 201 and the object 202.
  • FIG. 3 is a diagram illustrating an example in which the video shift unit 104 shifts the image generated by the video rendering unit 103.
  • Reference numerals 301 and 311 denote displays of the video display unit 11.
  • Reference numeral 302 denotes a horizontal center line of the display 301, and reference numeral 312 denotes a center line of the display 311.
  • 303 represents the user's left eye and 313 represents the user's right eye.
  • 304 represents the center line of the left eye 303, and 314 represents the center line of the right eye 313.
  • Reference numeral 305 denotes an object corresponding to 202 in FIG. 2 rendered by the video rendering unit 103. In the case of 20 in FIG.
  • an image in which the object 305 is displayed on the center line 302 is generated on the display 301, and the image shift unit 104 applies a rightward shift thereto.
  • the video shift unit 104 adds a leftward shift to the video displayed on the display 311.
  • the shift amount can be performed based on the visual field set by the visual field setting unit 101, the inter-pupil distance set by the inter-pupil distance setting unit 102, and the distance between the object in the 3D pseudo space rendered by the video rendering process 103 and the camera.
  • FIG. 4 is a diagram in which the setting of the 3D pseudo space in FIG. 2 is changed.
  • the object 402 is obtained by moving the object 202 in the horizontal direction.
  • Reference numeral 403 denotes the distance between the camera 201 and the object 402.
  • Reference numeral 404 denotes a line in the same direction as the horizontal direction of the camera 201 and passes through the object 402.
  • a 1 (xx 0 ) + a 0 . x 0 and the coefficient a i is a value determined by the image display device 11 can be known from the output obtained by inputting the information of the image display device 11 in the database.
  • the database may be in the video generation device 10.
  • the database may be on the server.
  • the video rendering unit 103 performs rendering once, and generates two videos that can obtain the effect of congestion based on the copy, thereby reducing the load on the processor. can do.
  • the video shift performed by the video shift unit 104 may be performed when the processor load exceeds a threshold.
  • the video rendering process 103 can prepare two cameras for the left eye and the right eye in the 3D pseudo space, and render two videos.
  • An image generated by the left-eye camera can be displayed on the display 301 of FIG. 3, and an image generated by the right-eye camera can be displayed on the display 311 of FIG. In this case, the processing of the video shift unit 104 is skipped.
  • the video rendering unit 103 may calculate the processor load.
  • the video generation device 10 performs rendering using one camera in the 3D pseudo space, and generates a video with congestion by shifting the rendered video in the horizontal direction. In the present embodiment, a case will be described in which a plurality of target objects exist in the 3D pseudo space.
  • FIG. 5 is an example in which an object 501 further exists in the 3D pseudo space of FIG.
  • the sense of distance between the camera 201 and the object 501 is not correct.
  • the video rendering unit 103 in FIG. 1 detects that the number of objects in the 3D pseudo space is two or more, it sets not the camera 201 but also a left eye camera and a right eye camera. Based on these, a left-eye video and a right-eye video can be generated. By doing so, the sense of distance of the object 501 can be correctly expressed.
  • the object 202 and the object 501 can include depth on information.
  • the video rendering unit 103 in FIG. 1 can determine whether to use the left-eye camera and the right-eye camera based on whether the number of objects for which depth-on information is set is one. For example, in FIG. 5, when depth on information is set for the object 202 and depth on information is not set for the object 501, the video rendering unit 103 renders a video based on the camera 201, and The shift unit 104 shifts the video, thereby generating a left-eye video and a right-eye video. By doing so, the load on the processor can be reduced.
  • a program that operates in the video generation device, the video generation method, and the video generation program according to one aspect of the present invention is a program that controls a CPU or the like so as to realize the functions of the above-described embodiments according to one aspect of the present invention.
  • a program that causes a computer to function Information handled by these devices is temporarily stored in the RAM at the time of processing, then stored in various ROMs and HDDs, read out by the CPU, and corrected and written as necessary.
  • a semiconductor medium for example, ROM, nonvolatile memory card, etc.
  • an optical recording medium for example, DVD, MO, MD, CD, BD, etc.
  • a magnetic recording medium for example, magnetic tape, Any of a flexible disk etc.
  • the program when distributing to the market, can be stored and distributed on a portable recording medium, or transferred to a server computer connected via a network such as the Internet.
  • the storage device of the server computer is also included in one embodiment of the present invention.
  • part or all of the video generation method and the video generation program in the above-described embodiments may be realized as an LSI that is typically an integrated circuit.
  • Each functional block of the receiving apparatus may be individually formed as a chip, or a part or all of them may be integrated into a chip. When each functional block is integrated, an integrated circuit controller for controlling them is added.
  • the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor.
  • an integrated circuit based on the technology can also be used.
  • the present invention is not limited to the above-described embodiment. It goes without saying that the video generation method of the present invention is not limited to application to eyeglass-type terminals, but can be applied to portable devices, wearable devices, and the like.
  • One embodiment of the present invention is suitable for use in a video generation method and a video generation program.
  • One embodiment of the present invention is used in, for example, a communication system, a communication device (for example, a mobile phone device, a base station device, a wireless LAN device, or a sensor device), an integrated circuit (for example, a communication chip), a program, or the like. be able to.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

L'invention concerne un procédé de génération vidéo qui génère une vidéo en faisant en sorte qu'une caméra dans un pseudo-espace tridimensionnel (3D) effectue une projection en perspective d'un objet dans le pseudo-espace 3D, et affiche la vidéo sur un dispositif d'affichage de vidéo, ledit procédé de génération vidéo consistant : en une étape de réglage de champ de vision selon laquelle le champ de vision de la caméra est réglé sur la base d'informations de champ de vidéo qui sont entrées par le dispositif d'affichage vidéo ; en une étape de réglage de distance interpupillaire selon laquelle une distance interpupillaire est réglée ; en une étape de représentation vidéo selon laquelle une vidéo de référence est représentée sur la base du champ de vision et d'une relation de position entre l'objet et la caméra ; et en une étape de décalage vidéo selon laquelle deux vidéos sont générées en décalant la vidéo de référence dans une direction latérale sur la base du champ de vision, de la distance interpupillaire, et de la relation de position entre l'objet et la caméra.
PCT/JP2018/011071 2017-04-05 2018-03-20 Dispositif de génération vidéo, procédé de génération vidéo, et programme de génération vidéo Ceased WO2018186169A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017075213A JP2020098945A (ja) 2017-04-05 2017-04-05 映像生成装置、映像生成方法および映像生成プログラム
JP2017-075213 2017-04-05

Publications (1)

Publication Number Publication Date
WO2018186169A1 true WO2018186169A1 (fr) 2018-10-11

Family

ID=63713220

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/011071 Ceased WO2018186169A1 (fr) 2017-04-05 2018-03-20 Dispositif de génération vidéo, procédé de génération vidéo, et programme de génération vidéo

Country Status (2)

Country Link
JP (1) JP2020098945A (fr)
WO (1) WO2018186169A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116076071A (zh) * 2020-06-03 2023-05-05 杰瑞·尼姆斯 二维图像捕获系统和三维数字图像的显示

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11195131A (ja) * 1997-12-26 1999-07-21 Canon Inc 仮想現実方法及び装置並びに記憶媒体
JP2002073003A (ja) * 2000-08-28 2002-03-12 Namco Ltd 立体視画像生成装置及び情報記憶媒体
JP2014192550A (ja) * 2013-03-26 2014-10-06 Seiko Epson Corp 頭部装着型表示装置および頭部装着型表示装置の制御方法
JP2014199617A (ja) * 2013-03-29 2014-10-23 株式会社バンダイナムコゲームス 画像生成システム及びプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11195131A (ja) * 1997-12-26 1999-07-21 Canon Inc 仮想現実方法及び装置並びに記憶媒体
JP2002073003A (ja) * 2000-08-28 2002-03-12 Namco Ltd 立体視画像生成装置及び情報記憶媒体
JP2014192550A (ja) * 2013-03-26 2014-10-06 Seiko Epson Corp 頭部装着型表示装置および頭部装着型表示装置の制御方法
JP2014199617A (ja) * 2013-03-29 2014-10-23 株式会社バンダイナムコゲームス 画像生成システム及びプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116076071A (zh) * 2020-06-03 2023-05-05 杰瑞·尼姆斯 二维图像捕获系统和三维数字图像的显示

Also Published As

Publication number Publication date
JP2020098945A (ja) 2020-06-25

Similar Documents

Publication Publication Date Title
US20180192022A1 (en) Method and System for Real-time Rendering Displaying Virtual Reality (VR) On Mobile Using Head-Up Display Devices
CN107590771B (zh) 具有用于在建模3d空间中投影观看的选项的2d视频
US9241155B2 (en) 3-D rendering for a rotated viewer
US20120044241A1 (en) Three-dimensional on-screen display imaging system and method
TW202332263A (zh) 立體影像播放裝置及其立體影像產生方法
CN101180891A (zh) 立体图像显示装置、立体图像显示方法及计算机程序
CN110892717A (zh) 图像处理器和图像处理器的控制方法
CN106797462B (zh) 多视图图像显示设备及其控制方法
US9167225B2 (en) Information processing apparatus, program, and information processing method
JP2022051978A (ja) 画像処理装置、画像処理方法、及び、プログラム
CN111656409B (zh) 信息处理装置和信息处理方法
KR20170065208A (ko) 3d 이미지 처리 방법 및 장치, 및 그래픽 처리 장치
US9225968B2 (en) Image producing apparatus, system and method for producing planar and stereoscopic images
CN112752085A (zh) 基于人眼跟踪的裸眼3d视频播放系统及方法
CN114513646B (zh) 一种三维虚拟场景中全景视频的生成方法及设备
WO2018186169A1 (fr) Dispositif de génération vidéo, procédé de génération vidéo, et programme de génération vidéo
US11477419B2 (en) Apparatus and method for image display
WO2012021129A1 (fr) Rendu 3d pour un spectateur effectuant une rotation
KR102223339B1 (ko) 증강 현실 비디오 게임을 제공하는 방법, 디바이스 및 시스템
WO2018186168A1 (fr) Dispositif, procédé et programme de génération de vidéo
EP4328657B1 (fr) Procédé et dispositif informatique pour la génération de scène 3d
US10757401B2 (en) Display system and method for display control of a video based on different view positions
CN103313075A (zh) 图像处理装置及方法以及记录图像处理程序的记录介质
TWI879032B (zh) 立體顯示系統
JP2020167657A (ja) 画像処理装置、ヘッドマウントディスプレイ、および画像表示方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18781423

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18781423

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP