CN120111200A - 3D display image generation method and related equipment - Google Patents
3D display image generation method and related equipment Download PDFInfo
- Publication number
- CN120111200A CN120111200A CN202510063417.0A CN202510063417A CN120111200A CN 120111200 A CN120111200 A CN 120111200A CN 202510063417 A CN202510063417 A CN 202510063417A CN 120111200 A CN120111200 A CN 120111200A
- Authority
- CN
- China
- Prior art keywords
- image
- display
- color image
- virtual viewpoint
- alpha channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/156—Mixing image signals
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
本申请公开了一种3D显示图像生成方法及相关设备，涉及裸眼3D显示技术领域，包括：获取待叠加的背景层图像、中间层图像与前景层图像，基于2D+Alpha通道的图像处理方式，对所述中间层图像进行处理，得到中间层对应的第一透明彩色图像，基于2D+Alpha通道+深度图的图像处理方式，对所述前景层图像进行处理，得到第一图像，对所述第一图像执行抠像操作与虚拟视点生成操作，得到虚拟视点彩色图像，对所述背景层图像、所述第一透明彩色图像与所述虚拟视点彩色图像执行图像融合操作，得到3D显示图像。本申请基于2D+Alpha通道+深度图的图像处理方式对各层图像进行处理，并将处理后的图像进行融合，降低生成3D显示图像的成本。
The present application discloses a 3D display image generation method and related equipment, which relates to the field of naked-eye 3D display technology, including: obtaining a background layer image, an intermediate layer image and a foreground layer image to be superimposed, processing the intermediate layer image based on the image processing method of 2D+Alpha channel to obtain a first transparent color image corresponding to the intermediate layer, processing the foreground layer image based on the image processing method of 2D+Alpha channel+depth map to obtain a first image, performing a matting operation and a virtual viewpoint generation operation on the first image to obtain a virtual viewpoint color image, performing an image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image. The present application processes each layer of images based on the image processing method of 2D+Alpha channel+depth map, and fuses the processed images to reduce the cost of generating 3D display images.
Description
Technical Field
The invention relates to the technical field of naked eye 3D display, in particular to a 3D display image generation method and related equipment.
Background
In the age of informatization and digitalization, along with the development of society, 2D display is gradually unable to meet the needs of human beings, and 3D display has become a new research target of researchers and a new development trend in the display field.
In the related art, the multi-view grating naked eye 3D display can enable multiple viewers to simultaneously view stereoscopic images in a larger viewing angle, and can feel shocking visual experience without any vision-assisting equipment, so that the multi-view grating naked eye 3D display is attractive. However, in the actual 3D display image generation process, stereoscopic images of multiple viewpoints need to be acquired in real time, and it is common practice to simultaneously capture the same scene by using N cameras arranged in parallel, so as to obtain N images of different positions (viewpoints), and the manufacturing cost is high. In addition, in recent years, naked eye 3D LED large screens based on visual 3D are widely applied, arc corner designs are generally adopted, the design avoids picture folding caused by corner folding, so that pictures can be seamlessly connected and smoother, construction of space sense and atmosphere sense is ensured, pictures have good immersion sense, and the single-view 3D content customization cost is very high. Therefore, how to generate 3D display images at low cost and meet the single-view 3D and multi-view 3D display requirements at the same time is an urgent problem to be solved.
The foregoing is provided merely for the purpose of facilitating understanding of the technical scheme of the present application and is not intended to represent an admission that the foregoing is related art.
Disclosure of Invention
The application mainly aims to provide a 3D display image generation method and related equipment, which aim to solve the technical problems of how to generate a 3D display image with low cost and simultaneously meet the single-view 3D and multi-view 3D display requirements.
In order to achieve the above object, the present application provides a method for generating a 3D display image, the method comprising:
Acquiring a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
Processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer;
processing the foreground layer image based on an image processing mode of 2D+alpha channel+depth map to obtain a first image, and executing a matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image;
and performing image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
In an embodiment, before the step of performing an image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image, the method includes:
Acquiring a preset 3D display image resolution;
And cutting the background layer image based on the resolution to obtain a target background layer image.
In an embodiment, the step of performing an image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image further includes:
and performing image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
In an embodiment, the step of performing an image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image further includes:
2D overlapping is carried out on the target background layer image and the first transparent color image, so that a first overlapped image is obtained;
Acquiring grating parameters to determine the number l 1 of the RGB sub-pixels covered in the horizontal direction in one grating period;
And determining a display value V of each sub-pixel, and obtaining a 3D display image based on l 1 sub-pixels, the display value V of each sub-pixel and the first superposition image.
In an embodiment, the step of processing the intermediate layer image by the image processing mode based on 2d+alpha channel to obtain a first transparent color image corresponding to the intermediate layer further includes:
performing two-way processing on the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a left-way image and a right-way image;
Setting a foreground corresponding to the left image as a first color image, and setting a foreground corresponding to the right image as a first Alpha channel image;
and obtaining a first transparent color image based on the first color image and the first Alpha channel image.
In an embodiment, the step of processing the foreground layer image based on the image processing mode of 2d+alpha channel+depth map to obtain a first image, performing a matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image further includes:
Based on an image processing mode of 2D+alpha channels, performing three paths of processing on the foreground layer image to obtain an upper path image, a middle path image and a lower path image;
Setting a foreground corresponding to the upper path image as a second color image, setting a foreground corresponding to the middle path image as a second Alpha channel image, and setting a foreground corresponding to the lower path image as a depth image;
Obtaining a first image based on the second color image, the second Alpha channel image and the depth map image;
Obtaining a second transparent color image based on the second color image and a second Alpha channel image;
and generating a virtual viewpoint color image based on the second transparent color image and the depth map image.
In an embodiment, the step of generating a virtual viewpoint color image based on the second transparent color image and the depth map image further includes:
Dividing the second transparent color image into a plurality of groups of image blocks with the size of n multiplied by n, wherein the value of n is set in advance;
Acquiring preset features of the second transparent color image, and calculating cosine similarity of the image blocks based on the preset features to obtain attention scores of each image block;
Reconstructing the preset features based on the attention score of each image block to obtain target features;
And obtaining a virtual viewpoint color image based on the target feature, the second transparent color image and the depth map image.
In addition, to achieve the above object, the present application also proposes a 3D display image generation apparatus, the 3D display image generation apparatus including:
the acquisition module is used for acquiring a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
the first image processing module is used for processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer;
The second image processing module is used for processing the foreground layer image based on an image processing mode of 2D+alpha channel+depth map to obtain a first image, and performing image matting operation and virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image;
And the fusion module is used for performing image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
In an embodiment, the 3D display image generating apparatus further includes a trimming module including:
The first acquisition unit is used for acquiring a preset 3D display image resolution;
and the cutting unit is used for cutting the background layer image based on the resolution ratio to obtain a target background layer image.
In one embodiment, the fusion module comprises:
and the fusion unit is used for performing image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
In an embodiment, the fusion module further comprises:
the superposition unit is used for carrying out 2D superposition on the target background layer image and the first transparent color image to obtain a first superposition image;
the determining unit is used for acquiring grating parameters to determine the number l 1 of the RGB sub-pixels covered in the horizontal direction in one grating period;
the first obtaining unit is configured to determine a display value V of each sub-pixel, and obtain a 3D display image based on l 1 sub-pixels, the display value V of each sub-pixel, and the first superimposed image.
In an embodiment, the first image processing module includes:
The first processing unit is used for performing two-way processing on the middle layer image based on an image processing mode of a 2D+alpha channel to obtain a left-way image and a right-way image;
The first setting unit is used for setting the foreground corresponding to the left image as a first color image and setting the foreground corresponding to the right image as a first Alpha channel image;
and the second obtaining unit is used for obtaining a first transparent color image based on the first color image and the first Alpha channel image.
In an embodiment, the second image processing module includes:
the second processing unit is used for performing three paths of processing on the foreground layer image based on the image processing mode of the 2D+alpha channel to obtain an upper path image, a middle path image and a lower path image;
A second setting unit, configured to set a foreground corresponding to the upper road image as a second color image, set a foreground corresponding to the middle road image as a second Alpha channel image, and set a foreground corresponding to the lower road image as a depth map image;
a third obtaining unit, configured to obtain a first image based on the second color image, the second Alpha channel image, and the depth map image;
A fourth obtaining unit, configured to obtain a second transparent color image based on the second color image and a second Alpha channel image;
And the generating unit is used for generating a virtual viewpoint color image based on the second transparent color image and the depth map image.
In an embodiment, the second image processing module further comprises:
a blocking unit, configured to divide the second transparent color image into a plurality of groups of image blocks with size of n×n, where a value of n is set in advance;
The second acquisition unit is used for acquiring preset features of the second transparent color image, and calculating cosine similarity of the image blocks based on the preset features so as to obtain attention scores of each image block;
a reconstruction unit, configured to reconstruct the preset feature based on the attention score of each image block, to obtain a target feature;
and a fifth obtaining unit, configured to obtain a virtual viewpoint color image based on the target feature, the second transparent color image, and the depth map image.
In addition, in order to achieve the above object, the present application also proposes a 3D display image generating device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the 3D display image generating method as described above.
Furthermore, to achieve the above object, the present application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the 3D display image generation method as described above.
Furthermore, to achieve the above object, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the 3D display image generating method as described above.
One or more technical schemes provided by the application have at least the following technical effects:
The application provides a 3D display image generation method and related equipment, which relate to the technical field of naked eye 3D display, and in the related technology, three-dimensional images of a plurality of viewpoints are required to be acquired in real time, the common practice is that N cameras which are arranged in parallel are used for shooting the same scene at the same time, so that N images of different positions (viewpoints) are obtained, and compared with high manufacturing cost, in the application, firstly, a background layer image, an intermediate layer image and a foreground layer image to be overlapped are acquired, then, the intermediate layer image is processed based on an image processing mode of a 2D+alpha channel, so that a first transparent color image corresponding to the intermediate layer is obtained, further, based on an image processing mode of a 2D+alpha channel+depth map, the foreground layer image is processed, so that a first image is obtained, a matting operation and a virtual viewpoint generation operation are carried out on the first image, so that a virtual viewpoint color image is obtained, and finally, an image fusion operation is carried out on the background layer image, the first transparent color image and the virtual viewpoint color image so that a 3D display image is obtained.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a method for generating a 3D display image according to an embodiment of the present application;
Fig. 2 is a schematic diagram of a multi-layer image fusion naked eye 3D display according to a first embodiment of the present application;
FIG. 3 is a schematic diagram of an intermediate layer "2D+alpha channel" video image according to a first embodiment of the 3D display image generation method of the present application;
FIG. 4 is a schematic diagram of a foreground layer "2D+alpha channel+depth map" video image according to a first embodiment of the 3D display image generation method of the present application;
FIG. 5 is a schematic flow chart of a second embodiment of a 3D display image generating method according to the present application;
fig. 6 is a schematic block diagram of a 3D display image generating apparatus according to an embodiment of the present application;
fig. 7 is a schematic device structure diagram of a hardware running environment related to a 3D display image generating method according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the technical solution of the present application and are not intended to limit the present application.
For a better understanding of the technical solution of the present application, the following detailed description will be given with reference to the drawings and the specific embodiments.
The main solutions of the embodiments of the present application are:
in this embodiment, for convenience of description, the following description will be made with the 3D display image generating apparatus as an execution subject.
In the prior art, in the actual process of generating the 3D display image, three-dimensional images of multiple viewpoints are required to be acquired in real time, the common practice is to shoot the same scene simultaneously by using N cameras arranged in parallel, so that N images of different positions (viewpoints) are obtained, the manufacturing cost is high, and in addition, the content customization cost is very high in consideration of the wide application of single-viewpoint 3D LED large screens based on visual 3D in recent years. Therefore, how to generate 3D display images at low cost and meet the single-view 3D and multi-view 3D display requirements at the same time is an urgent problem to be solved.
The application provides a solution, which comprises the steps of obtaining a background layer image, an intermediate layer image and a foreground layer image to be overlapped, processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer, processing the foreground layer image based on an image processing mode of the 2D+alpha channel+depth map to obtain a first image, executing an image matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image, and executing an image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image. According to the application, the images of all layers are processed based on the image processing mode of 2D+alpha channel+depth map, and the processed images are fused, so that the cost for generating the 3D display image is reduced.
It should be noted that, the execution body of the embodiment may be a computing service device having functions of data processing, network communication, and program running, such as a tablet computer, a personal computer, a mobile phone, or an electronic device, a 3D display image generating device, or the like capable of implementing the above functions. The present embodiment and the following embodiments will be described below by taking a 3D display image generating apparatus as an example.
Based on this, an embodiment of the present application provides a method for generating a 3D display image, and referring to fig. 1, fig. 1 is a schematic flow chart of a first embodiment of the method for generating a 3D display image according to the present application.
In this embodiment, the method for generating a 3D display image includes steps S10 to S40:
step S10, obtaining a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
It should be noted that, the execution body of the embodiment may be a computing service device having functions of data processing, network communication, and program running, such as a tablet computer, a personal computer, a mobile phone, or an electronic device, a 3D display image generating device, or the like capable of implementing the above functions. The present embodiment and the following embodiments will be described below by taking a 3D display image generating apparatus as an example.
In this embodiment, the specific application scenario may be:
After the 3D display image generating device receives the task of generating the 3D display image, the 3D display image generating device acquires the background layer image, the intermediate layer image and the foreground layer image to be superimposed, and then the 3D display image generating device processes the background layer image, the intermediate layer image and the foreground layer image to be superimposed to generate the 3D display image.
Referring to fig. 2, fig. 2 depicts a 3D display schematic diagram of multi-layer image fusion in the present invention, where 101 is a background layer 2D camera capturing video/local 2D video image (background layer image), 102 is an intermediate layer "2d+alpha channel" video image, time, weather information (intermediate layer image), and 103 is a foreground layer "2d+alpha channel+depth map" video image, where the background layer and intermediate layer fused image is a 2D display, and the foreground layer fused image is a 3D display.
Step S20, processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer;
Specifically, the step of processing the intermediate layer image in the 2d+alpha channel-based image processing manner to obtain a first transparent color image corresponding to the intermediate layer further includes steps S21 to S23:
step S21, performing two-way processing on the middle layer image based on an image processing mode of 2D+alpha channels to obtain a left-way image and a right-way image;
step S22, setting the foreground corresponding to the left image as a first color image, and setting the foreground corresponding to the right image as a first Alpha channel image;
Step S23, obtaining a first transparent color image based on the first color image and the first Alpha channel image.
It should be noted that, in this embodiment, in order to ensure the transparent effect of the intermediate layer, the intermediate layer is encoded by adopting a 2d+alpha channel mode, and the video content of the 2d+alpha channel can be compressed by adopting a two-way splicing mode in consideration of compatible playing of the video. Fig. 3 depicts a schematic view of a multi-layer image fusion naked eye 3D display in the present invention. Wherein 301 is a left image background, which is black screen material, 302 is a left image foreground which is a color image, and the right image is the same as 301, the background is also black screen material, 303 is an Alpha channel image of the foreground to-be-scratched object. And then converting YUV format obtained by decoding the '2D+alpha channel' video into RGB format. Aiming at information such as middle layer time and weather, text information or BMP or PNG picture formats with Alpha channels can be directly obtained. Assuming that C is the color value of the pixel, F, B represents the foreground color value and the background color value of the pixel in the respective channel, α is the opacity value of the pixel in the channel, and then c=αf+ (1- α) B is present, where the value range of α is [0,1], when α=1, it is indicated that the pixel belongs to the foreground, when α=0, it is indicated that the pixel belongs to the background, and when 0< α <1, it is indicated that the value is a mixture of the foreground and the background. Under the RGB color model, it can be described as:
Step S30, processing the foreground layer image based on an image processing mode of 2D+alpha channel+depth map to obtain a first image, and executing an image matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image;
Specifically, the image processing method based on 2d+alpha channel+depth map processes the foreground layer image to obtain a first image, performs an image matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image, and further includes steps S31 to S35:
Step S31, based on the image processing mode of 2D+alpha channel, performing three paths of processing on the foreground layer image to obtain an upper path image, a middle path image and a lower path image;
Step S32, setting the foreground corresponding to the upper road image as a second color image, setting the foreground corresponding to the middle road image as a second Alpha channel image, and setting the foreground corresponding to the lower road image as a depth image;
Step S33, obtaining a first image based on the second color image, the second Alpha channel image and the depth map image;
Step S34, obtaining a second transparent color image based on the second color image and the second Alpha channel image;
and step S35, generating a virtual viewpoint color image based on the second transparent color image and the depth map image.
In this embodiment, to ensure a 3D transparent display effect of the foreground layer, the foreground layer is encoded in a mode of "2d+alpha channel+depth map", and in consideration of compatible playback of video, the video content of "2d+alpha channel+depth map" may be compressed in a three-way splicing mode. Fig. 4 depicts a schematic diagram of a video image of a foreground layer "2d+alpha channel+depth map" in the present invention, wherein 401 is an upper image background, which is a black screen material, 402 is a color image of a foreground object to be scratched, a middle image background is also a black screen material, 403 is an Alpha channel image of a foreground object to be scratched, a lower image background is also a black screen material, and 404 is a depth image of the foreground object to be scratched. And then converting YUV format obtained by decoding the '2D+alpha channel+depth map' video into RGB format.
In this embodiment, a virtual viewpoint color image is generated by real-time matting of a video image of a foreground layer "2d+alpha channel+depth map" and virtual viewpoint generation. In order to ensure real-time image matting of a foreground layer object, firstly, a '2D+alpha channel' image is processed, and then, a transparent color image obtained by the '2D+alpha channel' processing and a depth image are subjected to virtual viewpoint generation, so that a virtual viewpoint color image is obtained.
And S40, performing image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
Specifically, before the step of performing the image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain the 3D display image, the method includes steps S41 to S42:
step S41, obtaining a preset resolution of a 3D display image;
And step S42, cutting the background layer image based on the resolution to obtain a target background layer image.
It can be appreciated that the background layer image is a video acquired by a 2D camera, and in order to reduce the delay of displaying the image, a MJPEG compression format is usually selected, so that the decoded image format is an RGB format. If the 2D camera collects video and only supports video compression formats such as h.264/h.265 to output or is a local 2D video, the YUV format obtained by decoding the 2D video needs to be converted into an RGB format, and the corresponding conversion relationship is as follows:
R=Y+1.402(V-128)
G=Y-0.344(U-128)-0.174(V-128)
B=Y+1.772(U-128)
It can be understood that, because the original aspect ratio of the common 2D camera acquired video/local 2D video is inconsistent with the display aspect ratio of the 3D display terminal, if the picture is not cut, the picture deformation or black edge phenomenon will occur, and in order to meet the matching display of the background layer 2D camera acquired video/local 2D video image and the naked eye 3D display terminal such as LCD, LED, etc., the background layer 2D camera acquired video/local 2D video image needs to be cut.
Assuming that the 3D display terminal has a resolution of M 1×N1, where M 1 is a horizontal resolution, N 1 is a vertical resolution, the 2D camera captures video/local 2D video has a resolution of M 2×N2, where M 2 is a horizontal resolution, M 2 is a vertical resolution, whenWhen the video is acquired by the 2D camera/the local 2D video is cut, the horizontal resolution M 2 is kept unchanged, the vertical resolution is cut off in the up-down direction, and whenWhen the video is acquired by the 2D camera/local 2D video clipping mode, the vertical resolution N 2 is kept unchanged, and the left and right directions of the horizontal resolution are clipped
Specifically, the step of performing an image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image further includes step S43:
And step S43, performing image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
Specifically, the step of performing an image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image further includes steps a 01-a 03:
Step A01, carrying out 2D superposition on the target background layer image and the first transparent color image to obtain a first superposition image;
Step A02, obtaining grating parameters to determine the number l 1 of the RGB sub-pixels covered in the horizontal direction in one grating period;
and A03, determining a display value V of each sub-pixel, and obtaining a 3D display image based on the l 1 sub-pixels, the display value V of each sub-pixel and the first superposition image.
In this embodiment, when the video image of the foreground layer 2d+alpha channel+depth map is subjected to real-time matting and virtual viewpoint generation, and the transparent color image and the depth map obtained by processing the 2d+alpha channel are subjected to virtual viewpoint generation, although the virtual viewpoint color image can be obtained, the filling of the cavity region and the inconsistency of time domain information are often caused when the texture structure of the cavity region is processed by the conventional image filling method, so that the quality of the virtual viewpoint image is poor. In order to further improve the image filling quality of the cavity area, aiming at the correlation of local features of the image filled by the cavity in the airspace, multi-scale feature aggregation is adopted, and semantic coding under multiple scales is carried out for balancing accuracy and complexity.
In this embodiment, first, 2D superimposition is performed on the background layer and the intermediate layer video image in the RGB space, and then naked eye 3D synthesis is performed on the virtual viewpoint transparent video image layer generated by the foreground. According to grating parameters, namely the inclination angle, the horizontal intercept of the grating element and the number of views, firstly, the number l 1,l1 of the RGB sub-pixels covered by one grating period in the horizontal direction is determined to satisfy the following conditions:
wherein, And l is the horizontal physical intercept of one grating period, and t is the horizontal physical intercept of the grating element.
Calculating from which disparity map N the display value of each sub-pixel in the composite map comes, where N satisfies the formula:
Wherein,% represents the remainder, x is the coordinate position of the RGB sub-pixels, K off is the horizontal displacement amount representing the left upper edge of the 2D display screen and the edge point of the grating unit, and n is the total view point number of the grating.
Finally, the display value V of each sub-pixel in the composite graph is determined, preferably, in order to improve the display quality, V satisfies the formula:
Wherein int (N) is an integer part for obtaining N value, fra (N) is a decimal part of N value, V [ int (N) ] is a display value of a sub-pixel at the (x, y) th parallax map of int (N), and V [ int (N) +1] is a display value of a sub-pixel at the (x, y) th parallax map of int (N) +1. When 1 is less than or equal to N < N, the display value of the sub-pixel is obtained by interpolation of the sub-pixel display values at the same coordinates of the int (N) th disparity map and the int (N) +1 th disparity map according to the proportions 1-fra (N) and fra (N), when N is less than or equal to N < (n+0.85), the display value of the sub-pixel is determined by the sub-pixel at the N-th disparity map (x, y), and when (n+0.85) is less than or equal to N < (n+1), the display value of the sub-pixel is determined by the sub-pixel at the 1 st disparity map (x, y).
One or more technical schemes provided by the application have at least the following technical effects:
The application provides a 3D display image generation method and related equipment, which relate to the technical field of naked eye 3D display, and in the related technology, three-dimensional images of a plurality of viewpoints are required to be acquired in real time, the common practice is that N cameras which are arranged in parallel are used for shooting the same scene at the same time, so that N images of different positions (viewpoints) are obtained, and compared with high manufacturing cost, in the application, firstly, a background layer image, an intermediate layer image and a foreground layer image to be overlapped are acquired, then, the intermediate layer image is processed based on an image processing mode of a 2D+alpha channel, so that a first transparent color image corresponding to the intermediate layer is obtained, further, based on an image processing mode of a 2D+alpha channel+depth map, the foreground layer image is processed, so that a first image is obtained, a matting operation and a virtual viewpoint generation operation are carried out on the first image, so that a virtual viewpoint color image is obtained, and finally, an image fusion operation is carried out on the background layer image, the first transparent color image and the virtual viewpoint color image so that a 3D display image is obtained.
In the second embodiment of the present application, the same or similar content as in the first embodiment of the present application may be referred to the above description, and will not be repeated. On this basis, referring to fig. 2, the step of generating a virtual viewpoint color image based on the second transparent color image and the depth map image further includes steps B01 to B04:
Step B01, dividing the second transparent color image into a plurality of groups of image blocks with the size of n multiplied by n, wherein the value of n is set in advance;
step B02, obtaining preset features of the second transparent color image, and calculating cosine similarity of the image blocks based on the preset features to obtain attention scores of each image block;
Step B03, reconstructing the preset features based on the attention score of each image block to obtain target features;
and step B04, obtaining a virtual viewpoint color image based on the target feature, the second transparent color image and the depth map image.
For example, in the present embodiment, the cosine similarity is calculated according to a 6×6-sized image blockAssuming F i is the ith image block of a given feature F, the attention score for each image block is obtained asThe features are reconstructed from the attention profile:
Where F i is the i-th image block of the reconstructed feature F rec.
When reconstructing feature acquisition, using voids of different expansion rates to acquire multi-scale features is:
Wherein Conv k(Frec) represents a hole convolution of different expansion rates, where k may be 1, 2, 4, 8. Polymerizing the multi-features to obtain polymerized multi-scale features:
In the application, the jump connection is adopted to prevent the input information loss, and the input loss cost of a given characteristic F is as follows:
Fconcact＝Fc-F
In this embodiment, in order to ensure continuity of the hole area filling image, the loss cost of the feature F and the time domain and space domain characteristics of the hole image are combined, and the consistency of the features of the hole boundary and the interior is ensured through spatial equalization.
It should be noted that the foregoing examples are only for understanding the present application, and are not meant to limit the method for generating a 3D display image according to the present application, and more forms of simple transformation based on the technical concept are all within the scope of the present application.
The present application also provides a 3D display image generating apparatus, referring to fig. 6, the 3D display image generating apparatus includes:
the acquisition module 10 is used for acquiring a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
the first image processing module 20 is configured to process the intermediate layer image based on an image processing manner of 2d+alpha channel, so as to obtain a first transparent color image corresponding to the intermediate layer;
The second image processing module 30 is configured to process the foreground layer image based on an image processing manner of 2d+alpha channel+depth map, to obtain a first image, and perform an image matting operation and a virtual viewpoint generating operation on the first image, to obtain a virtual viewpoint color image;
And a fusion module 40, configured to perform an image fusion operation on the background layer image, the first transparent color image, and the virtual viewpoint color image, to obtain a 3D display image.
In an embodiment, the 3D display image generating apparatus further includes a trimming module including:
The first acquisition unit is used for acquiring a preset 3D display image resolution;
and the cutting unit is used for cutting the background layer image based on the resolution ratio to obtain a target background layer image.
In one embodiment, the fusion module comprises:
and the fusion unit is used for performing image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
In an embodiment, the fusion module further comprises:
the superposition unit is used for carrying out 2D superposition on the target background layer image and the first transparent color image to obtain a first superposition image;
the determining unit is used for acquiring grating parameters to determine the number l 1 of the RGB sub-pixels covered in the horizontal direction in one grating period;
the first obtaining unit is configured to determine a display value V of each sub-pixel, and obtain a 3D display image based on l 1 sub-pixels, the display value V of each sub-pixel, and the first superimposed image.
In an embodiment, the first image processing module includes:
The first processing unit is used for performing two-way processing on the middle layer image based on an image processing mode of a 2D+alpha channel to obtain a left-way image and a right-way image;
The first setting unit is used for setting the foreground corresponding to the left image as a first color image and setting the foreground corresponding to the right image as a first Alpha channel image;
and the second obtaining unit is used for obtaining a first transparent color image based on the first color image and the first Alpha channel image.
In an embodiment, the second image processing module includes:
the second processing unit is used for performing three paths of processing on the foreground layer image based on the image processing mode of the 2D+alpha channel to obtain an upper path image, a middle path image and a lower path image;
A second setting unit, configured to set a foreground corresponding to the upper road image as a second color image, set a foreground corresponding to the middle road image as a second Alpha channel image, and set a foreground corresponding to the lower road image as a depth map image;
a third obtaining unit, configured to obtain a first image based on the second color image, the second Alpha channel image, and the depth map image;
A fourth obtaining unit, configured to obtain a second transparent color image based on the second color image and a second Alpha channel image;
And the generating unit is used for generating a virtual viewpoint color image based on the second transparent color image and the depth map image.
In an embodiment, the second image processing module further comprises:
a blocking unit, configured to divide the second transparent color image into a plurality of groups of image blocks with size of n×n, where a value of n is set in advance;
The second acquisition unit is used for acquiring preset features of the second transparent color image, and calculating cosine similarity of the image blocks based on the preset features so as to obtain attention scores of each image block;
a reconstruction unit, configured to reconstruct the preset feature based on the attention score of each image block, to obtain a target feature;
and a fifth obtaining unit, configured to obtain a virtual viewpoint color image based on the target feature, the second transparent color image, and the depth map image.
The 3D display image generation device provided by the application can solve the technical problem of 3D display image generation by adopting the 3D display image generation method in the embodiment. Compared with the prior art, the 3D display image generating device provided by the application has the same beneficial effects as the 3D display image generating method provided by the above embodiment, and other technical features in the 3D display image generating device are the same as the features disclosed by the method of the above embodiment, and are not repeated herein.
The application provides a 3D display image generating device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the 3D display image generating method in the first embodiment.
Referring now to fig. 7, a schematic diagram of a 3D display image generating apparatus suitable for use in implementing embodiments of the present application is shown. The 3D display image generating apparatus in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal DIGITAL ASSISTANT: personal digital assistants), PADs (Portable Application Description: tablet computers), PMPs (Portable MEDIA PLAYER: portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The 3D display image generating apparatus shown in fig. 5 is only one example, and should not bring any limitation to the functions and the use scope of the embodiment of the present application.
As shown in fig. 7, the 3D display image generating apparatus may include a processing device 1001 (e.g., a central processor, a graphics processor, etc.), which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access Memory (RAM: random Access Memory) 1004. In the RAM1004, various programs and data required for the operation of the 3D display image generating device are also stored. The processing device 1001, the ROM1002, and the RAM1004 are connected to each other by a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. In general, a system including an input device 1007 such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc., an output device 1008 including a Liquid crystal display (LCD: liquid CRYSTAL DISPLAY), a speaker, a vibrator, etc., a storage device 1003 including a magnetic tape, a hard disk, etc., and a communication device 1009 may be connected to the I/O interface 1006. The communication means 1009 may allow the 3D display image generating device to communicate with other devices wirelessly or by wire to exchange data. Although a 3D display image generating device having various systems is shown in the figures, it should be understood that not all of the illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. The above-described functions defined in the method of the disclosed embodiment of the application are performed when the computer program is executed by the processing device 1001.
The 3D display image generation device provided by the application can solve the technical problem of 3D display image generation by adopting the 3D display image generation method in the embodiment. Compared with the prior art, the beneficial effects of the 3D display image generating device provided by the application are the same as those of the 3D display image generating method provided by the above embodiment, and other technical features of the 3D display image generating device are the same as those disclosed by the method of the above embodiment, and are not repeated herein.
It is to be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon for performing the 3D display image generating method in the above-described embodiments.
The computer readable storage medium provided by the present application may be, for example, a USB flash disk, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access Memory (RAM: random Access Memory), a Read-Only Memory (ROM), an erasable programmable Read-Only Memory (EPROM: erasable Programmable Read Only Memory or flash Memory), an optical fiber, a portable compact disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this embodiment, the computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, RF (Radio Frequency) and the like, or any suitable combination of the foregoing.
The above-mentioned computer-readable storage medium may be contained in the 3D display image generating apparatus or may exist alone without being incorporated in the 3D display image generating apparatus.
The computer-readable storage medium described above carries one or more programs that, when executed by the 3D display image generating apparatus, cause the 3D display image generating apparatus to:
Acquiring a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
Processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer;
processing the foreground layer image based on an image processing mode of 2D+alpha channel+depth map to obtain a first image, and executing a matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image;
and performing image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of remote computers, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN: local Area Network) or a wide area network (WAN: wide Area Network), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present application may be implemented in software or in hardware. Wherein the name of the module does not constitute a limitation of the unit itself in some cases.
The readable storage medium provided by the application is a computer readable storage medium, and the computer readable storage medium stores computer readable program instructions (i.e. a computer program) for executing the 3D display image generation method, so that the technical problem of 3D display image generation can be solved. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the application are the same as those of the 3D display image generation method provided by the embodiment, and are not described in detail herein.
The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of a 3D display image generation method as described above.
The computer program product provided by the application can solve the technical problem of generating the 3D display image. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the 3D display image generating method provided by the above embodiment, and are not described herein.
The foregoing description is only a partial embodiment of the present application, and is not intended to limit the scope of the present application, and all the equivalent structural changes made by the description and the accompanying drawings under the technical concept of the present application, or the direct/indirect application in other related technical fields are included in the scope of the present application.
Claims (10)
1. A 3D display image generation method, characterized in that the 3D display image generation method comprises:
Acquiring a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
Processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer;
processing the foreground layer image based on an image processing mode of 2D+alpha channel+depth map to obtain a first image, and executing a matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image;
and performing image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
2. The method for generating a 3D display image according to claim 1, wherein before the step of performing an image fusion operation on the background layer image, the first transparent color image, and the virtual viewpoint color image to obtain a 3D display image, the method comprises:
Acquiring a preset 3D display image resolution;
Cutting the background layer image based on the resolution ratio to obtain a target background layer image;
The step of performing an image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image further includes:
and performing image fusion operation on the target background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
3. The method for generating a 3D display image according to claim 2, wherein the step of performing an image fusion operation on the target background layer image, the first transparent color image, and the virtual viewpoint color image to obtain a 3D display image further comprises:
2D overlapping is carried out on the target background layer image and the first transparent color image, so that a first overlapped image is obtained;
Acquiring grating parameters to determine the number l 1 of the RGB sub-pixels covered in the horizontal direction in one grating period;
And determining a display value V of each sub-pixel, and obtaining a 3D display image based on l 1 sub-pixels, the display value V of each sub-pixel and the first superposition image.
4. The method for generating a 3D display image according to claim 1, wherein the step of processing the intermediate layer image by using the image processing method based on 2d+alpha channel to obtain a first transparent color image corresponding to the intermediate layer further comprises:
performing two-way processing on the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a left-way image and a right-way image;
Setting a foreground corresponding to the left image as a first color image, and setting a foreground corresponding to the right image as a first Alpha channel image;
and obtaining a first transparent color image based on the first color image and the first Alpha channel image.
5. The method for generating a 3D display image according to claim 1, wherein the step of processing the foreground layer image based on the image processing mode of 2d+alpha channel+depth map to obtain a first image, performing a matting operation and a virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image further comprises:
Based on an image processing mode of 2D+alpha channels, performing three paths of processing on the foreground layer image to obtain an upper path image, a middle path image and a lower path image;
Setting a foreground corresponding to the upper path image as a second color image, setting a foreground corresponding to the middle path image as a second Alpha channel image, and setting a foreground corresponding to the lower path image as a depth image;
Obtaining a first image based on the second color image, the second Alpha channel image and the depth map image;
Obtaining a second transparent color image based on the second color image and a second Alpha channel image;
and generating a virtual viewpoint color image based on the second transparent color image and the depth map image.
6. The 3D display image generation method according to claim 5, wherein the step of generating a virtual viewpoint color image based on the second transparent color image and a depth map image further comprises:
Dividing the second transparent color image into a plurality of groups of image blocks with the size of n multiplied by n, wherein the value of n is set in advance;
Acquiring preset features of the second transparent color image, and calculating cosine similarity of the image blocks based on the preset features to obtain attention scores of each image block;
Reconstructing the preset features based on the attention score of each image block to obtain target features;
And obtaining a virtual viewpoint color image based on the target feature, the second transparent color image and the depth map image.
7. A 3D display image generation apparatus, characterized in that the 3D display image generation apparatus includes:
the acquisition module is used for acquiring a background layer image, an intermediate layer image and a foreground layer image to be overlapped;
the first image processing module is used for processing the intermediate layer image based on an image processing mode of a 2D+alpha channel to obtain a first transparent color image corresponding to the intermediate layer;
The second image processing module is used for processing the foreground layer image based on an image processing mode of 2D+alpha channel+depth map to obtain a first image, and performing image matting operation and virtual viewpoint generating operation on the first image to obtain a virtual viewpoint color image;
And the fusion module is used for performing image fusion operation on the background layer image, the first transparent color image and the virtual viewpoint color image to obtain a 3D display image.
8. A 3D display image generating device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the 3D display image generating method according to any one of claims 1 to 6.
9. A storage medium, characterized in that the storage medium is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, realizes the steps of the 3D display image generation method according to any one of claims 1 to 6.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the steps of the 3D display image generation method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510063417.0A CN120111200A (en) | 2025-01-15 | 2025-01-15 | 3D display image generation method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510063417.0A CN120111200A (en) | 2025-01-15 | 2025-01-15 | 3D display image generation method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN120111200A true CN120111200A (en) | 2025-06-06 |
Family
ID=95885734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510063417.0A Pending CN120111200A (en) | 2025-01-15 | 2025-01-15 | 3D display image generation method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120111200A (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101902657A (en) * | 2010-07-16 | 2010-12-01 | 浙江大学 | A method for generating virtual multi-viewpoint images based on depth map layering |
| CN102239506A (en) * | 2008-10-02 | 2011-11-09 | 弗兰霍菲尔运输应用研究公司 | Intermediate view synthesis and multi-view data signal extraction |
| KR20130051119A (en) * | 2011-11-09 | 2013-05-20 | 광주과학기술원 | Image display apparatus and method for operating the same |
| CN104869386A (en) * | 2015-04-09 | 2015-08-26 | 东南大学 | Virtual viewpoint synthesizing method based on layered processing |
| CN105791803A (en) * | 2016-03-16 | 2016-07-20 | 深圳创维－Rgb电子有限公司 | A display method and system for converting two-dimensional images into multi-viewpoint images |
| CN109561296A (en) * | 2017-09-22 | 2019-04-02 | 佳能株式会社 | Image processing apparatus, image processing method, image processing system and storage medium |
| WO2019142794A1 (en) * | 2018-01-22 | 2019-07-25 | シャープ株式会社 | Image generating device, display processing device, image reproducing system, image generating method, control program, and recording medium |
| CN111988598A (en) * | 2020-09-09 | 2020-11-24 | 江苏普旭软件信息技术有限公司 | Visual image generation method based on far and near view layered rendering |
| CN115176459A (en) * | 2021-01-25 | 2022-10-11 | 京东方科技集团股份有限公司 | Virtual viewpoint synthesis method, electronic device, and computer-readable medium |
| CN115941920A (en) * | 2022-11-23 | 2023-04-07 | 马凯翔 | Naked eye 3D video generation method, device, equipment and storage medium |
| WO2023235273A1 (en) * | 2022-06-02 | 2023-12-07 | Leia Inc. | Layered view synthesis system and method |
| US20240096020A1 (en) * | 2022-09-16 | 2024-03-21 | Electronics And Telecommunications Research Institute | Apparatus and method for generating moving viewpoint motion picture |
| US20240378789A1 (en) * | 2021-06-03 | 2024-11-14 | Koninklijke Philips N.V. | Depth segmentation in multi-view videos |
| US20240430395A1 (en) * | 2023-06-26 | 2024-12-26 | Adeia Guides Inc. | Viewpoint synthesis with enhanced 3d perception |
-
2025
- 2025-01-15 CN CN202510063417.0A patent/CN120111200A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102239506A (en) * | 2008-10-02 | 2011-11-09 | 弗兰霍菲尔运输应用研究公司 | Intermediate view synthesis and multi-view data signal extraction |
| CN101902657A (en) * | 2010-07-16 | 2010-12-01 | 浙江大学 | A method for generating virtual multi-viewpoint images based on depth map layering |
| KR20130051119A (en) * | 2011-11-09 | 2013-05-20 | 광주과학기술원 | Image display apparatus and method for operating the same |
| CN104869386A (en) * | 2015-04-09 | 2015-08-26 | 东南大学 | Virtual viewpoint synthesizing method based on layered processing |
| CN105791803A (en) * | 2016-03-16 | 2016-07-20 | 深圳创维－Rgb电子有限公司 | A display method and system for converting two-dimensional images into multi-viewpoint images |
| CN109561296A (en) * | 2017-09-22 | 2019-04-02 | 佳能株式会社 | Image processing apparatus, image processing method, image processing system and storage medium |
| WO2019142794A1 (en) * | 2018-01-22 | 2019-07-25 | シャープ株式会社 | Image generating device, display processing device, image reproducing system, image generating method, control program, and recording medium |
| CN111988598A (en) * | 2020-09-09 | 2020-11-24 | 江苏普旭软件信息技术有限公司 | Visual image generation method based on far and near view layered rendering |
| CN115176459A (en) * | 2021-01-25 | 2022-10-11 | 京东方科技集团股份有限公司 | Virtual viewpoint synthesis method, electronic device, and computer-readable medium |
| US20240378789A1 (en) * | 2021-06-03 | 2024-11-14 | Koninklijke Philips N.V. | Depth segmentation in multi-view videos |
| WO2023235273A1 (en) * | 2022-06-02 | 2023-12-07 | Leia Inc. | Layered view synthesis system and method |
| US20240096020A1 (en) * | 2022-09-16 | 2024-03-21 | Electronics And Telecommunications Research Institute | Apparatus and method for generating moving viewpoint motion picture |
| CN115941920A (en) * | 2022-11-23 | 2023-04-07 | 马凯翔 | Naked eye 3D video generation method, device, equipment and storage medium |
| US20240430395A1 (en) * | 2023-06-26 | 2024-12-26 | Adeia Guides Inc. | Viewpoint synthesis with enhanced 3d perception |
Non-Patent Citations (4)
| Title |
|---|
| HO SUB LEE, ET AL.: "Foreground-based depth map generation for 2D-to-3D conversion", 《2015 IEEE ISCAS》, 30 July 2015 (2015-07-30) * |
| YANG SHUAI, ET AL.: "LayerPano3D:layered 3D Panorama for hyper-immersive scene generation", 《ARXIV》, 23 August 2024 (2024-08-23) * |
| YU HONGXING, ET AL.: "WonderWorld:interactive 3D scene generation from a signgle image", 《ARXIV》, 13 June 2024 (2024-06-13) * |
| 韩冬雪: "多视点裸眼3D电视的虚拟视点生成方法研究", 《中国优秀硕士学位论文全文库（电子期刊）》, 15 September 2019 (2019-09-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10235795B2 (en) | Methods of compressing a texture image and image data processing system and methods of generating a 360 degree panoramic video thereof | |
| US20180174619A1 (en) | Interface for application-specified playback of panoramic video | |
| CN110663068B (en) | Coordinate mapping for rendering panoramic scenes | |
| CN105447898A (en) | Method and device for displaying 2D application interface in virtual real device | |
| CN106375748A (en) | Method and apparatus for splicing three-dimensional virtual reality panoramic view, and electronic device | |
| KR102059732B1 (en) | Digital video rendering | |
| US20170186243A1 (en) | Video Image Processing Method and Electronic Device Based on the Virtual Reality | |
| CN109983500A (en) | Reproject a flat-panel projection of a panoramic video image for rendering by the app | |
| US20130027389A1 (en) | Making a two-dimensional image into three dimensions | |
| CN111091491B (en) | Panoramic video pixel redistribution method and system for equidistant cylindrical projection | |
| CN108833877A (en) | Image processing method and device, computer installation and readable storage medium storing program for executing | |
| CN107197135A (en) | A kind of video generation method, player method and video-generating device, playing device | |
| CN114782648A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| US20130120527A1 (en) | Electronic apparatus and display control method | |
| US12190431B2 (en) | Image processing systems and methods | |
| CN115293994B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN114500970A (en) | Panoramic video image processing and displaying method and device | |
| CN115801983A (en) | Image superposition method and device and electronic equipment | |
| CN120111200A (en) | 3D display image generation method and related equipment | |
| CN111726598B (en) | Image processing method and device | |
| CN118413745A (en) | Method, device, equipment and storage medium for playing panoramic video | |
| US20250148699A1 (en) | Volumetric immersive experience with multiple views | |
| JP7214926B1 (en) | Image processing method, device, electronic device and computer-readable storage medium | |
| CN119991526B (en) | City-level digital twin-oriented collaborative rendering method, device and storage medium | |
| CN115442580B (en) | Naked eye 3D picture effect processing method for portable intelligent equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |