TW201803342A - Apparatus and method for combining with wavelet transformer and edge detector to generate a depth map from a single image can convert 2D planar imaging signals into 3D stereoscopic image signals - Google Patents

Abstract

An apparatus and method for combining with wavelet transformer and edge detector to generate a depth map from a single image provide a conversion system for converting 2D planar imaging signals into 3D stereoscopic image signals. In the depth map generated by the depth map establishment method of the invention, each image block is respectively assigned with a depth value. The method of the invention comprises the following steps of inputting an original image (S1), executing wavelet transformation and edge detection on the original image (S2), establishing a defocused map according to the wavelet transforming results and the edge detecting results (S3), executing depth prediction according to the wavelet coefficients of the wavelet transformation (S4), corresponding the depth value to the defocused map to execute depth diffusion (S5), and finally generating a depth map (S6).

Description

Device and method for establishing single image depth map by combining wavelet transformation and edge detection

The invention relates to a videoconferencing system, in particular to a device and a method for generating a depth map, which are used to convert two-dimensional image data into three-dimensional image data.

Since the launch of the Avatar 3D movie in 2009, people have begun to pursue the entertainment effects of 3D display technology. The 3D broadcast of the World Cup in 2010 and the virtual reality helmet in 2016 all show that our entertainment industry has changed from 2D. Like 3D, people are no longer satisfied with the image effects brought by 2D, and began to pursue 3D display technology. At present, because of the commercialization of 3D display technology and services related to 3D content are also increasing, the relative users for 3D Demand has also increased. However, there has been no significant progress in the development of 3D content. In contrast, there are quite a large number of 2D images or videos, and the images taken by individuals are also 2D images, waiting to be effectively used. To convert to 3D video applications.

For this reason, some inventors have invented, for example, Chinese Patent Publication No. CN 103559701 A "Depth estimation method of two-dimensional single-view image based on DCT coefficient entropy", which proposes to predict depth in a single image with depth of field. It captures every pixel in the image to be processed. Take the pixel as the center to capture the window as a sub-image. After wavelet transformation of these sub-images, quantize the wavelet coefficient value in the image, then calculate its coefficient entropy as the blur of the pixel, and then pass through The linear mapping maps the entropy value to an 8-bit depth range to get a one-pixel depth map. In addition, some inventors have proposed Chinese Patent Publication No. CN 10247539B "Method for 2D to 3D Video Image", which uses wavelet transform to predict the depth of a single image with depth of field, and performs wavelet transform on the original image to Extract the high-frequency coefficients in the image, divide the image into several blocks, and then count the number of non-zero coefficients in each block as the blur of the block. At the same time, based on the color characteristics of the original image, the original image is processed. The color is divided into three types of pixel sets, and then the ambiguity of each pixel set is compared with a statistical average. The pixel set corresponding to the maximum value is used as the foreground, the pixel set corresponding to the second largest value is regarded as the middle shot, and the minimum corresponding pixel value is Considered as the background, the depth of the foreground, the middle scene and the background are given different depth values by the system of preset depth of field to obtain the depth map.

According to the known techniques disclosed above, the prior art and the known depth map generation method have related disadvantages. For example, for a window centered on a pixel, the size of the window setting needs to be manually set, and it cannot be automatically adjusted according to different images. . Furthermore, using the color characteristics of the original image to segment the image into a collection of three types of pixels, and only dividing the image into the foreground, mid-range, and background, it is clear that the rich images we usually see have different levels of depth information, leading to Unable to produce the correct depth map.

In view of this, the inventors have been engaged in related industries and research based on many years of research and analysis of the existing method of generating depth maps, and hope to invent the depth of improving the known shortcomings. A method for generating a map. For this reason, the main purpose of the present invention is a device and method for creating a single image depth map by combining wavelet transformation and edge detection without human intervention and conforming to the depth information viewed by the human eye.

To achieve the above object, the device and method for creating a single image depth map by combining wavelet transformation and edge detection according to the present invention have an image acquisition unit for acquiring or inputting an original image; an image analysis unit And an information link with the image capturing unit for performing image analysis on the original image captured or input by the image capturing unit, wherein the image analysis can execute an image analysis algorithm and the image analysis algorithm The method can be wavelet transform, edge detection, etc .; an image synthesis unit is linked to the image analysis unit. After the image analysis unit executes the image analysis algorithm, the image synthesis unit can perform the image synthesis by analyzing the results of the image analysis unit. To generate a defocus map; a depth calculation unit that is informationally linked to the image synthesis unit; after the image synthesis unit generates the defocus map, the depth calculation unit performs a depth prediction based on the analysis result of the image analysis unit Algorithm and map the result of depth prediction to the defocus map so that the depth calculation unit can The results correspond to the defocus map, performing a deep diffusion algorithm, and the depth of diffusion algorithm may be a Laplacian interpolation technique or a global interpolation algorithm, it is not limited thereto.

1‧‧‧ Device for creating a single image depth map by combining wavelet transform and edge detection

11‧‧‧Image Acquisition Unit

12‧‧‧Image Analysis Unit

13‧‧‧Image Synthesis Unit

14‧‧‧Deep Computing Unit

S1‧‧‧ Input original image steps

S2‧‧‧Image analysis steps

S22‧‧‧Edge detection steps

S23‧‧‧ Wavelet conversion steps

S231‧‧‧Convert to grayscale image

S232‧‧‧Find the local maximum straight step

S233‧‧‧ local maximum corresponding to wavelet coefficient value

S234‧‧‧Threshold calculation result

S3‧‧‧Steps to create a defocus map

S4‧‧‧ depth prediction steps

Steps in S41‧‧‧Histogram Local Maximum

S42‧‧‧ Create window steps based on number

S43‧‧‧ Steps for calculating the ambiguity of the wavelet transform result

S44‧‧‧Depth prediction result steps

S5‧‧‧Deep diffusion step

S6‧‧‧ Generate depth map step

FIG. 1 is a structural diagram of the present invention.

Figure 2 is a flowchart of the steps of the present invention.

Figure 3 is a schematic diagram of the implementation of the present invention.

Figure 4 is a schematic diagram (I) of the implementation of the present invention.

FIG. 5 is a schematic diagram (2) of the implementation of the present invention.

FIG. 6 is a schematic diagram of an embodiment of the present invention.

FIG. 7 is a schematic diagram (1) of an embodiment of the present invention.

FIG. 8 is a schematic view (two) of the implementation of the present invention.

In the description of the following description, the term "depth map" refers to a two-dimensional matrix of depth values, and each depth value in the matrix corresponds to the relative position of a scene, and each depth value represents a specific reference position The distance to the relative position of the scene, if each pixel of a 2D image has its own depth value, the 2D image can be displayed using 3D technology.

In order to allow your reviewers to have a better understanding and understanding of the purpose, technical means, and effect of the present invention, we would like to explain it with preferred embodiments and drawings.

Please refer to "Fig. 1", which shows a schematic diagram of the structure of the present invention. As shown in the figure, the apparatus 1 for creating a single image depth map by combining wavelet transformation and edge detection according to the present invention is mainly composed of an image capture It is composed of an acquisition unit 11, an image analysis unit 12, an image synthesis unit 13, and a depth calculation unit 14. The image acquisition unit 11 can acquire an original image, and the original image is described. It is a 2D image or video. The image analysis unit 12 is information-linked to the image capture unit 11 to receive the original image and execute a plurality of image analysis algorithms. The image analysis algorithm may be One or a combination of a wavelet transform algorithm, an edge detection algorithm, and the wavelet transform algorithm may be discrete wavelet transform or continuous wavelet transform, and the edge detection algorithm may be Roberts Cross operator, Prewitt operator Operator, Sobel operator, Canny operator, compass operator, Marr-Hildreth, and wavelet transform. Any edge detection algorithm that can detect the original image is within the scope of the present invention, but it is not This is the limit. The image synthesizing unit 13 and the image analyzing unit 12 are information-linked to perform an image synthesizing of the image results analyzed by the image analyzing unit 12 to generate a defocus image. The depth calculation unit 14 and the image synthesis unit 13 are informationally linked. After performing a depth prediction algorithm based on the image results analyzed by the image analysis unit 12, the depth prediction algorithm is used to obtain the results through the image synthesis unit 13 After synthesis, the depth calculation unit 14 then executes a depth diffusion algorithm, and the depth diffusion algorithm may be a Laplacian interpolation technique or a global interpolation algorithm, and is not limited to this. The original image is paired with a depth map.

Carrying on the above, please refer to "Figure 2", which shows a flowchart of the steps of the present invention. As shown in the figure, the implementation steps of the present invention are as follows: an input original image step S1, which is an image capture The original image input by the unit 11. An image analysis step S2, which includes a wavelet conversion step S23 and an edge detection step S22, and performs a wavelet conversion analysis and an edge detection analysis on the original image, wherein the wavelet conversion step S23 performs a wavelet conversion Algorithm to generate a wavelet transform analysis result, and the wavelet transform algorithm may be one of a discrete wavelet transform or a continuous wavelet transform, and is not limited thereto, and the edge detection step S22 is performed An edge detection algorithm to generate an edge detection result, and the edge detection algorithm can be Roberts Cross operator, Prewitt operator, Sobel operator, Canny operator, compass operator, Marr-Hildreth, One of the wavelet transforms, and please refer to "Figure 3" for the matching. The figure shows the implementation of the present invention. The figure shows the results of the binarization of the wavelet transform analysis. A step S3 of establishing a defocus map is the result of the wavelet transform analysis by the image synthesis unit 13 and the The results of edge detection are synthesized to generate a defocus image. Please refer to "Figure 4" for the matching. The figure shows the implementation schematic diagram (1) of the present invention, and the defocus image is shown in the figure. The synthesis is that the result of the edge detection corresponds to the result of the wavelet transform analysis to extract the coefficient values of the pixels in the edge detection result to the result of the wavelet transform analysis. A depth prediction step S4 is to perform a depth prediction algorithm for the depth calculation unit 14 to perform a depth prediction on the original image according to the wavelet transform result of the original image according to the image analysis step S2. The depth calculation unit 14 After executing the depth prediction algorithm, the defocus map generated in step S3 of establishing a defocus map is synthesized through the image synthesis unit 13 to generate a defocused depth map. The synthesis is that the result of the depth prediction corresponds to the result of the defocus map, and the result of the depth prediction is replaced into the defocus map. A depth diffusion step S5 is to perform a depth diffusion algorithm based on the defocused depth map, and the depth diffusion algorithm may be the Laplacian interpolation technique or the global interpolation algorithm. Finally, a depth map generation step S6 is performed. After the depth calculation unit 14 executes the depth diffusion algorithm, a depth map will be generated. Please refer to "Figure 5", which shows the implementation schematic diagram (2) of the present invention. The depth map is shown in the figure.

Following the above description, please also refer to "Figure 6". The figure is a schematic diagram of an embodiment of the present invention. As shown in the figure, the step flow of the depth prediction algorithm performed by the depth prediction step S4 is: The number of local maximums of the histogram step S41 is to find the number of peaks of the grayscale value histogram of the original image in the grayscale value histogram, and a window is established based on the number of steps S42, based on the number of peaks A calculation window is established, and a pair of wavelet transformation results is used to perform the ambiguity calculation S43. Based on the wavelet transformation result, a neighborhood calculation is performed with the center pixel of the window as the center, and the wavelet transformation result of the neighborhood of the center pixel is calculated. Step S44 of the depth prediction result is to perform depth prediction based on the result of calculating the pixel neighborhood of the center point, that is, the blur degree.

其中，f(x)為所擷取之所有係數值，並將所有係數值中，取前三大之係數值，透過方程式(1)進行閥值之計算，當閥值計算完成後，滿足下列函式：

其中，I(m,n)為小波轉換結果，Th為計算之閥值，並請搭配參閱「第3圖」，圖中所示為本發明之實施示意圖，如圖所示為小波轉換後並二值化之結果，即為尚未設立閥值之結果，並請參閱「第8圖」，圖中所示為本發明之實施立示意圖(二)，如圖所示為閥值設立後之結果，將大於等於閥值之結果設為255，即白色部分，小於閥值之結果為0，即黑色部分。 Following the above description, please also refer to "Figure 7". The figure shows the schematic diagram (I) of the embodiment of the present invention. As shown in the figure, the wavelet transformation step S23 further includes a wavelet transformation threshold setting step. The steps include a step S231 of converting to a grayscale image, which converts the original image into a grayscale image. A step S232 of finding a local maximum is to establish a grayscale value histogram based on the grayscale image of the original image, and find the grayscale value where the peak is located in the grayscale value histogram. A local maximum corresponds to the wavelet coefficient value step S233. All grayscale values of the found peak are located at positions corresponding to the original image to the coefficient values in the wavelet transform result, and all coefficient values are set. Take it out. A threshold value calculation result step S234 is to set the wavelet transform threshold value through numerical analysis through numerical analysis. The numerical analysis can be Simpson's law, and the threshold value calculation function is as follows:

Among them, f (x) is all the captured coefficient values, and among all the coefficient values, the first three major coefficient values are taken, and the threshold value is calculated through equation (1). When the threshold value calculation is completed, the following Function:

Among them, I (m, n) is the result of wavelet transformation, Th is the calculated threshold value, and please refer to "Figure 3" for matching. The result of binarization is the result that the threshold has not been established. Please refer to "Figure 8". The figure shows the schematic diagram of the implementation of the present invention (II). , Set the result greater than or equal to the threshold to 255, that is, the white part, and the result less than the threshold to 0, that is, the black part.

In summary, the device and method for creating a single image depth map by combining wavelet transformation and edge detection according to the present invention are mainly based on an image analysis algorithm to perform image analysis on an original image, so that a depth calculation unit can After the depth prediction algorithm is executed, the depth diffusion algorithm is executed to generate the depth map. Since the image analysis algorithm is fast and accurate, and does not require complicated calculations, the measurement efficiency is good, and because the present invention does not need to be complicated and A huge calculation, so the cost is relatively reduced, and it can achieve the main purpose of the present invention without artificial intervention, and in accordance with the depth information viewed by the human eye, a device that creates a single image depth map by combining wavelet transformation and edge detection, and Its method.

Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the scope of patent application for the present invention. Anyone skilled in this art can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention is not limited thereto.

S1‧‧‧ Input original image steps

S2‧‧‧Image analysis steps

S22‧‧‧ Wavelet conversion steps

S23‧‧‧Edge detection steps

S3‧‧‧Steps to create a defocus map

S4‧‧‧ depth prediction steps

S5‧‧‧Deep diffusion step

S6‧‧‧ Generate depth map step

Claims (10)

A device for creating a single image depth map by combining wavelet transformation and edge detection, which includes: an image acquisition unit for inputting an original image; an image analysis unit that is information-linked to the image acquisition unit, and An image analysis algorithm is executed to analyze the original image input by the image capture unit; an image synthesis unit is information-linked with the image analysis unit to analyze the original image by the image analysis unit. Analyze the results, perform an image synthesis on the plurality of analysis results to generate a defocus map; a depth calculation unit, which is information-linked to the image synthesis unit, and can perform a depth prediction algorithm based on the plurality of analysis results to complete Then, a depth diffusion algorithm is executed, and a depth map can be generated after the depth diffusion algorithm is executed. The device for creating a single image depth map by combining wavelet transformation and edge detection as described in item 1 of the scope of patent application, wherein the image analysis algorithm may be one of a wavelet transformation algorithm or an edge detection algorithm. One or a combination. The device for creating a single image depth map by combining wavelet transformation and edge detection as described in item 2 of the scope of the patent application, wherein the wavelet transformation algorithm can be one of discrete wavelet transformation or continuous wavelet transformation. The device for creating a single image depth map by combining wavelet transformation and edge detection as described in item 2 of the scope of the patent application, wherein the edge detection algorithm can be Roberts Cross operator, Prewitt operator, Sobel operator, Canny operator, compass operator, Marr-Hildreth, wavelet transform. The device for creating a single image depth map by combining wavelet transformation and edge detection as described in item 1 of the scope of patent application, wherein the depth prediction algorithm executed by the depth calculation unit is based on the steps of the original image A number of local maximums of a grayscale value histogram, a window is created, and a blur calculation is performed according to the result of the image analysis. The device for establishing a single image depth map by combining wavelet transformation and edge detection as described in item 3 of the scope of the patent application, wherein the wavelet transformation algorithm further includes a step of establishing a wavelet transformation threshold. A method for establishing a single image depth map by combining wavelet transformation and edge detection, which is applied to a device for establishing a single image depth map by combining wavelet transformation and edge detection, and includes an image acquisition unit, an image analysis unit, An image synthesis unit and a depth calculation unit, and each of the units is an information link with each other. The device for combining a wavelet transformation and edge detection to create a single image depth map includes a method of inputting an original image, and The image capture unit inputs an original image; an image analysis step, and the image analysis unit executes a plurality of image analysis algorithms to analyze the original image; a step of establishing a defocus image, and after the image analysis step is completed, The image synthesis unit performs an image synthesis on the result of the original response image analyzed by the image analysis step to generate a defocus map; a depth prediction step, after the defocus map is generated, the depth calculation unit is based on the The result of the image analysis step executes a depth prediction algorithm; a depth diffusion step, when the depth calculation After execution of the depth prediction algorithm, performing a deep diffusion algorithm; At the step of generating a depth map, a depth map is generated after the depth diffusion algorithm is completed. The method for establishing a single image depth map by combining wavelet transformation and edge detection as described in item 8 of the scope of patent application, wherein the image analysis algorithm may be one of a wavelet transformation algorithm or an edge detection algorithm. One or a combination. The method for establishing a single image depth map by combining wavelet transformation and edge detection as described in item 8 of the scope of the patent application, wherein the steps of the depth prediction algorithm are based on a part of the grayscale value histogram of the original image A maximum number is established, and a window is created, and a blur calculation is performed according to the result of the image analysis. The method for establishing a single image depth map combining wavelet transformation and edge detection as described in item 8 of the scope of patent application, wherein the depth diffusion algorithm may be a Laplacian interpolation technique or a global interpolation algorithm. One of them.