TWI462053B - Method and apparatus for image processing - Google Patents

Description

Image processing method and device

The present invention is a method for performing image processing via a circuit model simulation operation, and embodiments include image processing operations related to depth data generation, image smoothing, and image resolution scaling.

In today's fast-changing technology era, stereoscopic multimedia systems are gradually being valued by the industry. In general, in the application of stereoscopic video/video, how to convert two Dimensional (2D) image content into three-dimensional (Three Dimensional, 3D) stereo image and two-view image matching (Stereo Matching) Image processing technology has always been the core technology of stereoscopic imaging that is urgently needed in the industry.

In the prior art, the 2D to 3D technology converts a traditional single-view image into a multi-view content, thereby providing a stereoscopic image content provided by the user; and the dual-view image comparison technology is first calculated based on the dual-view image. The depth map is generated, and then the multi-view image is generated according to Depth Image Based Rendering (DIBR).

In general, the accuracy of depth data has a decisive influence on the quality of stereoscopic image data. Accordingly, how to design an image processing method that can generate accurate and higher depth data is one of the directions that the industry is constantly striving for.

According to a first aspect of the present invention, an image processing method is provided, comprising the following steps. First, input data including multiple pieces of original data is received. Then the original data is converted to generate multiple converted analog voltage signals. At least one analog circuit model is then formed, including at least one spatial data node, at least one diffusion node, and at least one connection element, the at least one connection element being coupled to at least one of the spatial data nodes and at least one of the diffusion nodes. And then supplying part or all of the converted analog voltage signal to the at least one diffusion node, and diffusing part or all of the converted analog voltage signal to the at least one diffusion node via the at least one connection element to obtain at least one diffusion node Diffusion of imitation voltage signals. The processed image data is then generated based on at least one of the diffused imitation voltage signals.

According to a second aspect of the present invention, an image processing apparatus includes an input unit, a conversion unit, an analog unit, and a control unit. The input unit receives input data including a plurality of original materials. The conversion unit generates a plurality of converted analog voltage signals for the original data conversion. The analog unit establishes at least one analog circuit model including at least one data node, at least one diffusion node, and at least one connection element, the at least one connection element being coupled to at least one of the spatial data node and at least one of the diffusion nodes. The control unit provides part or all of the converted analog voltage signal to the at least one diffusion node, and diffuses part or all of the converted analog voltage signal to the at least one diffusion node via the at least one connection element to obtain the at least one diffusion node At least one diffused imitation voltage signal. The analog unit generates the processed image data according to at least one of the diffused imitation voltage signals.

In order to better understand the above and other aspects of the present embodiments, the preferred embodiments are described below, and in conjunction with the drawings, the detailed description is as follows:

The image processing apparatus and method of the present embodiment apply a circuit model simulation operation to perform related image processing operations.

Please refer to FIG. 1 , which is a flowchart of an image processing method according to an embodiment of the invention. The image processing method of this embodiment includes the following steps. First, as step (a), the input data Di including a plurality of original materials is received. Then, as in step (b), the original data is converted to generate a plurality of converted analog voltage signals v. And then, as in step (c), establishing at least one analog circuit model, including at least one spatial data node, at least one diffusion node, and at least one connecting component, wherein the at least one connecting component is coupled to the at least one spatial data node and the at least one Between the diffusion nodes.

And then, according to step (d), supplying part or all of the converted analog voltage signal v to the at least one diffusion node, and diffusing part or all of the converted analog voltage signal to the at least one diffusion node via the at least one connecting component. At least one diffusion node obtains at least one diffused analog voltage signal v_diff. Then, in step (e), the processed image data o is generated according to at least one of the diffused imitation voltage signals v_diff.

Next, a series of embodiments will be taken to further explain the image processing method of this embodiment.

First embodiment

The image processing method of the embodiment is applied to a two-dimensional (2D) three-dimensional (3D) technique for generating depth distribution data according to initial depth data.

Please refer to FIG. 2, which is a schematic diagram of input data according to a first embodiment of the present invention. The image processing method of this embodiment generates the depth distribution data Do based on the input data Di. For example, the input data Di includes m×n pen original data Di(1,1), Di(1,2), Di(1,3), . . . , Di(m,n), where m and n are greater than The natural number of 1. The input data Di corresponds to the image data DV displayed in the display, and the image data DV includes m×n pen element data I(1,1), I(1,2), . . . , I(m,n), where m× The n-th original data Di(1,1)-Di(m,n) respectively correspond to m×n-pixel data I(1,1)-I(m,n) of m×n pixels displayed in the display. ). In this embodiment, the input data Di is the initial depth data corresponding to the image data DV, and the values of the m×n pen original data Di(1,1)-Di(m,n) respectively indicate the m×n pen pixels. The depth corresponding to the data; the depth distribution data Do is the depth distribution data with higher fineness generated according to the initial depth data of the image data DV.

Accordingly, the image processing method of the present embodiment is applied to generate three-dimensional (3D) stereoscopic image data according to two-dimensional (2D) image data DV and initial depth data, in other words, to perform 2D image content. Transfer processing of 3D stereoscopic images.

For example, the original data Di(1,1)-Di(m,n) respectively includes mxn pen 8-bit data, in other words, each piece of original data Di(1,1)-Di(m,n) has a A numeric value from 0-255. When the original data Di(1,1)-Di(m,n) corresponds to a larger digit value, it indicates that the corresponding pixel data I(1,1)-I(m,n) has a shallow depth. When each of the original data Di(1,1)-Di(m,n) corresponds to a smaller digit value, it indicates that the corresponding pixel data I(1,1)-I(m,n) has a deeper depth.

Please refer to FIG. 3, which is a flow chart of an image processing method according to a first embodiment of the present invention. First, as step (a), the initial depth data corresponding to the image data DV is received, and the m×n pen pixel depth data is used as the m×n pen original data Di(1,1)-Di(m,n). . Then, as in step (b), each m×n pen original data Di(1,1)-Di(m,n) is converted to correspond to the m×n pen original data Di(1,1)-Di(m). , n) respectively generate m×n pen-converted analog voltage signals SV(1,1), SV(1,2), . . . , SV(m,n). For example, the conversion step of step (b) directly uses the digital value of each piece of original data Di(1,1)-Di(m,n) as the converted analog voltage signal SV(1,1)-SV. The digital voltage value of (m, n).

Next, as in step (c), an analog circuit model M including at least one spatial data node, at least one diffusion node, and at least one connecting element is established. In an operation example, the analog circuit model M, as shown in Fig. 5, includes m × n sub-circuit models M(1, 1), M(1, 2), ..., M(m, n) having similar circuit structures. ), which respectively correspond to m×n pen original data Di(1,1)-Di(m,n). Since each m×n sub-circuit model has a similar circuit structure, next, the sub-circuit model M(i, j) corresponding to the original data Di(i, j) in the analog circuit model is taken as an example to simulate the circuit model. Each sub-circuit model M(1,1)-M(m,n) in M is further described, wherein i and j are natural numbers less than or equal to m and natural numbers less than or equal to n, respectively.

Referring to FIG. 4, a circuit diagram of a sub-circuit model M(i, j) according to a first embodiment of the present invention is shown. The sub-circuit model M(i,j) includes a spatial data node NS(i,j), a diffusion node ND(i,j), a spatial data connection element RS, and z diffusion connection elements RD1, RD2, . . . , RDz, where z It is a natural number, wherein the spatial data connection element RS and the diffusion connection elements RD1 - RDz are, for example, resistance model elements. In the step (c), the spatial data connection element RS is coupled between the spatial data node NS(i,j) and the diffusion node ND(i,j), and one of the z diffusion connection elements RD1-RDz is coupled to The diffusion node ND(i,j) is coupled to a diffusion node of another sub-circuit model in the analog circuit model M.

In the example of Fig. 4, z is equal to 4; and in step (c), the other ends of the diffusion connecting elements RD1-RD4 are respectively coupled to the sub-circuit models M(i-1, j), M(i, J-1), M(i, j+1) and diffusion nodes ND(i-1,j), ND(i,j-1), ND(i,j+) in M(i+1,j) 1) and ND(i+1,j). Similarly, in step (c), all m×n diffusion nodes ND(1,1)-ND(m) in m×n sub-circuit models M(1,1)-M(m,n) , n) are connected to each other via corresponding diffusion connection elements such that each sub-circuit model M(1,1)-M(m,n) in the analog circuit model M is connected in series to form a node and a resistance network, as shown in FIG. Show.

For example, the resistance values of the spatial data diffusion connection elements RS(1,1)-RS(m,n) in all the analog circuit models M(1,1)-M(m,n) are substantially equal, and The set value given to the user.

For example, the resistance value ω _diffuse of each z diffusion connection elements RD1-RDz in the analog circuit model M(i,j) satisfies:

Where α and β are predetermined parameters; C _t is the color information of the pixel data corresponding to the original data Di(i, j); C _n is the diffusion node connected to each diffusion connecting element RD1-RDz (ie ND(i-1) , j), ND(i, j-1), ND(i, j+1), and ND(i+1, j)), the color information of the pixel data corresponding to each piece of original data. For example, the color information C _t and C _{n of the} pixel data can be obtained by summing the absolute values of the sub-pixel data of the respective pixels in the corresponding pixel data.

Then, as in step (d), the converted pseudo-voltage signals SV(1,1)-SV(m,n) corresponding to the m×n pen original data Di(1,1)-Di(m,n) are respectively supplied to m×n spatial data nodes NS(1,1)-NS(m,n), thereby driving m×n sub-circuit models via voltage diffusion operations between spatial data connection elements and diffusion connection elements in the analog circuit model M M(1,1)-M(m,n) voltage level redistribution, so that the diffusion node ND(1,1)-ND(m,n) respectively obtains m×n pen diffusion-like voltage signal SVD(1 , 1), SVD (1, 2), ..., SVD (m, n).

Then, as in step (e), the depth profile data Do is generated from the m×n pen diffusion analog voltage signal SVD(1,1)−SVD(m,n).

In the image processing method of the present embodiment, only the step (a) directly uses the digital value of each of the original data Di(1,1)-Di(m,n) as the converted analog voltage signal SV (1). The case of the digital voltage value of 1)-SV(m, n) is taken as an example. However, the image processing method of the present embodiment is not limited thereto.

In other examples, when the image data DV is a dynamic video data, the image processing method in this embodiment may also perform a corresponding converted analog voltage signal SV (x, according to the original data Di(x, y) according to the following equation. y) operation:

Sv(x,y)=γ×Di _pre (x,y)+(1-γ)×Di(x,y)

Where x and y are respectively a natural number less than or equal to m and a natural number less than or equal to n; γ is a predetermined parameter; Di _pre (x, y) is a period corresponding to the previous frame time (Frame Time) In the image data, corresponding to the depth data of the pixel data I(x, y); wherein x and y are natural numbers less than or equal to m and less than or equal to n, respectively. Through the foregoing operation, the image processing method of the embodiment can enhance the depth contrast of the depth distribution data Do.

In still another example, the image processing method of this embodiment can also achieve the effect of emphasizing the depth of the moving object by superimposing an enhanced voltage on the spatial data node NS (which originally has the converted analog voltage signal SV). For example, the image processing method of this embodiment performs the operation of superimposing the enhanced voltage on the spatial data node NS according to the following equation:

SV(x,y)=Di(x,y)+min(δ,κ|C _pre (x,y)-C _cur (x,y)|)

Where min(δ, κ|C _pre (x, y)-C _cur (x, y)|) is the enhanced voltage value to be superimposed on the converted pseudo-voltage signal SV; κ is a predetermined parameter; The upper limit value of the enhanced voltage value; C _pre (x, y) and C _cur (x, y) are the color of the pixel data at the (x, y) position of the previous frame time and the current frame time, respectively.

In the present embodiment, the sub-circuit model M(1,1)-M(m,n) including the number of the original data Di(1,1)-Di(m,n) is substantially included in the analog circuit model M. ), and respectively receive the data data NS(1,1)-NS(m,n) on the sub-circuit model M(1,1)-M(m,n) and the original data Di(1,1)- The case of converting the imitation voltage signal SV(1,1)-SV(m,n) corresponding to Di(m,n) is taken as an example. However, the image processing method of the embodiment is not limited thereto. In other examples, the analog circuit model generated by the image processing method in the step (c) may also include a sub-circuit model whose number is not equal to the number of original data; correspondingly, the user may also make the data node of the portion be in a floating state. (Floating) or means of discarding part of the original data to input part or all of the original data Di(1,1)-Di(m,n) into the analog circuit model M, thereby being processed by a similar operation Post-image data.

Second embodiment

The image processing method of this embodiment is applied to the dual-view image matching (Stereo Matching) technology for generating depth distribution data according to the first and second viewing angle image data.

Please refer to FIG. 6, which is a schematic diagram of input data according to a second embodiment of the present invention. In this embodiment, the input data Di' is parallax data corresponding to the first view image data DvL and the second view image data DvR, and the depth distribution data Do' is corresponding to the first or second view image data DvL and DvR. Depth distribution data. Accordingly, the image processing method of the present embodiment is applied to generate corresponding depth distribution data according to the parallax data of the first and second viewing angle image data DvL and DvR, in other words, to perform dual-view image comparison operation.

In more detail, the image processing method of the second embodiment is different from the image processing method of the first embodiment in that the step (a') further includes sub-steps as shown in Figs. 7A and 7B. First, as in step (a1), the first view image data and the second view image data DvL and DvR are received. For example, the first and second viewing angle image data DvL and DvR are image data corresponding to the left eye viewing angle and the right eye viewing angle, respectively.

Then, as step (a2), the w horizontal disparity values Dx1, Dx2, ..., Dxw are determined, and when the first view image data DvL has the kth horizontal disparity value Dxk with respect to the second view image data DvR, The first original disparity data (Disparity) DIS_k, k of the first view image data DvL and the second view image data DvR is an index of the image comparison window, and the value is greater than or equal to 1 and less than or equal to w Natural number. For example, the first original dissimilarity data DIS_k includes m×n original pixel dissimilarity data DIS(1,1, Dxk), DIS(1, 2, Dxk), ..., DIS(m, n, Dxk), wherein step (a2) is to find each m×n original pixel dissimilarity data DIS(1,1,Dxk)-DIS(m) of the first original dissimilarity data DIS_k by the following equation operation ,n,Dxk):

Where x and y are respectively a natural number less than or equal to m and a natural number less than or equal to n.

Then, as in step (a3), the first original dissimilarity data DIS_k is used as the input data Di', wherein the m×n pen original data Di'(1,1)-Di'(m,n) is the first original phase. In the heterogeneous data DIS_k, the first original pixel dissimilarity data DIS(1,1, Dxk) corresponding to m×n pixels I(1,1)-I(m,n) respectively -DIS(m,n,Dxk).

The image processing method of this embodiment performs steps corresponding to steps (b)-(d) of FIG. 3 correspondingly after step (a3): (b') for each m×n pen original data Di(1, 1) -Di(m,n) is converted to generate m×n pen-converted analog voltage signals SV'(1,1)-SV'(m,n); (c') corresponds to m×n original The data Di(1,1)-Di(m,n) generates an analog circuit model M', which includes the nodes formed by the diffusion nodes ND'(1,1)-ND'(m,n) and the corresponding diffusion connecting elements. And the resistor network; and (d') provide the converted analog voltage signal SV'(1,1)-SV'(m,n) to m×n spatial data nodes NS'(1,1)-NS', respectively (m,n), and obtain m from the diffusion nodes ND'(1,1)-ND'(m,n) of m×n sub-circuit models M′(1,1)-M′(m,n) ×n pen spreads the analog voltage signal SVD'(1,1,Dxk)-SVD'(m,n,Dxk).

For example, the sub-circuit model M'(1,1)-M'(m,n) of the present embodiment is different from the sub-circuit model M(1,1)-M(m,n) in the first embodiment. The resistance value ω _diffuse of each z diffusion connection elements RD1'-RDz' in the sub-circuit model M'(i,j) satisfies:

Wherein α, β, γ are predetermined parameters; G _s (x, y) is a gradient value of the first view image data DvL after the smoothing operation; C _t is the color information of the pixel data corresponding to each of the original materials; _n is the color information of the pixel data corresponding to the original data received by the sub-circuit model connected by the diffusion connecting element RD1'-RDz'.

After the step (d') and before the step (e'), the image processing method of the embodiment further includes steps (f), (g) and (h). In step (f), it is determined whether the value k is rounded up between 1 and the natural number between the dissimilarity search window parameters w; if not, it indicates that the image processing method of the embodiment has not been applied to all w levels. The disparity values Dx1, Dx2, ..., Dxw find the corresponding first original pixel dissimilarity data DIS_1-DIS_w (corresponding to the converted imitation voltage signal and the diffused imitation voltage signal). According to this, step (g) is performed to set the parameter k to a natural number between 1 and the dissimilarity finding window parameter w that has not been rounded, and repeat the operations of steps (a1)-(a3) to Find a first original dissimilarity data DIS_k corresponding to the next horizontal disparity value.

After finding the next first original dissimilarity data, the image processing method of the embodiment also repeats the operations of steps (b')-(d') correspondingly to generate the corresponding first original dissimilarity. The analog circuit model M' of the data is obtained, and the m×n pen diffusion-like voltage signal SVD'(1,1, Dxk)-SVD'(m, n, Dxk) corresponding to the next first original dissimilarity data is obtained.

The above part or all of the processing processes for all the w horizontal disparity values Dx1, Dx2, ..., Dxw to find the corresponding w original first pixel disparity data DIS_1-DIS_w may also be processed in parallel processing. .

When the value k corresponding to the natural number between the dissimilarity finding window parameter w is completed, it indicates that the image processing method of the embodiment has found corresponding correspondence for all w horizontal disparity values Dx1-Dxw. The first original dissimilarity data DIS_1-DIS_w, in other words, correspondingly finds the first original pixel dissimilarity data DIS(1,1,Dx1)-DIS(m, w×m×n n, Dx1), DIS(1,1, Dx2)-DIS(m,n,Dx2)...DIS(1,1,Dxw)-DIS(m,n,Dxw). According to the image processing method of this embodiment, step (h) is performed to find the w-split-like analog voltage signal corresponding to each m×n pen original data Di(1,1)-Di(m,n). Taking the (i,j) original data Di(i,j) of the m×n pen original data Di(1,1)-Di(m,n) as an example, the system corresponds to the following w pen diffusion. Imitation voltage signals SVD'(i,j,Dx1), SVD'(i,j,Dx2), SVD'(i,j,Dx3),...,SVD'(i,j,Dxw).

Correspondingly, the step (e') of the image processing method of the embodiment includes the step (e1) to find that each m×n pen original data Di′(1,1)-Di′(m,n) corresponds to In the pen-split-like voltage signal, the lowest-diffused pseudo-voltage signal SVD _min (1,1)-SVD _min (m,n) with the smallest voltage value is used to find the depth distribution data Do, respectively, with the original data Di'(1,1)-Di'(m,n) corresponds to the output pixel dissimilarity data Do'(1,1), Do'(1,2),...,Do'(m,n). Taking the (i,j) original data Di(i,j) of the m×n original data Di(1,1)-Di(m,n) as an example, the operation of the step (e1) may be as follows The equation represents:

Accordingly, the output pixel dissimilarity information Do'(i, j) corresponding to the original data Di(i, j) can be correspondingly obtained.

In another example, the step (e) further includes a step (e2) for generating a quadratic function curve according to the three diffused imitation voltage signals corresponding to the front and rear of the lowest diffused analog voltage signal, and using the quadratic function curve The minimum value is used to find the output pixel dissimilarity data Do'(1,1)-Do'(m,n) with accuracy below the decimal point. For an example of operation, the output pixel difference data Do'(i,j) corresponding to the (i,j) original data Di(i,j) (ie, the lowest diffusion analog voltage signal SVD _min (i) , j)) is a diffused analog voltage signal SVD'(i, j, 5) corresponding to the horizontal disparity value Dx5. The operation in step (e2) will correspond to the diffused analog voltage signals SVD'(i,j,4), SVD'(i,j,5) and SVD'(i,j) of the horizontal disparity values Dx4, Dx5 and Dx6. , 6) Substituting into the following equations to find the parameters a, b, and c correspondingly:

Ax ² +bx+c=y

The output pixel dissimilarity data Do'(i,j) corresponding to the original data Di(i,j) is equal to:

Accordingly, according to the operations of the foregoing steps (a')-(e'), (f)-(h), the image processing method of the embodiment may be correspondingly based on the first view image data DvL (for example, a left-eye view image) Data) The depth distribution data Do' is generated with respect to the first original dissimilarity data DIS_1-DIS_w of the second view image data DvR (for example, the right eye view image data).

In the present embodiment, the depth distribution data Do' is generated based on the first original dissimilarity data DIS_1-DIS_w of the first view image data DvL with respect to the second view image data DvR only by the image processing method of the embodiment. The case is described as an example. However, the image processing method of this embodiment is not limited thereto. In another example, the image processing method in this embodiment can be correspondingly based on the second view image data DvR (for example, the right eye view image data) relative to the first view image data, according to operations similar to those of the foregoing embodiment. The second original dissimilarity data DIS'_1-DIS'_w of the DvL (for example, the left-eye viewing image data) is used to generate the depth distribution data Do'.

In still another example, the image processing method of the embodiment can compare the depth distribution data Do′ and the second perspective image data DvR of the second perspective image data DvR with respect to the first perspective image data DvL. The consistency of the depth distribution data Do" of the image data DvL of one view image is used to improve the accuracy of the depth distribution data. For example, for the first view depth distribution data Do' and the second view depth distribution data Do" Keep the output pixel dissimilarity data in which the following equations are met:

In other words, only the output pixel dissimilarity data which coincides with each other in the depth distribution data Do' is left.

For the output pixel dissimilarity data that does not satisfy the above equation, any image fill-in (Inpainting) technique can be used to fill up the data. This example replaces the corresponding output pixel dissimilarity data with the minimum value of the output pixel dissimilarity data whose position on the left and right sides is closest and satisfies the above equation (ie, is retained).

In the present embodiment, the m×n sub-circuit models M′(1,1)-M′(m,n) established in the step (b′) only by the image processing method have the same as shown in FIG. 3 . The case of the circuit structure is taken as an example. However, the image processing method of this embodiment is not limited thereto. In another example, each of the m×n sub-circuit models M′(1,1)-M”(m,n) established by the image processing method further includes a time data node and a time data diffusion connection element. In the analog circuit model M"(1,1)-M"(m,n), the (i,j)th analog circuit model M"(i,j), and the circuit shown in FIG. The structure is different in that it further includes a time data node NT(i, j) and a time data diffusion connecting element RT(i, j), as shown in FIG.

The time data diffusion connecting element RT(i,j) is coupled between the time data node NT(i,j) and the diffusion node ND(i,j), and the time data diffusion connecting element RT(i,j) has a resistance value ω _Time meets:

Where λ and σ are predetermined parameters; C _cur is the color information of the corresponding pixel data (for example, each of the first-view image data DvL) in the current frame time; C _pre is the original The color information of the corresponding pixel data in the previous frame time. Accordingly, the image processing method of the embodiment may also generate the depth distribution data Do' by referring to the color information related to the different frame frames before and after.

In addition, Monocular Cues (for example, Linear Perspective) can be applied to assist in the operation of generating the depth profile data Do'. In addition, semi-global and global alignment mechanisms are also used. For example, Dynamic Programming or Belief Propagation can also be applied to the image processing method of the embodiment to improve the alignment accuracy of the first and second viewing angle image data DvL and DvR.

In this embodiment, the case where the first and second view image data DvL and DvR are directly applied by the image processing method to perform the correlation comparison operation is taken as an example. However, the image processing method in this embodiment is not limited. herein. In other examples, in order to simplify the overall amount of data calculation, the image processing method of this embodiment may perform a resolution reduction operation before performing the comparison operation on the first and second viewing angle image data DvL and DvR; In other words, the comparison operation is performed based on the first and second viewing angle image data DvL and DvR after the resolution is reduced, and the depth distribution data having a low resolution is obtained. Thereafter, the depth distribution data is enlarged to obtain depth distribution data of the same resolution when the amount of data calculation is greatly reduced.

For example, the foregoing operation for amplifying the depth distribution data in the embodiment may be implemented by the image processing method flow step as shown in FIG. 1. In an operation example, the depth distribution data to be amplified has, for example, s×t pen original data, and is intended to be enlarged into an enlarged depth distribution data having a resolution equal to s′×t′, where s, t, s′ and t' is a natural number greater than 1, and s and t satisfy: s<s' and t<t', respectively. In this example of operation, the image processing method of the present embodiment generates an analog circuit model including s×t data nodes and s′×t′ diffusion nodes, and operates on s′×t′ diffusion via voltage diffusion. The node obtains the s'×t' pen diffusion node to diffuse the imitation voltage signal. Accordingly, the image processing method of the embodiment can also perform the data resolution amplification operation correspondingly through the foregoing circuit simulation operation.

Third embodiment

The image processing method of this embodiment generates depth distribution data corresponding to the image according to the data input by the user. The difference between the first embodiment and the second embodiment is that the image processing method of the embodiment further provides a user interface to receive a user operation event provided by the user; and the image processing method of the embodiment is provided by the user. The user operates the event to selectively increase or decrease the nodes and the diffusion connection elements of the m×n sub-circuit models, or selectively set the voltage signals and the diffusion connection elements corresponding to the spatial data nodes in the m×n sub-circuit models. Resistance value. In addition, the image processing method of the embodiment further correspondingly drives the voltage level re-allocation corresponding to the analog circuit model, so as to respectively obtain the user-controlled diffusion-like voltage signal on the diffusion node of the analog circuit model.

For example, the aforementioned user interface can provide an image segmentation image processing tool and a brush tool. The image segmentation tool selectively performs an object segmentation operation on the input data in response to a user-triggered user operation event to find an object distribution information in the input data; the brush tool allows the user to selectively assign numerical information. Give the corresponding input data. The image processing method in this embodiment can further increase or decrease the data node or the diffusion node by referring to the foregoing information in the operation step (c) of performing the establishment of the analog circuit model, or perform resistance values on the diffusion connection component and the spatial data connection component. set up.

In an operation example, in the operation step (c) of establishing an analog circuit model, the image processing method first performs resistance value setting on the diffusion connection element with reference to the object allocation information. When the object allocation information indicates that the two diffusion nodes belong to the same object segmentation, the image processing method of the embodiment uses the same method as the first embodiment to set the resistance value of the diffusion connecting element during the period; when the object is allocated When the information indicates that the two diffusion nodes belong to different object segments, the image processing method in this embodiment correspondingly sets the resistance value of the diffusion connection element therebetween to a maximum value to ensure that the diffusion nodes between different object segments are separated. Has a lower voltage spread situation.

Then, in the operation step (c) of establishing the analog circuit model, the image processing method then establishes the data node and the data connection component according to the numerical data assigned by the user (corresponding to the specific diffusion node); For a diffusion node that does not assign any original data, the image processing method does not perform related data node and data connection component establishment operations. Accordingly, the image processing method may refer to the reference object segmentation information and the user's numerical data assignment information in step (c) to establish a corresponding analog circuit model for performing corresponding image processing operations.

Fourth embodiment

The image processing method of this embodiment is applied to an image smoothing (Smooth) application for generating smoothed image data for input image data.

The image processing method of this embodiment is configured to generate image smoothing data Do" according to the input data Di". For example, the input data Di" includes m×n pen pixel data I(1,1)-I(m,n) corresponding to m×n pixels, including the first pixel data Isub1(1, 1) -Isub1(m,n), and the image smoothing data Do" includes the m×n pen smoothing sub-pixel data Ism1(1,1)-Ism1(m,n) corresponding to the m×n pixels .

Please refer to FIG. 9, which is a flowchart of an image processing method according to a fourth embodiment of the present invention. First, as step (a), the first pixel data Isub1(1,1)-Isub1(m,n) of the m×n pen is received and used as input data. Then, as in step (b), the first pixel data Isub1(1,1)-Isub1(m,n) of each m×n pen is converted to respectively generate an m×n pen-converted voltage signal SV ( 1,1)-SV(m,n).

Then, as in step (c), the first pixel data Isub1(1,1)-Isub1(m,n) corresponding to the m×n pen generates an analog circuit model M′”. For example, the analog circuit model M′” Including m × n sub-circuit models M'"(1,1)-M'"(n,m), each m×n analog circuit models M'"(1,1)-M'"(n,m) The data node NS, the diffusion node ND, the data diffusion connection element RS and the x diffusion connection elements RD1-RDx, the data diffusion connection element RS is coupled between the data node NS and the diffusion node ND, and one end of each of the x diffusion connection elements RD The other end is coupled to the diffusion node ND, and the other end is coupled to another sub-circuit model of the m×n sub-circuit models M′′(1,1)−M′′(m,n), where x is a natural number.

Then, as in step (d), the converted pseudo-voltage signal SV(1,1)-SV(m, corresponding to the first pixel data Isub1(1,1)-Isub1(m,n) of each m×n pen is converted. n) supplied to the data node NS to drive voltage level re-distribution of m×n sub-circuit models M′"(1,1)-M'"(m,n), respectively, for m×n sub-circuit models M The diffusion node ND of '"(1,1)-M'"(n,m) respectively obtains an m×n pen-diffused pseudo-voltage signal SVD(1,1)-SVD(m,n).

Then, as in step (e), the m×n pen-spreading pseudo-voltage signal SVD(1,1)-SVD(m,n) is generated to include the m×n pen smoothing sub-pixel data Ism1(1,1)-Ism1( m, n) image smoothing data Do".

In one example, each m×n pen pixel data I(1,1)-I(m,n) includes, for example, a second pixel data and a third pixel data Isub2(1,1)-Isub2, respectively. (m, n) and Isub3(1,1)-Isub3(m,n), and the image processing method of the present embodiment is further found, for example, via substantially similar operational steps to steps (a)-(e) above. Corresponding smoothing sub-pixel data Ism2(1,1)-Ism2(m,n) and Ism3(1,1)-Ism(m,n).

Fifth embodiment

For example, the foregoing operation for amplifying the depth distribution data in the embodiment may be implemented by the image processing method flow step as shown in FIG. 1. In an operation example, the depth distribution data to be amplified has, for example, s×t pen original data, and is intended to be enlarged into an enlarged depth distribution data having a resolution equal to s′×t′, where s, t, s′ and t' is a natural number greater than 1, and s and t satisfy: s<s' and t<t', respectively.

In this example of operation, the image processing method of the present embodiment generates an analog circuit model including s×t data nodes and s′×t′ diffusion nodes, and operates on s′×t′ diffusion via voltage diffusion. The node obtains the s'×t' pen diffusion node to diffuse the imitation voltage signal. Accordingly, the image processing method of the embodiment can also perform the data resolution amplification operation correspondingly through the foregoing circuit simulation operation.

Accordingly, the user can also use the image processing method of the embodiment to implement the resolution scaling operation of the image data by selectively setting the number of the diffusion nodes and the data nodes in the analog circuit model.

According to the image processing method described in the foregoing embodiments of the present invention, it can be implemented by a plurality of well-developed computer readable programs and recorded in corresponding computer readable media. In this manner, the user can access the computer readable medium by using a computer processor to execute the image processing method according to the foregoing embodiments of the present invention according to the computer readable program stored therein.

For example, the image processing method according to each of the foregoing embodiments of the present invention can be implemented by the image processing apparatus 1 shown in FIG. In more detail, the image processing apparatus 1 includes an input unit 10, a conversion unit 20, an analog unit 30, and a control unit 40. In an operation example, each input unit 10, conversion unit 20, analog unit 30, and control unit 40 in the image processing apparatus 1 is a software module. In other words, each unit is executed by a processor. to realise.

The input unit 10 receives input data Di including a plurality of pieces of original data. The converting unit 20 converts the original data to generate a plurality of converted analog voltage signals v. The analog unit 30 establishes a corresponding analog circuit model including at least one spatial data node, at least one diffusion node, and at least one connecting element. The control unit 40 converts some or all of the analog voltage signals to the at least one diffusion node, and diffuses part or all of the converted analog voltage signals to the at least one diffusion node via the at least one connection component to obtain the at least one diffusion node. At least one diffused imitation voltage signal v_diff. The control unit generates the processed image data o according to at least one of the diffused imitation voltage signals v_diff.

The foregoing embodiments of the present invention relate to an image processing method. Compared with some conventional depth data generating methods, the image processing method of the embodiment of the present invention has the advantage of producing more accurate stereoscopic image data; compared with the conventional image smoothing method, the image processing method of the embodiment of the present invention The division has the advantage of effectively smoothing the input image.

In summary, although the patent specification has been disclosed above in the preferred embodiments, it is not intended to limit the disclosure. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the disclosure. Therefore, the scope of protection of this disclosure is subject to the definition of the scope of the appended claims.

DV. . . video material

Di. . . Input data

I(1,1)-I(m,n), I1(1,1)-I1(m,n), I2(1,1)-I2(m,n). . . Pixel data

Di(1,1)-Di(m,n). . . Source material

M(i,j), M"(i,j)...analog circuit model

NS(1,1)-NS(m,n). . . Spatial data node

ND(1,1)-ND(m,n). . . Diffusion node

RS. . . Spatial data diffusion connecting element

RD1-RDx. . . Diffusion connection element

Dx1-Dxw. . . Horizontal disparity value

NT. . . Time data node

RT. . . Time data diffusion connecting element

DvL, DvR. . . First and second perspective image data

1. . . Image processing device

10. . . Input unit

20. . . Conversion unit

30. . . Analog unit

40. . . control unit

FIG. 1 is a flow chart of an image processing method according to an embodiment of the invention.

2 is a schematic diagram showing input data according to a first embodiment of the present invention.

FIG. 3 is a flow chart showing an image processing method according to a first embodiment of the present invention.

Fig. 4 is a circuit diagram showing a sub-circuit model in accordance with a first embodiment of the present invention.

Fig. 5 is a circuit diagram showing an analog circuit model in accordance with a first embodiment of the present invention.

Figure 6 is a schematic diagram showing input data according to a second embodiment of the present invention.

7A and 7B are flowcharts showing an image processing method according to a second embodiment of the present invention.

Figure 8 is a circuit diagram showing a sub-circuit model in accordance with a second embodiment of the present invention.

FIG. 9 is a flow chart showing an image processing method according to a fourth embodiment of the present invention.

FIG. 10 is a block diagram of an image processing apparatus according to an embodiment of the present invention.

(a)-(e). . . Steps

Claims (34)

An image processing method includes: receiving a plurality of original data; converting the original data to generate a plurality of converted analog voltage signals; and establishing an analog circuit model, the analog circuit model including at least one data node and at least one diffusion node And at least one connecting component, wherein the at least one connecting component is coupled to part or all of the at least one data node and the at least one diffusion node; and some or all of the converted analog voltage signals are provided to the at least one data node And diffusing part or all of the converted analog voltage signals to the at least one diffusion node via the at least one connecting component to obtain at least one diffusion-like voltage signal at the at least one diffusion node; and according to the at least one diffusion simulation The voltage signal generates at least one processed image data. The image processing method of claim 1, wherein the original data is depth information, and the depth information is related to an image data. The image processing method of claim 1, wherein the original data is depth information related to the received user input data. The image processing method according to claim 1, wherein the connecting element is an element having a resistance characteristic. The image processing method of claim 4, wherein the resistance value of the connecting component is related to a color information in an image data. The image processing method of claim 4, wherein the resistance value of the connecting component is related to a user input data. The image processing method of claim 4, wherein the resistance value of the connecting element is corrected by a smoothed image gradient. The image processing method of claim 1, wherein the data node is related to image pixel distribution or related to frame time. The image processing method of claim 1, wherein the original material is color information of an image. The image processing method of claim 1, wherein the step of receiving the plurality of original data further comprises: receiving an initial depth data as an input data, wherein the initial depth data includes the corresponding pixel number Some original materials. The image processing method of claim 10, wherein the step of establishing an analog circuit model further comprises: providing a plurality of the at least one diffusion node in the analog circuit model, corresponding to each of the at least one diffusion node, The at least one connecting element includes x diffusion connecting elements, one end of each of the x diffusion connecting elements is coupled to the corresponding one of the at least one diffusion node, and the other end is coupled to the other one of the at least one diffusion node, Where x is a natural number. The image processing method of claim 1, wherein the step of receiving the plurality of original data further comprises: (a1) receiving a first view image data and a second view image data; (a2) finding the first When the first view image data has a k-th horizontal disparity value Dxk relative to the second view image data, the first view image data and the second view image data are the first original disparity data (Disparity), k is a natural number greater than or equal to 1 and less than or equal to w; and (a3) using the first original dissimilarity data as the input data, wherein the original data is respectively the first original dissimilarity data The first original pixel dissimilarity data corresponding to the plurality of pixels of the plurality of pixels. The image processing method of claim 12, wherein the step of establishing an analog circuit model further comprises: providing a plurality of the at least one diffusion node in the analog circuit model, corresponding to each of the at least one diffusion node, The at least one connecting element includes x diffusion connecting elements, one end of each of the x diffusion connecting elements is coupled to the corresponding one of the at least one diffusion node, and the other end is coupled to the other one of the at least one diffusion node, Where x is a natural number. The image processing method of claim 12, further comprising: setting the value k to a natural number between 1 and the dissimilarity finding window parameter w, and correspondingly in sequence or parallel The processing manner repeats steps (a1)-(a3) to find out and the following first original dissimilarity data as the input data; wherein, in order to generate the first original dissimilarity data corresponding to the next one The analog circuit model and the at least one diffused analog voltage signal corresponding to the next first original dissimilarity data are obtained, and the correlation step is correspondingly repeated. The image processing method of claim 14, wherein the method further comprises: when the value k corresponding to the natural number between the dissimilarity finding window parameter w is completed, corresponding to the at least a diffusion node finds the at least one diffused analog voltage signal; wherein, in the step of generating at least one processed image data, the method includes: (e1) having a minimum voltage in the at least one diffused analog voltage signal according to the w pen One of the values of the minimum diffused imitation voltage signal finds the processed image data. The image processing method of claim 15, wherein the step (e1) further comprises: determining whether the minimum diffusion analog voltage signal corresponds to a reference diffusion analog voltage signal to perform the minimum diffusion analog voltage signal Verification, wherein the reference diffusion-like voltage signal is related to the dissimilarity data of the second view image data relative to the first view image data; and when the lowest diffusion-like voltage signal corresponds to the reference diffusion-like voltage signal, The minimum spread analog voltage signal derived by the reference diffusion analog voltage signal is used as the processed image data. The image processing method of claim 12, wherein the step of establishing an analog circuit model further comprises: providing a time data node in the at least one data node, corresponding to the time data node, the at least one connecting component A time data connection component is coupled, the time data connection component coupled between the time data node and the at least one diffusion node. The image processing method of claim 1, wherein the step of establishing an analog circuit model further comprises: selectively determining the at least one data node in the analog circuit model in response to a first user operation event, The at least one diffusion node and the number of the at least one connection component; in response to a second user operation event, selectively setting a voltage signal corresponding to the at least one data node and a resistance value of the at least one connection component in the analog circuit model . The image processing method of claim 1, wherein the step of receiving the plurality of original data further comprises: receiving a plurality of pixel data corresponding to the plurality of pixels, and using the secondary pixel data as Corresponding to the original data of the pixels; wherein the processed image data generated based on the original data of the plurality of pixel data is image smoothing processed image data. The image processing method of claim 1, wherein the step of receiving the plurality of original data further comprises: receiving a first frame data, and using the first frame data as the input data, the first The frame data corresponds to a first screen resolution; wherein the processed image data generated by the input data that is substantially the first frame data is image scaling processing corresponding to one of the second image resolutions. Image data. The image processing method of claim 1, wherein the step of establishing an analog circuit model further comprises: providing a spatial data node in the at least one data node, corresponding to the spatial data node, the at least one connection The component includes a spatial data connection component coupled between the spatial data node and the at least one diffusion node. An image processing device comprising: an input unit for receiving an input data, the input data comprising a plurality of original data; and a conversion unit for converting the original data to generate a plurality of converted analog voltage signals; An analog unit for establishing an analog circuit model, the analog circuit model including at least one data node, at least one diffusion node, and at least one connection component, wherein the at least one connection component is coupled to the at least one data node and the at least one diffusion a part or all of the nodes; and a control unit, providing part or all of the converted analog voltage signals to the at least one diffusion node, and converting the portions of the voltage signals by the at least one connecting component All of the diffusion nodes are diffused to the at least one diffusion node to obtain at least one diffusion-like voltage signal, and the control unit generates a processed image data according to the at least one diffusion-like voltage signal. The image processing device of claim 22, wherein the input unit receives an initial depth data and uses the initial depth data as the input data, wherein the initial depth data includes the pixel corresponding to the plurality of pixels Some original materials. The image processing device of claim 23, wherein the simulation unit provides a plurality of the at least one diffusion node in the analog circuit model; wherein, corresponding to each of the at least one diffusion node, the at least one connection component Including x diffusion connection elements, one end of each of the x diffusion connection elements is coupled to the corresponding at least one diffusion node, and the other end is coupled to another diffusion node of the at least one diffusion node, where x is natural number. The image processing device of claim 22, wherein the input unit comprises: a receiver for receiving a first view image data and a second view image data; and an operator to find the first When the first view image data has a k-th horizontal disparity value Dxk relative to the second view image data, the first view image data and the second view image data are the first original disparity data (Disparity), k is a natural number greater than or equal to 1 and less than or equal to w; wherein the processor uses the first original dissimilarity data as the input data, wherein the original data is in the first original dissimilarity data Corresponding to the first original pixel dissimilarity data of the plurality of pixels of the plurality of pixels. The image processing device of claim 25, wherein the simulation unit provides a plurality of the at least one diffusion node in the analog circuit model; wherein the at least one connection component corresponds to each of the at least one diffusion node Including x diffusion connection elements, one end of each of the x diffusion connection elements is coupled to the corresponding at least one diffusion node, and the other end is coupled to another diffusion node of the at least one diffusion node, where x is natural number. The image processing device of claim 25, wherein the control unit sets the value k to a natural number between 1 and the dissimilarity finding window parameter w, and correspondingly in sequence or parallel The processing mode drives the input unit to find out and the following first original dissimilarity data is used as the input data; wherein the conversion unit and the control unit perform corresponding operations to generate corresponding to the next first original The analog circuit model of the dissimilarity data and the at least one diffused imitation voltage signal corresponding to the next first original dissimilarity data. The image processing device of claim 27, wherein when the value k corresponding to the natural number between the dissimilarity finding window parameter w is completed, the control unit corresponds to the at least A diffusion node finds the at least one diffusion-like voltage signal, and finds the processed image data according to a minimum diffusion-like voltage signal having a minimum voltage value among the at least one diffusion-like voltage signal. The image processing device of claim 28, wherein the control unit further determines whether the minimum diffusion analog voltage signal corresponds to a reference diffusion analog voltage signal to verify the minimum diffusion analog voltage signal, the reference diffusion The analog voltage signal is related to the dissimilarity data of the second view image data relative to the first view image data; wherein, when the minimum diffusion analog voltage signal corresponds to the reference diffusion analog voltage signal, the control unit uses the The minimum diffused imitation voltage signal derived from the diffused analog voltage signal is used as the processed image data. The image processing device of claim 25, wherein the simulation unit provides a time data node in the at least one data node, corresponding to the time data node, the at least one connection component comprising a time data connection component, The time data connection element is coupled between the time data node and the at least one diffusion node. The image processing device of claim 22, wherein the control unit selectively determines the at least one data node, the at least one diffusion node, and the analog circuit model in response to a first user operation event The control unit further selectively sets a voltage signal corresponding to the at least one spatial data node and a resistance value of the at least one connecting component in the analog circuit model in response to a second user operation event. The image processing device of claim 22, wherein the input unit receives a plurality of pixel data corresponding to the plurality of pixels, and uses the second pixel data as the pixel corresponding to the pixels. The original data; wherein the processed image data generated from the original data of the pseudo-pixel data is the image data after the image smoothing process. The image processing device of claim 22, wherein the input unit receives a first frame data, and uses the first frame data as the input data, wherein the first frame data corresponds to a first a picture resolution; wherein the processed image data generated by the input data that is substantially the first frame data is image data corresponding to one of the second picture resolutions. The image processing device of claim 22, wherein the simulation unit provides a spatial data node in the at least one data node, corresponding to the spatial data node, the at least one connection component comprising a spatial data connection component The spatial data connection element is coupled between the spatial data node and the at least one diffusion node.