KR101155227B1 - Apparatus and method for converting image reflecting directional texture information of original image - Google Patents

Abstract

Disclosed are an image converting apparatus and method considering the contour effect. The image input unit receives a target image, which is the basic image of the result image, and a source image, which is a reference image used to determine the texture of the result image, in shape and color. The distance function calculator calculates a value of a distance function defined as a sum of color factors, texture factors, and direction factors corresponding to the pixels to be processed sequentially in the result image to determine values of each pixel constituting the result image. . The result image generator generates a result image by determining a pixel value of a pixel that minimizes a result of the calculation of the distance function among candidate pixels determined on the source image as the pixel value of the pixel to be processed. According to the present invention, it is possible to generate a natural result image reflecting the texture of the source image accurately and reflecting the direction of the target image.

Description

Apparatus and method for converting image reflecting directional texture information of original image}

The present invention relates to an apparatus and method for converting an image, and more particularly, to an apparatus and method for converting a realistic image into a non-realistic image.

Painters draw a similar, stable brush stroke repeatedly on a two-dimensional canvas. As can be seen in Figure 1, the artworks can have very different forms depending on the artist's brush usage to create strokes of different lengths, directions and sizes from the material. In particular, the direction of the brush stroke is an important feature since the brush stroke follows a boundary for expressing the shape of the object. The main goal of Non-Photorealistic Rendering (NPR) is to imitate painting techniques. Non-realistic rendering can be categorized by visualization methods such as physical concurrency of brush operations through user interaction, stroke (brush-touch) based rendering, texture transitions based on target images, and the like. Among the non-realistic rendering techniques, the texture transfer technique receives the example source image S and the target image T and outputs the final result image R. Each pixel present in the target image T is updated by an optimal value among candidate sets constituting the example source image S. FIG. The resultant image R is obtained by synthesizing the color of the target image T and the texture of the source image S. FIG. Existing techniques detect the optimal texture from a candidate set using distance factors based on average color and spatial frequency to select candidate pixels. Although these factors are effective for the texture transition from the source image S to the target image T, the direction information of the target image T is lost. The use of obscure direction information makes it easier for the observer to reconstruct the shape of the object and create a more artistic impression. The importance of direction in painting is evident in Van Gogh's work in which the brush strokes clearly follow the direction of the outline. In the field of stroke-based rendering, this technique is widely studied to control the direction of stroke. Generally, the gradient of an image is considered to control the stroke direction, but the texture transition algorithm is a pixel-based approach that is difficult to consider.

The technical problem to be achieved by the present invention is to reflect the texture of the example source image while the resultant image has the orientation of the target image in consideration of both the directional component of the target image and the example source image and the contour effect based on the image tilt of the target image. There is provided an image conversion apparatus and method.

Another technical problem to be achieved by the present invention is to reflect the texture of the example source image while the resultant image has the orientation of the target image in consideration of both the directional component of the target image and the example source image and the contour effect based on the image tilt of the target image. The present invention provides a computer-readable recording medium that records a program that can be executed on a computer.

In order to achieve the above technical problem, an image conversion apparatus according to the present invention includes a target image, which is a basic image of a result image, and a source image, which is a reference image used to determine texture of the result image, in shape and color. Image input unit for receiving a; Distance function calculation for calculating a value of a distance function defined by Equation A below for a target pixel sequentially selected among pixels constituting the resultant image to determine a value of each pixel constituting the resultant image. part; And a result image generator configured to generate a result image by determining a pixel value of a pixel which minimizes a result of the calculation of the distance function among the candidate pixels determined on the source image as the pixel value of the pixel to be processed.

In order to achieve the above technical problem, an image conversion method according to the present invention includes (a) a reference image used to determine a target image, which is a basic image of a result image, and a texture of the result image in shape and color; Receiving a source image which is an image; (b) randomly selecting pixels of the source image to fill the pixels of the result image to initialize the result image; (c) sequentially selecting pixels to be processed from the upper left pixel to the lower right pixel of the resultant image, and calculating a value of a distance function defined by Equation A to determine a value of the selected pixel to be processed. ; And (d) determining a pixel value of a candidate pixel that minimizes a calculation result of the distance function among the candidate pixels determined on the source image as the pixel value of the pixel to be processed, wherein (c) and the Step (d) is repeatedly performed until a pixel value is determined for all pixels of the resultant image.

로 정의되며, 여기서, r은 상기 결과 영상을 구성하는 픽셀들 중에서 순차적으로 선택된 처리 대상 픽셀, q는 소스 영상 상에서 결정된 후보 픽셀, D_N(r,q)는 색상 인자로서 상기 목표 영상 상에서 처리 대상 픽셀에 해당하는 좌표를 중심으로 하는 사전에 설정된 크기의 사각형 내에 포함되는 픽셀들로 이루어진 사각형 이웃 집합에 속하는 픽셀들의 평균 픽셀값과 상기 소스 영상 상에서 처리 대상 픽셀에 해당하는 좌표를 중심으로 하는 사전에 설정된 크기의 사각형 내에 포함되는 픽셀들로 이루어진 사각형 이웃 집합에 속하는 픽셀들인 후보 픽셀들 각각에 대해 결정된 사각형 이웃 집합을 구성하는 픽셀들의 평균 픽셀값 사이의 유클리디안 거리, D_L(r,q)는 질감 인자로서 상기 결과 영상 상에서 상기 처리 대상 픽셀에 해당하는 좌표를 중심으로 하는 사전에 설정된 크기의 사각형 내에 속하는 픽셀들 중에서 상기 처리 대상 픽셀을 기준으로 좌상 방향에 위치하는 픽셀들로 이루어진 L형 이웃 집합 각각에 속하는 픽셀들의 표준 편차와 상기 후보 픽셀들 각각의 좌상 방향에 위치하는 픽셀들로 이루어진 L형 이웃 집합에 속하는 픽셀들의 표준 편차의 차이, D_I(r,q)는 방향 인자로서 상기 결과 영상 상에서 결정된 L형 이웃 집합에 속해 있는 보간된 경사 방향에 수직한 방향으로 놓여 있는 픽셀들로 구성되는 I형 이웃 집합에 속해 있는 픽셀들의 밝기 평균값과 상기 후보 픽셀들 각각의 밝기값의 차이, w_L은 상기 질감 인자에 대한 가중계수, 그리고 w_I는 상기 방향 인자에 대한 가중계수이다.Equation A is

Where r is a processing target pixel sequentially selected from the pixels constituting the resultant image, q is a candidate pixel determined on the source image, and D _N (r, q) is a color factor to be processed on the target image. An average pixel value of pixels belonging to a rectangular neighborhood set consisting of pixels included in a rectangle of a preset size centered on a coordinate corresponding to a pixel, and a coordinate centered on a coordinate corresponding to a pixel to be processed on the source image. Euclidean distance, D _L (r, q), between average pixel values of pixels constituting the rectangular neighborhood set determined for each of the candidate pixels that are pixels belonging to the rectangular neighborhood set of pixels included in the rectangle of the set size Denotes a texture factor that is centered on coordinates corresponding to the pixel to be processed on the resultant image. Standard deviations of pixels belonging to each of the L-shaped neighbor sets including pixels located in an upper left direction with respect to the processing target pixel among pixels belonging to a rectangle having a preset size and located in the upper left direction of each of the candidate pixels. The difference in the standard deviation of pixels belonging to the L-shaped neighbor set of pixels, D _I (r, q), is a direction factor and lies in a direction perpendicular to the interpolated oblique direction belonging to the L-shaped neighbor set determined on the resultant image. The difference between the brightness averages of the pixels belonging to the set of I-type neighbors and the brightness values of each of the candidate pixels, w _L is a weighting factor for the texture factor, and w _I is a weighting factor for the direction factor. Coefficient.

According to the image converting apparatus and the method according to the present invention, it is possible to generate a natural result image in which the direction component of the target image is properly expressed while accurately reflecting the texture of the source image. The degree of reflection of the direction component of the target image in the resultant image can be selectively determined by changing the weighting factor for the direction factor. In addition, by adjusting the brightness value of each pixel constituting the result image by using the brightness change value of each pixel extracted from the source image, it is possible to create a more visually excellent image by highlighting the outline of the result image.

1 is a view showing an example of a work of art having a very different form according to the artist's brush usage techniques,
2 is a diagram showing the configuration of a preferred embodiment of an image conversion apparatus according to the present invention;
3 is a diagram for explaining a stroke-based rendering technique and a pixel-based rendering technique;
4 shows an example of a rectangular neighborhood set and an L-shaped neighborhood set for pixel x,
FIG. 5 is a diagram for explaining a defect appearing in a resultant image by using edge pixels of a source image; FIG.
6 illustrates a non-edge image obtained by subtracting an edge image from a source image;
7 is a diagram illustrating an original gradient image and an interpolated gradient image;
8 shows an example of a type I neighbor set;
9 is a view showing a specific process of calculating the direction factor D _I (rq),
10A is a graph showing the average brightness change obtained by averaging the rate of change at each distance;
10B is a diagram illustrating a process of detecting a change in brightness of each pixel according to a distance from an edge from a source image;
11 is a graph illustrating average pixel brightness change versus distance of pixels from the nearest edge according to each source image;
12 is a diagram illustrating a circular flow and interpolated flow of respective source images;
FIG. 13 is a diagram illustrating a process of calculating a distance difference (DoD) value from an original direction and an interpolated direction.
14 is a diagram illustrating a direction factor weighting factor w _I estimated from DoD values calculated from various images;
15 is a flowchart illustrating a process of performing a preferred embodiment of the image conversion method according to the present invention;
FIG. 16 illustrates various types of source images; FIG.
17A is a view showing the resultant image generated by the image converting apparatus and method according to the present invention, using the Eiffel Tower photo and the portrait of Van Gogh as the image of FIG. 16 as the target image and the source image, respectively;
FIG. 17B is a view showing result images obtained by using the Eiffel Tower photo and the images (c), (j) and (e) of FIG. 16 as source images, and changing the weighting factors of the direction factors;
FIG. 18A is a view showing result images obtained by applying the source images shown in FIG. 16 to a Eiffel tower photograph by a conventional high speed texture transfer technique; FIG.
18B is a view showing result images obtained by applying the source images shown in FIG. 16 to a picture of the Eiffel Tower by the image converting apparatus and method according to the present invention;
FIG. 19 is a view showing result images obtained by using the image (a) of FIG. 16 as a source image while varying the direction factor weighting factor w _I for the Eiffel Tower photograph by the image converting apparatus and method according to the present invention _; FIG.
20 shows that the landscape image is the target image and the source image is the image (g), (f) and (b) of FIG. A view showing the resultant image generated by the image conversion apparatus and method according to the present invention;
21 is a view illustrating image conversion results by various techniques;
22A and 22B show a resultant image obtained by using the images (g) and (i) of FIG. 16 as source images, respectively;
FIG. 23 is a view illustrating result images obtained by using the image (c) of FIG. 16 as a source image for the direction texture transition technique and the conventional pixel-based texture transition technique according to the present invention; FIG.
24 is a view showing result images obtained by using the image of FIG. 16 (b) as a source image for the direction texture transition technique and the patch-based technique proposed by Wang according to the present invention;
25 is a diagram showing the result of applying the texture transfer technique to texture synthesis according to the present invention;
FIG. 26 is a diagram illustrating a result of applying a texture transfer technique to various images according to the present invention; FIG.
27 is a diagram showing the result of applying the contour effect proposed in the texture transfer technique according to the present invention to a kurtosis band image.
28 is a diagram illustrating the results of applying the texture transfer technique according to the present invention to the King Sejong image.

Hereinafter, exemplary embodiments of an image conversion apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a diagram showing the configuration of a preferred embodiment of an image conversion apparatus according to the present invention.

2, the image conversion apparatus according to the present invention includes an image input unit 210, a distance function calculator 220, a result image generator 230, and a contour effect unit 240.

The image input unit 210 is a means for receiving the target image T and the source image S. The image input unit 210 receives a target image T and a source image S through a communication network such as the Internet, or selects images selected by a user from among images stored in separate storage devices. It is inputted with (S). In this case, the target image T is an image that is the basis of the result image R in form and color, and the source image S is a reference image used to determine the texture of the result image R. FIG.

The distance function calculator 220 calculates a value of a distance function for determining a value of each pixel constituting the resultant image R. In the present invention, the distance function is composed of a color factor, a texture factor and a direction factor. The present invention relates to a technique for transferring the texture taken from the target image (T) to the source image (S) while following the inclination information of the source image (S). In addition, the present invention extends and uses a fast texture transfer technique having a fast presentation technique metric. In this case, the present invention introduces the concept of an I-type neighbor in order to evaluate the similarity of the neighbor considering the inclination direction.

In stroke-based rendering, the basic unit is stroke, and the resultant image R is filled with strokes. These techniques can consider the shape of the object by rotating the stroke based on the gradient value of the image. In contrast, the texture transfer algorithm generates the resulting image R by filling each pixel. Because each pixel has only one color value, representing the direction of the image is more difficult than stroke-based rendering. The pixel-based technique may express the direction of the image by filling each pixel with similar colors located along the tilt direction of the image, as shown in FIG. 3. The present invention is constructed based on a fast texture transition algorithm, which is a pixel-based technique.

4 shows examples of rectangular neighbor sets and L-shaped neighbor sets for pixel x. Referring to FIG. 4, the rectangular neighbor set for pixel x is composed of pixels located in a rectangular area of a constant size with respect to pixel x, and the L-shaped neighbor set is a component of the rectangular neighbor set of pixel x. Pixels located on the upper left side are used as components. If the source image S, the target image T to be filtered and the result image R are all single channel images, the fast texture transfer algorithm performs the result image R based on the distance function by the following process. Determine the pixels on the image.

First, the resultant image R is initialized by filling randomly selected pixels from the source image S. In this case, when the function g is a function of continuously tracking the mapping of the pixel coordinates, the pixel r of the resultant image R obtains a value from the pixel g (r) of the source image S. Next, the resultant image R is sequentially scanned from the upper left to the lower right, generating a candidate set Q for each pixel of the resultant image R, and then selecting a minimum distance factor from the generated candidate set Q. The pixel r of the resultant image R is updated. To this end, the distance function calculator 120 first generates a candidate set Q. The candidate set Q may be defined as follows.

Where r is a pixel selected in the result image R, g (r) is a pixel of the source image S mapped to pixel r selected in the result image R, and t is in the L-type neighbor set Is the pixel.

Additional candidate pixels are added to a predefined probability p to expand the search space. Since this is based on a random area, an area (indicated by a square box) having an intensity value similar to that of the edge of the source image S is often shown by the use of the boundary pixels of the source image S, as shown in FIG. 5. ) Causes a defect. To eliminate this defect, use only pixels that are not at the boundary. To this end, edge pixels are removed from the source image S, and candidate sets are selected from the remaining pixels. This process is illustrated in FIG. 6.

Next, the distance function calculator 220 calculates a distance function for selecting an optimal candidate pixel. Therefore, the distance of each candidate pixel is calculated using the following equation, and the pixel of the source image S mapped to the selected pixel r is determined in the result image R which is the pixel of the minimum distance.

In Equation 2, D (r, q) is defined as follows.

Where D _N (r, q) is the Euclidean distance between color average values of pixels belonging to a rectangular neighbor set, D _L (r, q) is the difference in standard deviations of pixels belonging to an L-shaped neighbor set, D _I ( r, q) is a direction factor, w _L is a weighting factor for a texture factor having a value between 0 and 1, and w _I is a weighting factor for a direction factor having a value between 0 and 1.

The calculation process for each factor shown in Equation 3 is as follows.

First, a rectangular neighbor set consisting of pixels included in a rectangle (for example, 5 × 5 pixel size) centered on coordinates corresponding to a processing target pixel on a target image T and a source image S is determined. Then, D _N (r, q), which is the Euclidean distance between color average values of pixels belonging to each set of square neighborhoods represented by the following equation, is calculated.

놈을 의미하고, n_r과 n_q는 다음의 수학식으로 정의되는 목표 영상(T) 내의 사각형 이웃 집합에 속하는 픽셀들의 색상 평균값과 소스 영상(S) 내의 대응되는 사각형 이웃 집합에 속하는 픽셀들의 색상 평균값을 의미한다. here, ??? ₂ is

N _r and n _q are the color mean values of pixels belonging to the rectangular neighbor set in the target image T and the colors of the pixels belonging to the corresponding rectangular neighbor sets in the source image S defined by the following equation. Mean average value.

If the size of the rectangle for determining the rectangle neighbor set is set to 5 × 5 pixel size, 24 color factor values corresponding to the number of pixels belonging to the neighbor set determined on the source image S are determined. In this case, the pixels belonging to the neighbor set determined on the source image S are elements of the candidate pixel set Q. In Equations 4 and 5, n _r is an average pixel value of pixels constituting a rectangular neighborhood set determined for a pixel having a coordinate corresponding to the coordinate of the pixel to be processed on the target image T. In addition, in Equations 4 and 5, n _q is an average pixel value of pixels constituting the rectangular neighbor set determined for each candidate pixel constituting the candidate pixel set Q. Thus, equations (4) and (5) derive 24 color factor values corresponding to each candidate pixel.

Next, among the pixels belonging to the rectangular neighbor set determined on the resultant image R, an L-shaped neighbor set consisting of pixels located in the upper left direction with respect to the pixel to be processed (that is, belonging to a rectangular neighbor set having a size of 5 × 5 pixels) Among the pixels, a set consisting of two pixels located on the left side of the pixel to be processed and ten pixels located on two upper rows) is determined. The standard deviation value of the pixels belonging to the L-type neighboring set determined for each resultant image R is calculated, and then D _L (r, which is the difference between the standard deviations of the pixels belonging to each L-type neighboring set defined by the following equation: , q) is calculated.

놈을 의미하고, |?|는 집합 크기를 의미하고, H_r과 H_q는 다음의 수학식으로 정의된다.here, ??? ₂ is

The norm, |? | Means the set size, H _r and H _q is defined by the following equation.

If the size of the rectangle for determining the rectangle neighbor set is set to 5 × 5 pixel size, 24 texture factor values corresponding to the pixels belonging to the neighbor set determined on the source image S are determined. In this regard, Equations 6 and 7 will be described in detail. Elements belonging to L (q) are elements of the candidate pixel set Q determined on the source image S. FIG. Accordingly, Hq is a difference between an average pixel value of twelve pixels belonging to L-type neighbor sets set on the source image S based on each candidate pixel, and a pixel value of the corresponding candidate pixel. As a result, equations (6) and (7) yield 24 texture factor values corresponding to the respective candidate pixels.

Finally, for each pixel of the resultant image R, the resultant image R is a pixel on the source image S that minimizes the distance function D (r, q) defined by the following equation while sequentially scanning from left to right. ) To replace the processing target pixel.

Here, w _L is a weighting coefficient having a value between 0 and 1, and is generally set to 1.

The distance function calculation unit 220 included in the image conversion apparatus according to the present invention calculates a value by using a new distance function modified from the distance function of the fast texture transfer method as described above. Existing fast texture transfer algorithms are useful for expressing consistent texture results. However, this technique considers only the distance between the mean value and the variance, and cannot express the direction information appearing in the target image T clearly.

The present invention considers the direction of the target image (T) to solve the problem of this fast texture transfer algorithm. The I-type neighbor set is composed of pixels lying in a direction perpendicular to the interpolated oblique direction belonging to the L-type neighbor set. In this case, image flow information is calculated to find an I-type neighbor set corresponding to each pixel. In order to extract consistent gradient information of the target image T, a known edge tangent flow (ETF) algorithm is applied. This technique has the advantage of preserving the edge orientation properly while keeping the edges around important features smooth. The ETF algorithm uses two parameters, radius and iteration, to adjust the smoothness of the direction field. In a preferred embodiment the radius and the number of repetitions are set to 6 and 3, respectively. 7 shows ETF interpolation results using Line Integral Convolution (LIC), which is a vector visualization radix. The left image of FIG. 7 is an original gradient image, and the middle image is an interpolated gradient image with radius and repetition frequency set to 6 and 3, respectively. The right image is interpolated gradient image with radius and repetition frequency set to 16 and 3, respectively. to be.

In stroke-based rendering, the resulting ETF is used to determine the position of the brush stroke. According to the present invention, the texture stroke may be expressed in the interpolated oblique direction of the target image T by considering the I-type neighbor set. At this time, the direction factor D _I (rq) is defined by the following equation.

Meanwhile, the average brightness value of the pixels existing in the I-type neighbor set defined in the result image may be calculated based on the interpolated slope value, and the calculated average brightness value and each pixel q located in the example source image S may be calculated. The distance between them can be calculated. If the distance between the average brightness value and each pixel q is small, this means that pixel q has a color similar to the pixels included in the type I neighbor set. In contrast, if the distance between the average brightness value and each pixel q is large, this means that the pixel q varies greatly in terms of color and pixels included in the type I neighbor set. Therefore, in order to maintain the consistency of the gradient information, the pixel having the minimum distance must be found. The range of the direction factor D _I (rq) is defined between 0 and 1. 8 shows an example of a type I neighbor set. Referring to FIG. 8, the pixels constituting the I-type neighbor set are N × N Sobel masks (for example, 5 × N) for pixels belonging to the L-type neighbor set around the pixel r present in the resultant image R. A color defined by the color direction (ie, θ = tan ⁻¹ (g (x) / g (y)) obtained based on g (x) and g (y) calculated using a 5-pixel Sobel mask) Direction).

9 shows a specific process for calculating the direction factor D _I (rq). Referring to FIG. 9, first, a pixel existing in an oblique direction obtained by applying a 5 × 5 pixel Sobel mask centering on the processing target pixel r on the resultant image R (in the upper direction of the processing target pixel r in FIG. 9). Determines the set of type I neighbors whose elements are the two pixels located in the thick rectangle. Next, an average brightness value of two pixels belonging to the I-type neighbor set on the resultant image R is calculated. Next, the Euclidean distance of the average brightness value of two pixels belonging to the I-type neighbor set on the resultant image R and the brightness value of the pixels belonging to the neighbor set determined on the source image S are calculated. Accordingly, the equation 24 yields 24 direction factor values corresponding to the candidate pixels.

The direction factor thus calculated is introduced as an element of the distance function for determining the value of the pixel to be processed of the resultant image R. Therefore, Equation 8, which is a distance function defined in the existing fast texture transfer method, is modified in the form of Equation 3 in the present invention. In Equation 3, each of the factors needs to be normalized in order to prevent a situation in which the influence of a certain factor among the factors of the distance function becomes overwhelming. The distance function calculator 220 calculates a value of the new distance function defined by Equation 3 below. The values of the calculated distance function are values corresponding to each of the candidate pixels determined on the source image S. FIG. Of the values listed in Table 1 below, q4 is the optimal pixel among the calculated pixels.

Candidate pixels D _N D _L D _I D (r, q) q1 0.42 0.3 0.6 1.32 q2 0.4 0.3 0.53 1.23 q3 0.9 0.65 0.1 1.65 q4 0.2 0.27 0.4 0.87 … … … … …

The result image generator 230 generates a result image by determining each pixel of the result image to a value that minimizes a result of the calculation of the distance function input from the distance function calculator 220 based on the target image and the source image. At this time, the result image generator 230 initializes the result image by filling the pixels randomly selected from the source image as described above, and then determines the pixel values by sequentially scanning each pixel from the upper left to the lower right. In this process, the result image generator 230 determines, as a pixel value of the processing target pixel, a pixel that minimizes the distance function value calculated by the distance function calculator 220 among candidate pixels determined on the source image.

The contour effect unit 240 provides a contour enhancement effect to the resultant image R based on the analysis of the edge effect of the source image S in order to express a more natural visual effect. To this end, the contour effect unit 240 first extracts the edge information from the source image (S). In this process, consistent line description techniques are used. On the other hand, when it is difficult to detect accurate boundary information from the source image S used as the reference image, the edge image is manually corrected. The contour effect unit 240 calculates an edge distance map from the detected edge image and uses pixels located near the edge. To this end, the contour effect unit 140 calculates an edge map by averaging the difference in brightness values according to the distance between the pixels constituting the source image S and each edge pixel constituting the edge image.

Next, the contour effect unit 240 analyzes the brightness change according to the distance of each pixel from the pixel closest to the edge. The brightness change information thus obtained is a value obtained by averaging the difference between the brightness values according to the distance between the pixels constituting the source image S and each edge pixel constituting the edge image, constituting the source image S. The average brightness change rate for each of the pixels. 10A is a graph showing the average brightness change obtained by averaging the rate of change at each distance. In this way, by obtaining the average brightness value at each distance and connecting each point, it is possible to obtain a picture with a linear decrease or increase in each pixel region as shown in FIG. 10B. In FIG. 10B, the left image is the source image S, the center image is an enlarged image of the box region of the left image, and the right image is an enlarged image of the box region of the center image. The contour effect unit 240 adjusts the brightness according to the edge distance information for each edge pixel of the resultant image R based on the graph derived from the source image S as shown in FIGS. 10A and 10B. . That is, the contour effect unit 240 changes the brightness value of each pixel constituting the resultant image R based on the average brightness change rate obtained for each pixel constituting the source image S, resulting in the resultant image R. The contour effect of the resulting image R is imparted by expressing the outline of. As shown in FIG. 11, the graph may have various forms depending on the material.

Meanwhile, the weighting factor w _I for the direction factor defined in Equation 3 may be automatically determined based on the source image S. To this end, the image conversion apparatus according to the present invention may further include a weighting factor determiner 140. The weighting factor determination unit 250 determines a weighting factor for the direction factor by using a weighting factor determination function that is experimentally set corresponding to the characteristics of the source image S. FIG.

Although there are some differences depending on the painting tools, the source image S has one stroke flow. For example, in sketches or paintings of Van Gogh, strokes are consistent based on the shape of the object. However, painting tools such as dot painting and watercolors are difficult to find. In order to express the stroke flow similarly to the stroke flow of the source image S, the present invention uses a weighting factor w _I for the direction factor. The degree of the direction effect in the resultant image R can be adjusted according to the value of the weighting factor w _I for this direction factor. If the value of the weighting factor w _I for the directional factor is set large, the directional component has a greater influence on the resultant image R, and thus the stroke texture with a more consistent direction based on the slope of the source image S. This expressed result image R can be obtained.

The value of the weighting factor w _I for this direction factor may be automatically set based on the source image S. If the value of the weighting factor w _I for the direction factor can be estimated, the user can reduce the effort to find the optimal weighting factor to obtain the resultant image R with the flow of the source image S. In order to automatically estimate the value of the weighting factor w _I for the direction factor from the source image S, a difference of direction (DOD) expressed by the following equation is defined.

는 소스 영상(S)의 원 경사 정보이며,

는 소스 영상(S)의 보간된 경사 정보이다. Here, x is the coordinate of the pixel in the source image (S),

Is the original tilt information of the source image (S),

Is interpolated slope information of the source image S. FIG.

In the present invention, the original tilt information is calculated using the Sobel filter. We use the ETF interpolation technique with a radius of 6 and an iteration of 3 for the interpolated direction. The range of directions is from 0 ° to 360 °. Since 0 ° is equal to 360 °, the average direction is calculated circularly. 12 illustrates a circular flow and interpolated flow of two different source images. In line drawing, the DoD value is not large because the flow of the original image slope is already consistent. In contrast, DoD has a large value in watercolors. Thus low DOS values are derived from line drawing and high DoD values are derived from watercolor. DoD has a value between 0 and 180. 13 shows an example for calculating a DoD value.

14 is a graph illustrating DoD values calculated from various images. Referring to FIG. 14, the DoD value is 30 or less in the strong line drawing, and the DoD value is 60 or more in the cell animation image with little flow in the stroke direction. In the case of oil paintings, watercolors and pastels, the DoD value is generally between 30 and 40. From the results shown in FIG. 14, when the DoD value is 30 or less, the weighting factor w _I for the direction factor is assigned to 1, and when the DoD value is 40 or more, the weighting factor w _I for the direction factor is 0. If the DoD value is between 30 and 40, it is allocated while decreasing linearly from 1 to 0. Therefore, the weighting factor determiner 250 may allocate the weighting factor w _I according to the direction factor according to the type of the source image S according to the following equation.

Here, a is a value obtained by subtracting a lower limit from an upper limit of a range preset for the direction factor by a value obtained by subtracting a lower limit from an upper limit of the direction difference value, and b is an upper limit value of the direction difference value by the direction difference value The upper limit of is divided by the lower limit of the lower limit.

From these results, the weighting factor w _I can be estimated by the direction factor of the target image T. If a value higher than the calculated weighting factor w _I is used, the direction larger than the flow of the source image S is obtained. Can express the effect.

15 is a flowchart illustrating a process of performing a preferred embodiment of the image conversion method according to the present invention.

Referring to FIG. 15, when the target image T and the source image S0 are input through the image input unit 210, the result image generator 230 randomly selects pixels of the source image S to generate a result image ( The resultant image generator 230 sequentially determines pixel values from the upper left pixel to the lower right pixel of the resultant image R in operation S1510. In this case, the resultant image generator 230 ) Indicates the coordinates of the pixel to be processed for which the pixel value is to be determined and information about the pixels included in a rectangle (for example, 5 × 5 pixel size) of a predetermined size centered on the pixel (ie, pixel coordinates and pixel values). Provides the processing target pixel information including the distance to the distance function calculator 220. Of course, the result image generator 230 performs coordinates of each pixel constituting the result image R generated while performing the result image generation process. And pixel values in a separate storage (not shown) A function calculating unit 220 may be used to read out necessary information from the storage unit.

Next, the distance function calculator 220 may use Equation 3 corresponding to the processing target pixel based on the processing target pixel information provided from the result image generator 230 and the target image and the source image provided from the image input unit 210. The value of the new distance function defined as is calculated (S1520). In this process, the distance function calculation unit 220 calculates the value of the distance function by calculating D _N corresponding to the color factor, D _ㅣ corresponding to the texture factor, and D _I corresponding to the direction factor among the components of Equation 3 do. Next, the result image generator 230 may select a pixel that minimizes the distance function value calculated by the distance function calculator 220 with respect to the pixel to be processed among the pixel values constituting the source image S. It is determined by the pixel value (S1530). Next, the result image generator 230 checks whether the determination of the pixel value for the last pixel of the result image S is completed (S1540). If the determination of the pixel value for the last pixel of the resultant image S is completed, the resultant image generator 230 outputs the final resultant image S (S1550).

Next, the contour effect unit 240 extracts edge information from the target image T and the source image S (S1560). At this time, if it is difficult to detect accurate boundary information from the source image S, the edge image is manually corrected. The contour effect unit 240 calculates an edge distance map based on the detected edge information, and analyzes the change in brightness according to the distance of each pixel from the pixel closest to the edge (S1570). Next, the contour effect unit 240 has an edge distance with respect to each edge pixel of the final resultant image R output by the resultant image generator 230 based on the brightness change information derived from the source image S. FIG. The brightness is adjusted according to the information (S1580).

Table 2 below describes the parameters used in the present invention, and Table 3 lists the size of the image and the rendering time according to the parameters.

parameter Explanation Neighbor size
p
Repeat count
w _L
w _I The size of the pixels in the considered area
The probability of an additional candidate being added
Number of iterations
Weighting Factor of Texture Factor D _L
Weighting Factor of Direction Factor D _I

The size of the target image (T) Neighbor size Processing time (ms) 640 × 480
640 × 480
1024 × 768
1024 × 768 5
7
5
7 43
73
76
132

Including the preprocessing time, when the image conversion according to the present invention is performed using a Pentium 2.8 GHz quad-core PC having 4 GB of memory for a VGA image size of which the neighbor size is 5 and the repetition frequency is set to 1, the processing time is about 4 seconds. This takes

16 illustrates various types of source images. Figure 16 (a) the image of Van Gogh's oil painting "self-portrait", (b) the image of Emil Nolde's "Autumn Sea", (c) image of Gustav Dora's "Don Quixote" line drawing, (d) The image is line drawing, (e) the image is watercolor, (f) the image is charcoal, (g) the image is watercolor, (h) the image is line drawing, (i) the image is Sumi-e painting, and (j) the image. Is pastel.

FIG. 17A is a view showing a result image generated by the image converting apparatus and method according to the present invention, using the Eiffel Tower photo and the portrait of Van Gogh as the image of FIG. 16 as the target image and the source image, respectively. The weighting value of the fragrance factor used at this time is 0.6. Referring to FIG. 17A, it can be seen that the resultant image obtained by the image converting apparatus and method according to the present invention properly reflects the direction of the target image while maintaining the texture of the source image. In FIG. 17A, an image disposed at a lower portion of the target image is a direction map of the target image, and the direction value obtained by performing Sobel masking on each pixel to visually indicate the direction component of the target image is line integral convolution ( Line Integral Convolution (LIC). FIG. 17B is a diagram showing the resultant images obtained by using the Eiffel Tower photo and the images (c), (j) and (e) of FIG. 16 as source images, and changing the weighting factors of the direction factors. The image (a) of FIG. 17B is a source image, and the weighting coefficient of the direction factor is 0.7 with respect to the image (c) of FIG. 16, and the image (b) is 0.8. And (c) the image showing the resultant image generated by the image conversion apparatus and method according to the present invention with the weighting factor of the direction factor of 0.8 for the (e) image of FIG. 17A and 17B, the texture of the stroke follows the flow of clouds and the Eiffel Tower.

FIG. 18A shows the result images obtained by using the (a) image, (c) image, (j) image, and (e) image of FIG. 16 as a source image for the Eiffel Tower photograph by a conventional high speed texture transfer technique. have. Referring to FIG. 18A, it can be seen that the result images generated by the conventional fast texture transfer technique reflect the texture of the source images to some extent, but the direction components of the target image are not properly represented. In addition, Figure 18b is a result obtained by using the image (a), (c), (j) and (e) of Figure 16 as a source image for the Eiffel Tower photo by the image conversion apparatus and method according to the present invention The images are shown. Referring to FIG. 18B, except for the image of FIG. 16 (j), the resultant images generated by the present invention accurately reflect the texture of the source image while the direction component of the target image is accurately reflected. You can see that it is properly represented.

FIG. 19 illustrates the resultant images obtained by using the image (a) of FIG. 16 as a source image while varying the weighting factor w _I for the direction factor with respect to the Eiffel Tower photograph by the image converting apparatus and method according to the present invention. . Referring to FIG. 19, it can be seen that the resultant images generated by the present invention more strongly reflect the direction component of the target image as the value of the weighting factor w _I for the direction factor increases.

20 shows that the landscape image is the target image and the source image is the image (g), (f) and (b) of FIG. After that, the resultant image generated by the image conversion apparatus and method according to the present invention is illustrated. Referring to FIG. 20, it can be seen that the characteristics of the source image appear in each image. In particular, in the image (c) of FIG. 20, the sky is represented by white pixels, and in the image (d), the sky is represented by a regular pattern. This defect is derived from a source image having a small white area, thus filling a large white area of the target image with a small number of candidates.

21 illustrates image conversion results by various techniques. (A) of FIG. 21 is a resultant image obtained by using the conventional ashkmin technique after setting the weighting factor for the direction factor to 0, and a certain defect exists because a random selection technique is used to obtain a candidate set. do. In addition, the image (b) of FIG. 21 is a result image obtained by using the direction texture transition method according to the present invention after setting the weighting factor for the direction factor to 0.4. In addition, the image (c) of FIG. 21 is a resultant image obtained by the direction texture transition technique according to the present invention by selecting a candidate set from the non-edge region after setting the weighting factor for the direction factor to 0.7. ) Is a resultant image obtained by adding a contour effect to image (c) of FIG. 21.

FIG. 22A is a resultant image obtained by using the image (g) of FIG. 16 as a source image. FIG. 22A (a) is a resultant image obtained without the contour effect. . In addition, FIG. 22B is a resultant image obtained by using the image of (i) of FIG. 16 as a source image, (a) of FIG. 22B is a resultant image obtained without the contour effect, and (b) the resultant image of applying the contour effect. to be. Referring to FIGS. 22A and 22B, it can be seen that a better quality image can be obtained by applying the contour effect to the resultant image based on the analysis of the edge effect of the source image.

FIG. 23 is a diagram illustrating result images obtained by using the image (c) of FIG. 16 as a source image for the direction texture transition technique and the conventional pixel-based texture transition technique according to the present invention. Referring to FIG. 23, the images (b) and (c) of FIG. 23 are the resulting images obtained using the method of Herzmann and the method of Ashkhmin, respectively, and are obtained by using the present invention. It can be confirmed that the texture of the source image is transferred to the result image more effectively than the result images obtained by these conventional techniques. FIG. 24 is a diagram illustrating result images obtained by using the image (b) of FIG. 16 as a source image for the direction texture transition technique and the patch-based technique proposed by Wang according to the present invention. Referring to FIG. 24, it can be seen that the resultant image (b) obtained by using the present invention has better texture effect than the resultant image obtained by the royal technique (a). Furthermore, while Wang's technique takes about 55 seconds to generate the result image, the present invention has the advantage that the result image can be generated with only about 6 seconds of processing time, and the present invention is easy to implement.

FIG. 25 is a diagram illustrating the results of applying the texture transfer technique to texture synthesis according to the present invention. In the texture transition study, only one channel (brightness) was used, and the final result is generated by combining the color of the target image and the updated brightness channel. In contrast, the texture synthesis study used all channels of the source image (ie, YIQ color space) and updated the result image. In the present invention, when the patch texture is in the form of petals, the orientation factor distorts the shape of the patch, and thus it is difficult to obtain good results. However, as shown in FIG. 25, when the patch does not have a specific shape, the present invention may also obtain good results. In addition, the quality of the resultant image varies depending on the observer, and the oblique flow is shown in the image (b) of FIG. 25 which is the result obtained by the present invention.

FIG. 26 is a diagram illustrating a result of applying a texture transfer technique to various images according to the present invention. FIG. (A) and (d) of FIG. 26 are result images obtained by using (d) of FIG. 16 as a source image, and (b) and (c) of FIG. 26 are images of (a) of FIG. The resulting image is obtained using. Referring to FIG. 26, it can be seen that the direction component existing in each target image is reflected in the result image, and the texture of the source image is appropriately displayed in the result image. 27 is a diagram showing the result of applying the contour effect proposed in the texture transfer technique according to the present invention to the kurtosis band image. (A), (b) and (c) of FIG. 27 are result images obtained by using (h), (i) and (g) images of FIG. 16, respectively, as source images. Referring to FIG. 27, each of the resulting images properly reflects the texture of the source image, and it can be confirmed that a new image effect exists due to the contour effect.

28 is a diagram illustrating the results of applying the texture transfer technique according to the present invention to the King Sejong image. In FIG. 28, (a) image is the target image, (b) image is the source image, (c) the image is not applied the contour effect, and (d) the image is applied the contour effect. As can be seen from FIG. 28, in the texture transfer technique according to the present invention, the direction component of the target image and the texture component of the source image are properly reflected in the resultant image, and further, another visual effect may be obtained when the contour effect is applied. .

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the above-described specific preferred embodiments, and the present invention belongs to the present invention without departing from the gist of the present invention as claimed in the claims. Various modifications can be made by those skilled in the art, and such changes are within the scope of the claims.

Claims (14)

An image input unit configured to receive a target image, which is a basic image of a result image, and a source image, which is a reference image, used to determine a texture of the result image in shape and color;
Distance function calculation for calculating a value of a distance function defined by Equation A below for a target pixel sequentially selected among pixels constituting the resultant image to determine a value of each pixel constituting the resultant image. part; And
And a result image generator configured to generate a result image by determining a pixel value of a pixel that minimizes a result of the distance function among the candidate pixels determined on the source image as a pixel value of the pixel to be processed. Video conversion device.
Equation A

Here, r is a processing target pixel sequentially selected from the pixels constituting the resultant image, q is a candidate pixel determined on the source image, and D _N (r, q) is a color factor corresponding to the processing target pixel on the target image. A rectangle of a preset size centered on an average pixel value of pixels belonging to a neighboring set of pixels included in a rectangle of a preset size centered on a coordinate and a coordinate corresponding to a pixel to be processed on the source image The Euclidean distance, D _L (r, q), between the average pixel values of the pixels constituting the rectangular neighborhood set determined for each of the candidate pixels that are pixels belonging to the rectangular neighborhood set of pixels included in the pixel is a texture factor. Preset based on the coordinates corresponding to the processing target pixel on the result image Standard pixels of pixels belonging to each of the L-shaped neighbor sets including pixels located in an upper left direction with respect to the processing target pixel among pixels belonging to a rectangle of a size, and pixels located in an upper left direction of each of the candidate pixels. The difference in the standard deviation of the pixels belonging to the formed L-shaped neighbor set, D _I (r, q), is a direction factor and the pixels lying in the direction perpendicular to the interpolated oblique direction belonging to the L-shaped neighbor set determined on the resultant image. The difference between the brightness average values of the pixels belonging to the I-type neighbor set, and the brightness values of each of the candidate pixels, w _L is a weighting factor for the texture factor, and w _I is a weighting factor for the direction factor. The method of claim 1,
The distance function calculator is configured to determine, on the resultant image, pixels existing in an oblique direction obtained by applying a Sobel mask having a preset size to each of the pixels constituting the L-type neighbor set determined on the resultant image. And an element of a neighbor set. 3. The method according to claim 1 or 2,
And a weighting coefficient determiner configured to determine a value of a weighting factor for the direction factor within a preset range according to a direction difference value defined by Equation B below with respect to the source image. :
Equation B

Where x is the coordinate of the pixel in the source image S,

Is the circle tilt information of the source image S,

Is the interpolated slope information of the source image S. The method of claim 3, wherein
And the weighting coefficient determining unit determines a value of the weighting coefficient with respect to the direction factor by Equation C below:
Equation C

Here, a is a value obtained by subtracting a lower limit from an upper limit of a range preset for the direction factor by a value obtained by subtracting a lower limit from an upper limit of the direction difference value, and b is an upper limit value of the direction difference value by the direction difference value The upper limit of is divided by the lower limit of the lower limit. 3. The method according to claim 1 or 2,
An edge image is generated by extracting edge information from the source image, and each of the results image is configured based on brightness change information between pixels constituting the source image and edge pixels constituting the edge image. And a contour effect unit for correcting brightness values of the pixels to give a contour effect to the resultant image. 6. The method of claim 5,
The brightness change information is obtained by averaging the difference in brightness values according to the distance between the pixels constituting the source image and each edge pixel constituting the edge image to average brightness for each pixel constituting the source image. The image conversion device, characterized in that the rate of change. The method according to claim 6,
And the contour effect unit changes the brightness value of each pixel constituting the resultant image based on the average change rate of brightness to give an outline effect to the resultant image. (a) receiving a target image, which is a basic image of a result image, and a source image, which is a reference image used to determine a texture of the result image, in shape and color;
(b) randomly selecting pixels of the source image to fill the pixels of the result image to initialize the result image;
(c) sequentially selecting pixels to be processed from the upper left pixel to the lower right pixel of the resultant image, and calculating a value of a distance function defined by Equation A to determine a value of the selected pixel to be processed. ; And
(d) determining a pixel value of a candidate pixel that minimizes a calculation result of the distance function among candidate pixels determined on the source image as the pixel value of the processing target pixel;
The steps (c) and (d) are repeatedly performed until a pixel value is determined for all pixels of the resultant image.
Equation A

Here, r is a processing target pixel sequentially selected from the pixels constituting the resultant image, q is a candidate pixel determined on the source image, and D _N (r, q) is a color factor corresponding to the processing target pixel on the target image. A rectangle of a preset size centered on an average pixel value of pixels belonging to a neighboring set of pixels included in a rectangle of a preset size centered on a coordinate and a coordinate corresponding to a pixel to be processed on the source image The Euclidean distance, D _L (r, q), between the average pixel values of the pixels constituting the rectangular neighborhood set determined for each of the candidate pixels that are pixels belonging to the rectangular neighborhood set of pixels included in the pixel is a texture factor. Preset based on the coordinates corresponding to the processing target pixel on the result image Standard pixels of pixels belonging to each of the L-shaped neighbor sets including pixels located in an upper left direction with respect to the processing target pixel among pixels belonging to a rectangle of a size, and pixels located in an upper left direction of each of the candidate pixels. The difference in the standard deviation of the pixels belonging to the formed L-shaped neighbor set, D _I (r, q), is a direction factor and the pixels lying in the direction perpendicular to the interpolated oblique direction belonging to the L-shaped neighbor set determined on the resultant image. The difference between the brightness average values of the pixels belonging to the I-type neighbor set, and the brightness values of each of the candidate pixels, w _L is a weighting factor for the texture factor, and w _I is a weighting factor for the direction factor. The method of claim 8,
In the step (c), the pixels existing in the oblique direction obtained by applying a Sobel mask having a preset size to each of the pixels constituting the L-type neighbor set determined on the resultant image are determined on the resultant image. The image conversion method, characterized in that determined as a member of the type I neighbor set. The method according to claim 8 or 9,
and (e) determining a value of a weighting factor for the direction factor within a preset range according to a direction difference value defined by Equation B for the source image. Way:
Equation B

Where x is the coordinate of the pixel in the source image S,

Is the circle tilt information of the source image S,

Is the interpolated slope information of the source image S. The method according to claim 8 or 9,
(f) extracting edge information from the source image to generate an edge image, and constructing the resultant image based on brightness change information between pixels constituting the source image and each edge pixel constituting the edge image; And correcting the brightness values of the respective pixels to give a contour effect to the resultant image. 12. The method of claim 11,
The brightness change information is obtained by averaging the difference in brightness values according to the distance between the pixels constituting the source image and each edge pixel constituting the edge image to average brightness for each pixel constituting the source image. The image conversion method, characterized in that the rate of change. 13. The method of claim 12,
In the step (f), the image conversion method, characterized in that to give the contour effect to the result image by changing the brightness value of each pixel constituting the result image based on the average brightness change rate. A computer-readable recording medium having recorded thereon a program for executing the image conversion method according to claim 8 or 9.

Images