CN1224242C - Method for fast picking up picture with any image as background in digital image process - Google Patents

Abstract

The present invention discloses a method for quickly picking up images in digital image processing by using any image as a background, which comprises the steps: 1) region segmentation is manually carried out for images; each image is divided into three parts: a foreground region, an unknown region and a background region; 2) according to the divided regions, the color of any point c in the unknown region is set as C, and the foreground color component F and the background color component B of the point c are calculated; 3) the values of F and B are adjusted by the measure of color adjustment according to the estimated foreground color component F and the background color component B; finally, the alpha value of the point c is estimated. Images can be picked up and processed effectively by the method for picking up the images; simultaneously, the speed of picking up images is greatly improved, the calculation cost is low, and the present invention has good practicability value.

Description

A Fast Cutout Method Using Arbitrary Image as Background in Digital Image Processing

Technical field

The invention relates to a fast image-cutting method with an arbitrary image as the background in digital image processing.

Background technique

Cutout technology is an image processing technique that separates the foreground part of any image from the background. It has extensive and in-depth applications in the production of film and television special effects and special effects. According to whether there are restrictions on the image background, the matting technology can be roughly divided into two categories:

There are restrictions on the foreground or background of the image and can be divided into blue screen matting and difference matting. The blue screen cutout technology has certain restrictions on the background of the image. The background is usually a fixed color, usually blue or green, or multiple images with the same foreground are prepared, and the background colors in these images are used to achieve accurate cutout. The purpose of taking the foreground. The difference matting technology needs to prepare another background image, and the alpha value is estimated by comparing the color difference between the background image and the corresponding point in the original image.

A class of natural image matting with no restrictions on the image background. It does not require the background of the image, and only needs one image.

The matting problem can be defined as: For any point c on a given image, find the foreground color F and alpha value α contained in the color C of point c. The difficulty of the matting problem is that for any point c on the image, its F and α solutions are not unique, and we need to find the most reasonable solution from countless pairs of solutions.

The blue screen cutout technology is simple, the amount of calculation is small, and the cutout effect is good. But it has its fatal weakness, that is, it has certain restrictions on the color of the background of the image. Under normal circumstances, the background is required to be blue or green. When applying this technology, a person needs to run around with a blue background. At the same time, it generally requires the RGB components of the foreground color to be distributed in a certain proportion. These make the blue screen cutout Technology has brought great inconvenience in specific applications.

Natural image matting techniques include Knockout method, Ruzon&Tomasi method, Hillman method and Chuang method. Natural image matting can generally be divided into three steps:

1. Regional segmentation. In general, manual segmentation is the main method. Because the accuracy of region segmentation has a great influence on the matting effect. Manually segmenting the region can have better accuracy. Generally, the region segmentation of an image takes about 2-3 minutes.

2. Foreground and background color estimation. The Knockout method uses the weighted average of the points in the adjacent area to estimate, the method is simple, and the calculation amount is small; the Ruzon&Tomasi method, the Hillman method and the Chuang method all use the law of statistics, the method is complicated, and the calculation amount is large.

3. Alpha value estimation. Use the estimated foreground and background colors to estimate the alpha value. Among the known methods, except for the Knockout method, the other methods do not adjust the foreground and background colors. However, the adjustment made by the Knockout method is too rough, resulting in poor matting effect.

Contents of the invention

The purpose of the present invention is to provide a fast method for cutting out images with any image as the background in digital image processing.

In the digital image processing, the fast matting method with any image as the background is to divide the image into three parts: the foreground area, the unknown area and the background area, and it is characterized in that: its steps are:

1) According to the segmented area, for any point c in the unknown area, set its color as C, and calculate its initial foreground and background color components F and B; the calculation of said initial foreground and background color components F and B Yes: For any point c in the unknown area, find out the points f' and b' closest to point c on the foreground contour line and background contour line, and the distances between point f' and b' from point c are d ₁ and d ₂ , given a positive real number θ (1.0<θ≤10.0), with point c as the center, and θd ₁ and θd ₂ as the radius, make two concentric circles C ₁ and C ₂ , set them in circle C ₁ All points inside and located on the foreground contour line are f ₁ , f ₂ , ..., f _k , and the distances of these points from point c are d ₁₁ , d ₁₂ , ..., d _1k , which are inside the circle C ₂ and located at All points on the background contour line are b ₁ , b ₂ , ..., b ₁ , and the distances from these points to point c are d ₂₁ , d ₂₂ , ..., d ₂₁ , and f _i (i=1, 2 ,...,k) the weighted average of point colors $\stackrel{\&OverBar;}{f}=\underset{i=1}{\overset{k}{\mathrm{\&Sigma;}}}{w}_{1i}{f}_i,b_j(j=\mathrm{1,2},\&CenterDot;\&Center\; Dot;\&CenterDot;,l)$ weighted average of point colors $\stackrel{\&OverBar;}{B}=\underset{j=1}{\overset{l}{\mathrm{\&Sigma;}}}{w}_{2j}{b}_j,$ in $w_{1i}=\frac{\mathrm{\&theta;}}{\mathrm{\&theta;}-1}-\frac{1}{\mathrm{\&theta;}-1}\&Center\; Dot;\frac{d_{1i}}{d_1}(i=\mathrm{1,2},\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,k),w_{2j}=\frac{\mathrm{\&theta;}}{\mathrm{\&theta;}-1}-\frac{1}{\mathrm{\&theta;}-1}\&Center\; Dot;\frac{d_{2j}}{d_2}(j=\mathrm{1,2},\&Center\; Dot;\&Center\; Dot;\&Center\; Dot;,l),$ F and B are the estimated initial foreground and background color components;

2) According to the estimated initial foreground and background color components F and B, take color adjustment measures to adjust to the values of F and B, and finally estimate the alpha value of point c. The estimated alpha value is: the steps are: 1 ) Determine the value of the color difference parameter ρ according to the color brightness difference between the foreground object and the background object in the image, 2) adjust the values of F and B to F and B according to the relationship between the color brightness ratio λ of F and B and the color difference parameter ρ , 3) According to the adjusted F and B, by the formula $\mathrm{\&alpha;}=\frac{(C-B)\&CenterDot;(f-B)}{{\left|\right|f-B\left|\right|}^2}$ Calculate the alpha value α of a point c in the unknown area.

The invention has the advantages of speed block and good effect. Previous natural image matting techniques put most of their energy on the second step, that is, how to estimate the foreground and background colors, while ignoring the third step, that is, the estimation of alpha. Focusing on step 2 and ignoring the results of step 3 lead to the emergence of various complex color estimation models. Although the matting effect is improved to a certain extent, it also greatly increases the amount of calculation, which makes the natural image matting technology in practice. Applications are very limited.

The present invention breaks through this framework and designs a cutout technology with a small amount of calculation and good cutout effect to meet actual needs. Its color estimation model is very simple, and the alpha value estimation scheme is ingenious.

Table 1 lists the processing time required for different matting examples on different machines. Compared with the Chuang method with the best matting effect in the world today, the speed of the present invention is 10-12 times higher than it, but the matting effect is not reduced. Chuang's method takes about 120 seconds to process the Syringe image in the environment where the CPU is P3 1.0G and the RAM is 512M. Use method of the present invention under the condition that machine condition is slightly worse than Chuang method, speed is about 11 times of Chuang method.

Table 1 Cutting time of different cutout examples in different environments CPU: Celeron 400RAM: 320M CPU: P3 600 RAM: 512M CPU: P4 1.8GRAM: 256M Syringes 16.484 seconds 10.485 seconds 5.680 seconds Feather_edge 25.696 seconds 16.474 seconds 8.844 seconds Galadriel 4.266 seconds 2.754 seconds 1.406 seconds Gandalf 5.618 seconds 3.635 seconds 1.891 seconds Tiger 5.658 seconds 3.666 seconds 1.922 seconds Water 9.303 seconds 5.988 seconds 3.281 seconds

By applying the invention, the foreground part in any image can be extracted quickly and effectively. A large number of examples prove that the present invention well solves the contradiction between the speed and the effect existing in the cutout, and has very good practical value.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention;

Figure 2(a)(b) depicts a schematic diagram of region division and estimation of foreground and background color components;

Figure 3(a)(b)(c)(d) is a schematic diagram of several relative positions of colors F, B and C in the color space;

Figure 4(a)(b) is a schematic diagram of the color adjustment scheme in the present invention;

Fig. 5 is the schematic diagram of the effect comparison about the example Syringe image of the present invention and other matting methods;

Fig. 6 is a schematic diagram of the effect comparison between the present invention and the Chuang method on the example Feather_edge image;

Fig. 7 is a schematic diagram of some other effects of the present invention.

Detailed ways

The principle of the fast matting method with an arbitrary image as the background in digital image processing is: for any point c in the transition area between the foreground and the background (that is, the unknown area in the area segmentation), the distance from the point c in the image space is Color-weighted average of some nearest foreground and background points F and B is used as the foreground and background color components of point c, and then an appropriate alpha value estimation scheme is used to estimate the alpha value α of point c. This method does not need to establish a complex model to calculate the foreground and background colors, and the color adjustment scheme is also very simple. Therefore, this method is a fast method for image matting.

First, in the second step of the matting method, the model for calculating the foreground color component F and the background color component B is simple and practical. This method only calculates the points on the contour lines contained in the two circle areas, and the calculation of the weight is linearly related to the distance between these points and point c. The weight of the nearest point is the largest, and the weight is linear as the distance increases. decrease. In the matting method of the present invention, the color estimation model in the second step is simplified as much as possible.

Second, in the third step, the color adjustment scheme of our method is simple and intuitive. This scheme keeps the ratio of the RGB components of the color unchanged, uses the characteristic that the human eye is not very sensitive to small changes in color brightness, adjusts the brightness of the foreground and background colors to a minimum extent, and finally uses the adjusted foreground and background colors to estimate the The alpha value of point C. The details of the program are as follows:

In the three-dimensional RGB color space, the color of a point can be expressed as a point or a vector. In the color three-dimensional coordinates, point O is the coordinate origin, which represents black. Any two points P ₁ , P ₂ , |P ₁ P ₂ | in a given color space represent the length of the line segment P ₁ P ₂ . Suppose any point c in the unknown area, its color is C, and the estimated initial foreground and background color components are B, F. In the three-dimensional color space, for the convenience of expression, we define as follows:

Cutout bias: The distance between point C and line segment BF is called cutout bias. In Figure 3(a), point C is located on line segment BF, and the matting deviation is 0 at this time; in Figure 2(b), point C is outside line segment BF, and line segment CC′ is perpendicular to line segment BF and intersects at point C ’, at this time, the matting deviation is the length |CC′| of the line segment CC′.

Regular point and irregular point: When point C is not too far from the line segment BF, we call point C at this time a regular point, otherwise it is an irregular point. The ratio of the lengths of line segment CC' to BF reflects the regularity of point C to a certain extent.

When point C just falls on the line segment BF, this situation is the most ideal situation of point C. From the color composition equation can be deduced:

$\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; )</span>}{{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In formula (1), the vector (C-B) corresponds to the directed line segment BC, the vector (F-B) corresponds to the directed line segment BF, and α is the ratio of the length of the projection of the line segment BC on BF to the length of BF. When point C is on the line segment BF, the matting deviation is 0, and the length of the projection of line segment BC on BF is the length of line segment BC. In most cases, the matting deviation is not 0, and the calculation of α by formula (1) will have a certain deviation. So the problem is transformed into how to reduce the value of the matting deviation as reasonably as possible. The color adjustment scheme of the present invention is based on the following observations:

Observation 1: When point C is a regular point, the error of the alpha value calculated by formula (3) can be ignored. (as shown in Figure 3(b));

Observation 2: When the angle ∠BCF is an obtuse angle, the distance between point C and line segment BF is not too far.

Observation 3: When the length of segment OC is between segments OB and OF, point C is not too far from segment BF.

Point C that satisfies observations 2 and 3 is a regular point. Not all rule points satisfy observations 2 or 3. After estimating the initial foreground and background color components F and After B, adjust in color space by the method that the present invention introduces F and The position of B can convert most of the irregular C points into regular points.

Definition of the present invention: F=( F _r , F _g , F _b ), B=( B _r , B _g , B _b ), C=(C _r , C _g , C _b ), wherein A _r , A _g , A _b is the component of color A on three channels of RGB, where A=F, B, C, calculate l _F = F _r + F _g + F _b , l _B = B _r + B _g + B _b , l _C =C _r +C _g +C _b , the ratio $\mathrm{\&lambda;}=l_{\stackrel{\&OverBar;}{f}}/l_{\stackrel{\&OverBar;}{h}},$ At this time, it is divided into three situations for discussion: ① If $\mathrm{\&lambda;}>1/\mathrm{\&rho;}$ and l _C > l _F , or $\mathrm{\&lambda;}\&le;1/\mathrm{\&rho;}$ And l _C < l _F , then adjust F to F=( Fr×l _C /l _F , F _g ×l _C /l _F , F _b ×l _C /l _F ), B is not adjusted, that is, B= B, ② if $\mathrm{\&lambda;}>1/\mathrm{\&rho;}$ and l _C < l _B , or $\mathrm{\&lambda;}\&le;1/\mathrm{\&rho;}$ And l _C >l _B , adjust B to B=( B _r ×l _C /l _B , Bg×l _C /l _B , B _b ×l _C /l _B ), F is not adjusted, that is, F= F , ③ In the rest of the cases, F and B are not adjusted, that is, F = F, B = B. Substitute the adjusted F and B into formula (1) to calculate the alpha value α.

The color adjustment scheme of the present invention is only for the initial foreground and background color components F and The brightness of B was modified minimally, without changing their color. In general, the human eye is not very sensitive to small changes in the brightness of the color, so it can be replaced by F and B F and B to calculate the alpha value. This replacement maintains the continuity of the estimated value of α and does not cause a great impact on the human eye. The α value calculated by F and B is closer to the real α value.

Fig. 1 is a detailed flow chart of the present invention. First, the input image is divided into three parts: foreground area, background area and unknown area; secondly, according to the divided area, for each point in the unknown area, its foreground and background color components are preliminarily estimated ; Finally, the present invention uses a color adjustment scheme to adjust the estimated foreground and background color components to estimate its alpha value.

Fig. 2(a) is an example of region division in the present invention; Fig. 2(b) describes how to estimate the foreground and background color components of points in unknown regions in the present invention. The points on the red foreground contour line contained in the smaller circle C ₁ are all the points needed to calculate the foreground component, and the points on the green background contour line contained in the larger circle C ₂ That is all the points needed to compute the background component.

The point C in Figure 3(a) is just on the line segment BF; the point C in Figure 3(b) is outside the line segment BF, but very close to the line segment BF; in Figure 3(c), l _C is not between l _B and l _F , but ∠ BC F is an obtuse angle; in Figure 3(d), l _C is between l _B and l _F , but ∠ BC F is an acute angle.

Fig. 4 is a schematic diagram of a color adjustment scheme in the present invention. Here we only discuss the case when l _B <l _F. At this time, if l _C is not between l _B and l _F , there are two situations: l _C <l _B <l _F (as shown in Figure 4(a)) and l _B <l _F <l _C ( As shown in Figure 4(b)). In Figure 4(a), find a point F on the extension line of BF, so that l _F = l _C , so that l _B < l _C ≤ l _F , satisfying observation 3, it can be considered that point C is a regular point at this time , you can apply the formula (1) to calculate the alpha value. The processing of Fig. 4(b) is similar.

This specification has cited 6 implementation examples altogether. In the implementation examples cited in this specification, the values of the parameters are all θ=2.0 and ρ=1.428. Fig. 5 and Fig. 6 are embodiment examples 1 and 2, and Fig. 7 includes embodiment examples 3-6.

Example 1

The enlarged image 1 in Figure 5 is composed of three parts, the left is the original image, the middle is the grayscale image, and the right is the composite image with a black background. The Knockout method did not pick out some hair strands in the enlarged image 1, and the results of some points on the enlarged image 1 were distorted. In the Ruzon & Tomasi method, some hair strands are severely broken, and there are obvious discontinuities in the composite image. The effect of Chuang's method is better, but on closer inspection, it can be found that there is a slight discontinuity at the bottom of the enlarged image 2, and there is also a slight discontinuity in the enlarged image 1. The method according to the invention does not have the above-mentioned disadvantages in this example.

Example 2

In Figure 6, the Chuang method zooms in on the area inside the two ellipses in Figure 1, resulting in errors or impurities. Inside the upper small ellipse, a part of the hair was forcibly cut off, and the cutout result here is wrong, while inside the lower big ellipse, a piece of impurity appeared. The method of the present invention then handles these problems well.

Embodiment 3～6

In Figure 7, (a) is an example of Gandalf, (b) is an example of Galadriel, (c) is an example of Tiger, and (d) is an example of Water.

The main points of the fast matting method with an arbitrary image as the background in digital image processing are:

1. Manually divide the area to ensure that the contour line passes through the appropriate area and has sufficient accuracy;

2. The sample points used in color estimation are points on the contour line inside the circle, and the length of the radius of the two circles is generally 1.5 to 3.0 times the shortest distance;

3. Adjust the estimated foreground and background colors to turn point C into a regular point;

4. Use the adjusted color to estimate the alpha value of point c.

The accuracy of region division has a great influence on the accuracy of natural image matting. The following points should be paid attention to when matting: ① Make sure that all points inside the foreground contour line are foreground points, and all points outside the background contour line are background points, and no intersection is allowed, otherwise the matting accuracy of the intersection area will be greatly affected; ②Due to the influence of light and air, the boundaries of objects in the human eye are always a little blurred. The foreground and background contour lines should not be too close to the boundary of the object, and a certain amount of room should be left to ensure that the transition area does not fall into the foreground or background area. The distance from the boundary to the two contour lines is approximately equal; ③ try to keep the contour lines from passing through areas with sharp color mutations.

The sample points used in the estimation of the foreground and background colors in the present invention are only the points on the contour line, which greatly reduces the number of sample points and improves the calculation speed.

The color adjustment scheme of the present invention is the focus of the whole method. Its core is to convert the point C from an irregular point to a regular point, so that the solution of α can use the formula (1). In the case that the RGB ratio of the color remains unchanged, the brightness of the color is adjusted to the minimum, and the human eye is not sensitive to small-scale changes in the brightness of the color, and the adjusted foreground and background colors replace the original estimated color to find Get a more reasonable alpha value.

Claims (4)

1. in a kind of digital image processing, take arbitrary image as the fast matting method of background, carry out area division to image, it is divided into three parts: foreground area, unknown area and background area, it is characterized in that: its steps are: 1) According to the segmented area, for any point c in the unknown area, set its color as C, and calculate its initial foreground and background color components F and B; the calculation of said initial foreground and background color components F and B Yes: For any point c in the unknown area, find out the points f' and b' closest to point c on the foreground contour line and background contour line, and the distances between point f' and b' from point c are d ₁ and d ₂ , given a positive real number θ, where, 1.0<θ≤10.0, with point c as the center, and θd ₁ and θd ₂ as the radius, make two concentric circles C ₁ and C ₂ , set them at circle C All points inside ₁ and located on the foreground contour line are f ₁ , f ₂ , ..., f _k , and the distances from these points to point c are d ₁₁ , d ₁₂ , ..., d _1k , inside circle C ₂ and All points on the background contour line are b ₁ , b ₂ , ..., b _l , and the distances from these points to point c are d ₂₁ , d ₂₂ , ..., d _2l , and f _i is calculated, where i= 1, 2, ..., k, weighted average of point colors $\stackrel{\&OverBar;}{f}=\underset{i=1}{\overset{k}{\mathrm{\&Sigma;}}}{w}_{1i}{f}_i,$ b _j , where, j=1, 2, ..., l, the weighted average of point colors $\stackrel{\&OverBar;}{B}=\underset{j=1}{\overset{l}{\mathrm{\&Sigma;}}}{w}_{2j}{b}_j,$ in $w_{1i}=\frac{\mathrm{\&theta;}}{\mathrm{\&theta;}-1}-\frac{1}{\mathrm{\&theta;}-1}\&Center\; Dot;\frac{d_{1i}}{d_1},$ In the formula, i=1, 2,..., k, $w_{2j}=\frac{\mathrm{\&theta;}}{\mathrm{\&theta;}-1}-\frac{1}{\mathrm{\&theta;}-1}\&Center\; Dot;\frac{d_{2j}}{d_2},$ In the formula, j=1, 2, ..., 1, F and B are exactly estimated initial foreground and background color components; 2) According to the estimated initial foreground and background color components F and B, take color adjustment measures to adjust to the values of F and B, and finally estimate the alpha value of point c. The estimated alpha value is: the steps are: 1 ) Determine the value of the color difference parameter ρ according to the color brightness difference between the foreground object and the background object in the image, 2) adjust the values of F and B to F and B according to the relationship between the color brightness ratio λ of F and B and the color difference parameter ρ , 3) According to the adjusted F and B, by the formula $\mathrm{\&alpha;}=\frac{(C-B)\&CenterDot;(f-B)}{{\left|\right|f-B\left|\right|}^2}$ Calculate the alpha value α of a point c in the unknown area. 2. in a kind of digital image processing according to claim 1, take arbitrary image as the fast matting method of background, it is characterized in that said region segmentation is: manually draw two contour lines at the foreground edge of image, one is The foreground contour line, the points inside this contour line are the points of the foreground area, the other contour line is the background contour line, and the points outside this contour line are the points of the background area, located in these two contour lines The points within are the points of the unknown area. When manually drawing the contour line, try to make the unknown area not contain the points of the foreground area or the background area. 3. in a kind of digital image processing according to claim 1, take arbitrary image as the background fast matting method, it is characterized in that the value of color difference parameter ρ is: when the brightness of foreground object is bigger than background object, the value of ρ The value range is 1.0<ρ≤2.0; when the brightness of the foreground object is smaller than that of the background object, the value range of ρ is ρ>2.0. 4. in a kind of digital image processing according to claim 1, take arbitrary image as the fast matting method of background, it is characterized in that the adjustment of F and B is: Let F=( F _r , F _g , F _b ), B=( B _r , B _g , B _b ), C=(C _r , C _g , C _b ), where A _r , A _g , A _b are the components of color A on the three channels of RGB, where A= F, B, C, calculate l _F ＝ F _r + F _g + F _b , l _B ＝ B _r + B _g + B _b , l _C ＝ C _r + C _g + C _b , ratio λ = l _F /l _B , at this time it is divided into three situations for discussion: ①If λ>1/ρ and l _C >l _F , or λ≤1/ρ and l _C <l _F , then adjust F to F=( F _r ×l _C /l _F , F _g ×l _C /l _F , F _b ×l _C /l _F ), B is not adjusted, that is, B= B, ②If λ>1/ρ and l _C <l _B , or λ≤ 1/ρ and l _C ＞l _B , adjust B to B=( B _r ×l _C /l _B , B _g ×l _C /l _B , B _b ×l _C /l _B ), F is not adjusted, and That is F=F, ③ in the rest of the cases, F and B are not adjusted, that is F=F, B=B.

Images