CN106023089B - A kind of image repair method based on Block- matching - Google Patents

Abstract

The present invention relates to a kind of image repair methods based on Block- matching, include the following steps:1）Calculate the priority of area to be repaired marginal point, and the gradient amount of introduction point and invariant improve priority.2）To be filled piece is established according to the highest point of priority；3）It is according to searching for Best exemplar patch in image known region with Given information in be filled piece.4）The pixel value of Best exemplar patch is extracted, and calculates the confidence value of Best exemplar patch central pixel point；5）The corresponding pixel value of Best exemplar patch is copied to to be filled piece of corresponding position, and the confidence level of point is updated to the confidence value of Best exemplar patch central pixel point, forms new area to be repaired；6）Execute above-mentioned steps 1）~5）, until area to be repaired is stuffed entirely with finishing.The above invention achieves good repairing effect, is suitable for various image repairs and works.

Description

A Method of Image Restoration Based on Block Matching

technical field

The present invention relates to an image restoration method, in particular to an image restoration method based on block matching.

Background technique

With the development and progress of computer technology, computer-based digital image restoration technology has attracted more and more attention. At the same time, digital image restoration technology has been widely used in repairing lost information in pictures, removing text and redundant objects in pictures or videos, etc. In the image restoration technology, the picture is divided into known area and unknown area. The unknown area can be the part lost after the picture is damaged, or it can be the part we need to remove. The image restoration is realized by transferring the pixels of the known area to the unknown area through a certain method, and the requirement for the restoration result is to meet the requirements of human vision. At present, image repair methods are mainly divided into two categories: one uses pixels as the operation object, and repairs unknown areas through diffusion based on physical and mathematical knowledge. This method is very suitable for small-scale image restoration, but large-scale image restoration The result is unsatisfactory; another type takes pixel blocks as the operation object, and transfers the pixel information of the known area to the unknown area to achieve repair by performing global or local matching on the pixel block. can achieve good results, but the restoration success rate based on this method is very low.

In the prior art, the image repair method based on block matching calculates and sorts the priority of all points in the area to be filled, selects the point with the highest priority, uses this point to establish the block to be filled, and then uses the known pixels in the filled block The points and the pixels in the sample block in the known area are matched through the SSD algorithm. After the matching is successful, the sample block is copied to the block to be filled to realize the transmission of image information and complete image restoration. The algorithm also has defects such as high matching error rate and the algorithm is easy to lose its function. These defects make the algorithm not always obtain satisfactory repair results.

Contents of the invention

In view of the above problems, the purpose of the present invention is to provide a block matching image restoration method that improves the matching success rate and ensures that the algorithm works normally at any time.

In order to achieve the above object, the present invention takes the following technical solutions: a method for image restoration based on block matching, comprising the following steps:

1) Calculate any pixel on the boundary of the area to be repaired in the image to be repaired priority, including the following steps: ① Divide the image to be repaired into known regions and the area to be repaired Ω two parts, is the boundary of the area to be repaired; ② Calculate the boundary of the area to be repaired any pixel Confidence value of :

,

In the formula, is a fixed constant, for the absolute value of the point gradient magnitude, is the block to be repaired, for and Any pixel in the intersecting area; ③ Calculate the pixel data item ;④Calculate pixels priority ;

2) Calculate the pixel points according to the step 1) The priority to determine the block to be filled first to be repaired;

3) Search all sample blocks in the known area, and find the best sample block to be filled according to the set matching conditions;

4) Extract the pixel value of the best sample block, and calculate the confidence value of the center pixel of the best sample block;

5) Copy the pixel value corresponding to the best sample block to the corresponding position of the block to be filled, and set The confidence of the point is updated to the confidence value of the center pixel of the best sample block to form a new area to be repaired;

6) Repeat steps 1) to 5) above until the area to be repaired is completely filled.

In the step 1) ② It is a fixed constant between 0 and 1. Experiments show that a value of 0.35 can satisfy most repairs. The existence of the fixed constant expands the range of the confidence item value to negative numbers, and the algorithm formula will not stop working because the confidence item approaches 0.

In the step 1) ② for The absolute value of the gradient of a point, which represents the characteristics of the block to be filled centered on this point, The larger the feature, the more obvious the feature, that is, the block to be filled with the more obvious feature has a higher priority and is filled first.

The steps 1) ③ Calculate the pixel points data item : , where, represent pixels Isoluminance line vector at , Represent a point The unit normal vector at , Indicates the normalization factor.

The step 3) searches all sample blocks in the known area, and finds the best sample block of the block to be filled according to the set matching conditions, including the following content: calculate the relationship between the block to be filled and each sample block in the known area The average SSD distance, the sample block with the smallest calculated average SSD distance is used as the best sample block, and the information of the sample block is filled into the block to be filled.

The block to be filled with sample block The formula for calculating the mean SSD distance of is:

In the formula, , , Represent the mean value of the brightness of different color channels in each pixel in the block to be filled and the sample block, and the sample block described in this formula is a certain pixel in a known area A rectangular block with a center point.

The present invention has the following advantages due to the adoption of the above technical scheme:

1. Since the present invention introduces a fixed constant item when calculating the confidence of a pixel point, the confidence value breaks through the limit of 0 by periodically subtracting the fixed constant, and can become a negative value, and a negative value is also a scalar value that can be used for all edge points Sort the confidence of each block, so that the calculated priority value can represent the correct block filling order, and prevent the occurrence of the "dropping effect". Compared with the existing technology, it can not only correctly generate the information required for the damaged area, but also And effectively avoid the "dropping effect" effect, so that the calculated pixel priority is more in line with the actual repair sequence, ensuring the accuracy of image repair;

2. The present invention introduces the absolute value item of the gradient value of the point, which represents the absolute value of the gradient of all edge points. The point with a large absolute value of the gradient value is located in the area where the pixel changes significantly, that is, the block to be filled centered on this point has obvious identifiable features, and the block with this feature can be used as the block to be filled to make the image matching process happen. The probability of error is lower, so that the restoration work of the image has a higher success rate. The invention can be widely used in the process of image restoration.

Description of drawings

Fig. 1 is a schematic diagram of the principle of image restoration in the present invention.

Fig. 2 is a schematic diagram of priority calculation in the present invention.

Fig. 3 is a schematic flowchart of the image restoration method of the present invention.

Fig. 4 is a schematic diagram of the search mechanism of the optimal block to be filled in the present invention.

Fig. 5 is a schematic diagram of the repairing effect of the present invention, Fig. 5(a) is a schematic diagram of the graph before repairing, and Fig. 5(b) is a schematic diagram of the effect after repairing.

Fig. 6(a) is a schematic diagram of a confidence curve in the prior art, the abscissa is the number of iterations, and the ordinate is the confidence.

Fig. 6(b) is a schematic diagram of the confidence curve calculated by the method of the present invention, the abscissa is the number of iterations, and the ordinate is the confidence.

Fig. 7(a) is a schematic diagram of block filling priority in the prior art, the abscissa is the number of iterations, and the ordinate is the block filling priority.

Fig. 7(b) is a schematic diagram of the filling priority calculated by the method of the present invention, the abscissa is the number of iterations, and the ordinate is the block filling priority.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

Object removal is a typical application in the technical field of image restoration, and its task is to restore the lost image information in the specified area of the image as naturally as possible. The present invention uses object removal as an example to illustrate the specific process of the image repair method. Object removal is aimed at The information defect area in the image is filled with the information of the known area in the image, so that the restored image is natural and real, and meets the visual psychological requirements of people.

As shown in Figure 1, where I is the entire image, Represents the known area, and Ω represents the unknown area. Copy the information in the known area to the unknown area to make the whole image real and natural. This is the basic principle of image restoration.

as shown in picture 2, represents the boundary of the area to be filled, for The block to be filled with the point as the center, is the edge point The orthogonal vector of the gradient vector at , is the point with the edge Intersecting unit vectors, represent the known area of represent In the unknown area, the priority is calculated according to the above parameters.

The schematic diagram of the repair process is shown in Figure 3, comprising the following steps:

1. Calculate the boundary of the area to be repaired in the image to be repaired all pixels on priority, including the following steps:

1) Divide the image to be repaired into known regions and the area to be repaired two parts, Indicates the area to be repaired the boundaries of

2) Calculate the boundary of the area to be repaired any pixel Confidence of :

(1)

(2)

In the above formula (1), It is a fixed constant 0.35, which is used to expand the value range of the confidence item to a negative number to avoid repair errors caused by the confidence item tending to 0;

In the above formula (1), for the point The absolute value of the gradient value at Whether the features of the feature are obvious, the block to be filled with obvious features is filled first;

In the above formula (2), for Confidence of a point in the area around the point, when point at When in the area The default is 1, when point at When in the area Default is 0, for block the number of interior points;

In the above formula (2), for The original confidence of the point, because will gradually become smaller and approach 0, resulting in item becomes 0, resulting in a fix error;

3) Calculate the pixel points data item :

(3)

In formula (3), used to measure pixels The edge strength at The direction and intensity of the isolux line at the point, represent pixels Isoluminance line vector at , Show point The unit normal vector at , Indicates the normalization factor;

4) According to the pixel point Confidence of and data item The product of computing pixels priority :

(4).

2. Calculate the boundary of the area to be repaired according to the above step 1 upper pixel The priority to determine the block to be filled first to be repaired.

The invention makes the blocks to be filled with obvious identification features have a higher priority order, and can take priority over the blocks to be filled without features when being filled. The obvious recognition feature mentioned here refers to the obvious structure or obvious edge point feature, by introducing the point gradient To illustrate the apparent degree of structure, the gradient represent The change degree of the pixel value of the points around the point, and the greater the change degree of the pixel value, the The color change around the point is more obvious, The larger the value, the greater the change in pixel values around the point, and the more obvious the structural features can be identified. As shown in Figure 4, the figure is the block to be filled with obvious identification features, for value to select the block to fill in. The black part in the figure is the structural part of the image, and there are obvious pixel value changes at the edge of the structure, that is, the gradient here Larger and more recognizable features.

3. Search all sample blocks in the known area, and find the best sample block for the block to be filled according to the set matching conditions, including: calculating the block to be filled and all sample blocks in the known area ( … ) of the mean SSD distance (Sumof Squared Differences, difference square sum), this mean SSD distance is used as the measurement block to be filled The similarity with each sample block, select the sample block with the smallest distance from the mean SSD of the block to be filled as the block to be filled The optimal matching is fast.

Among them, the average SSD distance is the mean value of the brightness of each color channel to reflect the average similarity between the block to be filled and the sample block, and the block to be filled with any sample block ( … ) The mean SSD distance is:

(5)

In the formula, , , Represent the mean value of the brightness of different color channels in each pixel in the block to be filled and the sample block, and the sample block described in this formula is a certain pixel in a known area is the center point, a rectangular block with a size of 9 pixels and 9 pixels, the block to be filled can be calculated according to the formula (5) The mean SSD distance from any other sample block.

4. Extract the pixel value of the best sample block, and calculate the confidence value of the center pixel of the best sample block.

5. Copy the pixel value corresponding to the best sample block to the block to be filled corresponding position of the The confidence at the point is updated to the confidence value of the center pixel of the best sample block, and a new area to be repaired is formed at this time.

6. Perform the above steps 1 to 5 until the area to be repaired is completely filled (as shown in Figure 5 (a) and Figure (b) are the effect diagrams of the repaired picture).

As shown in Figures 6-7, in summary, after the processing of the present invention, it can be seen that the "dropping effect" effect has been resolved through the processing of the present invention, because the value of the confidence item of the edge point will no longer show a trend In the case of a certain value, although there will be negative numbers, negative numbers are also scalars and do not affect the ordering of the confidence of all edge points, so this effect has been well resolved. And by increasing the priority of the sample block with obvious characteristics, the efficiency and success rate of finding the best sample block are higher, so that the algorithm has a better repair result.

The above-mentioned embodiments are only used to illustrate the present invention, wherein each step of the method etc. can be changed to some extent, and all equivalent transformations and improvements carried out on the basis of the technical solution of the present invention should not be excluded from the protection scope of the present invention outside.

Claims (7)

1. a kind of image repair method based on Block- matching, includes the following steps：

1) priority for calculating any pixel p on the area to be repaired boundary of complex pattern to be repaired, includes the following steps：

1. complex pattern to be repaired is divided into known region Φ and area to be repaired Ω two parts, δ Ω are the boundary of area to be repaired Ω；

2. calculating the confidence level R of any pixel P on the δ Ω of area to be repaired boundary_c(p)：R_c(P)=(C (P)-λ)+| I_P|,In formula, λ is fixed constant, Ψ_pFor multiblock to be repaired, q Ψ_pWith any picture of Φ intersecting areas Vegetarian refreshments, when q points are located at Ψ_PC (q) is defaulted as 1 when in the regions ∩ Φ, when q points are located at Ψ_PC (q) is defaulted as 0 when outside the regions ∩ φ, | I_P| for the absolute value of the gradient at point P；

3. calculating the data item D (p) of pixel p；

4. calculating priority P (p)=R of pixel p_c(p)D(p)；

2) priority of the pixel p calculated according to the step 1) determines to be filled piece repaired at first；

3) all sample blocks in known region are searched for, according to the matching condition of setting, find to be filled piece of Best exemplar patch；

4) pixel value of Best exemplar patch is extracted, and calculates the confidence level of Best exemplar patch central pixel point；

5) the corresponding pixel value of Best exemplar patch is copied to to be filled piece of corresponding position, and the confidence level of p points is updated to The confidence level of Best exemplar patch central pixel point forms new area to be repaired；

6) above-mentioned steps 1 are executed)~5), until area to be repaired is stuffed entirely with finishing.

2. a kind of image repair method based on Block- matching as described in claim 1, it is characterised in that：λ in the step 1) For fixed constant 0.35, the value range for expanding confidence level is avoided repairing caused by confidence level tends to 0 and be lost to negative It loses.

3. a kind of image repair method based on Block- matching as described in claim 1, it is characterised in that：In the step 1) | I_P | it is Ψ_PGradient amount absolute value, for indicating Ψ_PFeature it is whether apparent, with to be filled piece of obvious characteristic preferential quilt Filling.

4. a kind of image repair method based on Block- matching as described in claim 1, it is characterised in that：The step 1) is 2. counted Calculate the confidence level R of pixel p_c(P)：R_c(p)=(C (p)-λ)+| I_P|, it is used for the confidence level of to be filled piece of tabular form.

5. a kind of image repair method based on Block- matching as claimed in claim 2, it is characterised in that：The step 1) is 3. counted Calculate the data item D (p) of pixel p：In formula,Indicate the isophote vector at pixel p, n_pIndicate that the unit normal vector at point p, α indicate normalization factor.

6. a kind of image repair method based on Block- matching as claimed in claim 1 or 2 or 3 or 4, it is characterised in that：It is described Step 3) searches for all sample blocks in known region, according to SSD distance algorithms, finds to be filled piece of Best exemplar patch and fills out It fills.

7. a kind of image repair method based on Block- matching as claimed in claim 5, it is characterised in that：The to be filled piece of Ψ_p Calculation formula with the mean value SSD distances of any sample block is：

In formula,The equal of different color channels brightness in each pixel is indicated in be filled piece and sample block respectively It is worth, the described sample block in this formula is in known region with a certain pixel q_iCentered on the rectangular block put, ψ_pIt indicates It is to be matched fast centered on marginal point p, Ψ_qiIt indicates with all the points q of position and known region_iCentered on sample block.