CN105550981A - Image registration and splicing method on the basis of Lucas-Kanade algorithm - Google Patents

Abstract

The invention discloses an image registration and splicing method based on the Lucas-Kanade algorithm. The method includes: preprocessing the original image, due to differences in shooting angles, shooting distances, and lighting when shooting, it may cause gaps between two images. The parameters do not match, so the image is preprocessed before image registration; image registration, as a key step in image stitching technology, the quality of image stitching is largely determined by the accuracy of image registration; Image fusion, the purpose of fusion is to transform the registered images to be stitched into a certain coordinate system through the coordinate transformation relationship for pixel-level fusion, so as to display the scene information more comprehensively. Aiming at the problem that the homography matrix estimation accuracy is not high caused by fewer feature points and noise interference, the present invention innovatively proposes to use the LK image alignment algorithm and its modified algorithm to optimize the homography matrix estimation. The iterative optimization of the image minimizes the mean square error of the image, improves the accuracy of the homography matrix estimation, and then improves the effect of image registration and splicing fusion.

Description

An Image Registration and Stitching Method Based on Lucas-Kanade Algorithm

technical field

The invention relates to the technical field of image processing, in particular to an image registration and splicing method based on the Lucas-Kanade algorithm.

Background technique

Image mosaic (ImageMosaic) technology refers to the coordinate registration of multiple image sequences that are captured by the camera on the same scene and have overlapping areas with each other, and then form a wide viewing angle, distortion-free, New images with higher resolution and more image information. Image stitching technology has been applied in the field of remote sensing technology earlier, but it is limited to the shooting angle, and the camera can only capture a local area of a certain scene. However, the scene images in local areas cannot fully display the scene information, so it is necessary to perform spatial registration, fusion and stitching of multiple local scene images taken under different imaging conditions to form a complete remote sensing image with high resolution and wide viewing angle. In recent years, with the rapid development of information technology and electronic technology, fields such as medical images, aerial photography, and daily life needs require high-resolution images. If it is obtained by means of a panoramic camera or a wide-angle camera, the captured image Not only is the resolution limited, but image edges are often distorted, deformed, blurred, etc. In addition, such methods cannot be popularized due to the expensiveness and inoperability of these professional equipment. However, image stitching technology can obtain high-resolution, undistorted wide-view images that satisfy multiple fields and multiple scenes by analyzing and processing images and applying cutting-edge registration and stitching methods. Wavelet analysis is a time-frequency localized analysis method with a fixed window size (that is, window area) but a variable shape, that is, it has higher frequency resolution and lower time resolution in the low frequency part, and has Higher time resolution and lower frequency resolution. However, in practical applications, it is often desired to improve the frequency resolution of the high-frequency band.

The present invention is aimed at the precise registration and panoramic image stitching problems in the current image mosaic technology, analyzes and studies each step in the mosaic technology in detail, and proposes an effective improvement method based on the deficiencies of the current technology. Therefore, image stitching technology has broad application fields and prospects, and it is of great significance to study image registration and stitching methods with high performance, strong real-time performance, and suitable for most scenes.

Contents of the invention

In order to overcome at least one defect (deficiency) of the above-mentioned prior art, the present invention provides an image registration and splicing method based on the Lucas-Kanade algorithm, which can better realize image registration and splicing.

In order to solve the problems of the technologies described above, the technical solution of the present invention is as follows:

An image registration and mosaic method based on the Lucas-Kanade algorithm, comprising:

1) Preprocessing the initial image;

2) The Lucas-Kanade algorithm is used to register the preprocessed image, and the specific method is:

(1) Extract Harris corner points from the two images 1I and 2I to be registered respectively;

(2) Apply the NCC matching algorithm to initially match the corner points in the two images;

(3) Apply the RANSAC algorithm to remove outliers and obtain the optimal homography matrix H;

(4) convert the two images to be registered into grayscale images respectively;

(5) Take the H obtained in step (3) as the initial value, and set a convergence precision ε and the maximum number of iterations, apply the reverse composition algorithm to align the grayscale images of 1I and 2I until convergence, and obtain the refined homography matrix H;

3) The weighted average method is used for image fusion on the registered images.

Preferably, the preprocessing includes operations of smoothing and filtering, histogram matching, and image enhancement transformation on the image.

Preferably, the process of image fusion using the weighted average method for the images after registration is:

Szeliski weighted and averaged the corresponding pixels of each overlapping frame by using a "hat function"; let f ₁ and f ₂ be two images that have been registered and ready to be stitched, and f represents the fused and stitched image, then :

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&cap;</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right.$

Wherein, α ₁ and α ₂ are the weight values of pixels corresponding to overlapping positions in the two images to be fused respectively, and both satisfy α ₁ +α ₂ =1, 0<α ₁ <1, 0<α ₂ <1. _In the overlapping area, as the pixel position changes, the weights α1 and _α2 will also change accordingly.

The weighted average method of overlapping areas is used to achieve smooth transitions and eliminate stitching seams to obtain seamless, high-resolution mosaics.

Compared with the prior art, the beneficial effect of the technical solution of the present invention is: the image registration and splicing method of the present invention can effectively eliminate the influence of special scenes on the splicing effect, has strong applicability, and can splice images without distortion, without faults, High-resolution wide-view images outperform existing image registration and stitching techniques.

Description of drawings

Figure 1 is an image sequence diagram of Sun Yat-sen University archway;

Figure 2 is a seamless splicing diagram of Sun Yat-sen University archway.

Figure 3 is a flow chart of image stitching.

Figure 4 is a graph of experimental parameters.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The invention provides an image processing method, which can better realize image registration and splicing.

The present invention adopts the Lucas-Kanade algorithm and the Lucas-Kanade image alignment algorithm, including the original algorithm (forward additive algorithm) and improved algorithm (forward composition algorithm and reverse composition algorithm) of the algorithm.

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention.

Fig. 3 is the concrete realization flow chart of hybrid image denoising method of the present invention; A kind of image registration and mosaic method based on Lucas-Kanade algorithm comprises:

1) Preprocessing the initial image;

2) The Lucas-Kanade algorithm is used to register the preprocessed image, and the specific method is:

(1) Extract Harris corner points from the two images 1I and 2I to be registered respectively;

(2) Apply the NCC matching algorithm to initially match the corner points in the two images;

(3) Apply the RANSAC algorithm to remove outliers and obtain the optimal homography matrix H;

(4) convert the two images to be registered into grayscale images respectively;

(5) Take the H obtained in step (3) as the initial value, and set a convergence precision ε and the maximum number of iterations, apply the reverse composition algorithm to align the grayscale images of 1I and 2I until convergence, and obtain the refined homography matrix H;

3) The weighted average method is used for image fusion on the registered images.

For verifying the superiority of the present invention, carry out following experiment:

The experiment adopts VisualStudio2010 development environment, OpenCV library and c language programming to realize the simulation code, and the experiment is carried out on the computer with IntelCore3.4GHz processor and 4GB memory. The settings of steps such as Harris corner detection, NCC matching, and RANSAC algorithm in the experiment are shown in Figure 4. The maximum number of iterations of the global alignment algorithm is 100, and the convergence value is 0.001. The image sequence Figure 1 of Sun Yat-sen University archway is spliced, and the splicing result is shown in Figure 2.

It can be seen that the method based on the Lucas-Kanade reverse composition algorithm proposed by the present invention can effectively align two images and obtain a seamless, high-resolution mosaic result image, which has advantages.

Apparently, the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, rather than limiting the implementation of the present invention. For those of ordinary skill in the art, other changes or changes in different forms can be made on the basis of the above description. It is not necessary and impossible to exhaustively list all the implementation manners here. All modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (3)

1., based on image registration and the joining method of Lucas-Kanade algorithm, it is characterized in that, comprising:

1) to initial pictures pre-service;

2) carry out registration to the image Lucas-Kanade algorithm after pre-service, concrete mode is:

(1) Harris angle point is extracted respectively to two width image 1I and 2I subject to registration;

(2) apply NCC matching algorithm and initial matching is carried out to the angle point in two width images;

(3) apply RANSAC algorithm and remove exterior point, and obtain optimum homography matrix H;

(4) respectively two images subject to registration are converted to gray level image;

(5) H obtained with (3) step for initial value, and sets the convergence precision ε of and maximum iterations, applies the gray level image of reverse composition algorithm alignment 1I and 2I until convergence, the homography matrix H after obtaining refinement;

3) image co-registration is carried out to the imagery exploitation method of weighted mean after registration.

2. the image registration based on Lucas-Kanade algorithm according to claim 1 and joining method, is characterized in that, pre-service comprises the operation to the smoothing filtering of image, Histogram Matching, image enhaucament conversion.

3. the image registration based on Lucas-Kanade algorithm according to claim 1 and joining method, is characterized in that, the process of the imagery exploitation method of weighted mean after registration being carried out to image co-registration is:

Szeliski, by use one " hat shape function ", is weighted on average to the respective pixel of each overlapping frame; If f ₁and f ₂be two width registration prepare well splice image, f represents the image after anastomosing and splicing, then have:

Wherein, α ₁and α ₂be the weighted value that two width need lap position respective pixel in the image merged respectively, both meet α ₁+ α ₂=1,0< α ₁<1,0< α ₂<1, in overlapping region, along with the change of location of pixels, weights α ₁and α ₂also corresponding change is had.