CN104766319B - Lifting night takes pictures the method for image registration accuracy - Google Patents

Abstract

A method for improving the registration accuracy of photographed images at night, specifically comprising: inputting images to be registered; image grayscale; histogram equalization; detecting feature points; matching feature points; Matching pair; determine whether the feature points of the image taken under flashlight conditions are too concentrated, if so, perform feature point matching pair equalization, otherwise, directly use the detected feature point matching pair to solve the affine transformation matrix; feature point matching pair equalization use feature point matching to list equations, solve equations, and obtain affine transformation matrix; register images. The present invention can adaptively add matching pairs according to the position information of the detected feature point matching pairs, so that the distribution of feature points is more uniform, avoiding the over-fitting problem caused by too concentrated feature points, and improving the accuracy of nighttime photographed images. Registration accuracy.

Description

A method to improve the registration accuracy of images taken at night

technical field

The invention belongs to the technical field of image processing, and further relates to a method for improving the registration accuracy of images taken at night in the technical field of image registration. The present invention is used in registration preprocessing for fusion of a single-frame flashlight image and multi-frame non-flashlight images to improve the quality of photographing at night, can effectively avoid ghosting and blurring, and greatly improve the quality of image fusion.

Background technique

At present, there are mainly frequency-domain-based methods and time-domain-based methods for image registration at night. The most representative method based on the time domain is the feature-based method. This method extracts feature points from the image to be registered according to the feature point extraction algorithm, and uses the feature points to perform image registration. The advantages of the feature-based method are scale invariance, strong robustness, and insensitivity to weak uniform illumination changes. However, this method still has shortcomings. When taking pictures at night, the foreground will be very bright and the background will be very dark when the flash is on, and the overall image will be dark when there is no flash. There are very few detected feature points, which are not enough to calculate the affine transformation matrix, or they are concentrated in a relatively small area, which will cause over-fitting when calculating the affine transformation matrix.

A feature-based Point image registration method. This method first extracts the feature points of the image to be registered according to the BP-SIFT (Belief Propagation Scale InvariantFeature Transform) algorithm, then matches the feature points to obtain the matching pair, obtains the transformation matrix according to the matching pair, and finally performs re-registration according to the transformation matrix. Sampling to get the final registered image. This method has a good registration effect on images to be registered with little change in illumination, but there is still a disadvantage that for images to be registered with large changes in illumination intensity, there will be few feature points extracted, or the extracted The feature points are too concentrated, resulting in over-fitting and reducing the registration accuracy.

An image registration method is disclosed in the patent "Image registration method based on improved Harris corner points" (application date: September 4, 2009, application number: 200910195131.9, publication number: 101655982) filed by Shanghai Jiaotong University. This method calculates the scale space of the image to be registered and obtains the Harris corner points in the scale space, uses the affine shape modification technology to iteratively process the Harris corner points in the scale space, and uses the descriptor and matching method to match the feature points. Matching achieves registration. The disadvantage of this method is that when this method is applied to the image registration of night shooting with and without flash, it does not consider the uneven distribution of illumination, resulting in uneven distribution of detected feature points , which causes overfitting, resulting in the image being registered only in a certain area.

Contents of the invention

The object of the present invention is to propose a method for improving the registration accuracy of images with and without a flash to address the above-mentioned deficiencies in the prior art.

According to the problem that the foreground is too bright and the background is too dark when taking pictures at night when the flashlight is turned on, and the overall darkness of the picture is taken under the condition of no flashlight, the present invention first performs histogram equalization on the two images to be registered to improve The gray distribution is concentrated, the contrast of the image is enhanced, and then the scale-invariant feature transformation SIFT feature point detection and matching is performed, and the random sampling consistent Ransac algorithm is used to remove the mismatching points. In order to avoid the over-fitting problem caused by the over-concentration of feature points, Using the obtained matching pairs, adaptively add matching pairs to make the distribution of feature points more uniform, and finally use these matching pairs to obtain an affine transformation matrix and register images.

To achieve the above object, the present invention comprises the following main steps:

(1) Input the image to be registered:

Input one image taken with flash and one without flash to be registered respectively;

(2) Image grayscale:

According to the following formula, the images taken under the conditions of the flash light and without the flash light to be registered are respectively grayscaled:

Among them, Y _i represents the gray value of the i-th pixel in the image taken under the condition of the flash light and without the flash light to be registered, and i represents the pixel point of the image shot under the condition of the flash light and without the flash light to be registered B, G, and R represent the blue, green, and red channels of the images taken under the condition of the flash light and without the flash light to be registered respectively, and B _i represent the channels of the images shot under the condition of the flash light to be registered and without the flash light The blue channel of the i-th pixel of the image, G _i represents the green channel of the i-th pixel of the image to be registered under the flash condition and no flash condition, and R _i represents the flash condition and no flash condition to be registered The red channel of the i-th pixel of the image captured under the condition;

(3) Histogram equalization:

According to the following formula, the histogram equalization is performed on the images taken under the condition of the flash light and without the flash light to be registered respectively:

s _x = int[(L-1)*p _x +0.5];

Among them, p _x represents the cumulative sum of the probability values of the final gray level of the brightness channel matrix, x represents the gray value of the brightness channel matrix, and the value range of x is 0 to 255, ∑ represents the summation operation, and f represents the brightness The gray level of the channel matrix, f=0,1,2,...,x, g(f) represents the probability value of the final gray level of the brightness channel matrix, and s _x represents the brightness channel matrix after histogram equalization The mapping value of the middle gray value x, int means the rounding operation, L means the maximum value of the gray level of the brightness channel matrix;

(4) Detection of feature points:

(4a) For the images taken under the conditions of the flash light to be registered and without the flash light, the images obtained by filtering with Gaussian filters of different scales form a sub-octave; by analogy, the images taken under the flash light conditions to be registered and The images taken under the condition of no flash are down-sampled once, twice, and three times respectively, and similar filtering operations are performed to obtain a Gaussian pyramid layer, and the adjacent layers are subtracted to obtain a differential Gaussian pyramid;

(4b) In the differential Gaussian pyramid, compare the pixel point on the middle layer with the size of 8 adjacent pixel points of the same scale layer, and the size of the pixel point and the size of 18 adjacent pixel points of the adjacent scale layer above and below , if the value of the pixel point on the intermediate layer is the maximum value or the minimum value, the pixel point is used as a candidate feature point;

(4c) remove noise-sensitive low-contrast candidate feature points and candidate feature points with unstable edge responses, and the rest are final feature points;

(4d) Calculate the gradient direction of the neighborhood pixel with the final feature point as the center, and express it with a histogram. The peak value of the histogram represents the main direction of the gradient of the neighborhood pixel of the final feature point, and take the main direction of the neighborhood pixel gradient as the The direction of the final feature point;

(4e) Take the final feature point as the center, select a 16×16 neighborhood, and divide the neighborhood into 16 4×4 sub-regions, and calculate 0°, 45°, 135°, 180° on each sub-region °, 225°, 270°, 315°, 360° total gradient cumulative value in 8 directions to generate a 128-dimensional feature vector;

(5) Match feature points:

For each final feature point in the image taken under the flash condition, use the Euclidean distance to find the two feature points closest to the final feature point of the image taken under the flash condition in the image taken without the flash condition. Among the feature points, if the ratio of the shortest distance to the next shortest distance is less than 0.4, the final feature point of the image taken under the flash condition matches the nearest point in the image taken under the flash condition, otherwise it does not match;

(6) Use random sampling consistent RANSAC algorithm to eliminate wrong feature point matching pairs;

(7) Judging whether the feature points of the image taken under the condition of the flashlight meet the judgment condition, if so, perform step (8), otherwise, perform step (9);

(8) Feature point matching pair equalization:

(8a) According to the following formula, calculate the average offset of the feature point matching pairs in the column direction and the row direction of the image taken under the condition of the flashlight and without the flashlight to be registered:

Among them, _Δx represents the average offset of the matching pairs of feature points in the column direction of the image to be registered under the flash condition and without the flash condition, and x represents the image taken under the flash condition and without the flash condition to be registered The column direction of the feature points of the image, n represents the total number of matching pairs of feature points of the image to be registered under the condition of the flash and without the flash, i represents the number of matching pairs of the feature points of the image to be registered under the condition of the flash and without the flash The serial number of the feature point matching pair of the image, Indicates that the feature points of the i-th image to be registered under the condition of the flashlight and the image taken without the flashlight match the column coordinates of the feature points of the image taken under the condition of the flashlight, Indicates the feature point column coordinates of the _i -th image to be registered under the condition of the flashlight and without the flashlight in the matching pair of feature points of the image taken under the condition of the flashlight without the flashlight, Δy represents the condition of the flashlight to be registered and without the flashlight The average offset of the feature point matching pairs in the row direction of the image taken under the condition of Represents the row coordinates of the feature points of the i-th image to be registered under the condition of the flashlight and the image taken under the condition of no flashlight matching the feature points of the image taken under the condition of the flashlight, Represent the row coordinates of the feature points of the i-th image to be registered under the condition of the flashlight and the feature point of the image taken under the condition of the flashlight without the flashlight;

(8b) According to the following formula, the image taken under the condition of the flashlight to be registered is divided into M×M sub-blocks of the same size:

Among them, HW represents the width of the sub-block, Represents the rounding down operation, W represents the width of the image captured under the flash condition, M represents the number of sub-blocks in each row of the image captured under the flash light, HH represents the height of the sub-block, and H represents the height of the image captured under the flash condition ;

(8c) According to the following formula, calculate the row coordinates and column coordinates of the feature points to be added to the image taken under the flashlight:

in, Represents the column coordinates of the feature points to be added, x represents the column direction of the feature points to be added, k represents the sequence number of the feature points to be added in the image taken under flash conditions, k=(i×M+j)×N×N +i1×N+j1, i represents the number of the row corresponding to the sub-block of the same size, i=0,1,2,...,M-1, i1 represents the row corresponding to the feature point to be added in the sub-block Number, i1=0,1,2,...,N-1, j represents the number of the corresponding column of the sub-block of the same size, j=0,1,2,...,M-1, j1 represents The number of the corresponding column of the feature point to be added in the sub-block, j1=0,1,2,...,N-1, HW indicates the width of the sub-block, D _x indicates the column between the feature points to be added direction distance, W represents the width of the image captured under the flashlight condition, M represents the number of subblocks in each row of the image captured under the flashlight, and N represents the number of feature points added to each row of each subblock of the image captured under the flashlight, Represents the row coordinates of the feature points to be added, y represents the row direction of the feature points to be added, HH represents the height of the sub-block, D _y represents the distance in the row direction between the feature points to be added, and H represents the shooting under flash conditions the height of the image, H represents the height of the image taken under the flashlight condition, M represents the subblock number of each row of the image taken under the flashlight, and N represents the number of feature points added to each row of each subblock of the image taken under the flashlight;

(8d) According to the following formula, calculate the column coordinates and row coordinates of the feature points to be added to the image taken under the condition of no flashlight:

in, Represents the column coordinates of the feature points to be added to the image taken under the condition of no flash, x represents the column direction of the feature points to be added, k represents the serial number of the feature points to be added, Indicates the column coordinates of the feature points to be added to the images captured under flash conditions, _Δx represents the average offset in the column direction of the matching pairs of feature points of images captured under flash conditions and without flash conditions, Indicates the row coordinates of the feature points to be added to the image taken under the condition of no flashlight, Represents the row coordinates of the feature points to be added to the image taken under the flashlight condition, and _Δy represents the average offset in the row direction of the feature point matching pair of the image taken under the flashlight condition and without the flashlight condition to be registered;

(9) utilize step (6) and the feature point matching that step (8) obtains to list equation, solve equation group, obtain affine transformation matrix H;

(10) Registration image:

(10a) According to the following formula, calculate the pixel of the position (i, j) of the image captured under the flashlight condition after registration, and the position of the image captured under the corresponding flashlight condition after mapping:

Among them, i' represents the column coordinates of the image pixels taken under the condition of flashlight, H ^-1 _1,1 represents the first row and first column element of the inverse matrix of the affine transformation matrix, H ^-1 _1,2 represents the affine transformation matrix The element in the first row and the second column of the inverse matrix, H ^-1 _1,3 means the element in the first row and the third column of the inverse matrix of the affine transformation matrix, H ^-1 _2,1 means the inverse matrix of the affine transformation matrix The element in the first column of the second row, H ^-1 _2,2 means the element in the second row and the second column of the inverse matrix of the affine transformation matrix, H ^-1 _2,3 means the second row and the second column of the inverse matrix of the affine transformation matrix Three columns of elements, H ^-1 _3,1 represents the third row and first column element of the inverse matrix of the affine transformation matrix, H ^-1 _3,2 represents the third row and second column element of the inverse matrix of the affine transformation matrix, H ^-1 _3,3 represents the third row and third column element of the inverse matrix of the affine transformation matrix, i represents the column coordinates of the image pixels captured under the flashlight condition after registration, and j represents the shooting under the flashlight condition after registration The row coordinates of the image pixels of j' represent the row coordinates of the image pixels captured under the flashlight condition;

(10b) According to the following formula, calculate the pixel value of the pixel at position (i, j) of the image captured under the condition of the flashlight after registration:

R _i,j =α ₁ ×F _Ii,Ij +α ₂ ×F _Ii,Ij+1 +α ₃ ×F _Ii+1,Ij +α ₄ ×F _Ii+1,Ij+1 ;

where R _i,j represents the pixel value of the image captured under the registered flash condition, i represents the column coordinate of the image pixel captured under the registered flash condition, and j represents the image captured under the registered flash condition The row coordinates of the pixel, α ₁ represents the weight of the pixel in the upper left corner closest to the image pixel captured under the flash condition, F _{Ii, Ij} represents the pixel value of the pixel in the upper left corner closest to the image pixel captured under the flash condition, and Ii represents The integer part of the column coordinates of the image pixel captured under the flash light condition, Ij represents the integer part of the row coordinates of the image pixel captured under the flash light condition, _α2 represents the weight of the pixel in the lower left corner closest to the image pixel captured under the flash light condition, F _Ii,Ij+1 represents the pixel value of the pixel in the lower left corner closest to the image pixel captured under the flash condition, α ₃ represents the weight of the pixel in the upper right corner closest to the image pixel captured under the flash condition, F _{Ii+1, Ij} represents the pixel value of the pixel in the upper right corner closest to the image pixel captured under the flash condition, α ₄ represents the weight of the pixel in the lower right corner closest to the image pixel captured under the flash condition, F _{Ii+1, Ij+1} represents the distance The pixel value of the nearest bottom right pixel to the image pixel taken under flash conditions.

Compared with the prior art, the present invention has the following advantages:

First, because the present invention adopts the histogram equalization preprocessing method before the registration of the image to be registered that is shot at night, it overcomes the problem of the prior art that there are too few feature points extracted for the image to be registered that varies greatly in light intensity Insufficiency of the method makes the present invention have the advantages of being able to detect more feature points and improve registration accuracy.

Second, because the present invention uses the standard deviation information of the column coordinates and row coordinates of the feature point matching pairs to set the judging conditions for whether the feature point matching pairs are concentrated, it overcomes the inability to effectively deal with the excessive concentration of feature point matching pairs in the prior art Due to the shortcomings of the situation, the present invention has the advantage of being able to judge whether the matching pairs are too concentrated, and then perform feature point equalization processing to improve registration accuracy.

Third, because the present invention can equalize the matching pair of feature points according to the position information of the detected matching pair of feature points, it overcomes the feature extraction of images to be registered that vary greatly in light intensity in the prior art. The problem of too concentrated points makes the present invention have the advantages of avoiding the problem of over-fitting when solving the affine transformation matrix and improving the registration accuracy.

Description of drawings

Fig. 1 is a flowchart of the present invention;

Fig. 2 is a simulation diagram of the present invention.

detailed description

The present invention will be described in further detail below in conjunction with the accompanying drawings.

With reference to accompanying drawing 1, the step that the present invention realizes is described in further detail:

Step 1, input the image to be registered.

Input one image taken with flash and one without flash to be registered.

Step 2, image grayscale.

According to the following formula, the images taken under the conditions of the flash light and without the flash light to be registered are respectively grayscaled:

Among them, Y _i represents the gray value of the i-th pixel in the image taken under the condition of the flash light and without the flash light to be registered, and i represents the pixel point of the image shot under the condition of the flash light and without the flash light to be registered B, G, and R represent the blue, green, and red channels of the images taken under the condition of the flash light and without the flash light to be registered respectively, and B _i represent the channels of the images shot under the condition of the flash light to be registered and without the flash light The blue channel of the i-th pixel of the image, G _i represents the green channel of the i-th pixel of the image to be registered under the flash condition and no flash condition, and R _i represents the flash condition and no flash condition to be registered The red channel of the i-th pixel of the image captured under the condition.

Step 3, histogram equalization.

According to the following formula, the histogram equalization is performed on the images taken under the condition of the flash light and without the flash light to be registered respectively:

s _x = int[(L-1)*p _x +0.5];

Among them, p _x represents the cumulative sum of the probability values of the final gray level of the brightness channel matrix, x represents the gray value of the brightness channel matrix, and the value range of x is 0 to 255, ∑ represents the summation operation, and f represents the brightness The gray level of the channel matrix, f=0,1,2,...,x, g(f) represents the probability value of the final gray level of the brightness channel matrix, and s _x represents the brightness channel matrix after histogram equalization The mapping value of the middle gray value x, int means the rounding operation, L means the maximum value of the gray level of the brightness channel matrix.

Step 4, detect feature points.

Images taken under the conditions of the flash light and no flash light to be registered, the images obtained by filtering with Gaussian filters of different scales form a sub-octave octave; The images taken below are down-sampled once, twice, and three times, and similar filtering operations are performed to obtain a Gaussian pyramid layer, and the adjacent layers are subtracted to obtain a differential Gaussian pyramid.

In the differential Gaussian pyramid, compare the size of the pixel on the middle layer with the 8 adjacent pixels of the same scale layer, and the size of the 18 adjacent pixels of the pixel and its upper and lower adjacent scale layers, if the middle If the value of the pixel point on the layer is the maximum value or the minimum value, then the pixel point is used as a candidate feature point.

According to the following method, the low-contrast candidate feature points that are sensitive to noise and the candidate feature points with unstable edge responses are removed, and the remaining feature points are the final feature points.

The method of removing low-contrast candidate feature points that are sensitive to noise is as follows.

Step 1: Calculate the feature point position with sub-pixel precision according to the following formula:

Among them, X' represents the position of the feature point with sub-pixel precision, D represents the difference Gaussian DOG space, and X represents the position of the feature point.

Step 2: According to the following formula, calculate the value of the difference Gaussian space at the position of the feature point with sub-pixel precision:

Among them, D(X′) represents the value of the differential Gaussian space at the position of the feature point with sub-pixel precision, D(X) represents the value of the differential Gaussian space at the position of the feature point, D represents the differential Gaussian DOG space, and X represents the feature point location.

The third step: retain the feature points that satisfy the condition of |D(X′)|≥0.03, and eliminate the feature points that do not meet the condition.

The method of removing candidate feature points with unstable marginal responses is as follows.

Step 1: Calculate the Hessian matrix according to the following formula:

Among them, H represents the local curvature matrix of the differential Gaussian space, D _xx represents the second-order partial derivative of the differential Gaussian space in the column direction of the candidate feature point, D _xy represents the differential Gaussian space in the column direction and row direction of the candidate feature point The second-order partial derivative of D _yy represents the second-order partial derivative of the differential Gaussian space in the row direction of the candidate feature point, x represents the column number of the candidate feature point, and y represents the row number of the candidate feature point.

Step 2: According to the following formula, calculate the ratio of the large eigenvalue to the small eigenvalue of the Hessian matrix H:

Among them, r represents the ratio of the large eigenvalue to the small eigenvalue of the Hessian matrix H, α represents the large eigenvalue of the Hessian matrix H, and β represents the small eigenvalue of the Hessian matrix H.

Step 3: Determine whether the Hessian matrix Hessian satisfies the following conditions:

Among them, Tr(H) represents the trace of the Hessian matrix H, Det(H) represents the determinant of the Hessian matrix H, and r represents the ratio of the large and small eigenvalues of the Hessian matrix H.

Step 4: Keep the candidate feature points that meet the above conditions, and eliminate the candidate feature points that do not meet the conditions.

According to the following formula, calculate the gradient direction of the neighborhood pixel with the final feature point as the center, and express it in a histogram. The peak value of the histogram represents the main direction of the gradient of the neighborhood pixel of the final feature point, and take the main direction of the neighborhood pixel gradient as The orientation of this final feature point:

Among them, m(x,y) represents the modulus value of the gradient of the neighboring pixel, L(x+1,y) represents the value of the pixel on the right side of the neighboring pixel in Gaussian space, and L(x-1,y) represents the neighboring pixel The value of the pixel on the left in Gaussian space, L(x,y+1) represents the value of the pixel below the neighboring pixel in Gaussian space, L(x,y-1) represents the value of the pixel above the neighboring pixel in Gaussian space, x represents The column number of the neighboring pixel, y represents the row number of the neighboring pixel, θ(x, y) represents the direction of the gradient of the neighboring pixel, and arctan represents the arctangent operation.

With the final feature point as the center, select a 16×16 neighborhood, and divide the neighborhood into 16 4×4 sub-regions, and calculate 0°, 45°, 135°, 180°, 225° on each sub-region °, 270°, 315°, and 360°, the cumulative gradient values in 8 directions can generate a 128-dimensional feature vector.

Step 5, matching feature points.

For each final feature point in the image taken under the flash condition, use the Euclidean distance to find the two feature points closest to the final feature point of the image taken under the flash condition in the image taken without the flash condition. Among the feature points, if the ratio of the closest distance to the next closest distance is less than 0.4, the final feature point of the image taken under the flash condition matches the nearest point in the image taken without the flash condition, otherwise it does not match.

Step 6, use the following random sampling consistent RANSAC algorithm to eliminate false feature point matching pairs.

Step 1: Randomly select 4 feature point matching pairs from the set of feature point matching pairs.

The second step: List the equations according to the selected four matching pairs of feature points, and solve the equations to obtain the affine transformation matrix.

Step 3: According to the affine transformation matrix and the Euclidean distance error metric function, search for a consensus set that satisfies the current affine transformation matrix from the feature point matching pair set.

Step 4: Determine whether the number of elements in the current consistent set is greater than the number of elements in the optimal consistent set. If so, update the current consistent set to the optimal consistent set; otherwise, keep the original optimal consistent set.

Step 5: Update the current error probability P.

Step 6: Determine whether the updated error probability P is greater than the allowable minimum error probability, if so, perform the first step, otherwise, use the optimal consistent set as the final feature point matching pair.

Step 7, judging whether the feature points of the image captured under the condition of the flash lamp meet the following judging conditions, if so, go to step (8), otherwise, go to step (9).

Among them, V _x represents the degree to which the column coordinates of the feature points of the image captured under the flash light condition deviate from the average value, W represents the width of the image captured under the flash light condition, and V _y represents the deviation of the feature point row coordinates of the image captured under the flash light condition from the average value The degree, H indicates the high of the image captured under the flash condition.

Step 8, feature point matching pair equalization.

According to the following formula, calculate the average offset of the feature point matching pairs in the column direction and the row direction of the images taken under the flashlight condition and without the flashlight condition to be registered:

Among them, _Δx represents the average offset of the matching pairs of feature points in the column direction of the image to be registered under the flash condition and without the flash condition, and x represents the image taken under the flash condition and without the flash condition to be registered The column direction of the feature points of the image, n represents the total number of matching pairs of feature points of the image to be registered under the condition of the flash and without the flash, i represents the number of matching pairs of the feature points of the image to be registered under the condition of the flash and without the flash The serial number of the feature point matching pair of the image, Indicates that the feature points of the i-th image to be registered under the condition of the flashlight and the image taken without the flashlight match the column coordinates of the feature points of the image taken under the condition of the flashlight, Indicates the feature point column coordinates of the _i -th image to be registered under the condition of the flashlight and without the flashlight in the matching pair of feature points of the image taken under the condition of the flashlight without the flashlight, Δy represents the condition of the flashlight to be registered and without the flashlight The average offset of the feature point matching pairs in the row direction of the image taken under the condition of Represents the row coordinates of the feature points of the i-th image to be registered under the condition of the flashlight and the image taken under the condition of no flashlight matching the feature points of the image taken under the condition of the flashlight, Indicates the row coordinates of the feature points of the i-th to-be-registered image captured under the flash condition and without the flash condition in the matching pair.

According to the following formula, the image taken under the flashlight condition to be registered is divided into M×M sub-blocks of the same size:

Among them, HW represents the width of the sub-block, Represents the rounding down operation, W represents the width of the image captured under the flash condition, M represents the number of sub-blocks in each row of the image captured under the flash light, HH represents the height of the sub-block, and H represents the height of the image captured under the flash condition .

According to the following formula, calculate the row coordinates and column coordinates of the feature points to be added to the image taken under the flashlight:

in, Represents the column coordinates of the feature points to be added, x represents the column direction of the feature points to be added, k represents the sequence number of the feature points to be added in the image taken under flash conditions, k=(i×M+j)×N×N +i1×N+j1, i represents the number of the row corresponding to the sub-block of the same size, i=0,1,2,...,M-1, i1 represents the row corresponding to the feature point to be added in the sub-block Number, i1=0,1,2,...,N-1, j represents the number of the corresponding column of the sub-block of the same size, j=0,1,2,...,M-1, j1 represents The number of the corresponding column of the feature point to be added in the sub-block, j1=0,1,2,...,N-1, HW indicates the width of the sub-block, D _x indicates the column between the feature points to be added direction distance, W represents the width of the image captured under the flashlight condition, M represents the number of subblocks in each row of the image captured under the flashlight, and N represents the number of feature points added to each row of each subblock of the image captured under the flashlight, Represents the row coordinates of the feature points to be added, y represents the row direction of the feature points to be added, HH represents the height of the sub-block, D _y represents the distance in the row direction between the feature points to be added, and H represents the shooting under flash conditions the height of the image, H represents the height of the image captured under the flashlight condition, M represents the number of sub-blocks in each row of the image captured under the flashlight, and N represents the number of feature points added to each row of each sub-block in the image captured under the flashlight.

According to the following formula, calculate the column coordinates and row coordinates of the feature points to be added to the image taken under the condition of no flash:

in, Represents the column coordinates of the feature points to be added to the image taken under the condition of no flash, x represents the column direction of the feature points to be added, k represents the serial number of the feature points to be added, Indicates the column coordinates of the feature points to be added to the images captured under flash conditions, _Δx represents the average offset in the column direction of the matching pairs of feature points of images captured under flash conditions and without flash conditions, Indicates the row coordinates of the feature points to be added to the image taken under the condition of no flashlight, Indicates the row coordinates of the feature points to be added to images captured under flash conditions, and _Δy represents the average offset in the row direction of the feature point matching pairs of images captured under flash conditions and without flash conditions to be registered.

Step 9, use the matching pairs of feature points obtained in step (6) and step (8) to list equations, solve the equation system, and obtain the affine transformation matrix H.

Step 10, register images.

According to the following formula, calculate the pixel of the position (i, j) of the image captured under the flash light condition after registration, and the corresponding position of the image captured under the flash light condition after mapping:

Among them, i' represents the column coordinates of the image pixels taken under the condition of flashlight, H ^-1 _1,1 represents the first row and first column element of the inverse matrix of the affine transformation matrix, H ^-1 _1,2 represents the affine transformation matrix The element in the first row and the second column of the inverse matrix, H ^-1 _1,3 means the element in the first row and the third column of the inverse matrix of the affine transformation matrix, H ^-1 _2,1 means the inverse matrix of the affine transformation matrix The element in the first column of the second row, H ^-1 _2,2 means the element in the second row and the second column of the inverse matrix of the affine transformation matrix, H ^-1 _2,3 means the second row and the second column of the inverse matrix of the affine transformation matrix Three columns of elements, H ^-1 _3,1 represents the third row and first column element of the inverse matrix of the affine transformation matrix, H ^-1 _3,2 represents the third row and second column element of the inverse matrix of the affine transformation matrix, H ^-1 _3,3 represents the third row and third column element of the inverse matrix of the affine transformation matrix, i represents the column coordinates of the image pixels captured under the flashlight condition after registration, and j represents the shooting under the flashlight condition after registration The row coordinates of the image pixels, j' represents the row coordinates of the image pixels captured under flash conditions.

According to the following formula, the pixel value of the pixel at position (i, j) of the image taken under the condition of the flash light after registration is calculated:

R _i,j =α ₁ ×F _Ii,Ij +α ₂ ×F _Ii,Ij+1 +α ₃ ×F _Ii+1,Ij +α ₄ ×F _Ii+1,Ij+1 ;

where R _i,j represents the pixel value of the image captured under the registered flash condition, i represents the column coordinate of the image pixel captured under the registered flash condition, and j represents the image captured under the registered flash condition The row coordinates of the pixel, α ₁ represents the weight of the pixel in the upper left corner closest to the image pixel captured under the flash condition, F _{Ii, Ij} represents the pixel value of the pixel in the upper left corner closest to the image pixel captured under the flash condition, and Ii represents The integer part of the column coordinates of the image pixel captured under the flash light condition, Ij represents the integer part of the row coordinates of the image pixel captured under the flash light condition, _α2 represents the weight of the pixel in the lower left corner closest to the image pixel captured under the flash light condition, F _Ii,Ij+1 represents the pixel value of the pixel in the lower left corner closest to the image pixel captured under the flash condition, α ₃ represents the weight of the pixel in the upper right corner closest to the image pixel captured under the flash condition, F _{Ii+1, Ij} represents the pixel value of the pixel in the upper right corner closest to the image pixel captured under the flash condition, α ₄ represents the weight of the pixel in the lower right corner closest to the image pixel captured under the flash condition, F _{Ii+1, Ij+1} represents the distance The pixel value of the nearest bottom right pixel to the image pixel taken under flash conditions.

The simulation effect of the present invention will be further described in conjunction with accompanying drawing 2 below.

1. Simulation data:

The test images to be processed used in the simulation are one frame of images with the flash on and one frame without the flash. The size of the image is 5312×2988. The image has three color channels of R, G, and B, and the level of each channel is 256 .

2. Simulation results and analysis:

Accompanying drawing 2 is the simulation result figure of the present invention, and wherein, accompanying drawing 2 (a) is the photo taken under the open flashlight to be registered; Accompanying drawing 2 (b) is the photo taken under the non-open flashlight as reference frame; Accompanying drawing 2 (c) is the effect diagram of fusion of two frames after traditional SIFT algorithm registration; Accompanying drawing 2 (d) is the effect diagram of fusion of two frames after registration of the present invention.

Comparing the four sub-pictures in Attachment 2, it can be seen that the lights on the floors in the fusion effect picture after registration and fusion using the traditional SIFT algorithm appear ghosting, and the vertical bars between the windows on the second floor on the left appear broken. The phenomenon of driving and the car downstairs have ghosts. However, the present invention uses the preprocessing step of histogram equalization to greatly increase the number of detected matching pairs, and adopts the algorithm of adaptive matching pair addition to solve the problem of local overfitting, thereby effectively solving the above-mentioned ghosting problem.

In summary, it can be seen that the present invention can improve the registration accuracy of photos taken under flashlight and photos taken without flashlight, and overcome the problem of ghosting when the general SIFT algorithm is applied to the above situation.

Claims (6)

1. A method for improving the registration accuracy of photographed images at night, comprising the steps of: (1) Input the image to be registered: Input one image taken with flash and one without flash to be registered respectively; (2) Image grayscale: According to the following formula, the images taken under the conditions of the flash light and without the flash light to be registered are respectively grayscaled: ${\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2365</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 23434</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 6969</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 32768</span>}}<span\; class="notranslate">\; ;</span>$ Among them, Y _i represents the gray value of the i-th pixel in the image taken under the condition of the flash light and without the flash light to be registered, and i represents the pixel point of the image shot under the condition of the flash light and without the flash light to be registered B, G, and R represent the blue, green, and red channels of the images taken under the condition of the flash light and without the flash light to be registered respectively, and B _i represent the channels of the images shot under the condition of the flash light to be registered and without the flash light The blue channel of the i-th pixel of the image, G _i represents the green channel of the i-th pixel of the image to be registered under the flash condition and no flash condition, and R _i represents the flash condition and no flash condition to be registered The red channel of the i-th pixel of the image captured under the condition; (3) Histogram equalization: According to the following formula, the histogram equalization is performed on the images taken under the condition of the flash light and without the flash light to be registered respectively: ${\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; x</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ;</span>$ s _x = int[(L-1)*p _x +0.5]; Among them, p _x represents the cumulative sum of the probability values of the final gray level of the brightness channel matrix, x represents the gray value of the brightness channel matrix, and the value range of x is 0 to 255, ∑ represents the summation operation, and f represents the brightness The gray level of the channel matrix, f=0,1,2,...,x, g(f) represents the probability value of the final gray level of the brightness channel matrix, and s _x represents the brightness channel matrix after histogram equalization The mapping value of the middle gray value x, int means the rounding operation, L means the maximum value of the gray level of the brightness channel matrix; (4) Detection of feature points: (4a) For the images taken under the conditions of the flash light to be registered and without the flash light, the images obtained by filtering with Gaussian filters of different scales form a sub-octave; by analogy, the images taken under the flash light conditions to be registered and The images taken under the condition of no flash are down-sampled once, twice, and three times respectively, and similar filtering operations are performed to obtain a Gaussian pyramid layer, and the adjacent layers are subtracted to obtain a differential Gaussian pyramid; (4b) In the differential Gaussian pyramid, compare the pixel point on the middle layer with the size of 8 adjacent pixel points of the same scale layer, and the size of the pixel point and the size of 18 adjacent pixel points of the adjacent scale layer above and below , if the value of the pixel point on the intermediate layer is the maximum value or the minimum value, the pixel point is used as a candidate feature point; (4c) remove noise-sensitive low-contrast candidate feature points and candidate feature points with unstable edge responses, and the rest are final feature points; (4d) Calculate the gradient direction of the neighborhood pixel with the final feature point as the center, and express it with a histogram. The peak value of the histogram represents the main direction of the gradient of the neighborhood pixel of the final feature point, and take the main direction of the neighborhood pixel gradient as the The direction of the final feature point; (4e) Take the final feature point as the center, select a 16×16 neighborhood, and divide the neighborhood into 16 4×4 sub-regions, and calculate 0°, 45°, 135°, 180° on each sub-region °, 225°, 270°, 315°, 360° total gradient cumulative value in 8 directions to generate a 128-dimensional feature vector; (5) Match feature points: For each final feature point in the image taken under the flash condition, use the Euclidean distance to find the two feature points closest to the final feature point of the image taken under the flash condition in the image taken without the flash condition. Among the feature points, if the ratio of the shortest distance to the next shortest distance is less than 0.4, the final feature point of the image taken under the flash condition matches the nearest point in the image taken under the flash condition, otherwise it does not match; (6) Use random sampling consistent RANSAC algorithm to eliminate wrong feature point matching pairs; (7) Judging whether the feature points of the image taken under the condition of the flashlight meet the judgment condition, if so, perform step (8), otherwise, perform step (9); (8) Feature point matching pair equalization: (8a) According to the following formula, calculate the average offset of the feature point matching pairs in the column direction and the row direction of the image taken under the condition of the flashlight and without the flashlight to be registered: ${\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; no</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; PF</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; PN</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; no</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; PF</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; PN</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ;</span>$ Among them, _Δx represents the average offset of the matching pairs of feature points in the column direction of the image to be registered under the flash condition and without the flash condition, and x represents the image taken under the flash condition and without the flash condition to be registered The column direction of the feature points of the image, n represents the total number of matching pairs of feature points of the image to be registered under the condition of the flash and without the flash, i represents the number of matching pairs of the feature points of the image to be registered under the condition of the flash and without the flash The serial number of the feature point matching pair of the image, Indicates that the feature points of the i-th image to be registered under the condition of the flashlight and the image taken without the flashlight match the column coordinates of the feature points of the image taken under the condition of the flashlight, Indicates the feature point column coordinates of the _i -th image to be registered under the condition of the flashlight and without the flashlight in the matching pair of feature points of the image taken under the condition of the flashlight without the flashlight, Δy represents the condition of the flashlight to be registered and without the flashlight The average offset of the feature point matching pairs in the row direction of the image taken under the condition of Represents the row coordinates of the feature points of the i-th image to be registered under the condition of the flashlight and the image taken under the condition of no flashlight matching the feature points of the image taken under the condition of the flashlight, Represent the row coordinates of the feature points of the i-th image to be registered under the condition of the flashlight and the feature point of the image taken under the condition of the flashlight without the flashlight; (8b) According to the following formula, the image taken under the condition of the flashlight to be registered is divided into M×M sub-blocks of the same size: Among them, HW represents the width of the sub-block, Represents the rounding down operation, W represents the width of the image captured under the flash condition, M represents the number of sub-blocks in each row of the image captured under the flash light, HH represents the height of the sub-block, and H represents the height of the image captured under the flash condition ; (8c) According to the following formula, calculate the row coordinates and column coordinates of the feature points to be added to the image taken under the flashlight: ${\mathrm{<span\; class="notranslate">\; PF</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ;</span>$ ${\mathrm{<span\; class="notranslate">\; PF</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; ;</span>$ in, Represents the column coordinates of the feature points to be added, x represents the column direction of the feature points to be added, k represents the sequence number of the feature points to be added in the image taken under flash conditions, k=(i×M+j)×N×N +i1×N+j1, i represents the number of the row corresponding to the sub-block of the same size, i=0,1,2,...,M-1, i1 represents the row corresponding to the feature point to be added in the sub-block Number, i1=0,1,2,...,N-1, j represents the number of the corresponding column of the sub-block of the same size, j=0,1,2,...,M-1, j1 represents The number of the corresponding column of the feature point to be added in the sub-block, j1=0,1,2,...,N-1, HW indicates the width of the sub-block, D _x indicates the column between the feature points to be added direction distance, W represents the width of the image captured under the flashlight condition, M represents the number of subblocks in each row of the image captured under the flashlight, and N represents the number of feature points added to each row of each subblock of the image captured under the flashlight, Represents the row coordinates of the feature points to be added, y represents the row direction of the feature points to be added, HH represents the height of the sub-block, D _y represents the distance in the row direction between the feature points to be added, and H represents the shooting under flash conditions the height of the image, H represents the height of the image taken under the flashlight condition, M represents the subblock number of each row of the image taken under the flashlight, and N represents the number of feature points added to each row of each subblock of the image taken under the flashlight; (8d) According to the following formula, calculate the column coordinates and row coordinates of the feature points to be added to the image taken under the condition of no flashlight: ${\mathrm{<span\; class="notranslate">\; PN</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; PF</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ;</span>$ ${\mathrm{<span\; class="notranslate">\; PN</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; PF</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; ;</span>$ in, Represents the column coordinates of the feature points to be added to the image taken under the condition of no flash, x represents the column direction of the feature points to be added, k represents the serial number of the feature points to be added, Indicates the column coordinates of the feature points to be added to the images captured under flash conditions, _Δx represents the average offset in the column direction of the matching pairs of feature points of images captured under flash conditions and without flash conditions, Indicates the row coordinates of the feature points to be added to the image taken under the condition of no flashlight, Represents the row coordinates of the feature points to be added to the image taken under the flashlight condition, and _Δy represents the average offset in the row direction of the feature point matching pairs of the images taken under the flashlight condition and without the flashlight condition to be registered; (9) utilize step (6) and the feature point matching that step (8) obtains to list equation, solve equation group, obtain affine transformation matrix H; (10) Registration image: (10a) According to the following formula, calculate the pixel of the position (i, j) of the image captured under the flashlight condition after registration, and the position of the image captured under the corresponding flashlight condition after mapping: ${\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; ,</span>}<span\; class="notranslate">\; =</span>\frac{{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}}}{{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; ;</span>$ ${\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; ,</span>}<span\; class="notranslate">\; =</span>\frac{{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}}}{{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; ;</span>$ Among them, i' represents the column coordinates of the image pixels taken under the condition of flashlight, H ^-1 _1,1 represents the first row and first column element of the inverse matrix of the affine transformation matrix, H ^-1 _1,2 represents the affine transformation matrix The element in the first row and the second column of the inverse matrix, H ^-1 _1,3 represents the element in the first row and the third column of the inverse matrix of the affine transformation matrix, H ^-1 _2,1 represents the inverse matrix of the affine transformation matrix The element in the first column of the second row, H ^-1 _2,2 means the element in the second row and the second column of the inverse matrix of the affine transformation matrix, H ^-1 _2,3 means the second row and the second column of the inverse matrix of the affine transformation matrix Three columns of elements, H ^-1 _3,1 represents the third row and first column element of the inverse matrix of the affine transformation matrix, H ^-1 _3,2 represents the third row and second column element of the inverse matrix of the affine transformation matrix, H ^-1 _3,3 represents the third row and third column element of the inverse matrix of the affine transformation matrix, i represents the column coordinates of the image pixels captured under the flashlight condition after registration, and j represents the shooting under the flashlight condition after registration The row coordinates of the image pixels of j' represent the row coordinates of the image pixels captured under the flashlight condition; (10b) According to the following formula, calculate the position (i, j) pixel value of the image captured under the condition of the flash light after registration: R _i,j =α ₁ ×F _Ii,Ij +α ₂ ×F _Ii,Ij+1 +α ₃ ×F _Ii+1,Ij +α ₄ ×F _Ii+1,Ij+1 ; where R _i,j represents the pixel value of the image captured under the registered flash condition, i represents the column coordinate of the image pixel captured under the registered flash condition, and j represents the image captured under the registered flash condition The row coordinates of the pixel, α ₁ represents the weight of the pixel in the upper left corner closest to the image pixel captured under the flash condition, F _{Ii, Ij} represents the pixel value of the pixel in the upper left corner closest to the image pixel captured under the flash condition, and Ii represents The integer part of the column coordinates of the image pixel captured under the flash light condition, Ij represents the integer part of the row coordinates of the image pixel captured under the flash light condition, _α2 represents the weight of the pixel in the lower left corner closest to the image pixel captured under the flash light condition, F _Ii,Ij+1 represents the pixel value of the pixel in the lower left corner closest to the image pixel captured under the flash condition, α ₃ represents the weight of the pixel in the upper right corner closest to the image pixel captured under the flash condition, F _{Ii+1, Ij} represents the pixel value of the pixel in the upper right corner closest to the image pixel captured under the flash condition, α ₄ represents the weight of the pixel in the lower right corner closest to the image pixel captured under the flash condition, F _{Ii+1, Ij+1} represents the distance The pixel value of the nearest bottom right pixel to the image pixel taken under flash conditions. 2. the method for promoting the registration accuracy of photographed images at night according to claim 1, characterized in that: the specific steps of the method for removing noise-sensitive low-contrast candidate feature points described in step (4c) are as follows: Step 1: Calculate the feature point position with sub-pixel precision according to the following formula: ${\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>{\frac{{<span\; class="notranslate">\; \&part;</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; \&part;</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\frac{<span\; class="notranslate">\; \&part;</span>\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; \&part;</span>\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ;</span>$ Among them, X' represents the position of the feature point with sub-pixel precision, D represents the difference Gaussian DOG space, and X represents the position of the feature point; Step 2: According to the following formula, calculate the value of the difference Gaussian space at the position of the feature point with sub-pixel precision: $\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}{\frac{<span\; class="notranslate">\; \&part;</span>\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; \&part;</span>\mathrm{<span\; class="notranslate">\; x</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ;</span>$ Among them, D(X′) represents the value of the differential Gaussian space at the position of the feature point with sub-pixel precision, D(X) represents the value of the differential Gaussian space at the position of the feature point, D represents the differential Gaussian DOG space, and X represents the feature point position; The third step: retain the feature points that satisfy the condition of |D(X′)|≥0.03, and eliminate the feature points that do not meet the condition. 3. the method for promoting the registration accuracy of photographed images at night according to claim 1, characterized in that: the specific steps of the method for removing the candidate feature points with unstable edge responses described in step (4c) are as follows: Step 1: Calculate the Hessian matrix according to the following formula: $\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; x</span>}}& {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}}\end{array}\right]<span\; class="notranslate">\; ;</span>$ Among them, H represents the local curvature matrix of the differential Gaussian space, D _xx represents the second-order partial derivative of the differential Gaussian space in the column direction of the candidate feature point, D _xy represents the differential Gaussian space in the column direction and row direction of the candidate feature point The second-order partial derivative of , D _yy represents the second-order partial derivative of the differential Gaussian space in the row direction of the candidate feature point, x represents the column number of the candidate feature point, and y represents the row number of the candidate feature point; Step 2: According to the following formula, calculate the ratio of the large eigenvalue to the small eigenvalue of the Hessian matrix H: $\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}<span\; class="notranslate">\; ;</span>$ Among them, r represents the ratio of the large eigenvalue to the small eigenvalue of the Hessian Hessian matrix H, α represents the large eigenvalue of the Hessian Hessian matrix H, and β represents the small eigenvalue of the Hessian Hessian matrix H; Step 3: Determine whether the Hessian matrix Hessian satisfies the following conditions: $\frac{\mathrm{<span\; class="notranslate">\; T</span>}\mathrm{<span\; class="notranslate">\; r</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; <</span>\frac{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; ;</span>$ Among them, Tr (H) represents the trace of Hessian Hessian matrix H, Det (H) represents the determinant of Hessian Hessian matrix H, and r represents the ratio of the large eigenvalue and small eigenvalue of Hessian Hessian matrix H; Step 4: Keep the candidate feature points that meet the above conditions, and eliminate the candidate feature points that do not meet the conditions. 4. The method for improving the registration accuracy of photographed images at night according to claim 1, characterized in that: the calculation described in the step (4d) takes the final feature point as the formula of the modulus and direction of the gradient of the center neighborhood pixels as follows: $\begin{array}{c}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}\\ \mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arctan</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; ;</span>$ Among them, m(x, y) represents the modulus value of the gradient of the neighborhood pixel centered on the final feature point, and L(x+1, y) represents the value of the pixel on the right side of the neighborhood pixel centered on the final feature point in Gaussian space, L(x-1,y) represents the value of the left pixel of the neighborhood pixel centered on the final feature point in Gaussian space, and L(x,y+1) represents the value of the pixel below the neighborhood pixel centered on the final feature point in Gaussian space value, L(x,y-1) represents the value of the upper pixel in the Gaussian space with the final feature point as the center of the neighborhood pixel, x represents the column number of the neighborhood pixel with the final feature point as the center, and y represents the final feature point as the The row number of the central neighborhood pixel, θ(x,y) indicates the direction of the gradient of the central neighborhood pixel with the final feature point as the center, and arctan indicates the arctangent operation. 5. the method for promoting the registration accuracy of photographed images at night according to claim 1, is characterized in that: the concrete steps of random sampling consistent RANSAC algorithm described in step (6) are as follows: Step 1: Randomly select 4 feature point matching pairs from the set of feature point matching pairs; The second step: list the equations according to the selected four feature point matching pairs, solve the equations, and obtain the affine transformation matrix; Step 3: According to the affine transformation matrix and the Euclidean distance error measurement function, find the consensus set Consensus that satisfies the current affine transformation matrix from the feature point matching pair set; Step 4: Determine whether the number of elements in the current consistent set is greater than the number of elements in the optimal consistent set, if so, update the current consistent set to the optimal consistent set, otherwise, keep the original optimal consistent set; Step 5: Update the current error probability P; Step 6: Determine whether the updated error probability P is greater than the allowable minimum error probability, if so, perform the first step, otherwise, use the optimal consistent set as the final feature point matching pair. 6. The method for improving the registration accuracy of photographed images at night according to claim 1, characterized in that: the judging condition that the feature points of the image captured under the condition of the flash light described in step (7) satisfy the judgment condition refers to satisfying the following conditions simultaneously situation: ${\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; <</span>\frac{\mathrm{<span\; class="notranslate">\; W</span>}}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; <</span>\frac{\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; ;</span>$ Among them, V _x represents the degree to which the column coordinates of the feature points of the image captured under the flash light condition deviate from the average value, W represents the width of the image captured under the flash light condition, and V _y represents the deviation of the feature point row coordinates of the image captured under the flash light condition from the average value The degree, H indicates the high of the image captured under the flash condition.