WO2022127010A1 - Perspective image correction method - Google Patents

Abstract

Disclosed is a perspective image correction method, comprising the following steps: (1) mounting a flat plate structural member in an imaging path of an X-ray machine, wherein a plurality of sets of collinear mark points are provided on the flat plate structural member; (2) performing perspective imaging, specifically, extracting image coordinates of all of the mark points, and obtaining a central area and a peripheral area by means of partitioning; (3) using a first-order radial distortion correction model based on cross ratio invariance for the central area, obtaining a correction coefficient by means of calculation, and thereby obtaining theoretical distortion-free coordinates of all of the mark points in the central area by means of calculation; (4) obtaining theoretical distortion-free coordinates of the mark points in the peripheral area by means of calculation according to the cross ratio invariance and design parameters; and (5) establishing a correction model, and solving a high-order polynomial mapping model. According to the present invention, by means of providing a cover with a specific arrangement of steel balls covering an image multiplier end of an X-ray machine, a single perspective image is directly corrected, which has an important use value in a surgical robot system based on an image multiplying type X-ray machine.

Description

A kind of perspective image correction method technical field

The invention relates to the field of image processing, in particular to a perspective image correction method.

Background technique

As one of the large-scale medical equipment in the operating room, the C-arm X-ray machine has been widely used in various orthopedic operations because of its advantages of fluoroscopy imaging and storing the acquired images at any time. In recent years, orthopedic surgical robots with assisted positioning have also developed rapidly, and have been gradually applied in actual orthopedic clinical operations, while C-arm X-ray machines have become an indispensable and important imaging component in orthopedic surgical robotic systems. The accuracy also directly affects the navigation and positioning accuracy of the entire surgical robot system.

The shadow-enhancing X-ray machine has become the first choice of major surgical robot research and development companies because of its low price, stable function and convenient maintenance. However, the C-arm X-ray machine will definitely produce images during the imaging process through the lens in the intensifier. Distortion causes the position of each pixel in the original image to be shifted, which does not conform to the law of perspective imaging, which makes the image unable to be used in high-precision navigation and positioning occasions. How to correct or reduce the distortion in the X-ray machine perspective imaging process has also become It is one of the problems that the surgical robot system with high precision navigation and positioning must solve.

SUMMARY OF THE INVENTION

Purpose of the invention: In view of the above problems, the present invention proposes a perspective image correction method, which corrects the perspective image based on a distance-weighted polynomial fitting model, which has high correction accuracy and is easy to operate.

Technical solutions:

A perspective image correction method, comprising the steps of:

(1) Install a flat plate structure on the imaging path of the X-ray machine, and set several groups of collinear marking points on the flat plate structure;

(2) X-ray machine fluoroscopic imaging, extracting the image coordinates of all marked points, and partitioning the fluoroscopic image to obtain the central area of the set diameter and its outer area;

(3) The first-order radial distortion correction model based on the principle of cross-ratio invariance is adopted for the central area, and the correction coefficients of each straight line are obtained according to the image coordinates and design parameters of the marked points in the central area, and all markers in the central area are further obtained. The theoretical undistorted coordinates of the point;

(4) According to the principle of cross ratio invariance and design parameters, the theoretical undistorted coordinates of the marked points in the outer region are obtained;

(5) establishing a correction model from an undistorted image to a distorted image, and obtaining a high-order polynomial mapping model f ⁿ from an undistorted image to a distorted image;

(6) Apply the correction model to obtain an undistorted image.

The marking points are steel balls with a diameter of 2 mm.

In the step (2), the image coordinates of all marked points are extracted using canny operator, adaptive threshold segmentation method or edge detection algorithm.

In the step (3), the first-order radial distortion correction model based on the principle of cross-ratio invariance is adopted for the central region as follows:

Suppose the image coordinates of a mark point i in the central area are (X _1i , Y _1i ), and the theoretical undistorted position coordinates of this point are (X _1i' , Y _1i' ), then the distortion correction model is as follows:

Among them, X _1i represents the abscissa of a mark point i in the C ₁ area in the image coordinate system, Y _1i represents the ordinate of a mark point i in the C ₁ area in the image coordinate system, and r represents C ₁ The distance of a marker point i in the area from the origin in the image coordinate system; X _1i' represents the abscissa of the theoretical undistorted position of a marker point i in the C ₁ area in the image coordinate system, and Y _1i' represents C ₁ is the ordinate of the theoretically undistorted position of a marker i in the image coordinate system; _ki represents the distortion coefficient of the straight line where the marker i is located.

The details of establishing the high-order polynomial mapping model f ⁿ in the step (5) are as follows: let (x, y) be the coordinates of a pixel in the distorted image, and (x', y') be the corresponding theoretical undistorted pixels point coordinates, there is a relation:

Among them, a _ij and b _ij are polynomial coefficients, and n is the polynomial order;

The least squares solution method is used for the above-mentioned high-order polynomial model, and the target error function is the sum of squares of the fitting errors, that is, the following formula

That is, the fitting error square sums ε _x and ε _y made by the coefficients a _ij and b _ij are required to take the minimum value;

Each marker point has an image coordinate and a theoretical undistorted coordinate, that is, each marker point generates two differential equations to obtain an overdetermined equation system, and the coefficients a _ij and b _ij of the polynomial model can be obtained by solving the overdetermined equation system .

In the step (6), the actual undistorted image is subjected to distortion mapping to obtain the theoretical distortion position coordinates of the marker point, which is compared with the image coordinates of the marker point obtained in step (2), and the average value between the corresponding points of the two is compared. Pixel distance as correction error.

Beneficial effects: The present invention directly completes the correction for a single fluoroscopic image after the end cover of the X-ray intensifier has a cover with a specific steel ball arrangement, and establishes a correction model through the partition processing of the image, which greatly eliminates the large distortion of the image edge. Influence, improve the accuracy of intraoperative positioning. It has important application value in the surgical robot system based on the shadow-enhancing X-ray machine.

Description of drawings

FIG. 1 is a flow chart of the steps of the present invention.

FIG. 2 is a schematic diagram of a flat plate structure of the present invention.

FIG. 3 is a perspective view of the X-ray machine of the present invention for the bone model after the flat plate structure is installed.

FIG. 4 is a schematic diagram of partitioning an image.

Figure 5 is a schematic diagram of cross ratio invariance.

FIG. 6 is a schematic diagram of an undistorted image after correction.

Detailed ways

The present invention will be further illustrated below in conjunction with the accompanying drawings and specific embodiments.

The fluoroscopic image correction method of the present invention is processed based on a distance-weighted polynomial fitting model. Before fluoroscopic imaging, a flat plate structure member with steel balls arranged according to a designed rule is installed on the imaging path of the X-ray machine. It can be installed directly approximately parallel to the plane of the shadow intensifier. 2 is a schematic diagram of the flat plate structure of the present invention, the flat plate structure includes a flat plate and a support member supporting the flat plate around it, the flat plate of the flat plate structure is a circular plate, and a steel ball with a diameter of 2 mm is arranged on the flat plate as a marking point; The range with a diameter smaller than d ₁ on the plate is set as the central area, and the range with a diameter greater than d ₁ is set as the outer area; the specific steel ball distribution is: the steel balls are radially distributed with the central steel ball as the center, the steel balls in the central area are sparsely distributed, and the steel balls in the outer area are distributed. It is densely distributed, and every five steel balls arranged in the central area are collinear, and there are four groups of collinear steel balls; the steel balls in the outer area are arranged on the extension line of each collinear straight line.

In the present invention, a steel ball is arranged at the center of the flat plate, and with the steel ball as the central layer, at least two layers are arranged from the inside to the outside in the central area, and each layer is evenly arranged with 8 steel balls, and the outer layers are opposite to each other. The two steel balls are collinear with the steel balls in the center layer; the steel balls in the outer area are arranged in several layers from the inside to the outside, and the steel balls in each layer are evenly arranged, with an equal number of 32; and 8 steel balls are arranged in the center area. extension line.

Fig. 1 is the step flow chart of the present invention, as shown in Fig. 1, the present invention comprises the following steps:

(1) Using an imaging device with a flat plate structure for perspective imaging, as shown in Figure 3, the X-ray machine is used in practical applications to obtain a perspective image by perspective imaging;

在本发明中，提取方法有很多，常用的canny算子、自适应阈值分割或边缘检测等都可以实现，但需保证能达到足够的提取精度； (2) Accurately extract the coordinates of the marked points to obtain the actual coordinates of all the marked points in the perspective image

In the present invention, there are many extraction methods, and the commonly used canny operator, adaptive threshold segmentation or edge detection can be implemented, but it is necessary to ensure that sufficient extraction accuracy can be achieved;

(3) Considering that the distortion of the image is that the distortion in the center of the image is small, and the distortion far from the center of the image is large, if the distortion correction is only performed on the central area of the image, the commonly used correction model can achieve a better correction effect. For the overall distortion correction accuracy, it is a more reasonable choice to partition the image. Here, the entire fluoroscopic image is divided into C ₁ area and C ₂ area, as shown in Figure 4, which are a circular central area with a radius of d ₁ and an annular outer area with a radial range of d ₁ to d ₂ , and d ₁ =d ₂ /2; the C ₁ area is located in the center of the image and has small distortion, and the C ₂ area is located outside the image and has large distortion; the present invention first performs distortion correction on the C ₁ area containing small distortion to obtain The undistorted position of the marker point in the _C1 area, and the undistorted position of the outer marker point is calculated based on this, and the undistorted coordinate position of the marker point is the necessary input data for the distortion correction model;

(4) Establish a _first -order radial distortion correction model based on the _invariance of the cross ratio for the _C1 region. 4; As shown in Figure 4, each straight line l _j corresponds to the correction coefficient k _j ;

The invariance of the intersection ratio is a basic law in perspective projection. Referring to Figure 5, A, B, C, D on the straight line l, and the line connecting the ray source S and l ₀ intersect at A ₀ , B ₀ , C ₀ , D ₀ , the intersection ratio is defined as λ, and the following formula holds:

Then the undistorted position of the marked point in the _C1 area and the actual position of the marked point satisfy the law of cross ratio invariance. For example, for four consecutive marked points A, B, C, and D on a straight line in the _C1 area, it is assumed that there is no distortion in theory. The position coordinates are A', B', C', and D', because the spacing between the marked points can be calculated from the design parameters, and the cross ratio is a constant, there is the following relationship:

Accordingly, the actual image position coordinates of a certain mark point i in the _C1 area are set as (X _1i , Y _1i ), where X _1i represents the horizontal direction of a certain mark point i in the C ₁ area under the image coordinate system Coordinates, Y _1i represents the ordinate of a marker point i in the C ₁ area under the image coordinate system, r represents the distance of a marker point i in the C ₁ area from the origin in the image coordinate system; because the center of the perspective image The distortion of the area is small. Here, the first-order radial distortion correction is used to describe the distortion model of this area to achieve high accuracy. Let the theoretical undistorted position coordinates of this point be (X _1i' , Y _1i' ), where , X _1i' represents the abscissa of the theoretically undistorted position of a marked point i in the _C1 area under the image coordinate system, and Y _1i' represents a theoretically undistorted position of a marked point i in the _C1 area under the image coordinate system The ordinate of the position; the distortion correction model is as follows:

Among them, k _i represents the distortion coefficient of the straight line where the marker point i is located;

In the specific embodiment of the present invention, there are four straight lines in the C ₁ area of the fluoroscopic image, each straight line has 5 marking points, and the marking points on the flat plate structure are arranged at equal distances, then each straight line corresponds to a distortion coefficient k _j The theoretical undistorted positions of all marked points in the area _C1 are to be determined;

及其设计时的坐标

求解出与每条直线相对应的矫正系数k _j，并据此计算得到C ₁区域内所有标记点的理论无畸变位置坐标

(5) According to the distortion correction model established in step (4), combined with the image coordinates extracted from the marked points in the _C1 area

and its design coordinates

Solve the correction coefficient k _j corresponding to each straight line, and calculate the theoretical undistorted position coordinates of all marked points in the C ₁ area accordingly

根据平板结构件的设计参数得到C ₂区域内标记点与C ₁区域内标记点之间的拓扑关系，并据此计算得到C ₂区域内标记点的理论无畸变位置坐标

(6) According to the invariance of the cross ratio, the theoretical undistorted position coordinates of all the marked points in the _C1 area

According to the design parameters of the plate structure, the topological relationship between the marked points in the _C2 area and the marked points in the _C1 area is obtained, and based on this, the theoretical undistorted position coordinates of the marked points in the _C2 area are calculated.

结合步骤(2)得到的所有标记点在透视图像的实际坐标

建立由无畸变图像至畸变图像的矫正模型，即下式： (7) Calculate the theoretical undistorted position coordinates of all the marked points on the fluoroscopic image according to steps (5) and (6)

Combine the actual coordinates of all marked points in the perspective image obtained in step (2)

Establish a correction model from an undistorted image to a distorted image, that is, the following formula:

M′ _img = f ⁿ (M _img )

Among them, M′ _img is the distorted image, M _img is the undistorted image to be obtained, f ⁿ is the high-order polynomial mapping model between the undistorted image and the distorted image, and its model coefficients need to be solved accurately;

The established higher-order polynomial mapping model f ⁿ can be described as follows: Let (x, y) be the coordinates of a pixel in the distorted image, and (x', y') be the corresponding theoretical undistorted pixel coordinates, then there is a relationship Mode:

In formula (3), a _ij and b _ij are polynomial coefficients that need to be solved, and n is the polynomial order;

The least squares solution method is used for the above-mentioned high-order polynomial model, and the target error function is the sum of squares of the fitting errors, that is, the following formula

That is, the coefficients a _ij and b _ij need to be solved, so that the squared sums of fitting errors ε _x and ε _y take the minimum value;

Each marked point has an actual coordinate and a theoretical undistorted coordinate, that is, each marked point generates two differential equations to obtain an overdetermined equation system, and the coefficients a _ij and b _ij of the polynomial model can be obtained by solving the overdetermined equation system ;

将其与步骤(2)得到的标记点实际坐标

进行对比，

与

对应点之间的平均像素距离作为矫正误差，同时将矫正模型应用到整副图像上，得到矫正后无畸变图像，对图3进行矫正后的无畸变图像见图6，矫正误差为0.38pixel。 (8) Apply the correction model obtained in step (7), perform distortion mapping on the actual undistorted image, and obtain the theoretical distortion position coordinates of the marker point

Compare it with the actual coordinates of the marked point obtained in step (2)

comparing,

and

The average pixel distance between the corresponding points is used as the correction error. At the same time, the correction model is applied to the entire image to obtain a corrected undistorted image. The corrected undistorted image in Figure 3 is shown in Figure 6, and the correction error is 0.38 pixel.

The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the specific details of the above-mentioned embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations (such as quantity, shape, etc.) can be performed on the technical solutions of the present invention. , position, etc.), these equivalent transformations all belong to the protection scope of the present invention.

Claims (6)

A method for correcting a perspective image, comprising the steps of: (1) Install a flat plate structure on the imaging path of the X-ray machine, and set several groups of collinear marking points on the flat plate structure; (2) X-ray machine fluoroscopic imaging, extracting the image coordinates of all marked points, and partitioning the fluoroscopic image to obtain the central area of the set diameter and its outer area; (3) The first-order radial distortion correction model based on the principle of cross-ratio invariance is adopted for the central area, and the correction coefficients of each line are obtained according to the image coordinates and design parameters of the marked points in the central area, and all markers in the central area are further obtained. The theoretical undistorted coordinates of the point; (4) According to the principle of cross ratio invariance and design parameters, the theoretical undistorted coordinates of the marked points in the outer region are obtained; (5) establishing a correction model from an undistorted image to a distorted image, and obtaining a high-order polynomial mapping model f ⁿ from an undistorted image to a distorted image; (6) Apply the correction model to obtain an undistorted image. The method for correcting a fluoroscopic image according to claim 1, wherein the marking point is a steel ball with a diameter of 2 mm. The perspective image correction method according to claim 1, characterized in that: in the step (2), the image coordinates of all the marked points are extracted using a canny operator, an adaptive threshold segmentation method or an edge detection algorithm. The perspective image correction method according to claim 1, wherein: in the step (3), a first-order radial distortion correction model based on the principle of cross-ratio invariance is adopted for the central area as follows: Suppose the image coordinates of a mark point i in the central area are (X _1i , Y _1i ), and the theoretical undistorted position coordinates of this point are (X _1i' , Y _1i' ), then the distortion correction model is as follows:

Among them, X _1i represents the abscissa of a mark point i in the C ₁ area in the image coordinate system, Y _1i represents the ordinate of a mark point i in the C ₁ area in the image coordinate system, and r represents C ₁ The distance of a marker point i in the area from the origin in the image coordinate system; X _1i' represents the abscissa of the theoretical undistorted position of a marker point i in the C ₁ area in the image coordinate system, and Y _1i' represents C ₁ is the ordinate of the theoretically undistorted position of a marker i in the image coordinate system; _ki represents the distortion coefficient of the straight line where the marker i is located. The perspective image correction method according to claim 1, wherein: establishing the high-order polynomial mapping model f ⁿ in the step (5) is as follows: Let (x, y) be the coordinates of a pixel in the distorted image , (x', y') is the corresponding theoretical undistorted pixel coordinate, then there is a relational formula:

Among them, a _ij and b _ij are polynomial coefficients, and n is the polynomial order; The least squares solution method is used for the above-mentioned high-order polynomial model, and the target error function is the sum of squares of the fitting errors, that is, the following formula

That is, the fitting error square sums ε _x and ε _y made by the coefficients a _ij and b _ij are required to take the minimum value; Each marker point has an image coordinate and a theoretical undistorted coordinate, that is, each marker point generates two differential equations to obtain an overdetermined equation system, and the coefficients a _ij and b _ij of the polynomial model can be obtained by solving the overdetermined equation system . The perspective image correction method according to claim 1, wherein in the step (6), the actual undistorted image is subjected to distortion mapping to obtain the theoretical distortion position coordinates of the marker point, which are compared with the marker obtained in step (2). The image coordinates of the points are compared, and the average pixel distance between the two corresponding points is used as the correction error.

Images