WO2010038195A2

WO2010038195A2 - Method and system for removing butting or stitching artifacts from images

Info

Publication number: WO2010038195A2
Application number: PCT/IB2009/054266
Authority: WO
Inventors: Carlos Sousa; Mattieu Stefan De Villiers
Original assignee: Lodox Systems Pty Ltd
Current assignee: Lodox Systems Pty Ltd
Priority date: 2008-09-30
Filing date: 2009-09-30
Publication date: 2010-04-08
Anticipated expiration: 2011-03-30
Also published as: WO2010038195A3; ZA201103035B

Abstract

A method and system for removing butting or stitching artifacts from an image are disclosed. The system includes a locating module which is arranged to locate and identify at least one region in the image in which a butting artifact is present. A modeling module determines at least one model of a butting artifact as a function of electromagnetic absorption level, typically an x-ray absorption level, at the identified region. A subtraction module applies a subtraction algorithm to subtract the butting artifact from the image at the identified region based on the model of the butting artifact. The method may include applying linear and cubic interpolation algorithms to the butting artifact in the image to obtain respective linear and cubic interpolations of the butting artifact, or respective linear and cubic interpolation models.

Description

METHOD AND SYSTEM FOR REMOVING BUTTING OR STITCHING ARTIFACTS FROM IMAGES

BACKGROUND OF THE INVENTION

THIS invention relates to a method and a system for removing butting or stitching artifacts from images, in particular from x-ray images from scanning x-ray systems.

Imaging apparatus, in particular scanning x-ray systems, of the type disclosed in WO 00/53093, typically includes an x-ray source and an x-ray detector which are moved relative to a subject under examination in order to generate a composite image of the subject.

As described in WO 00/53093, a plurality of individual abutting camera arrays is provided in the x-ray system. Despite the camera arrays being in abutment with each other, there is typically a degradation of the generated image at a contact region of the camera arrays. The degradation or distortion of the image at the contact regions of the camera arrays is what is referred to as butting artifacts.

The object of the present invention is to provide a method to minimize, if not eliminate, such distortions or butting or stitching artifacts from images generated by scanning x-ray systems, for example. SUMMARY OF THE INVENTION

According to a first aspect of the invention there is provided a method for removing butting or stitching artifacts from an image, the method including:

identifying at least one region in the image in which a butting artifact is present;

determining at least one model of a butting artifact as a function of an electromagnetic absorption level at the identified region; and

applying a subtraction algorithm to subtract the butting artifact from the image at the identified region based on the model of the butting artifact.

The model may include at least a shape and an amplitude of the butting artifact.

The electromagnetic absorption level may be an x-ray absorption level.

As part of determining a model for the butting artifact, the method may comprise:

applying a linear interpolation algorithm to the butting artifact in the image to obtain a linear interpolation of the butting artifact or a linear interpolation model; and

applying a cubic interpolation algorithm to the butting artifact in the image to obtain a cubic interpolation of the butting artifact or a cubic interpolation model.

The method may further comprise: determining a blurring kernel footprint size and associating this with each pixel row index for the butting artifact;

determining linear butting artifact estimates, A_Lin, from differences between the linear interpolation of the butting artifact and the original image;

determining cubic butting artifact estimates, A_Cube, from differences between the cubic interpolation of the butting artifact and the original image;

determining, for each row, linear weighting values of the linear interpolation model, and

determining, for each row, cubic weighting values of the cubic interpolation model.

The step of determining linear weighting values may comprise determining the sum of the absolute differences between the linear interpolation model and the image, and inverting the sum into arrays of linear weighting values, Wu_n.

The step of determining cubic weighting values may comprise determining the sum of the absolute differences between the cubic interpolation model and the image, and inverting the sum into arrays of cubic weighting values,

Wcube-

The method may further comprise:

multiplying the linear butting artifact estimates, A_Un, for each row by the linear weighting values or factors Wu_n and accumulating these products, accAWLin, in a scan direction; multiplying the cubic butting artifact estimates, A_Cυbe, for each row by the cubic weighting values or factors W_Cube and accumulating these products, accAWCube; in the scan direction; and

accumulating the weighting values W_Un and W_Cub_e separately as accWLin and accWCube respectively.

The method may comprise:

determining, by using footprint indices for each row, the difference between the accumulated weighted linear butting artifact estimates, accAWLin, separated by the foot print size from high to low indices; and

dividing the determined difference by a corresponding difference in accumulated linear weighting factors, accWLin, to yield a local linear butting artifact estimate.

The method may comprise:

determining, by using footprint indices for each row, the difference between the accumulated weighted cubic butting artifact estimates, accAWCube, separated by the foot print size from high to low indices; and

dividing the determined difference by a corresponding difference in accumulated cubic weighting factors, accWCube, to yield a local cubic butting artifact estimate.

The method preferably includes determining an average of the local linear and cubic butting artifact estimates to obtain a final artifact estimate.

It will be appreciated that the butting artifact subtracted by the subtraction algorithm may be the final artifact estimate. According to a second aspect of the invention there is provided a system for removing butting or stitching artifacts from an image, the system including:

a locating module arranged to locate and identify at least one region in the image in which a butting artifact is present;

a modeling module arranged to determine at least one model of a butting artifact as a function of electromagnetic absorption level at the identified region; and

a subtraction module arranged to apply a subtraction algorithm to subtract the butting artifact from the image at the identified region based on the model of the butting artifact.

The modeling module may be arranged to apply a linear interpolation algorithm to the butting artifact in the image to obtain a linear interpolation of the butting artifact or a linear interpolation model.

The modeling module may be arranged to apply a cubic interpolation algorithm to the butting artifact in the image to obtain a cubic interpolation of the butting artifact or a cubic interpolation model.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 shows a camera of a camera array of a scanning x- ray system;

Figure 2 shows a front view of a camera array of the scanning x-ray system; Figure 3 shows a functional block diagram of a system in accordance with an example embodiment of the invention;

Figure 4 shows a graphical representation of an image profile, cubic interpolation profile, linear interpolation profile and true image profile in accordance with the invention;

Figure 5 shows a high level flow diagram of a method in accordance with an example embodiment; and

Figure 6 shows a flow diagram of the method of Figure 5 in greater detail.

DESCRIPTION OF PREFERRED EMBODIMENTS

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of an embodiment of the present disclosure. It will be evident, however, to one skilled in the art that the present disclosure may be practiced without these specific details.

Referring to Figures 1 and 2 of the drawings, a scanning x-ray system of the type described in WO 00/53093 typically includes a camera array 9 of cameras 10 arranged side-by-side to other similar cameras 10 (Figure 2). Each camera 10 typically comprises optical fibre bundles 12 which taper from a larger scintillator face 14 to a smaller Charge Coupled Device (CCD) face 16. The scintillator face 16 typically has a parallelogram shape such that when the cameras 10 are side-by-side there is a certain amount of overlap in data from one camera 10 to another. The overlap between adjacent cameras 10 ensures that there will be no or very little loss of data for any pixel column in the resultant x-ray image captured by the camera array 9. However, this arrangement often results in butting or stitching artifacts occurring at the region of the overlap.

It must be noted that the most significant properties of butting artifacts are that:

• different butting artifacts (horizontally separated in image) generally have different shapes; the shape of a particular butting artifact remains nearly constant for a relatively large range of absorption levels but is deformed if the gain compensation is inaccurate at the given absorption level; deformed butting artifacts cannot be predicted without knowing what the gain compensation (calibration) errors are;

• the shape and amplitude of the butting artifact remain relatively similar at different vertical offsets in the image if the absorption level is the same;

• the amplitude of the butting artifact, noise amplitude and gain compensation errors increase with absorption level (at a rate faster than an exponential rate in certain circumstances) butting artifact shapes, amplitudes as well as gain compensation accuracy vary for different machine settings; at high voltage and current settings of the machine, gain compensation errors and butt shape distortion occur at low absorption levels - it will be appreciated that this may be that the machine was not calibrated for this operating region; and occasionally spurious butting artifact amplitudes are encountered which are not predictable.

It will be understood that the above explanations attempt to provide a better understanding of the invention as described herein.

The invention is now described further with reference to Figure 3 of the drawings, wherein a system in accordance with the invention is generally indicated by reference numeral 20. In particular, the system 20 is a system for removing butting artifacts from x-ray images captured by the camera array 9 for example. Conveniently, the system 20 is therefore communicatively coupled to the camera array 9. In an example embodiment the system 20 is provided in a scanning x-ray system as hereinbefore described, typically at or in a digital signal processing module of the scanning x-ray system. In other example embodiments the system 20 does not form part of the scanning x-ray system and is operated in a post-processing fashion to remove butting artifacts from images obtained from the scanning x-ray system.

The system 20 comprises a plurality of components or modules which correspond to the functional tasks to be performed by the system 20. In this regard, "module" in the context of the specification will be understood to include an identifiable portion of code, computational or executable instructions, data, or computational object to achieve a particular function, operation, processing, or procedure. It follows that a module need not be implemented in software; a module may be implemented in software, hardware, or a combination of software and hardware. Further, the modules need not necessarily be consolidated into one device but may be spread across a plurality of devices typically in the scanning x-ray system.

It will be understood that in a preferred example embodiment, the system 20 operates on pixels of the images. In this regard, the system 20 operates on a matrix of pixels, comprising of rows and columns of pixels, associated with the image.

In particular, the system 20 includes a locating module 22 arranged to locate and identify at least one region in the image in which a butting artifact is present. The identified region may typically have a footprint size associated therewith which is determined by the locating module 22. Typically, the footprint size defines a continuous range of rows in a butting artifact space (which may be the identified region) where the shape of the butting artifact remains relatively similar to the butting artifact shape at this row location. It follows that the locating module 22 is arranged to locate and identify a row start and stop range of the identified region, in particular row indices of the row start and stop range.

Typically, the footprint size is chosen to be inversely proportional to the amount of variation in the scanning direction. In this regard, the scanning direction is the direction in which the x-ray source and an x-ray detector of the scanning x-ray system are moved relative to a subject under examination (also herein referred to as the y-direction). Also, fan-beam direction will be understood as the direction of the x-ray beams emitted by the x-ray source, the fan-beam direction being typically transverse to the scanning direction (also herein referred to as the x-direction). It follows that if there is little variation in the image in the scanning direction this range will be relatively large. If there is a lot of variation in the scanning direction, the range will be small.

Similarly, the range must be correspondingly larger if there is more variation in the fan beam direction of the true underlying image. This improves the accuracy of performance of the system 20.

The locating module 22 is further arranged to determine a variable footprint size. It will be appreciated that the variable footprint size takes into account variations of the overlap artifact in both the scanning direction (so called y- direction), and the fan beam direction (so called x-direction).

In this regard, the locating module 22 is arranged to determine the averages of absolute edge differences in a scanning direction across the butting artifact for a particular pixel row in the image and the averages are correspondingly associated with that row. The averages in the scanning direction are referred to as T_y .To make matters computationally efficient, these values are actually accumulated in the scanning direction yielding a lookup table γτ_y . In the fan beam direction, the locating module 22 is arranged to determine a similar value T_x which provides an indication of what amount of variation that occurs for a particular row in the fan beam direction for the underlying artifact-free image at a butting artifact. In particular the locating module 22 is arranged to determine T_x by determining an average of three independent techniques. The first technique calculates the sum of the absolute differences of adjacent pixels in the row of the artifact. This technique is biased by the artifact yet captures high variations in the underlying image which the other two techniques do not capture. The second technique determines the average slope at the left and right extremes of the butt and multiplies it by the butt width. The third technique simply calculates the absolute difference between the left and right intensity for that butt row. It follows that the locating module 22 uses the average of the three techniques above to determine T_x,

The foot print size is the calculated to be the row indices that accumulated T_x (∑T_X ) changes but the actual T_x amount upwards and downwards separately. These extents define the footprint upper and lower limits (accFootLow and accFootHigh). Footprint limits arrays (accFootLow and accFootHigh) which are determined by the locating module 22 are then smoothed for continuity.

The system 20 further includes a modeling module 24 arranged to determine at least one model of a butting artifact as a function of x-ray absorption level at the region identified by the locating module 22. The model includes at least a shape and amplitude of the butting artifact.

The system 20 also includes a subtraction module 26 arranged to apply a subtraction algorithm to subtract the butting artifact from the image at the identified region based on the model of the butting artifact.

It will be understood that the subtraction is "based" on the model of the butting artifact means that the subtraction module 26 is arranged to subtract the model from the image or alternately the subtraction module 26 is arranged to subtract the butting artifact from the image from data associated with or obtained from the model for example (explained in greater detail below).

In an example embodiment, with reference to Figure 4, the system 20 includes a linear interpolation module 28 arranged to apply a linear interpolation algorithm to the butting artifact in the image. The linear interpolation is illustrated accordingly in Figure 4. Similarly, the system 20 further includes a cubic interpolation module 30 arranged to apply a cubic interpolation algorithm to the butting artifact in the image (also illustrated graphically in Figure 4).

In an example embodiment, the linear and cubic interpolation modules 28 and 30 respectively are included in the modeling module 24 or are used by the modeling module 24 such that they facilitate the determination of the model of the butting artifact.

It will be noted that applying the linear interpolation algorithm may include the linear interpolation module 28 being arranged to perform a linear interpolation curve fit to the identified butt region. In a similar way applying the cubic interpolation algorithm may include the cubic interpolation module 30 being arranged to perform a cubic spline curve fit to the identified butt region.

The system 20 includes a butting artifact error estimation module 32 arranged to determine linear butting artifact error estimates from differences between the linear interpolation, as determined by the linear interpolation module 28, and the image. The butting artifact estimation module 32 is also arranged to determine cubic butting artifact error estimates from differences between the cubic interpolation of the butting artifact, as determined by the cubic interpolation module 30, and the image. (This is explained in greater detail below.) The linear and cubic butting artifact estimates are stored, for example, in arrays A_Un and A_Cube respectively for each row of the indentified butt region and for each row separately

The system 20 further includes a weighting module 34 arranged to determine a linear weighting value of the linear interpolation with respect to the image; and a cubic weighting value of the cubic interpolation with respect to the image. These weighting values are associated with a goodness of fit (sum of absolute differences) of each of these interpolations with respect to the image. In particular the weighting values are determined or calculated from the inverted sums of absolute differences and are stored in arrays W_Lln and W_CUbe respectively and for each row in the region.

It will be noted that the weighting module 34 is further arranged to multiply the linear butting artifact estimates for each row by the linear weighting values (ALin*WLin) and to accumulate these in the scan direction {accAWLin). It will be noted that the weighting module 34 is further arranged to multiply the cubic butting artifact estimates for each row by the cubic weighting values (ACube*WCube) and accumulate these in the scan direction {accAWCube). The weighting factors are accumulated likewise, separately as well {accWCube, accWLin).

It will be understood that in one example embodiment, the system 20 includes a database 38 arranged to store at least the weighted linear artifact estimates and the weighted cubic artifact estimates or their associated arrays. It follows that the database 38 is arranged to store a plurality of data, for example other arrays, not expressly mentioned herein but which may have to be stored for use by the system 20.

In order to improve the speed of implementation the multiplication of weighting factor and the artifact estimate quantities are accumulated in the scanning direction ∑(J¥_j-_m -^_j-_m ) ^an(* stored in the database 38. Simiiarly, the weighting factors are accumulated and stored into ∑(fiP}) also in the database 38.

Using the footprint limit indices associated with each row as hereinbefore described, the difference is taken of the accumulated weighted artifact estimates separately from high to low indices. These values are divided by the corresponding difference in accumulated weights from high to low indices. For the new artifact estimates, a local artifact estimate becomes:

where:

W = is the weighting factor for row j,

AJ = difference between the interpolated image and the original image; ni_j = lower footprint size for row j, and ri_j = upper foot print size for row j.

The same is done for the linear interpolation as well as the cubic spline interpolation version and the average of the two is taken to create the final or best fit artifact estimate. It follows that the subtraction module 26 is arranged to apply the subtraction algorithm to subtract the final or best fit artifact estimate from the image thereby effectively removing the butting artifact from the image.

Example embodiments will now be further described in use with reference to Figures 5 and 6. The example methods shown in Figures 5 and 6 are described with reference to Figure 3, although it is to be appreciated that the example methods may be applicable to other systems (not illustrated) as well. Referring now to Figure 5 of the drawings where a flow diagram of a method in accordance with an example embodiment is generally indicated by reference numeral 70.

It will be appreciated that the method 70 may operate at run-time or it may be a post-processing procedure.

The method 70 includes identifying, at block 72, at least one region in the image in which a butting artifact is present. This is typically done by way of the locating module 22 as hereinbefore described. In particular, this step includes determining a footprint size as described above.

The method 70 then includes determining, at block 74, at least one model of a butting artifact as a function of electromagnetic, in particular x-ray absorption level, at the identified region, the model including at least a shape and amplitude of the butting artifact. In an example embodiment, the modeling module 24 is arranged to determine the model of the butting artifact as hereinbefore described.

It follows that the method 70 includes applying, at block 76 a subtraction algorithm, by way of the subtraction module 26 for example, to subtract the butting artifact from the image at the identified region based on the model of the butting artifact. It will be appreciated that the model referred to here is the final or best artifact estimate as hereinbefore described.

Turning now to Figure 6 of the drawings, a method in accordance with a further example embodiment is generally indicated by reference numeral 80.

The method 80 includes determining, at block 82, a footprint size by way of the locating module 22 as hereinbefore described. In particular the method 70 includes determining a blurring kernel footprint size and associating this with each pixel row index for the butting artifact. The blurring kernel footprint size provides a measure of the continuous range of rows in the butt space where the butt shape remains relatively similar to the butt shape at this row location.

The method 80 includes further applying, at block 84, a linear interpolation algorithm, in particular a linear interpolation curve fit, to the butting artifact in the image by way of the linear interpolation module 28 thereby to obtain a linear interpolation of the butting artifact or a linear interpolation model.

Similarly the method 80 includes applying, at block 86, a cubic interpolation algorithm, in particular a cubic spline fit, to the butting artifact in the image by way of the cubic interpolation module 30 thereby to obtain a cubic interpolation of the butting artifact or a cubic interpolation model.

The method 80 also includes determining, at block 88, a cost function. In particular, the method 80 includes determining, by way of module 32, linear butting artifact estimates, A_Lin, from differences between the linear interpolation of the butting artifact and the original image; determining cubic butting artifact estimates, A_Cube, from differences between the cubic interpolation of the butting artifact and the original image; determining, for each row, linear weighting values of the linear interpolation model, and determining, for each row, cubic weighting values of the cubic interpolation model.

It will be appreciated that the step of determining linear weighting values comprises determining the sum of the absolute differences between the linear interpolation model and the image, and inverting the sum into arrays of linear weighting values, Wu_n-

Similarly the step of determining cubic weighting values comprises determining the sum of the absolute differences between the cubic interpolation model and the image, and inverting the sum into arrays of cubic weighting values, W_Cube- The method 80 also includes determining, at block 90, a weighted artifact error. In particular, the method 80 includes, for example, multiplying the linear butting artifact estimates, Au_n, for each row by the linear weighting values or factors W_Liπ, by way of the weighting module 34, and accumulating these products, accAWϋn, in the scan direction; multiplying the cubic butting artifact estimates, A_Cube, for each row by the cubic weighting values or factors W_Cube and accumulating these products, ace AWCu be, in the scan direction; and also accumulating the weighting values Wu_n and Wc_ube separately as accWLin and accWCube respectively.

The method 80 also includes determining, by using footprint indices for each row, the difference between the accumulated weighted linear butting artifact estimates, accAWLin, separated by the foot print size from high to low indices; and dividing the determined difference by a corresponding difference in accumulated linear weighting factors, accWLin, to yield a local linear butting artifact estimate.

Similarly, the method 80 also comprises determining, by using footprint indices for each row, the difference between the accumulated weighted cubic butting artifact estimates, accAWCube, separated by the foot print size from high to low indices; and dividing the determined difference by a corresponding difference in accumulated cubic weighting factors, accWCube, to yield a local cubic butting artifact estimate. This can be better understood by way of the following example equation for yielding the local cubic butting artifact estimate:

, , _ , _{r Π} (αccArtWCube[αccFootHigh[row]] - αccArtWCube[αccFooLov{row]])

LocαlArtCube[mw] = ^ — -. ²^ r-^

\αccWCube[αccFootHigh[row]] - αccWCube[αccFooLow{row]])

It will be appreciated that, despite the symbols denoting certain elements not being identical, the above equation corresponds to the equation hereinbefore described. It follows that the method includes determining an average of the local linear and cubic butting artifact estimates to obtain a final or best artifact estimate.

The method 80 therefore includes correcting, at block 92, the image by applying the subtraction algorithm to subtract the final or best artifact estimate from the image thereby effectively to remove the butting artifact from the image.

The method and system as herein before described provide a convenient and effective way to remove butting or stitching artifacts from images. The described algorithm can also be used to extrapolate data when missing data is present, such as photograph and image restoration when scratches are present.

Claims

1. A method for removing butting or stitching artifacts from an image, the method including:

2. A method according to claim 1 wherein the model includes at least a shape and an amplitude of the butting artifact.

3. A method according to claim 1 or claim 2 wherein the electromagnetic absorption level is an x-ray absorption level.

4. A method according to any one of claims 1 to 3 wherein, as part of determining a model for the butting artifact, the method comprises:

5. A method according to any one of claims 1 to 4, further including:

determining a blurring kernel footprint size and associating this with each pixel row index for the butting artifact;

determining linear butting artifact estimates, Au_n, from differences between the linear interpolation of the butting artifact and the original image;

6. A method according to claim 5 wherein the step of determining linear weighting values comprises determining the sum of the absolute differences between the linear interpolation model and the image, and inverting the sum into arrays of linear weighting values, W_Lin.

7. A method according to claim 5 or claim 6 wherein the step of determining cubic weighting values comprises determining the sum of the absolute differences between the cubic interpolation model and the image, and inverting the sum into arrays of cubic weighting values, W_Cuhe-

8. A method according to any one of claims 5 to 7, further including: multiplying the linear butting artifact estimates, A_Lin, for each row by the linear weighting values or factors W_Un and accumulating these products, accAWLin, in a scan direction;

multiplying the cubic butting artifact estimates, Ac_ube, for each row by the cubic weighting values or factors Wcute and accumulating these products, ace AWCu be, in the scan direction; and

accumulating the weighting values W_Un and W_Cube separately as accWLin and accWCube respectively.

9. A method according to any one of claims 5 to 8, including:

10. A method according to any one of claims 5 to 9, including:

11. A method according to any one of claims 5 to 10 including determining an average of the local linear and cubic butting artifact estimates to obtain a final artifact estimate.

12. A method according to claim 11 wherein the butting artifact subtracted by the subtraction algorithm is the final artifact estimate.

13. A system for removing butting or stitching artifacts from an image, the system including:

14. A system according to claim 13 wherein the modeling module is arranged to apply a linear interpolation algorithm to the butting artifact in the image to obtain a linear interpolation of the butting artifact or a linear interpolation model.

15. A system according to claim 13 or claim 14 wherein the modeling module is arranged to apply a cubic interpolation algorithm to the butting artifact in the image to obtain a cubic interpolation of the butting artifact or a cubic interpolation model.