JP2000342558A - Image positioning processor and inter-picture arithmetic processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently, stably and automatically process a precise positioning though there is relatively large positional deviation between the radiation images of a human body. SOLUTION: This positioning processor 1 processes the positioning between plural images including a common subject. The processor 1 has a partial area setting means 2 for setting plural partial areas in at least one image among the plural images, a partial similarity deciding means 3 for deciding similarity to a corresponding area in another image by each set partial area, a whole area similarity deciding means 4 for deciding whole area similarity by weight- adding decided partial area similarity through the use of weighting coefficient set to each partial area, an image converting condition deciding means 6 for deciding an image converting condition for positioning one image to the other image based on decided whole area similarity, and an image converting means 7 for positioning an image by converting the images based on the decided converting condition.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image alignment processing apparatus and an inter-image arithmetic processing apparatus, and more particularly to a technique for accurately and efficiently adjusting a relative displacement of a subject between a plurality of images.

[0002]

2. Description of the Related Art For example, radiographic images such as X-ray images are widely used for diagnosing diseases and the like. In order to obtain such X-ray images, X-rays transmitted through a subject are applied to a phosphor layer (fluorescent screen). ), Thereby producing visible light, and irradiating the visible light to a film using a silver salt in the same manner as in a normal photograph and developing the film.

However, in recent years, a method has been devised for directly taking out an image from a phosphor layer without using a film coated with a silver salt. This includes:
The radiation that has passed through the subject is absorbed by the phosphor, and then the phosphor is excited by, for example, light or heat energy to emit the radiation energy that the phosphor has accumulated due to the absorption as fluorescence. There is a method in which fluorescence is photoelectrically converted and A / D converted to obtain a digital image signal (see U.S. Pat. No. 3,859,527, JP-A-55-12144).

The thus obtained radiation image signal is output as it is or after being subjected to image processing to a silver halide film, a CRT or the like, and is visualized. Also,
A method of irradiating a silver halide film on which a radiation image is recorded with light from a light source such as a laser or a fluorescent lamp to obtain transmitted light of the silver halide film, and photoelectrically converting the transmitted light to obtain a radiation image signal. is there.

[0005] On the other hand, conventionally, there has been known a positioning process for adjusting a relative positional shift of a subject between a plurality of images including a common subject portion based on a template matching method. That is, the center of the template t (x, y) representing the target to be detected is overlapped with the point (i, j) in the image f (x, y), and t (x, y) is set.
This is a method of measuring the similarity between y) and the partial pattern of the image that overlaps with y), and using the value as a probability that the target exists at the point (i, j). In order to determine the position of the object, there is a method in which this operation is performed on all points of the image to find a position where the similarity is maximum, or to search for a position where the similarity exceeds a certain threshold value ( Introduction to Computer Image Processing, edited by Japan Industrial Technology Center, April 1, 1991, 1st edition, 9th printing, page 149).

Here, two images including a common subject portion are used, a partial image cut out from one image is regarded as a template t, and the other image is regarded as an image f, and the above-described processing is performed. Registration between images can be performed.

[0007]

By the way, the template matching method as a method of aligning two images uses a complicated template such as an X-ray image of a human body when a small template is used for the entire image. It is difficult to accurately position the entire image.

For example, between two chest images of the same patient taken at different times, various changes are included in addition to the pathological changes to be diagnosed. In the area, there is a reflection of an object other than the human body such as a shoulder rest and an ID information label, and the reproducibility of image information is extremely low because the posture of the subject's arm is not constant. With a large template,
There is also a problem that such low reproducibility regions are included in the template and the positioning error increases.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and performs an efficient and stable automatic alignment process even if there is a relatively large displacement between radiation images of a human body. It is an object of the present invention to provide an image registration processing device and an image-to-image processing device that enable processing.

[0010]

Means for Solving the Problems In order to solve the above problems, the present invention is configured as follows.

According to a first aspect of the present invention, there is provided an image alignment processing apparatus for performing an alignment process between a plurality of images including a common subject, wherein at least one of the plurality of images includes A partial area setting means for setting a plurality of partial areas; a partial area similarity determining means for determining a similarity between corresponding areas in other images for each of the set partial areas; The area similarity is weighted and added using a weighting factor set for each partial area, so that the entire area similarity determining means for determining the overall area similarity is determined based on the determined overall area similarity. Image conversion condition determining means for determining an image conversion condition for aligning one image with another image, and performing image conversion based on the determined image conversion condition, thereby performing image alignment. Positioning processing apparatus for an image, characterized in that it comprises a conversion means. ].

According to the first aspect of the present invention, a plurality of partial areas are set in at least one of the plurality of images, and each of the set partial areas corresponds to a corresponding one in another image. The overall area similarity is determined by determining the similarity between the area and the determined partial area similarity by weighting and adding using the weighting factor set for each partial area. Based on the overall area similarity determined, an image conversion condition for aligning one image with another image is determined, and image conversion is performed based on the determined image conversion condition, thereby obtaining a radiation image of a human body. Even if there is a relative misalignment between them, the process of positioning with high accuracy can be performed efficiently and stably automatically.

According to a second aspect of the present invention, there is provided a method for extracting an anatomical structure, wherein the common subject is a part of a human body and an anatomical structure of the human body is extracted from at least one of the plurality of images. 2. The apparatus according to claim 1, wherein said partial area setting means sets said plurality of partial areas based on an extraction result of said anatomical structure extracting means.
An image alignment processing apparatus according to claim 1. ].

According to the second aspect of the present invention, the common subject is a part of the human body, and the anatomical structure of the human body is extracted from at least one of a plurality of images, and the extraction is performed based on the extraction result. By setting multiple partial areas
Even if there is a relative displacement between radiation images of the human body,
It is possible to efficiently and stably automatically perform the process of positioning with high accuracy.

According to a third aspect of the present invention, there is provided an image processing apparatus, comprising: an attention position designating means for designating an attention position in at least one of the plurality of images; 3. The apparatus according to claim 1, wherein an area is set. ].

According to the third aspect of the present invention, the attention position is designated in at least one of the plurality of images, and the plurality of partial regions are set based on the designated attention position. By focusing on the position, it is possible to more accurately align the displacement.

According to a fourth aspect of the present invention, there is provided an image processing apparatus, comprising: an attention position designation means for designating an attention position in at least one of the plurality of images;
4. The weighting factor according to claim 1, wherein the weighting factor is set such that a weighting factor corresponding to a partial area including the designated target position is larger than a weighting factor corresponding to another partial area. An image alignment processing apparatus according to any one of the preceding claims. ].

According to the fourth aspect of the present invention, a position of interest is designated in at least one of the plurality of images, and a plurality of partial regions correspond to a partial region including the designated position of interest. By setting the weighting factor such that the weighting factor to be performed is larger than the weighting factor corresponding to the other partial area, it is possible to focus on the target position and more accurately perform the positional displacement.

According to a fifth aspect of the present invention, there is provided an image processing apparatus, comprising: "the image conversion means performs image conversion based on the determined image conversion condition, thereby performing fine position adjustment after coarse alignment for image alignment. The image alignment processing apparatus according to claim 1, wherein image alignment is performed in two stages of alignment. ].

According to the fifth aspect of the present invention, the accuracy of the alignment is improved in a relatively short processing time by performing the two-stage image alignment of the fine alignment after the coarse alignment. Can be.

According to a sixth aspect of the present invention, there is provided an image display apparatus for simultaneously displaying a plurality of images that have been aligned by the image alignment processing apparatus according to any one of the first to fifth aspects. An image alignment processing apparatus comprising: ].

According to the sixth aspect of the present invention, since a plurality of images that have been aligned by the image alignment processing device are displayed at the same time, diagnosis based on comparative image interpretation can be easily performed using images without positional deviation. Accurate and fast.

According to a seventh aspect of the present invention, there is provided an image processing apparatus for performing an operation between a plurality of images that have been aligned by the image alignment processing apparatus according to any one of the first to fifth aspects. An inter-image calculation processing device comprising an inter-calculation means. ].

According to the seventh aspect of the present invention, since calculation is performed between a plurality of images that have been aligned by the image alignment processing apparatus, fine alignment can be performed while reducing the calculation load. Accuracy is ensured.

According to an eighth aspect of the present invention, there is provided an image processing apparatus for performing an operation between a plurality of images that have been aligned by the image alignment processing apparatus according to any one of the first to fifth aspects. An inter-image calculation processing device comprising: inter-calculation means; and image display means for displaying a plurality of images having undergone alignment or an operation-processed image generated by inter-image calculation. ].

According to the eighth aspect of the present invention, since a plurality of aligned images or an operation-processed image generated by an inter-image operation is displayed, a comparative image interpretation and an image can be obtained by using an image having no positional deviation. Diagnosis based on the inter-operation result can be performed easily, accurately, and quickly.

[0027]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of an image registration processing apparatus and an inter-image calculation processing apparatus according to the present invention; however, the present invention is not limited to this embodiment.

FIG. 1 is a schematic configuration diagram of an image registration processing apparatus. The image alignment processing apparatus 1 is also an image-to-image arithmetic processing apparatus that performs image alignment for performing alignment processing between a plurality of images including a common subject.
The image registration processing apparatus 1 includes a partial area setting means 2, a partial area similarity determining means 3, an entire area similarity determining means 4, a weighting coefficient storage means 5, an image conversion condition determining means 6, and an image converting means 7. , An image storage means 8, an image display control means 9 and an image display means 10.

The partial area setting means 2 sets a plurality of partial areas in at least one of the plurality of images, in this embodiment, an image obtained from the image data 1. In the partial area similarity determination means 3, the image data 2 in another image, in this embodiment,
And the overall area similarity determining means 4 uses the determined partial area similarity by using a weighting factor set for each partial area. Then, the overall area similarity is determined by performing weighted addition. The weight coefficient is stored in the weight coefficient storage means 5 in advance. The image conversion condition determining means 6 determines an image conversion condition for aligning one image with another image based on the determined overall area similarity, and the image conversion means 7 determines the determined image conversion condition. The position of the image is adjusted by performing the image conversion based on.

Further, in this embodiment, the image conversion means 7 performs image conversion based on the determined image conversion conditions, so that after the coarse alignment for aligning the image, the fine alignment is performed in two stages. Can be performed, and the positioning accuracy can be improved in a relatively short processing time.

The image storage unit 8 stores an image obtained from the image data 2 and an image aligned by performing image conversion based on the determined image conversion condition. Is displayed on the image display means 10 on the basis of.

As described above, a plurality of partial areas are set in at least one of the plurality of images, and the degree of similarity between each of the set partial areas and a corresponding area in another image is determined. Then, the determined partial area similarity is weighted and added using a weighting factor set for each partial area to determine the overall area similarity, and based on the determined overall area similarity, By determining image conversion conditions for aligning one image with another image and performing image conversion based on the determined image conversion conditions, there is a relative displacement between radiographic images of a human body. However, it is possible to automatically and stably automatically perform the process of accurately positioning such a position shift.

Here, the input of the image data 1 and the image data 2 is performed, for example, by taking a radiographic image usually using an X-ray film. A laser digitizer is used to input these X-ray photographs to the image registration processing apparatus of this embodiment. In this method, a film is scanned with a laser beam, the amount of transmitted light is measured, and the value is converted from analog to digital to be input as digital image data.

For inputting an image, a device using an optical sensor such as a CCD can be used. Instead of reading a film, it is also possible to connect a photographing device capable of directly outputting a digital image using a stimulable phosphor as described in JP-A-55-12429. . In this case, a film is not required, and cost can be reduced.

Further, an X-ray image obtained from a flat panel detector (FPD) that captures an X-ray image with a plurality of two-dimensionally arranged detection elements and outputs the image as an electric signal can be input. For example, JP-A-6-34209
As described in Japanese Patent Application Publication No. 8 (1996) -1994, there is a method in which a photoconductive layer that generates electric charges according to the intensity of irradiated X-rays and a method that accumulates the generated electric charges in a plurality of two-dimensionally arranged capacitors. Used. Further, as described in JP-A-9-90048, a photodiode or the like in which X-rays are absorbed by a phosphor layer such as an intensifying screen to generate fluorescence and the intensity of the fluorescence is provided for each pixel. A method of detecting with a photodetector is also used. Other means for detecting fluorescence include CCD and CM.
There is also a method using an OS sensor. Further, a configuration in which an X-ray scintillator that emits visible light by X-ray irradiation, a lens array, and an area sensor corresponding to each lens is also used.

The image registration processing apparatus is used, for example, for registration between two chest X-ray images (time-sequential chest images) of the same patient photographed at different times, so that the fluctuation of the photographing conditions can be improved. And a highly accurate alignment process that is not easily affected by the body shape of the subject.
In the following, for two chest X-ray images (past and present images), the image portion corresponding to the most important lung field for diagnosis is emphasized, and at the same time, the background portion and the respiration / An image registration process based on the present invention using a weighted template matching method that reduces the weight of an image portion that is likely to change based on a heartbeat or the like will be described.

For example, the change between the time-series chest images of the same patient is not limited to the pathological change to be diagnosed.
Various changes are included. Generally, in the area outside the rib cage of the chest X-ray image, objects other than the human body, such as shoulder pads and ID information labels, are reflected, and the posture of the subject's arm is not constant. Very low. Looking inside the thorax, changes in the diaphragm height due to the inspiratory state of the subject and changes in the width of the heart due to the heartbeat phase can frequently occur. In addition, a change in the collarbone angle due to how the arm of the subject is raised may be observed.

Based on the above findings, the chest image is divided into partial areas A _{1 to} A ₅ as shown in FIG. 5 based on the following points (i) to (iv).

(I) Region with little change other than pathological change: middle lung (partial region A ₁ ) (ii) Region with partial change: upper lung (partial region A
₂ ), lower lung margin (partial area A ₃ ) (iii) Changeable area in most cases: near lower lung mediastinum (partial area A ₄ ) (iv) Area with almost no reproducibility: extrathoracic ( Partial region A ₅ ) The method of defining the partial region is as follows: draw a curve R ′ obtained by expanding the lung field contour R by a predetermined width in the left, right, and upward directions, and draw a straight line B connecting the lower ends of the lung field contours. an outer closed curve surrounded by B and the partial area a _5. Here, the lung field contour line R may be automatically determined using an anatomical structure extraction unit described later, or may be manually specified by a user.

The distance between the upper end line T and B of R 'is 1 from the top.
/ 5 and 2/3 in the horizontal direction, the uppermost partial area surrounded by R ′ is A ₂ , and the lower partial area is A ₂
Let it be 1. Further, the lower part of A ₁ is divided into three by １ ／ and ／ columns from the left of the section sandwiched by R ′ from left and right, the central partial area is A ₄ , and the partial areas on both sides are A ₃ And

Next, a weighted template matching method is used in which weighting is performed such that the contribution rate to the position adjustment becomes smaller in the order of the above classifications (i), (ii), (iii), and (iv). Align the images.

Using the past image as a template and the current image as a reference image, template matching is performed by translation and rotation of the template. Here, the similarity S, which is an index of the quality of the matching, is defined by Expression 1.

[0043]

(Equation 1)

Here, C _i is a normalized cross-correlation value between the partial area A _i (i = 1, 2,..., N) in the template and the corresponding area B _i in the reference image. Represent. _Wi
Is a weighting factor assigned to the region A _i , w _i (i =
1, 2,..., N) is 1.0. In this embodiment, the division number N of the area is set to 5, and the weighting factors w _i are respectively w ₁ = 0.445, w ₂ = 0.222, w ₃ = 0.
222, w ₄ = 0.111, w ₅ = 0. Normalized cross-correlation value, a pixel value of the j-th pixel in A _i A _i (j),
B pixel value of the j-th pixel in B _i the corresponding
_When expressed as _i (j), it is expressed by Expression 2.

[0045]

(Equation 2)

As described above, the average value m of the pixel values in each area is
By using cross-correlation values normalized using _Ai , m _Bi and standard deviations σ _Ai , σ _BI , differences in average density and gradation caused by differences in exposure conditions between two X-ray images can be reduced. Matching can be performed without being affected.

Based on the amount of translation and rotation of the template when S takes the maximum value, the entire past image is translated and rotated to align the images.

The partial area similarity is not limited to the above-described normalized cross-correlation value. For example, a normal cross-correlation value or a correlation value (Medica) of a phase term of a Fourier transform
Il Imaging Technology, Vol.
7, pp175-176, 1989) may be used as the partial area similarity.

In this embodiment, the image conversion is performed by translation and rotation. However, the invention is not limited to this, and the image conversion may be performed by using a linear conversion composed of a combination of rotation, enlargement / reduction, and translation. Good. Assuming that the x and y coordinates before the conversion are (x, y), the x and y coordinates after the conversion are (x ′, y ′), and the coefficients are a _ij and b _ij , the linear conversion of the two-dimensional image is expressed by Equation 3 It is expressed as

[0050]

(Equation 3)

Here, the following methods are used to determine the coefficients a _ij and b _ij in the equation (3). While changing the values of a _ij and b _ij little by little within a predetermined range,
, The past image is subjected to image conversion, and the similarity S is calculated using the converted image as a template and the current image as a reference image. The coefficient a _ij when S takes the maximum value,
The combination of b _ij is determined as an image conversion condition for positioning.

Here, in the image conversion,
Non-linear conversion may be used. That is, in the non-linear conversion, it is possible to distort an image, and thus it is possible to accurately position a complicated displacement of a human body. In the non-linear conversion of a two-dimensional image, for example, the coordinates before the conversion are (x,
y), assuming that the coordinates after the conversion are (x ′, y ′), the coordinates are expressed by a two-dimensional polynomial conversion represented by the following Expression 4.

[0053]

(Equation 4)

Here, the degree n of the polynomial is preferably 2 or more and 6 or less. The coefficients a _ij and b _ij in Equation 4 are determined in the same manner as in the case of the linear conversion. As described above, a plurality of partial areas are set in at least one of the plurality of images, and each of the set partial areas is set. In addition, the similarity between the corresponding region in another image is determined, and the determined partial region similarity is weighted and added using a weighting factor set for each partial region, so that the overall Determine the area similarity, determine an image conversion condition for aligning one image with the other image based on the determined overall area similarity, and perform image conversion based on the determined image conversion condition. By doing so, even if there is a relative displacement between the images in the chest image, it is possible to automatically and stably automatically perform the process of accurately positioning the structure of the lung field important for diagnosis.

Further, in this embodiment, by performing image conversion based on the determined image conversion conditions described above, after coarse positioning for performing image positioning, two-step image positioning for fine positioning is performed. Perform alignment.

Next, the fine positioning process performed after the rough positioning will be described. In the fine registration process, the image conversion is performed so that the structure of the common subject portion between the images is relatively finely adjusted based on the image data that has been subjected to the coarse registration process for performing rough positioning of the entire image. Expression to perform or image transformation
Determine the coefficients and the like.

Here, in the case of performing two-stage image alignment including coarse alignment and fine alignment, a reduced image in which the number of pixels is reduced by averaging thinning processing or the like is used in the coarse alignment stage. However, it is preferable because the amount of calculation is small and the processing speed is high. Further, the image used in the coarse registration stage may be an image that has been smoothed using an averaging filter, a median filter, a Gaussian filter, or the like. This is because it is only necessary to use the global features of the image, that is, only the low spatial frequency components, in the coarse alignment, and sufficient accuracy can be obtained with a reduced image or a smoothed image.

It is preferable to use linear conversion or conversion by a combination of translation and rotation as the image conversion in the coarse positioning. This is because it is only necessary to roughly match a large structure in an image in the coarse registration, and by using these image conversions, the amount of calculation can be reduced and the processing speed can be improved. On the other hand, it is preferable to use a non-linear conversion as the image conversion in the fine registration. In other words, the nonlinear transformation can distort the image, so it is possible to correct local misalignment that was not completely adjusted by coarse alignment, and accurately align complex misalignment of the human body. It is.

In the non-linear conversion of a two-dimensional image, for example, if the coordinates before the conversion are (x, y) and the coordinates after the conversion are (x ′, y ′), the two-dimensional polynomial conversion shown in the above equation (4) is used. expressed. Here, the degree n of the polynomial is preferably 4 or more and 10 or less. Further, in a substantially symmetrical human body part such as a chest, since the positional shift of the human body structure is also substantially symmetric, it is preferable to use an even function in which the order n of the polynomial is an even number.

In determining the coefficients (a _ij , b _ij in Equation 4) in the non-linear conversion, the coefficients are determined by integrating a plurality of movement amounts obtained by using a local matching method at a plurality of points. And good. With local matching, a small region of interest is selected in one image, and based on the structure in this region of interest, a region where the structure best matches near the corresponding position in the other image is searched, The movement amount corresponding to the center point of the region of interest is obtained based on the search result.

When the image conversion condition is represented by the two-dimensional polynomial of the above equation 4, as a specific method of determining the coefficients in the non-linear conversion, a number of regions of interest are obtained by local matching (x ', y'). Is used as a sample value, coefficients a _ij and b _ij are determined by approximation using the least square method or the like. The approximation may be performed independently in the x and y directions,
This may be performed using a distance in a two-dimensional plane.

As another method, (x ′, y ′) obtained by local matching is used as a sample value, and the space between samples is filled by interpolation processing such as linear interpolation or spline interpolation which is known in the art. x ′, y ′). These two types of methods may be used together to determine the final image conversion condition by combining the two results.

By repeatedly executing the fine positioning process based on the local matching two or more times,
A configuration for performing finer alignment may be employed. In this case, the size and setting position of the region of interest for local matching, the order of the image conversion formula, the coefficient determination method, and the like may be always the same in two or more fine alignment processes,
A different one may be used each time.

As a specific example of the fine positioning process based on the local matching, in the chest radiographic image, the contour of the lung field region is extracted, and a large number of points are matrixed over the entire lung field region of each of the current image and the past image. Place Next, a region of interest having a certain size is set around each point. At this time, the size of the region of interest in one image is set to be larger than the size of the region of interest in the other image, and the smaller region of interest is called a template and the larger region of interest is called a search region. As the shape of the region of interest, a square or another rectangle, a circle, a cross, or the like can be used.

Then, as shown in FIG. 6, by using a pair of the template and the search area at the corresponding position, the sub-area included in the search area and having the same shape as the template T is gradually moved. Search for a sub-region T ′ most similar to the template T. When the sub-region T ′ is searched, the moving amounts Δx and Δ with respect to the center coordinates (x, y) of the template T based on the center coordinates (x ′, y ′).
y is calculated based on the relations x ′ = x + Δy and y ′ = y + Δy. Here, as means for evaluating how similar a certain sub-region is to the template (similarity), a known SSDA method, a cross-correlation method, a Fourier transform phase correlation method, or the like can be used.

Based on the distributions of the movement amounts Δx and Δy determined for the plurality of templates, an image conversion expression represented by an n-th order polynomial as shown in Expression 4 is determined, and one of the image conversion expressions is determined based on the conversion expression. By performing image conversion,
Perform image alignment.

When determining the coefficients of the n-th order polynomial in Equation 4 from the distributions of the movement amounts Δx and Δy, the value of the movement amount with respect to the center point of all the templates subjected to local matching may be used. It is preferable to exclude values obtained from relatively poorly matching templates, as shown below.

For example, in the case of two time-series images, when a unique shadow due to a medical device such as a pacemaker is present in a part of only one of the images, a template including the unique shadow corresponds to the other image in the other image. Since no shading exists anywhere, the obtained movement amount has a low reliability value, and the similarity at that time is low. If such values are mixed in data for polynomial approximation or interpolation processing, it will adversely affect nearby data, and as a result, positioning accuracy will be reduced. Therefore, it is preferable that such a moving amount having poor matching is recognized and excluded from the calculation of the polynomial approximation or the interpolation processing.

In order to recognize a moving amount having poor matching,
For example, it is only necessary to select one having a similarity smaller than a predetermined threshold value when the movement amount is obtained. Alternatively, a value whose value is uniquely larger or smaller than the amount of movement obtained from an adjacent template may be selected. Further, the contribution of the movement amount to the calculation of the polynomial approximation may be defined as a weight coefficient represented by a value of 0 or more and 1 or less, and weighting may be performed according to the similarity. For example, when a cross-correlation value is used as the degree of similarity, it is preferable that the weighting coefficient be determined in advance so that the cross-correlation value increases. FIG. 7 shows an example of the relationship between the cross-correlation value and the weight coefficient.

As another method for recognizing a moving amount having poor matching, a moving vector (.DELTA.x, .DELTA.x) in a two-dimensional plane having moving amounts .DELTA.x and .DELTA.y as vector elements.
Δy) may be defined for all template center points and recognized based on vector length or direction. For example, a moving vector whose length or direction is significantly different from that of a moving vector obtained from an adjacent template may be selected. Alternatively, by statistically analyzing the directions (angles) of the movement vectors with respect to a plurality of template center points included in the whole or a part of the image, an image having an angle that is at least a certain distance from the average angle may be selected.

By the way, when the size of the template in the local matching is too small, the accuracy of the positioning is deteriorated. On the other hand, when the size of the template is too large, the calculation time is prolonged. It is desirable to determine the size of the template in consideration of the alignment accuracy and the calculation time. For example, when the effective pixel size is 0.7 mm, the size of one side of the template region is preferably 5 to 50 mm, and 1
More preferably, it is 5 to 40 mm. Similarly, if the size of the search region is too small, the positioning accuracy will decrease, while if the size of the search region is too large, the calculation time will increase and even the portion that changes over time will be aligned. Because of the possibility, it is preferable to set the size larger by about 10 to 40 mm than the size of the template. Further, if the distance between the center points of adjacent templates is too small, the total number of templates increases, so that the calculation time increases.On the other hand, if the distance between the center points is too large, the alignment accuracy decreases. It is preferable to set it to 5 to 50 mm.
Note that adjacent templates may overlap with each other. In addition, when the positioning is performed by a combination of translation and rotation in the coarse positioning process, the local matching in the fine positioning process causes the search to be performed only by the parallel movement of the sub-region in the search region. Is preferred. With such a configuration, the rotational displacement of the entire image can be corrected by one-stage processing, and the processing speed can be increased.

As described above, in the local matching processing of the present invention, rough alignment of the entire image has been completed in the coarse alignment. Therefore, in the fine alignment processing, only a small local distortion needs to be corrected. By limiting the size of the search area to a certain range and performing a search only by parallel movement, a highly accurate position can be obtained in a relatively short processing time despite the two-step alignment processing. Matching is possible.

FIG. 2 is a schematic configuration diagram of an image registration processing apparatus. In the image registration processing apparatus 1 of this embodiment, the same components as those of the image registration processing apparatus of FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. An anatomical structure extracting means for extracting an anatomical structure of the human body from at least one of a plurality of images, the partial region setting means 12 being a part of a human body; A plurality of partial areas are set based on the extraction result in 20.

As described above, the common subject is a part of the human body, the anatomical structure of the human body is extracted from at least one of a plurality of images, and a plurality of partial regions are set based on the extraction result. Thereby, even if there is a relative displacement between the radiation images of the human body, it is possible to automatically and stably automatically perform the process of positioning with high accuracy.

Here, the extraction of the anatomical structure (anatomical structure extracting means) will be described. In the case of a radiographic image formed according to the amount of radiation transmitted through the human body, the signal level locally changes in accordance with the anatomical structure of the human body. Can be extracted.

For example, in the breast radiation image shown in FIG. 8, the skin line L1 is detected, and the nipple position P1, which is recognized as the point closest to the right end of the skin line L1, is detected. The partial areas A ₁ , A ₂ , and A ₃ are set based on concentric circles centered at. The technique of detecting skin lines from breast images is described in the cited reference, Journal of Medical Image Information Society, Vol. 14, pp. 104-11
3, 19972.

Further, the extraction of the translocation of the lung field in the chest radiographic image can be performed using, for example, a method disclosed in Japanese Patent Application Laid-Open No. 63-240832. Specifically, attention is paid only to one row or column of the image data, and a point in the one-dimensional image data sequence where the relationship with the data before and after the specific pattern is a predetermined pattern is defined as the row or column. And the line connecting these points is determined as the contour of the lung field, and the specific pattern has a minimum point and a maximum slope. , The point at which the inclination is minimum, and the like.

As a method of extracting a rectangular area including a lung field in a chest radiographic image, see, for example,
There is a method disclosed in Japanese Patent No. 18578. Specifically, a projection value (a cumulative value in one direction of image data) is obtained in the vertical direction of the image. The point at which the projection value becomes the minimum value is defined as the median line, and the projection value and the predetermined threshold value are compared while moving outward from the median line. The left and right points are determined as the left and right ends of the lung field. Similarly, a projection value is determined for the horizontal direction of the image, and the upper and lower ends of the lung field are determined.

As a method for extracting a lung field contour and a rib position in a chest radiographic image, Japanese Patent Laid-Open No. 2-2501 is disclosed.
No. 80 is disclosed. In this method, lung field contours and rib positions are extracted based on vertical and horizontal profile information. In particular, in the extraction of ribs, the influence of the background is eliminated by polynomial approximation.

On the other hand, a method disclosed in Japanese Patent Application Laid-Open No. 4-341246 is a method for extracting bones such as lumbar vertebrae, iliac bones, and pelvis as anatomical structures from a radiation image of the abdomen. For example, in order to extract the iliac part in an abdominal radiation image, a profile indicating a signal change in the lateral direction of the image is created, and the iliac part is surrounded based on the number and position of the parts having the minimum value in the profile. The upper and lower line segments are obtained, and the iliac region is extracted. Also, a boundary signal value between the bone and the part other than the bone is obtained from a histogram or the like, and the boundary signal is used as a threshold value to binarize the image signal. It is possible to extract the bone region by dividing the region into the region and the other region.

FIG. 3 is a schematic configuration diagram of an image registration processing apparatus. In the image registration processing device 1 of this embodiment, the same components as those of the image registration processing device of FIG. At least one attention position designation means 30 for designating a notice position in at least one image is provided, and a plurality of partial areas are set based on the designated notice position. In addition, a weighting factor is set such that a weighting factor corresponding to a partial region including a designated target position among a plurality of partial regions is larger than a weighting factor corresponding to another partial region.

In this embodiment, the target position is designated by referring to FIG.
As shown in (1), while observing the image displayed on the image display means 10, the user specifies a target position P2 using a pointing device such as a mouse, and the partial regions A ₁ , A around the target position P2. _2, the a ₃ determines, determine the weighting factors such that _{W 1> W 2> W 3} . The weight coefficients are set to, for example, W ₁ = 0.7, W ₂ = 0.3, and W ₃ = 0.

As described above, the target position is specified in at least one of the plurality of images, and the plurality of partial regions are set based on the specified target position. Automatically and efficiently.

By setting the weighting factor so that the weighting factor corresponding to the partial region including the designated target position is larger than the weighting factor corresponding to the other partial region,
The process of accurately aligning the position of the target position
Automatic processing can be performed efficiently and stably.

An image display means 10 for simultaneously displaying a plurality of images which have been aligned by the image alignment processing apparatus of the embodiment shown in FIGS. 1 to 4 is displayed as shown in FIG. .

The image display control means 9 processes the image data to be output to the image display means 10 in order to display an image in accordance with a display format designated in advance. The processing of the image data includes gradation conversion such as enlargement / reduction processing for adjusting the image to a specified display size and window processing for adjusting the gradation of the image to the luminance characteristics of the display device. It is.

As the image display means, known image display means such as a CRT, a liquid crystal display, and a plasma display can be used. Among them, a high-definition and high-brightness CRT or a liquid crystal display dedicated to medical images is most preferable. High definition displays having a number of about 1000 × 1000 or more are preferred.

FIG. 10A shows the chest X-ray images of the same patient taken at different times side by side, and FIG. 10B shows the breast X-ray images of the same patient taken at different times in the same direction. 10C shows an example in which left and right breast X-ray images of the same patient photographed by the same examination are displayed side by side. In the case of FIG. 10 (c), the right and left images have opposite nipple directions. Therefore, one of the images is flipped left and right, the alignment of the present invention is performed, and the flipping is reversed. Perform the procedure.

Since a plurality of images that have been aligned by the image alignment processing apparatus are displayed at the same time,
When the interpreting doctor interprets the past image and the current image of the same patient while comparing them (comparative image reading), or when performing the image interpretation while comparing the left and right breasts of the same patient, it is necessary to compare the corresponding image parts. It can be done easily, accurately and quickly. This makes it possible to more accurately detect, for example, the lesion shadow X existing only in one of the two displayed images. In addition, for example, it becomes possible to more accurately diagnose a lesion shadow in which a disease state has progressed between a past image and a current image or in which a pathological condition has been improved.

The image registration processing apparatus of the embodiment shown in FIG. 4 has an inter-image calculation means 40 for performing an arithmetic operation between a plurality of aligned images. An operation is performed between a plurality of aligned images.

[0091] The image processing is given to the inter-image calculation (difference processing) shown by the inter-image calculation means 40, and image processing corresponding to the same anatomical structure is aligned between the time-series images to perform the difference processing. Thus, a difference image (time-series processed image) is generated in which the normal structure portion that does not change over time is canceled out and the portion that changes over time is selectively emphasized.

When performing two-step image alignment consisting of coarse alignment and fine alignment, at least one of the time-series images is image-converted based on the alignment information obtained by the coarse alignment processing. It is also possible to execute a fine alignment process between the time-series images subjected to the image conversion, or to generate a converted image obtained by performing the image conversion of the image data after obtaining the alignment information by the coarse alignment process. Instead, the fine positioning process may be executed based on the rough positioning information and the original image data. Similarly, at least one of the time-series images is image-converted based on the alignment information obtained by the fine alignment processing, and an inter-image calculation (difference processing) is performed based on the time-series image subjected to the image conversion. Alternatively, an inter-image operation may be performed based on the fine alignment information and the original image data without generating a converted image obtained by performing image conversion of the image data.

For example, the pixel values at the coordinates (x, y) of the past image before deformation and the past image after deformation are respectively represented by S
_p (x, y) and S _pw (x, y) and expressed, S _pw (x,
y) is determined by Equation 5.

[0094]

(Equation 5)

Here, Δx and Δy represent the amounts of movement with respect to the coordinates (x, y). Here, (x + Δx) or (y
When + Δy) is a non-integer, a virtual pixel value is determined by linear interpolation using the pixel values of the four nearest pixels. Finally, a difference image is obtained by calculating the difference between the pixel values of the corresponding pixels between the past image after deformation and the current image.

In the present invention, a density / tone correction process for adjusting the density / tone of the entire image to a standard density / tone characteristic may be performed before the position adjustment process. Specifically, a density / tone correction process as disclosed in US Pat. No. 5,224,177 can be used.
Alternatively, a method may be used in which an image is divided into a plurality of small areas and the pixel values of one image are corrected so that the statistical values of the pixel values in the corresponding small areas are equal. As the statistics, an average value, a variance value, or the like is used. Further, before performing the positioning process, a frequency emphasizing process for emphasizing or attenuating a specific spatial frequency component may be performed. This makes it possible to perform the alignment processing focusing on the structure including the emphasized frequency component or the frequency component that has not been attenuated. In addition, before performing the alignment processing, image processing may be performed to emphasize a specific structure such as a rib or a lung print using edge enhancement (Japanese Society of Radiological Technology, Vol. б0-б
8. 1999).

Further, the arithmetic processing image generated by the inter-image operation according to the present invention is subjected to gradation processing for changing the density and gradation and frequency emphasis processing for emphasizing or attenuating a specific spatial frequency component. Is also good. In addition, a process of masking or trimming an image part that is not required for image interpretation diagnosis, such as an image part outside the rib cage in a chest image, in an arithmetic processing image generated by an inter-image operation may be performed.

In the above-described embodiment, the example of the inter-image calculation for performing the difference processing between the radiation images of the same patient photographed at different times has been described. However, the present invention is not limited to this. From the radiation image in which the specific structure is emphasized, an image-to-image calculation process for extracting the specific structure by subtracting a radiation image in which a contrast agent is not injected, or radiation having a different energy distribution with respect to the same subject Or irradiating the radiation after passing through the subject to the two radiation detecting means while changing the energy distribution, and then performing a subtraction process after appropriately weighting the image signals of the two radiation images. You may apply to the energy subtraction process which extracts a specific structure.

The image display means 10 displays a plurality of aligned images or an arithmetically processed image generated by an inter-image operation. This display shows chest X-ray images C1 and C1 of the same patient taken at different times as shown in FIG.
C2 and their difference images C3 are displayed side by side.

In the difference image C3, since the difference is calculated between the aligned C1 and C2, the normal structure part of the chest such as bones and blood vessels which does not change between C1 and C2 is canceled. At the same time, the newly generated lesion shadow X is emphasized, so that the interpreting doctor can detect the lesion shadow more accurately. In addition, for example, it is possible to more accurately diagnose a lesion shadow in which a disease state progresses or is improved between C1 and C2, and this helps a quantitative diagnosis of a change amount.

Although FIGS. 10 and 11 show an example in which a plurality of images are simultaneously displayed on the same display screen, a configuration having image display means including a plurality of display screens may be employed. For example, the image display means has two display screens, a display screen 1 and a display screen 2, and has a configuration in which C 1 and C 2 in FIG. 11 are displayed side by side on the display screen 1 and C 3 is displayed on the display screen 2. There may be.

Generally, when the number of pixels of an image does not match the number of pixels of a displayable area on a display screen, the image is enlarged / reduced.
For all of the plurality of images to which the registration processing is applied, the size of the subject included in the images is substantially equal (so that the enlargement / reduction ratio for a full-sized human body is the same between the images). It is preferable to determine the enlargement / reduction ratio. On the other hand, the operation-processed image generated by the inter-image operation like the difference image in FIG. 11 does not need to be displayed in the same size as a plurality of images to which the alignment processing is applied. The enlargement / reduction ratio may be determined so as to be smaller than two original images. This is because an arithmetically processed image such as a difference image is displayed for the purpose of diagnostic interpretation diagnosis, and the final diagnosis is performed based on observation of the original image.

The display screen of the image display means 10 may be configured to display patient information, examination information, finding information, etc. together with the image. By connecting an external device that outputs a hard copy of an image in addition to the image display unit 10, a hard copy of a plurality of images that have been aligned according to the present invention or an arithmetically processed image generated by an inter-image operation can be obtained. It is good also as a structure obtained. For example, image data is output to a scanning laser exposure device called a laser imager via an output interface. This scanning laser exposure apparatus modulates the laser beam intensity based on image data, exposes it to a conventional silver halide photographic material or a thermally developed silver halide photographic material, and then performs an appropriate development process. A hard copy of the image is obtained. Further, a configuration in which a hard copy of an image is obtained by an inkjet printer, a thermal printer, or the like may be adopted. The hard copy of the image is preferably configured to output patient information, examination information, finding information, and the like together with the image.

[0104]

As described above, according to the first aspect of the present invention, an image conversion condition for aligning one image with another image is determined based on the determined overall area similarity. By performing image conversion based on the determined image conversion conditions, even if there is a relative displacement between radiation images of a human body, the process of accurately positioning can be automatically and stably automatically performed. .

According to the second aspect of the present invention, the common subject is a part of the human body, the anatomical structure of the human body is extracted from at least one of the plurality of images, and the plurality of anatomical structures are extracted based on the extraction result. By setting the partial area, even if there is a relative displacement between the radiation images of the human body, it is possible to automatically and stably automatically perform the process of performing the alignment with high accuracy.

According to the third aspect of the present invention, the attention position is designated in at least one of the plurality of images, and the plurality of partial regions are set based on the designated attention position. By doing so, the misalignment can be more accurately aligned.

According to the fourth aspect of the present invention, a position of interest is specified in at least one of a plurality of images, and a weighting factor corresponding to a partial region including the specified position of interest among the plurality of partial regions is specified. Is set to be larger than the weight coefficients corresponding to the other partial regions,
By focusing on the target position, the positional deviation can be adjusted more accurately.

According to the fifth aspect of the present invention, the accuracy of the alignment can be improved in a relatively short processing time by performing the two-stage image alignment of the fine alignment after the coarse alignment.

According to the sixth aspect of the present invention, since a plurality of images that have been aligned by the image alignment processing device are displayed at the same time, diagnosis based on comparative image reading can be easily and accurately performed using images without positional deviation. And can be performed quickly.

According to the seventh aspect of the present invention, since calculation is performed between a plurality of images that have been aligned by the image alignment processing device, the accuracy of fine alignment is ensured while reducing the calculation load. Is done.

According to the eighth aspect of the present invention, since a plurality of aligned images or an operation-processed image generated by an inter-image operation is displayed, the results of comparative image interpretation and inter-image operation can be obtained using images without positional deviation. Diagnosis can be performed easily, accurately, and quickly.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of an image registration processing device.

FIG. 2 is a schematic configuration diagram of another embodiment of an image registration processing apparatus.

FIG. 3 is a schematic configuration diagram of another embodiment of an image registration processing apparatus.

FIG. 4 is a schematic configuration diagram of another embodiment of an image registration processing apparatus.

FIG. 5 is a diagram illustrating a weighted template matching method.

FIG. 6 is a diagram illustrating a template and a search area used in local matching.

FIG. 7 is a diagram showing a relationship between a cross-correlation value and a weight coefficient.

FIG. 8 is a diagram illustrating detection of a skin line of a breast radiation image and setting of a partial region.

FIG. 9 is a diagram illustrating setting of a partial area.

FIG. 10 is a diagram in which chest X-ray images of the same patient taken at different times are displayed side by side.

FIG. 11 is a diagram in which chest X-ray images of the same patient taken at different times and their difference images are displayed side by side.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image registration processing apparatus 2 Partial area setting means 3 Partial area similarity determining means 4 Whole area similarity determining means 5 Weight coefficient storing means 6 Image conversion condition determining means 7 Image converting means 8 Image storing means 9 Image display controlling means DESCRIPTION OF SYMBOLS 10 Image display means 20 Anatomical structure extraction means 30 Attention position designation means 40 Inter-image calculation means

Claims (8)

[Claims] An image alignment processing apparatus for performing an alignment process between a plurality of images including a common subject, wherein a plurality of partial regions are set in at least one of the plurality of images. Area setting means, for each set partial area, a partial area similarity determining means for determining the similarity between the corresponding area in another image, and the determined partial area similarity, By weighting and adding using the weighting factor set for
Whole area similarity determining means for determining whole area similarity, and image conversion condition determining means for determining image conversion conditions for aligning one image with another image based on the determined whole area similarity And an image conversion unit that performs image conversion based on the determined image conversion condition to perform image alignment. 2. The anatomical structure extracting means for extracting an anatomical structure of a human body from at least one of the plurality of images, wherein the common subject is a part of a human body, The apparatus according to claim 1, wherein the setting unit sets the plurality of partial regions based on an extraction result obtained by the anatomical structure extracting unit. 3. An apparatus according to claim 2, further comprising an attention position specifying means for specifying an attention position in at least one of the plurality of images, wherein the plurality of partial areas are set based on the designated attention position. The image alignment processing apparatus according to claim 1 or 2, wherein: 4. An attention position designation means for designating an attention position in at least one image of the plurality of images, wherein the attention position designation means corresponds to a partial area including the designated attention position among the plurality of partial areas. 4. The image alignment process according to claim 1, wherein the weighting factor is set such that the weighting factor is larger than a weighting factor corresponding to another partial area. 5. apparatus. 5. The image conversion device according to claim 1, wherein the image conversion is performed based on the determined image conversion conditions, so that the image position is adjusted in two stages of fine alignment after coarse alignment for image alignment. The image registration processing apparatus according to claim 1, wherein registration is performed. 6. An image display device for simultaneously displaying a plurality of images that have been aligned by the image alignment processing apparatus according to any one of claims 1 to 5. Image alignment processing device. 7. An inter-image calculation means for performing an operation between a plurality of images that have been aligned by the image alignment processing apparatus according to any one of claims 1 to 5. Image processing device. 8. An inter-image calculating means for performing an arithmetic operation between a plurality of images which have been aligned by the image alignment processing apparatus according to any one of claims 1 to 5, An inter-image arithmetic processing apparatus, comprising: an image display unit that displays a plurality of performed images or an operation-processed image generated by an inter-image operation.