WO2012001594A1

WO2012001594A1 - Viewing frames of medical scanner volumes

Info

Publication number: WO2012001594A1
Application number: PCT/IB2011/052770
Authority: WO
Inventors: Dieter Geller; Yuechen Qian
Original assignee: Philips Intellectual Property and Standards GmbH; Koninklijke Philips Electronics NV
Current assignee: Philips Intellectual Property and Standards GmbH; Koninklijke Philips NV
Priority date: 2010-06-30
Filing date: 2011-06-24
Publication date: 2012-01-05
Anticipated expiration: 2012-12-30

Abstract

The invention relates to a system (100) for viewing a first image dataset and a second image dataset, the system (100) comprising a first viewer (120) for computing a first image from the first image dataset, and a second viewer (130) for computing a second image from the second image dataset, wherein the second image is based on the computed first image. The second image, e.g. a reference image computed from the second image dataset, may thus be adapted to show features which can be seen and compared to the features in the first image, e.g. a query image computed from the first image dataset. The system may be further adapted for displaying additional reference images computed from the second image dataset, based on the query image or on new query images selected, for example, by a user of the system. Every time a new query image is computed from the first image dataset, a corresponding reference image is computed from the second image dataset.

Description

Viewing frames of medical scanner volumes

FIELD OF THE INVENTION

The invention relates to retrieving a reference image dataset from a set of reference image datasets based on a query image dataset. BACKGROUND OF THE INVENTION

A radiologist investigating a 3-dimensional (3D) volume of a patient on the basis of a 3D image dataset acquired, for example, by a medical CT, MR or PET scanner, often needs to retrieve similar cases stored in a database, for reference. The database may be a PACS database, a personal folder of the radiologist comprising his personal reference cases, or a radiology database stored on a workstation available via the Internet. To evaluate whether a certain case comprised in a database is relevant for a case under investigation, the radiologist often has to scan many 3D database sets. This task is usually very time- consuming. Because the databases contain usually many cases and a huge amount of data, there is a need for supporting the physician in extracting the relevant database cases fast, reliably and mostly automatically.

SUMMARY OF THE INVENTION

It would be advantageous to have a system capable of assisting users in identifying a second image dataset in a set of 3-dimensional image datasets, based on a query image dataset.

Thus, in an aspect, the invention provides a system for viewing a first image dataset and a second image dataset, the system comprising:

a first viewer for computing a first image from the first image dataset; and a second viewer for computing a second image from the second image dataset, wherein the second image is based on the computed first image.

The second image, e.g. a reference image computed from the second image dataset, may thus be adapted to show features which can be seen and compared to the features in the first image, e.g. a query image computed from the first image dataset. The system may be further adapted for displaying additional reference images computed from the second image dataset, based on the query image or on new query images selected, for example, by a user of the system. Every time a new query image is computed from the first image dataset, a corresponding reference image is computed from the second image dataset.

Alternatively, the second image, e.g. a query image computed from the second image dataset, may be adapted to show a view corresponding to the first image, e.g. a reference image computed from the first image dataset. The system may be adapted to show relevant features of the reference image. The user can thus see whether the query image is similar to the reference image.

In an embodiment of the system, the first image dataset is a query image dataset and the second image dataset is an at least 3 -dimensional reference image dataset or vice versa, the first image dataset is a reference image dataset and the second image dataset is an at least 3 -dimensional query image dataset. The system is thus adapted for finding an at least 3 -dimensional image dataset relevant to an at least 2-dimensional dataset.

In an embodiment, the system further comprises a selector for selecting the reference image dataset from a set of image datasets, based on a predefined criterion. This is useful when searching for a reference image dataset in a database comprising a plurality of image datasets.

In an embodiment of the system, the selector is adapted for computing similarity between the query image dataset and the reference image dataset, and wherein the computed similarity satisfies the predefined criterion. The predefined criterion may be that a similarity function is greater than a predefined threshold similarity; in the case of another similarity function, the predefined criterion may be that the similarity function is less than a predefined threshold similarity.

In an embodiment of the system, the similarity is computed based on the query image and the reference image. Thus, the less relevant data comprised in the first and second image dataset is ignored when computing the value of similarity of the reference and query image datasets.

In an embodiment of the system, the first image dataset is computed based on viewing information comprised in a user input or in metadata included with the first image dataset. For example, the user may browse a stack of images (slices) comprised in the query image dataset, providing an input for determining the query image, wherein the query image dataset is the first image dataset. The system is then adapted for computing the reference image from the reference image data, based on the query image and/or the viewing information. In an embodiment of the system, the viewing information comprises a definition of a portion of the first image dataset for computing the first image. For example, the portion of the first image dataset may be a data slice. The information may comprise three landmarks defining three non-collinear points for defining the data slice. The fist image may be a MIP (maximum intensity projection) view computed from the data slice.

In an embodiment, the system further comprises an annotator for annotating the view of the first or second image dataset. The annotating is based on annotation information comprised in a user input or metadata included with the first or second image dataset. The annotation may help emphasizing similarities or differences between the first and second image and thus interpreting the computed images.

In an embodiment of the system, the reference image is a thumbnail image. Because thumbnail images are small relative to the size of a display, many thumbnail images can be shown to a user of the system in the display, thereby enabling the user to visually examine reference images.

In a further aspect, the invention provides a database comprising a system according to the invention.

In a further aspect, the invention provides an image acquisition apparatus comprising a system according to the invention.

In a further aspect, the invention provides a workstation comprising a system according to the invention.

In a further aspect, the invention provides a method of viewing a first image dataset and a second image dataset, the method comprising the steps of:

computing a first image from the first image dataset; and

computing a second image from the second image dataset, wherein the second image is based on the computed first image dataset.

In a further aspect, the invention provides a computer program product to be loaded by a computer arrangement, comprising instructions for viewing a first image dataset and a second image dataset, the computer arrangement comprising a processing unit and a memory, the computer program product, after being loaded, providing said processing unit with the capability to carry out steps of a method according to the invention.

It will be appreciated by those skilled in the art that two or more of the above- mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful. Modifications and variations of the database system, of the image acquisition apparatus, of the workstation, of the method, and/or of the computer program product, which correspond to the described modifications and variations of the system or of the method, can be carried out by a person skilled in the art on the basis of the description.

A person skilled in the art will appreciate that an image dataset in the claimed invention may be a 2-dimensional (2-D), 3-dimensional (3-D) or 4-dimensional (4-D) image dataset, acquired by various acquisition modalities such as, but not limited to, X-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Nuclear Medicine (NM).

The invention is defined in the independent claims. Advantageous embodiments are defined in the dependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will become apparent from and will be elucidated with respect to the implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein:

Fig. 1 shows a block diagram of an exemplary embodiment of the system; Fig. 2 shows an exemplary query image and an annotated reference image computed according to the invention;

Fig. 3 shows a flowchart of exemplary implementations of the method;

Fig. 4 schematically shows an exemplary embodiment of the database system; and

Fig. 5 schematically shows an exemplary embodiment of the image acquisition apparatus; and

Fig. 6 schematically shows an exemplary embodiment of the workstation. Identical reference numerals are used to denote similar parts throughout the

Figures. DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 schematically shows a block diagram of an exemplary embodiment of the system 100 for viewing a first image dataset and a second image dataset, the system 100 comprising:

a first viewer 120 for computing a first image from the first image dataset; and a second viewer 130 for computing a second image from the second image dataset, wherein the second image is based on the computed first image.

The exemplary embodiment of the system 100, wherein the first image dataset is a query image dataset and the second image dataset is an at least 3 -dimensional reference image dataset or vice versa, the first image dataset is a reference image dataset and the second image dataset is an at least 3 -dimensional query image dataset, further comprises:

a selector 110 for selecting the reference image dataset from a set of image datasets based on a predefined criterion;

an annotator 140 for annotating the view of the first or second image dataset, and wherein the annotating is based on annotation information comprised in a user input or metadata included with the first or second image dataset;

a control unit 160 for controlling the work of the system 100; a user interface 165 for communication between the user and the system 100; and

- a memory unit 170 for storing data.

In an embodiment of the system 100, there are three input connectors 181, 182 and 183 for the incoming data. The first input connector 181 is arranged to receive data coming in from a data storage means such as, but not limited to, a hard disk, a magnetic tape, a flash memory, or an optical disk. The second input connector 182 is arranged to receive data coming in from a user input device such as, but not limited to, a mouse or a touch screen. The third input connector 183 is arranged to receive data coming in from a user input device such as a keyboard. The input connectors 181, 182 and 183 are connected to an input control unit 180.

In an embodiment of the system 100, there are two output connectors 191 and 192 for the outgoing data. The first output connector 191 is arranged to output the data to a data storage means such as a hard disk, a magnetic tape, a flash memory, or an optical disk. The second output connector 192 is arranged to output the data to a display device. The output connectors 191 and 192 receive the respective data via an output control unit 190.

A person skilled in the art will understand that there are many ways to connect input devices to the input connectors 181, 182 and 183 and the output devices to the output connectors 191 and 192 of the system 100. These ways comprise, but are not limited to, a wired and a wireless connection, a digital network such as, but not limited to, a Local Area Network (LAN) and a Wide Area Network (WAN), the Internet, a digital telephone network, and an analog telephone network. In an embodiment of the system 100, the system 100 comprises a memory unit 170. The system 100 is arranged to receive input data from external devices via any of the input connectors 181, 182, and 183 and to store the received input data in the memory unit 170. Loading the input data into the memory unit 170 allows quick access to relevant data portions by the units of the system 100. The input data comprises the first and second image dataset. The memory unit 170 may be implemented by devices such as, but not limited to, a register file of a CPU, a cache memory, a Random Access Memory (RAM) chip, a Read Only Memory (ROM) chip, and/or a hard disk drive and a hard disk. The memory unit 170 may be further arranged to store the output data. The output data comprises the first and second image. The memory unit 170 may be also arranged to receive data from and/or deliver data to the units of the system 100 comprising the selector 110, the first viewer 120, the second viewer 130, the annotator 140, the control unit 160, and the user interface 165, via a memory bus 175. The memory unit 170 is further arranged to make the output data available to external devices via any of the output connectors 191 and 192. Storing data from the units of the system 100 in the memory unit 170 may advantageously improve performance of the units of the system 100 as well as the rate of transfer of the output data from the units of the system 100 to external devices.

In an embodiment of the system 100, the system 100 comprises a control unit 160 for controlling the system 100. The control unit 160 may be arranged to receive control data from and provide control data to the units of the system 100. For example, after selecting the second image dataset, the selector 110 may be arranged to provide control data "the second image dataset is identified" to the control unit 160, and the control unit 160 may be arranged to provide control data "compute the second image" to the second viewer 130. Alternatively, a control function may be implemented in another unit of the system 100.

In an embodiment of the system 100, the system 100 comprises a user interface 165 for communication between a user and the system 100. The user interface 165 may be arranged to receive a user input for calculating the first image from the first image dataset. Next, the user interface is adapted for displaying the first and second image. A person skilled in the art will understand that more functions may be advantageously implemented in the user interface 165 of the system 100.

In an embodiment, the invention is used by a physician for an assessment of the brain of a patient on the basis of a brain CT scan, referred to as a query image dataset. The system is employed for assisting the physician in finding a reference case described by a reference image dataset in a database of image datasets, namely, of brain CT scans. Both the query image dataset and the reference image dataset are 3-D image datasets.

In one embodiment, the physician determines a view of the query image dataset computed by the first viewer 120 of the system 100. The view can be a 2-D view computed from a slice of the query image dataset, using planar reformatting of the query image dataset and MIP, for example. The view is selected based on at least 3 non-collinear features of interest identified by the physician using the user interface 165. The feature locations are used to define the slice of the image dataset. The second viewer 130 of the system 100 is adapted for identifying, based on the features identified in the query image dataset, corresponding features in the reference image dataset using, for example, image registration of the query image dataset with the reference image dataset, or image segmentation such as, but not limited to, model-based segmentation of the reference image dataset. Based on the corresponding features, a slice of the reference image dataset is determined by the second viewer 130. Finally, the second viewer 130 is adapted for computing the reference image, i.e. a view of the reference image dataset corresponding to the query image, based on the determined slice of the reference image dataset, using planar reformatting of the reference image dataset and MIP. The second viewer 130 is further adapted for evaluating the quality of the identified corresponding features. If the

corresponding features of satisfactory quality cannot be found, the reference image dataset is declared "invalid" and the reference image is not computed from this invalid reference image dataset. Instead, the system 100 is arranged to continue processing another reference image dataset according to the embodiment of the invention.

Alternatively, the reference image may be based on viewing parameters included with the reference image dataset. The inclusion may be implemented by

incorporating a data structure with the viewing parameters in the reference image dataset or the reference image dataset may comprise information pointing to a data structure defining the viewing parameters. The viewing parameters may comprise locations of features in the reference image dataset. The first viewer 120 is arranged to read the viewing parameters and to render the first image, i.e. the reference image. For example, the viewing parameters may comprise a first viewing direction and a first transition function for DVR (direct volume rendering) of the reference image dataset. The first viewer 120 is arranged to render a DVR view of the reference image dataset, using the first viewing direction and the first transition function. The second viewer 130 of the system 100 is adapted for identifying a second viewing direction by registering the query image dataset with the reference image dataset and for computing a second transition function based on the first transition function and the histograms of the reference and query image dataset. Finally, the second viewer 130 is arranged to render a DVR view of the reference image dataset, using the second viewing direction and the second transition function. The second viewer 130 may be further adapted for evaluating the quality of the registration of the query image dataset with the reference image dataset. If the two registered images are not similar according to a similarity criterion, the reference image dataset is declared invalid and the query image is not computed from the poorly registered query image dataset. The system 100 is arranged to continue processing another reference image dataset according to the embodiment of the invention.

The skilled person will understand that there are many other ways of computing the query and reference images. These ways may employ various feature detection methods, image dataset registration methods or image rendering methods. The above embodiments show exemplary embodiments of the invention and must not be construed as limiting the scope of the claims.

In an embodiment, the system 100 comprises a selector 110 for selecting the reference image dataset from a set of image datasets, based on a predefined criterion. The predefined criterion may be based on similarity of the image dataset to the query image dataset registered with the image dataset or vice versa, of the query image dataset to the image dataset registered with the query image dataset. For example, if the similarity is greater than a predefined threshold similarity, the image dataset is selected as a reference image dataset and further processed by the first viewer 120 or second viewer 130. The skilled person will understand that there are many ways of evaluating similarity of two image datasets which can be used by the system 100 of the invention.

In an embodiment, the selector 110 is adapted for initially accepting each image dataset of the database as a reference image dataset. The first viewer 120 and the second viewer 130 are adapted for computing the first and second image. The selector 110 is then arranged to register the first and second image and to compute similarity of the two registered images. If the similarity of the two registered images does not satisfy the predefined condition, the reference image dataset is rejected.

In an embodiment, a plurality of reference image datasets is selected and the corresponding first and second images are computed. For example, the top ten reference image datasets in terms of similarity to the query image dataset may be selected by the selector 110 of the system 100. The first and second images computed on the basis of the query image dataset and the selected reference image datasets may be ranked according to the similarity score of each pair. The physician may view all pairs, beginning with the most similar pair. This may further reduce the time needed for finding the best reference image dataset.

In an embodiment, the system 100 further comprises an annotator 140 for annotating the view of the first or second image dataset. The annotating may be based on annotation information comprised in a user input. Alternatively, the annotation information may be comprised in metadata included with the first or second image dataset. The annotation information may include, but is not limited to, contours or arrows for indicating areas of interest, keywords, or links/pointers to additional information pertaining to the case depicted in the first or second image dataset. The annotation information also comprises information on annotation placement and the annotator 140 is adapted for placing the relevant annotations in the computed first and/or second image.

Fig. 2 shows an exemplary query image 22 and an annotated reference image 24 computed according to the invention. In addition, the reference image is annotated using an annotation contour 26 for indicating an area of interest. The reference image is further described by a keyword 28.

A person skilled in the art will appreciate that the system 100 may be a valuable tool for assisting a physician in many aspects of her/his job. Further, although the embodiments of the system are illustrated using medical applications of the system, non- medical applications of the system are also contemplated.

Those skilled in the art will further understand that other embodiments of the system 100 are also possible. It is possible, among other things, to redefine the units of the system and to redistribute their functions. Although the described embodiments apply to medical images, other applications of the system, not related to medical applications, are also possible.

The units of the system 100 may be implemented using a processor. Normally, their functions are performed under the control of a software program product. During execution, the software program product is normally loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, such as a ROM, hard disk, or magnetic and/or optical storage, or may be loaded via a network like the Internet. Optionally, an application-specific integrated circuit may provide the described functionality.

An exemplary flowchart of the method M of viewing a first image dataset and a second image dataset is schematically shown in Fig. 3. The first image dataset is a query image dataset and the second image dataset is an at least 3 -dimensional reference image dataset or vice versa, the first image dataset is a reference image dataset and the second image dataset is an at least 3 -dimensional query image dataset. The method M begins with selecting S 10 the reference image dataset from a set of image datasets, based on a predefined criterion. After selecting S10 the reference image dataset, the method M continues to computing S20 a first image from the first image dataset. After computing S20 the first image, the method M continues to computing S30 a second image from the second image dataset, wherein the second image is based on the computed first image dataset. Optionally, after computing S30 the second image, the method M continues to annotating S40 the view of the first or second image dataset. The annotating S40 is based on annotation information comprised in a user input or metadata included with the first or second image dataset. After computing S30 the second image or, optionally, after annotating S40 the view of the first or second image dataset, the method M terminates.

A person skilled in the art may change the order of some steps or perform some steps concurrently, using threading models, multi-processor systems or multiple processes without departing from the concept as intended by the present invention.

Optionally, two or more steps of the method M may be combined into one step. Optionally, a step of the method M may be split into a plurality of steps.

Fig. 4 schematically shows an exemplary embodiment of the database system 400 employing the system 100 of the invention, said database system 400 comprising a database unit 410 connected via an internal connection to the system 100, an external input connector 401, and an external output connector 402. This arrangement advantageously increases the capabilities of the database system 400, providing said database system 400 with advantageous capabilities of the system 100.

Fig. 5 schematically shows an exemplary embodiment of the image acquisition apparatus 500 employing the system 100 of the invention, said image acquisition apparatus 500 comprising an image acquisition unit 510 connected via an internal connection with the system 100, an input connector 501, and an output connector 502. This arrangement advantageously increases the capabilities of the image acquisition apparatus 500, providing said image acquisition apparatus 500 with advantageous capabilities of the system 100.

Fig. 6 schematically shows an exemplary embodiment of the workstation 600. The workstation comprises a system bus 601. A processor 610, a memory 620, a disk input/output (I/O) adapter 630, and a user interface (UI) 640 are operatively connected to the system bus 601. A disk storage device 631 is operatively coupled to the disk I/O adapter 630. A keyboard 641, a mouse 642, and a display 643 are operatively coupled to the UI 640. The system 100 of the invention, implemented as a computer program, is stored in the disk storage device 631. The workstation 600 is arranged to load the program and input data into memory 620 and execute the program on the processor 610. The user can input information to the workstation 600, using the keyboard 641 and/or the mouse 642. The workstation is arranged to output information to the display device 643 and/or to the disk 631. A person skilled in the art will understand that there are numerous other embodiments of the workstation 600 known in the art and that the present embodiment serves the purpose of illustrating the invention and must not be interpreted as limiting the invention to this particular embodiment.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim or in the description. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a programmed computer. In the system claims enumerating several units, several of these units can be embodied by one and the same record of hardware or software. The usage of the words first, second, third, etc., does not indicate any ordering. These words are to be interpreted as names.

Claims

CLAIMS:

1. A system (100) for viewing a first image dataset and a second image dataset, the system (100) comprising:

a first viewer (120) for computing a first image from the first image dataset; and

- a second viewer (130) for computing a second image from the second image dataset, wherein the second image is based on the computed first image.

2. A system (100) as claimed in claim 1, wherein the first image dataset is a query image dataset and the second image dataset is an at least 3 -dimensional reference image dataset or vice versa, the first image dataset is a reference image dataset and the second image dataset is an at least 3 -dimensional query image dataset.

3. A system (100) as claimed in claim 2, further comprising a selector (110) for selecting the reference image dataset from a set of image datasets, based on a predefined criterion.

4. A system (100) as claimed in claim 3, wherein the selector (110) is adapted for computing similarity between the query image dataset and the reference image dataset, and wherein the computed similarity satisfies the predefined criterion.

5. A system (100) as claimed in claim 4, wherein the similarity is computed based on the query image and the reference image.

6. A system (100) as claimed in claim 1, wherein the first image dataset is computed based on viewing information comprised in a user input or in metadata included with the first image dataset.

7. A system (100) as claimed in claim 6, wherein the viewing information comprises a definition of a portion of the first image dataset for computing the first image.

8. A system (100) as claimed in claim 7, wherein the portion of the first image dataset is a data slice.

9. A system (100) as claimed in any of claims 1 to 8, further comprising an annotator (140) for annotating the view of the first or second image dataset, and wherein the annotating is based on annotation information comprised in a user input or metadata included with the first or second image dataset.

10. A system (100) as claimed in claim 2, wherein the reference image is a thumbnail image.

11. A database (400) comprising a system (100) as claimed in claim 1.

12. An image acquisition apparatus (500) comprising a system (100) as claimed in claim 1.

13. A workstation (600) comprising a system (100) as claimed in claim 1.

14. A method (M) of viewing a first image dataset and a second image dataset, the method (M) comprising the steps of:

computing (S20) a first image from the first image dataset; and computing (S30) a second image from the second image dataset, wherein the second image is based on the computed first image dataset.

15. A computer program product to be loaded by a computer arrangement, comprising instructions for viewing a first image dataset and a second image dataset, the computer arrangement comprising a processing unit and a memory, the computer program product, after being loaded, providing said processing unit with the capability to carry out steps of a method as claimed in claim 14.