HK1215890B - System and method for reviewing and analyzing cytological specimens - Google Patents

Description

Systems and methods for examining and analyzing cytological specimens

Related applications

This application claims the benefit under 35 U.S.C. §119 of U.S. Provisional Application Serial No. 61/787,975, filed on March 15, 2013, the contents of which are hereby incorporated by reference herein in their entirety.

field

The present disclosure generally relates to systems and methods for examining and analyzing cytology specimens. In particular, the disclosed systems and methods facilitate the classification of cytology specimens.

background

Modern technology, including microprocessors, computer memory, computer displays, and user interfaces, continues to advance in related fields and can be used to solve problems and meet unmet needs in cytology. For example, U.S. Patent No. 8,041,091 describes an image analysis system used in ophthalmology, the contents of which are incorporated herein by reference as if fully set forth.

Current systems for examining and analyzing cytological specimens include relatively expensive examination microscopes or examination stations. Furthermore, current methods for examining and analyzing cytological specimens are typically labor-intensive and time-consuming. These demanding methods can result in errors in the examination and analysis of cytological specimens, such as false negatives and false positives.

Overview

The user interface of a cytology specimen inspection and analysis system can be improved to better facilitate inspection of cytology specimens and objects of interest ("OOIs") identified therein. Thus, improved systems and methods for quickly and intuitively analyzing OOIs and navigating through large amounts of image data may be highly desirable. Furthermore, improved systems and methods for analyzing cytology specimens that reduce the chances of false negatives and false positives occurring during the specimen slide inspection process may also be highly desirable. Improvements such as those listed above can make cytology inspection and analysis systems and methods simpler, more elegant, and suitable for more applications.

In one embodiment of the disclosed invention, a system for facilitating the inspection and analysis of cytological specimens includes at least one machine, each including a processor communicatively coupled to a storage device storing computer-executable instructions, which instructions, when executed by the processor, cause the processor to operate as: (i) a recording module configured to request and permit input of cytological specimen image data, the image data comprising a digital image of the cytological specimen; (ii) an image set module configured to analyze the input cytological specimen image data and generate a corresponding image set and analysis information thereof for individual specimen inspection based on characteristic attributes of an object of interest in the cytological specimen image; and (iii) an analysis tool configured to display the image set and analysis information generated by the image set module to a user, wherein the analysis tool is further configured to collect input and instructions from the user via one or more tools of a user interface and cause additional specimen image data to be acquired and/or analyzed by the recording module and/or the image set module.

In some embodiments of the inspection and analysis system, the input image data further includes one or both of characteristic attributes and positions of objects of interest in the cytological specimen images. In various embodiments, the image collection module is configured to analyze the input cytological specimen image data and generate corresponding image collections and analysis information thereof to further facilitate individual specimen inspection based on the positional information of the objects of interest in the cytological specimen images. When executed by the processor, the computer-executable instructions cause the processor to further operate as an image processing module configured to extract one or both of characteristic attributes and positions of objects of interest in the cytological specimen images.

In various embodiments, the image set module generates a corresponding image set in response to user input received by the analysis tool. In some embodiments, the inspection and analysis system further includes a graphical user interface. In certain embodiments, the inspection and analysis system further includes a voice recognition user interface. The image set module is configured to selectively enhance objects in the images of the image set. In various embodiments, the image set module enhances the objects depicted in the images of the image set by varying the contrast and/or brightness of the depicted objects. The analysis tool includes one or more of a login module, a homepage module, a patient module, a patient dashboard, and an image browser. The login module is configured to accept user credentials including a username and password. The homepage module includes a user dashboard, a patient list, a search page, and a new patient module, wherein the user dashboard includes a list of submitted analysis jobs and the status of the submitted analysis jobs. The patient dashboard includes cytology specimen images associated with the patient identified by the dashboard, and/or analysis results associated with the patient identified by the dashboard. The patient dashboard includes a means for submitting new analyses for the patient identified by the dashboard.

In some embodiments, a method for navigating through and examining cytological image data using a processor-controlled device includes images of a cytological specimen, the images including respective images of objects of interest therein, the method comprising: causing a plurality of images from the image data to be displayed on a display integrated with or otherwise operatively associated with the device, each of the displayed images in the plurality of images depicting a respective object of interest in the specimen; and selecting one of the displayed images via a user interface integrated with or otherwise operatively associated with the device, thereby causing the device to display on the display an image of at least a portion of the specimen, the at least portion of the specimen including the respective object of interest depicted in the selected image and adjacent objects in the specimen. The display includes the user interface. In some embodiments, the image of the at least portion of the specimen is displayed at a magnification appropriate for cytological examination of the respective object of interest depicted in the selected image in the plurality of images.

In various embodiments, a method for navigating through and inspecting cytological image data using a processor-controlled device further includes highlighting, in an image of at least a portion of a specimen, a corresponding object of interest depicted in a selected image from a plurality of images. The highlighting includes a change in contrast and/or brightness of the depicted object. The displayed plurality of images depicting the corresponding object of interest and the image of at least a portion of the specimen are displayed in different regions of a display.

In some embodiments, a method for navigating through and examining cytological image data using a processor-controlled device includes images of a cytological specimen, the images including respective images of objects of interest therein, the method comprising: causing a first plurality of images from the image data to be displayed on a display integrated with or otherwise operatively associated with the device, each of the displayed images in the first plurality of images depicting a respective object of interest in the specimen; and selecting one of the displayed images via a user interface operatively associated with the device, thereby causing the device to display a second plurality of images on the display, each of the second plurality of images depicting a cytological object having characteristics similar to the characteristics of the respective object of interest in the selected image from the first plurality of images. The second plurality of images is obtained from a library of previously classified cytological objects. The second plurality of images depicts additional cytological objects in the specimen.

In various embodiments, a method for navigating through and examining cytological image data using a processor-controlled device includes images of a cytological specimen, the images including respective images of objects of interest therein, the method comprising: (i) causing a first plurality of images from the image data to be displayed on a display integrated with or otherwise operatively associated with the device, each of the displayed images in the first plurality of images depicting a respective object of interest in the specimen; (ii) selecting one of the displayed images via a user interface integrated with or otherwise operatively associated with the device, thereby causing the device to display on the display a list of metric values calculated for the respective object of interest in the selected image; and (iii) selecting a metric value from the list via the user interface, thereby causing the device to display on the display a second plurality of images, each of the second plurality of images depicting a cytological object having a calculated metric value that is the same as or similar to the metric value selected from the list. In some embodiments, the second plurality of images is obtained from a library of previously categorized cytological objects. The second plurality of images depicts additional cytological objects in the specimen.

In various embodiments, an automated method for navigating through and interrogating cytological image data using a processor-controlled device, the image data comprising images of a cytological specimen, the images including respective images of objects of interest therein, the device comprising a user interface and a display integrated or otherwise operatively associated, the method comprising: in response to one or more user commands received via the user interface, displaying on the display a first plurality of images from the image data, each image in the first plurality of images depicting a respective object of interest in the specimen; detecting, via the user interface, a user selection of an image from the first plurality of images; and in response to the detected user selection, displaying on the display one or more of the following images: (a) an image of at least a portion of the specimen, the at least a portion of the specimen including the respective object of interest depicted in the selected image and adjacent objects in the specimen; and (b) a second plurality of images, each image in the second plurality of images depicting a cytological object having (i) similar characteristics to the characteristics of the respective object of interest in the selected image from the first plurality of images, or (ii) a calculated metric value that is the same as or similar to a selected metric value of the respective object of interest in the selected image. The second plurality of images is obtained from a library of previously classified cytological objects. The second plurality of images depicts additional cytological objects in the specimen. The image of at least a portion of the specimen is displayed at a magnification appropriate for cytological inspection of a corresponding object of interest depicted in a selected image from the plurality of images. In some embodiments, the automated method of navigating through and inspecting cytological image data using a processor-controlled device further comprises: highlighting, in the image of at least a portion of the specimen, the corresponding object of interest depicted in the selected image from the plurality of images.

In some embodiments, an automated method for navigating through and interrogating cytology image data using a processor-controlled device further comprises: receiving input representing a user-proposed classification of a displayed object of interest in a first plurality of images; and providing feedback regarding the user-proposed classification in light of previously determined classifications of selected objects of interest. In various embodiments, an automated method for navigating through and interrogating cytology image data using a processor-controlled device further comprises: receiving input representing a user-proposed classification of a displayed object of interest in a first plurality of images; and determining a percentage of matches between the user-proposed classification and the corresponding previously determined classification of the corresponding object of interest.

In some embodiments, a system for navigating through and inspecting input cytological specimen image data, the input cytological specimen image data comprising an image of a cytological specimen, the system comprising: a processor; a display integrated with or otherwise operatively associated with the processor; a user interface operatively coupled to the processor and the display, wherein the processor is configured to display on the display a first plurality of images from the image data, each image in the first plurality of images depicting a corresponding object of interest in the specimen; detect, via the user interface, a user selection of an image in the first plurality of images; and in response to the detected user selection, display on the display one or more of the following images: (a) an image of at least a portion of the specimen, the at least portion of the specimen including the corresponding object of interest depicted in the selected image, together with adjacent objects in the specimen; and (b) a second plurality of images, each image in the second plurality of images depicting a cytological object having (i) similar characteristics to the characteristics of the corresponding object of interest in the selected image from the first plurality of images, or (ii) a calculated metric value that is the same as or similar to a selected metric value from a list.

The input cytological specimen image data further includes one or both of characteristic attributes and positions of objects of interest in the cytological specimen image. The processor is programmed or otherwise configured to extract one or both of characteristic attributes and positions of objects of interest in the input cytological specimen image. The processor is programmed or otherwise configured to generate a first plurality of images in response to user input via a user interface. In some embodiments, the user interface includes a voice recognition system. The processor is configured to selectively enhance objects depicted in the displayed image. Objects depicted in the displayed image are enhanced by varying contrast and/or brightness. The processor is programmed or otherwise configured to acquire a second plurality of images from a library of previously classified cytological objects. The processor is programmed or otherwise configured to acquire a second plurality of images from a group of cytological objects in the specimen. The processor is programmed or otherwise configured to display an image of at least a portion of the specimen at a magnification suitable for cytological examination of the corresponding object of interest depicted in a selected image from the plurality of images. In some embodiments, the display includes a user interface. The processor is programmed or otherwise configured to display the first plurality of images along with an image of at least a portion of the specimen, or to display the second plurality of images in a different area of the display.

In various embodiments, a system for navigating through and interrogating input cytology specimen image data is further configured to: receive input representing a user-proposed classification of a displayed object of interest in a first plurality of images; and provide feedback regarding the user-proposed classification in light of previously determined classifications of selected objects of interest. The system for navigating through and interrogating input cytology specimen image data is further configured to: receive input representing a user-proposed classification of a displayed object of interest in a first plurality of images; and determine a percentage of matches between the user-proposed classification and the corresponding previously determined classification of the corresponding object of interest.

In various embodiments, a computer-assisted method for classifying an image of a cytological specimen includes the following actions: analyzing an image of a cytological specimen to identify an object of interest within the cytological specimen; displaying an image of the identified object of interest to an examiner; in response to input from the examiner selecting an object of interest, determining at least one other object of interest that is similar to the selected object of interest; and displaying images of the at least one other object of interest and the selected object of interest for the examiner to compare the objects of interest. The action of displaying the images of the at least one other object of interest and the selected object of interest is within a comparison view. The image of the at least one other object of interest is provided from the same cytological specimen. The image of the at least one other object of interest is provided from a database of previously stored objects of interest. The image of the selected object of interest has a first stain, and the image of the at least one other object of interest is the same selected object of interest having a second stain.

In various embodiments, the computer-assisted method for classifying an image of a cytology specimen further comprises: using the image of the selected object of interest to determine and store a classification of the selected object of interest, wherein the classification is determined by an examiner. In some embodiments, the computer-assisted method for classifying an image of a cytology specimen further comprises: using the image of the selected object of interest to determine and store a classification of the selected object of interest, wherein the classification is determined by a processor. In some embodiments, the analysis of the image of the cytology specimen is performed by the processor. In specific embodiments, the computer-assisted method for classifying an image of a cytology specimen further comprises: using the image of the selected object of interest to create a database of the classification of the selected object of interest. In some embodiments, the method further comprises: receiving the image of the cytology specimen from a remote workstation.

In a particular embodiment, a computer-assisted method for classifying an image of a cytological specimen includes the following actions: analyzing an image of a cytological specimen to identify an object of interest within the cytological specimen; displaying an image of the identified object of interest to an examiner; in response to input from the examiner selecting an object of interest, determining characteristics of the selected object of interest; determining at least one other object of interest having similar characteristics to the selected object of interest; and displaying images of the selected object of interest and the at least one other object of interest for the examiner to compare the objects of interest. The image of the at least one other object of interest is provided from the same cytological specimen. The image of the at least one other object of interest is provided from a database of previously stored objects of interest. The image of the identified object of interest has a first stain, and the image of the at least one other object of interest is the same identified object of interest having a second stain.

In various embodiments, the computer-assisted method for classifying images of cytological specimens further comprises: using images of identified objects of interest to determine and store a classification of the identified objects of interest, wherein the classification is determined by an examiner. In some embodiments, the computer-assisted method for classifying images of cytological specimens further comprises: using images of identified objects of interest to determine and store a classification of the identified objects of interest, wherein the classification is determined by a processor. In various embodiments, the analysis of the images of cytological specimens is performed by a processor. In specific embodiments, the computer-assisted method for classifying images of cytological specimens further comprises: using images of selected objects of interest to create a database of classifications of the selected objects of interest. In some embodiments, the computer-assisted method for classifying images of cytological specimens further comprises: receiving images of cytological specimens from a remote workstation.

In various embodiments, a computer-assisted method for navigating an image of a cytological specimen includes the following actions: analyzing an image of a cytological specimen to identify an object of interest within the cytological specimen; displaying an image of the corresponding identified object of interest to an examiner; and, in response to input from the examiner selecting an object of interest, displaying a field of view of the selected object of interest and adjacent objects of interest for the examiner to view in the field of view. The corresponding identified objects of interest are displayed to the examiner as a scroll bar. In various embodiments, the computer-assisted method for classifying an image of a cytological specimen further includes: using the image of the selected object of interest to determine and store a classification of the selected object of interest, wherein the classification is determined by the examiner. In other embodiments, the computer-assisted method for navigating an image of a cytological specimen further includes: using the image of the selected object of interest to determine and store a classification of the selected object of interest, wherein the classification is determined by a processor. In some embodiments, the analysis of the image of the cytological specimen is performed by a processor. In various embodiments, the computer-assisted method for navigating an image of a cytological specimen further includes: using the image of the selected object of interest to create a database of classifications of the selected object of interest. In some embodiments, the method further comprises receiving an image of the cytology specimen from a remote workstation.

In a particular embodiment, a system for navigating within an image of a cytological specimen comprises: at least one processor operatively connected to a memory; a user interface display; an identification component executed by the at least one processor, the identification component configured to identify an object of interest within the image; a user interface component executed by the at least one processor, the user interface component configured to display the object of interest within the user interface display, wherein the user interface component is configured to accept a user selection of the object of interest to display the selected object of interest and at least one other object of interest having similar features to the selected object of interest for comparison by an examiner.

In various embodiments, the user interface component is configured to display the selected object of interest and at least one other object of interest in a comparison view. The user interface component is configured to accept a user selection of the object of interest to display the selected object of interest in a field of view that includes the selected object of interest and adjacent objects of interest. The system is configured to accept a user selection of the object of interest and display a cell property menu of the object of interest. The system is configured to accept the selection of cell properties from the cell property menu and identify additional objects of interest with the selected cell properties. Cell properties include any one of morphological properties, stains (e.g., chromogenic, fluorescent, dual, etc.), cell size, nuclear/cytoplasmic ratio, optical density, outline regularity, color-based criteria, and nuclear density. In some embodiments, the system is configured to identify and display different images of the selected object of interest. In some embodiments, the system is configured to identify an object of interest with at least one of traditional stains, advanced stains, color images, and fluorescent stains, and is configured to accept a user selection of the object of interest and display an image of the corresponding object of interest with such stain.

Various aspects, embodiments, and advantages are discussed in detail below. The embodiments disclosed herein may be combined with other embodiments in any manner consistent with at least one of the principles disclosed herein, and references to "embodiments," "some embodiments," "alternative embodiments," "various embodiments," "one embodiment," etc. are not necessarily mutually exclusive and are intended to indicate that the particular features, structures, or characteristics being described may be included in at least one embodiment. The appearance of these terms herein does not necessarily all refer to the same embodiment. The features and advantages discussed in conjunction with any one or more embodiments according to one or more aspects are not intended to be excluded from similar roles in any other embodiment or aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate the design and utility of embodiments of the disclosed invention, wherein similar elements are referenced by common reference numerals. These drawings are not necessarily drawn to scale. To better understand how the above-described and other advantages and objects are achieved, a more detailed description of the embodiments will be presented, which is illustrated in the accompanying drawings. These drawings depict only typical embodiments of the disclosed invention and, therefore, are not to be considered limiting of its scope.

FIG1 is a screen shot of a display monitor of a cytology specimen inspection system showing an example mode of operation according to an embodiment of the disclosed invention;

FIG2 is another screen shot of a display monitor of a cytology specimen inspection system showing an example mode of operation in accordance with an embodiment of the disclosed invention;

FIG3A is another screen shot of a display monitor of a cytology specimen inspection system showing an example mode of operation in accordance with an embodiment of the disclosed invention;

FIG3B is another screen shot of a display monitor of a cytology specimen inspection system illustrating an example mode of operation in accordance with an embodiment of the disclosed invention;

FIG4 is a top view of a standard microscope slide carrying a cytology specimen;

FIG5A is a schematic diagram illustrating an example of image collection and a cytology specimen inspection system constructed in accordance with various embodiments of the disclosed invention;

FIG5B is another schematic diagram illustrating an example of image collection and a cytology specimen inspection system constructed in accordance with various embodiments of the disclosed invention; and

FIG6 is a schematic diagram of one example of a computer system that may perform the processes and functions disclosed herein.

Detailed description

For the following defined terms, these definitions shall apply, unless a different definition is given in the claims or elsewhere in this specification.

It is assumed herein that all numerical values are modified by the term "about", whether or not explicitly indicated. The term "about" generally refers to a range of numbers that one skilled in the art would consider equivalent to the recited value (i.e., having the same function or result). In many instances, the term "about" can include numbers that are rounded to the nearest significant figure.

The recitation of numerical ranges by endpoints includes all numbers within that range (eg, 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is generally employed in its sense including "and/or" unless the content clearly dictates otherwise.

Various embodiments of the disclosed invention are described below with reference to the accompanying drawings. It should be noted that the drawings are not drawn to scale and that elements of similar structure or function are represented by similar reference numerals throughout the drawings. It should also be noted that the drawings are intended only to facilitate the description of the embodiments. They are not intended to be an exhaustive description of the invention or to limit the scope of the invention, which is defined solely by the appended claims and their equivalents. In addition, the illustrated embodiments of the disclosed invention need not have all of the aspects or advantages shown. Aspects or advantages described in conjunction with a particular embodiment of the disclosed invention are not necessarily limited to that embodiment and, if not so illustrated, may be practiced in any other embodiment.

The cytological specimen inspection and analysis system 510 can receive a cytological specimen 412 on a microscope slide 414 and acquire and process an image of the received cytological specimen 412 (described below with reference to FIG. 4 ). Alternatively, the cytological specimen inspection and analysis system 510 can receive previously acquired and processed image data, including data related to, for example, previously identified objects of interest (“OOIs”) in the cytological specimen 412. In either case, the inspection device 522 described herein enables and facilitates detailed inspection and analysis of the cytological specimen 412. Cytological specimen image processing techniques for identifying and sorting OOIs based on possible features (e.g., size, color, optical density, and boundary regularity) are described in U.S. Patent No. 7,590,492, the contents of which are incorporated herein by reference in their entirety as if fully set forth.

In the embodiment depicted in Figures 1-3, the cytology specimen image inspection device 522 is a tablet computer with a display monitor 500, which may include a touch screen input device (e.g., a capacitive touch screen) or a user interface. The inspection device 522 has multiple operating modes. Depending on the command entered by the user, four examples of modes are: (1) a "take me there" navigation mode (Figure 1); (2) an "more like this" analysis tool mode (Figure 2); (3) an "analysis tool mode of similar metrics" (Figure 3A); and (4) "show me a reference image" (Figure 3B). The display on the display monitor 500 is divided into top and bottom windows 104, 106, but may have a gallery of side-by-side thumbnails or windows separated in other ways.

In FIG1-3B , detailed respective images (i.e., thumbnails 110) of a first plurality 108 of OOIs (e.g., 112) are displayed in a top window 104 of a display on a display monitor 100. The thumbnails 110 of the first plurality 108 are displayed serially, but may be displayed in any arrangement. Thumbnails 110 of OOIs (e.g., 112) may be displayed based on the likelihood that each OOI (e.g., 112) has a specific predetermined characteristic or a predetermined order selected by another user/cytotechnician. In some examples, the display shown on the display monitor is organized into an upper portion and a lower portion. Within the upper portion, a list of sorted thumbnails of OOIs may be displayed (e.g., at 108). A user may select an image from a display bar 109. The sorting within the displayed image may be based on the system-generated likelihood that the displayed image contains a feature or characteristic. In some examples, an examiner (e.g., a cytotechnician) may establish a characteristic of interest, and the system may display the images of the sorted OOIs in the upper portion of the display accordingly. Depending on the mode of the inspection device (e.g., 522 of FIG. 5A ), the bottom window 106 of the display monitor 100 displays various images. The magnification of the displayed image can be responsive to user selection. For example, the user can select from "1X," "4X," "20X," and "40X" at 111.

In some embodiments, a visual aid for determining the size of an object can be provided to the user. For example, a user interface can be configured to display concentric rings around the OOI. In some embodiments, the user interface can provide a region tool that is configured to be visible in the display in response to user selection. The concentric rings can each be marked with a corresponding size so that the inspector can establish the object size and/or size with greater accuracy. In some examples, the user can, for example, use input device 502 to select the concentric rings and position them on the display.

The cytotechnician can enter the "Take Me There" navigation mode depicted in FIG1 by clicking on a thumbnail (e.g., 110) of the OOI (e.g., 112) displayed in the top window 104. The cytotechnician can select the thumbnail (e.g., 110) using a touch screen input device (e.g., 502 of FIG5A ) of the tablet computer inspection device 522. In the case of other input devices (e.g., 502), selection can be made using a mouse (button clicks and/or mouse hovers), voice recognition, and/or eye movements using an augmented reality head-mounted display.

A mouse click or touch screen control (such as clicking on a thumbnail of an OOI (e.g., 112)) displays a larger field of view 114 focused on the OOI (e.g., 112). In some embodiments, the field of view display may be shown in a bottom window 106 displayed on the display monitor 100. The larger field of view 114 is a more conventional image of the cytology specimen 412, for example, a line scan or digital scan image as would be seen through a microscope. In some examples, the larger field of view 114 may be accessed from a database of cytology specimen images, and a "take me there" navigation mode may be used to visualize the OOI from within the database of images. The larger field of view 114 in the bottom window 106 allows the cytotechnician to view the OOI (e.g., 112) in the context of adjacent objects 116. This allows for manual comparison of features of the OOI (e.g., 112) and its neighbors 116 in the same field of view 114, thereby facilitating classification of the OOI (e.g., 112) and the cytology specimen 412.

According to one embodiment, the cytotechnician enters the "More Like This" analysis tool mode depicted in FIG2 by a mouse click or touch screen control (such as double-clicking the thumbnail 210A of the OOI 212 displayed in the top or top window 204 displayed on the display monitor 200). In the bottom window 206 of the display monitor 200, a mouse click or touch screen control (such as double-clicking the thumbnail (e.g., 210A of the OOI 212)) displays a second plurality 218 of thumbnails (e.g., 210B) of similar objects (e.g., 220). The similarity of the object (e.g., 220) to the selected OOI (e.g., 212) is determined by the characteristics used to identify the OOI (e.g., 112) or other user-selected characteristics. In this case, these characteristics are cells with a small diameter and a high nuclear/cytoplasmic ratio. Other characteristics may include color and boundary regularity.

According to one embodiment, the system determines similar objects from a specimen or a database of specimens. In some examples, a database of specimens may be predefined prior to inspection. In a further embodiment, the inspector may input reference images and/or add additional specimens for inspection. Based on the objects identified by the system in the specimen, the system generates a gallery for the user to inspect. For example, the system may be configured to generate an OOI gallery. Images in the OOI gallery may be automatically selected by the system and grouped or sorted based on cellular characteristics and/or the likelihood that a given OOI has cellular characteristics. For example, an automatic analysis tool may identify features within a cellular object, and the system may group/sort the OOIs to display based on the identified features and/or the likelihood that a given object includes the identified features. A sorting may be based on cell size or feature size, and the sorting may correspond to different categories of associations with these features (e.g., basal to surface).

The cytotechnician enters the "same calculated metric" analysis tool mode depicted in FIG3A by mouse click or touch screen control (such as double-clicking with two fingers on a thumbnail (e.g., 310) of an OOI (e.g., 312) displayed in the top window 304 of the display monitor 300). Double-clicking with two fingers on the thumbnail of the OOI (e.g., 312) opens a drop-down menu 322 of calculated metrics 324. In the bottom window 306 of the display monitor 300, selecting one or more calculated metrics 324 (in this case, cell size greater than 50 microns) displays a second plurality of thumbnails (e.g., 310) of similar objects (e.g., 320) having the selected calculated metrics. Other calculated metrics include nuclear/cytoplasmic ratio, optical density, and cell outline regularity. In some embodiments, the system is configured to dynamically generate cell size ranges based on inspection of the slide and the properties of the OOI found with the slide and/or digital image. In a further embodiment, each selection of a calculated metric may be configured to display a number of matching OOIs so that the inspector can identify whether a category/characteristic is significant within a set of OOIs.

In the "Show Reference Image" analysis tool mode depicted in Figure 3B, selecting an option from the menu in the top window causes cells of the type described in the menu to be displayed in the bottom window. In the illustrated example, in response to the menu selection, a field of view focused on LSIL cells is displayed in the bottom window.

Similar objects displayed in the "More Like This" or "Same Calculated Metric" analysis tool modes can be other objects on a microscope slide, previously inspected objects on other slides, or pre-classified library objects with known characteristics and diagnoses. Displaying similar objects for evaluation and comparison facilitates efficient classification of the entire cytology specimen (e.g., 412) or any image of a cytology specimen. For example, an examiner can identify an OOI as a possible glandular cell and request glandular cells from the classified library object in the user interface. Displaying the cells identified by the examiner side by side using the library object facilitates efficient and consistent classification.

In one embodiment, the touch screen 502 also allows the cytotechnician to mark an OOI (e.g., 112), for example, by two-finger tapping on a thumbnail in the top window 104 or an OOI (e.g., 112) in the field of view in the bottom window 106 of the display monitor 100. Additionally, the touch screen 502 may allow for “manual” navigation within the bottom window 106 of the display monitor 100 using touch and drag commands, for example, in a “take me there” navigation mode.

Because the selected OOI (e.g., 112) is displayed in context and/or with similar objects, the cytotechnician inspects the OOI (e.g., 112) and makes a decision about the level of cellular abnormality in the specimen 412, if any. The cytotechnician can electronically mark or otherwise note any OOI (e.g., 112) that is suspicious in the display. The inspection device 522 also enables the cytotechnician to return to a previously viewed OOI (e.g., 112) and manually move to (and view) other fields of view that do not cover the OOI (e.g., 112) (e.g., by touching and dragging). The cytological specimen inspection system 510 can also be used to perform quality control of the inspection process and training of cytotechnicians. Quality control can include randomly displaying thumbnails 110 of pre-classified OOIs (e.g., 112) (positive and negative) in the top window 104 of the display monitor 100, and tracking the classification of the OOI (e.g., 112) by the cytotechnician. By way of illustration, U.S. Patent Application Serial No. 13/427,251, the contents of which are incorporated herein by reference in their entirety as if fully set forth, describes a method for automatically seeding previously classified images among images of OOIs from a specimen for training and quality control. Training can include randomly displaying thumbnails (e.g., 110) of pre-classified OOIs (e.g., 112) for subsequent inspection. "More like this" and "same calculated metric" modes also facilitate training. Requesting the display of library images of known anomalies for comparison can also facilitate training.

The user interface of the analysis tool may also include one or more of a login module, a homepage module, a patient module, a patient dashboard, and an image browser. The login module may be configured to accept user identity credentials including a username and password. The homepage module may include a user dashboard, a patient list, a search page, and a new patient module. The user dashboard may include a list of submitted analysis jobs and the status of the submitted analysis jobs. The patient dashboard may include cytology specimen images associated with the patient identified by the dashboard, and/or analysis results associated with the patient identified by the dashboard. The patient dashboard may also include a means for submitting a new analysis for the patient identified by the dashboard.

As described above, the cytology specimen inspection system 510 can receive image data that has been previously acquired and processed, or it can receive a cytology specimen (e.g., 412) on a microscope slide 414, or it can collect images from other systems (such as an imaging flow cytometer and a line scanner). For illustrative purposes, the following embodiments are described where the system 510 processes an image of a cytology specimen 412. The following embodiments are shown by way of example, and these embodiments are not intended to limit the scope of the claims. Specifically, the following embodiments describe acquiring cytology image data from a specimen mounted on a slide. However, other techniques for imaging cytology specimens are known and may be equally employed. For example, an image of a cytology specimen 412 may originate from a flow cytometer or a PDMS biochip. See, for example, U.S. Patent No. 7,796,256, (“Oil-Immersion Enhanced Imaging Flow Cytometer”) and U.S. Patent Application Serial No. 12/740,087 (“Hybrid Microfluidic SPR and Molecular Imaging Device”), the contents of which are incorporated herein by reference in their entireties. Further exemplary procedures for collecting images using a line scanning device are disclosed in U.S. Patent Application Publication No. 2010/0238442 (“Serial-Line-Scan Encoded Multicolor Fluorescence Microscopy and Imaging Flow Cytology”), the contents of which are incorporated herein by reference in their entireties.

FIG5A depicts a cytology specimen inspection system 510 constructed in accordance with an embodiment of the present invention. The system 510 is configured to present a cytology specimen 412 located on a microscope slide 514 (best shown at 414 in FIG4 ) to a technician (such as a cytotechnician), who can then inspect an object of interest (OOI (e.g., 112)) located within the cytology specimen 412 as described above. The slide 414 is provided with fiducial markers 416, the function of which will be described in greater detail below.

While system 510 can be used to display any cytological specimen requiring further examination (or even non-biological specimens, such as computer chips), system 510 itself is particularly well suited for displaying cytological cervical or vaginal cellular material (such as that typically found on a cervical smear slide). In this case, the OOI (e.g., 112) takes the form of individual cells and cell clusters that are examined for the possible presence of abnormalities (such as malignancies or precursors to malignancies). Cytological specimen 412 will typically be placed on slide 414 as a thin cytological layer. Preferably, a cover glass (not shown) is adhered to specimen 412, thereby fixing specimen 412 in position on slide 414. Specimen 412 can be stained using any suitable stain, such as Papanicolaou stain. In other examples, specimens can be stained using advanced staining techniques. The system can store information about the stain type and, in some examples, can provide images of classified objects with multiple stain types. In a further embodiment, each specimen can be stained and imaged, faded, and re-stained to be imaged with another stain. The system can link cell objects so that a single object can be displayed using a variety of stains. Each user interface element can enable the examiner to select the display of the OOI and a specific stain. In addition, the user interface can be configured to display the OOI and a corresponding image with a corresponding stain (e.g., traditional stain, advanced stain, color stain, fluorescent stain, etc.).

The system 510 generally includes: (1) an imaging station 518 for acquiring images of cytological material contained on a slide 514 and generating electronic image data based on those images; (2) a computer 520 for filtering and processing the image data to identify OOIs (e.g., 112); and (3) a plurality of inspection devices 522 (3 shown), each of which provides a display monitor 500 for displaying the OOI (e.g., 112) for inspection by a cytotechnician and an input device 502 for allowing the cytotechnician to interact with the system 510.

According to some embodiments, the inspection system may access digital images captured from, for example, a line scan device. Digital images may also be captured by other systems, such as wide-field imaging devices that do not have a moving x/y stage. Digital images may be captured and stored as part of a database of cytological images.

Imaging station 518 is configured to image slides 514, which are typically contained within a cassette (not shown) along with other slides. During the imaging process, these slides are removed from their respective cassettes, imaged, and then returned to the cassette in a serial manner. In the illustrated embodiment, imaging station 518 is capable of processing up to 10 cassettes, each holding up to 25 slides, in approximately 16 hours. Again, the aforementioned "slide imaging" example is for purposes of illustration and not limitation, and cytological specimen images can be acquired using other known processes and devices.

The imaging station 518 includes a camera 524, a microscope 526, and a motorized stage 528. The camera 524 captures a magnified image of the slide 514 through the microscope 526. The camera 524 can be any of a variety of conventional cameras, such as a charge coupled device (CCD) camera, which can generate a digital output having sufficient resolution to allow processing of the captured image, for example, a digital image having a resolution of 640x480 pixels, either alone or in combination with other components (such as an analog-to-digital (A/D) converter). Preferably, each pixel is converted to an 8-bit value (0 to 255), depending on its transmittance, where "00000000" is the value assigned to the minimum amount of light passing through the pixel and "11111111" is the value assigned to the maximum amount of light passing through the pixel.

The slide 512 is mounted on a motorized stage 528 that scans the slide 514 relative to the viewing area of the microscope 526 while the camera 524 captures images across various areas of the cytology specimen (e.g., 412). Preferably, the shutter speed of the camera 524 is relatively high so as to maximize the scanning speed and/or number of images taken. The motorized stage 528 tracks the x-y coordinates of the image as it is captured by the camera 524. For example, encoders (not shown) can be coupled to respective motors of the motorized stage 528 to track the static distance traveled in the x and y directions during imaging. These coordinates are measured relative to the fiducial markers 416 affixed to the slide 414.

In some embodiments, the inspector can switch between a stored digital image of the OOI and a view of the OOI as it appears on the specimen. In some examples, the system can use the coordinate information to move the stage to a specified x-y coordinate for additional inspection of the OOI directly on the slide.

Computer 520 includes an image processor 530 configured to identify OOIs (e.g., 112) from image data captured by camera 524, and a memory 536 configured to store image data and information related to the OOIs (e.g., 112). Image processor 530 includes an input/output module 532, a processing module 534, and an analysis tool module 538. Input/output module 532 is configured to store image data (i.e., pixel data) captured from camera 524 in memory 536 and retrieve image data therefrom. Processing module 534 analyzes the image data to identify OOIs (e.g., 112), as described below. Processing module 534 also determines the likelihood that the OOIs (e.g., 112) have specific predetermined characteristics and generates a ranking based on the corresponding likelihood of the OOIs (e.g., 112). The ranking and x-y coordinates of the OOIs (e.g., 112), as well as a thumbnail image (e.g., 110) of each OOI (e.g., 112), are stored in memory 536. It should be appreciated that the functions performed by the respective processors and modules 530, 532, and 534 may be performed by a single processor or module, or alternatively, by more than three processors and modules. Likewise, it may be appreciated that the memory 536 may be divided into several memories.

The processing module 534 of the image processor 530 identifies OOIs (eg, 112) within the cytology specimen 412 by manipulating the digital image received from the camera 524 in an appropriate manner. In one embodiment, the processing module 534 does this using primary and secondary segmentation operations.

In the main segmentation operation, the processing module 534 removes artifacts from further consideration. The processing module 534 achieves this by masking pixels in the digital image data from further consideration due to their brightness, which pixels cannot be cell nuclei. The remaining pixels in the digital image form "blobs" with all possible shapes and sizes. The processing module 534 then performs an erosion process on these blobs to remove blobs that are only a few pixels in diameter and narrow fibers (strands) extending from the blobs or connecting adjacent blobs from further consideration. Then, depending on the number of pixels in the blobs, the processing module 534 determines whether each blob in the image is an individual object or a cluster object. For example, blobs with more than 500 pixels can be considered cluster objects, while blobs with 500 or less pixels can be considered individual objects. For individual objects, blobs that do not meet specific criteria related to total area, perimeter-to-area ratio, optical density standard deviation, and grayscale mean pixel value are not further considered.

In the secondary segmentation operation, processing module 534 removes spots that are unlikely to be individual cells or clusters of cells. For individual objects, processing module 534 performs a series of erosion operations, which remove small objects and eliminate projections from the remaining spots, and dilation operations, which remove holes from the remaining spots. For clustered objects, processing module 534 sharpens the edges of the objects to provide predefined boundaries. Processing module 534 then selects one or more individual objects with the highest integrated optical density from the predefined cluster objects. The individual objects extracted from the cluster objects are marked as cluster-extracted objects.

During the OOI identification operation, the processing module 534 measures various features of each of the individual objects and clustered objects, and then calculates an object score for each object based on the measured values of these features. In a further embodiment, the processing module may extract feature parameters of the individual and clustered objects based on the type of stain. The processing module may analyze information about the features obtained from the corresponding stain and use the feature information to calculate the object score. The feature information may be associated with the object and stored for later access. Based on the scores, the processing module 534 removes individual objects and clustered objects that may be artifacts. The remaining objects are considered OOIs (e.g., 112), where the individual objects represent individual OOIs (e.g., 112) ("IOOIs") and the clustered objects represent clustered OOIs ("COOIs"). The processing module 534 then evaluates the kernel integrated or average optical density of the OOIs (e.g., 112) and ranks the OOIs (e.g., 112) according to their optical density values. Although the integrated or average optical density is the characteristic evaluated in this embodiment, other object characteristics or combinations of object characteristics are also covered by the claims. In some embodiments, the processing module can evaluate OOIs based on any one or more of morphological characteristics, stains (e.g., chromogenic, fluorescent, dual, etc.), cell size, nuclear/cytoplasmic ratio, optical density, outline regularity, color-based criteria, and nuclear density. For each digital image, the input/output module 532 stores a thumbnail (e.g., 110) of the OOI (e.g., 112), along with its relative ranking, coordinates, and extracted features.

In the embodiment depicted in FIG5A , three inspection stations 522 are shown coupled to computer 520, enabling up to three cytotechnicians to simultaneously access relevant information stored in computer 520. Additional nearby or remote inspection stations or monitoring devices may also access relevant information stored in computer 520. It is noteworthy that system 510 can typically process slides 514 much faster than a cytotechnician can inspect them. Even if the system 510's specimen processing speed is slower than a cytotechnician's specimen inspection speed, system 510 can generally operate 24 hours a day, while a typical cytotechnician will only work an eight-hour day. Thus, the bottleneck in the screening process occurs at the human level (i.e., the detailed inspection of the cytological material contained on slides 514). As can be appreciated, the use of multiple inspection devices 522 alleviates this bottleneck, thereby providing a much more efficient process. However, the claims encompass a cytological specimen inspection system 510 that includes only a single inspection device 522. The number of inspection devices 522 connected to the computer 520 may be modified to accommodate the task at hand.

Suitable inspection devices 522 include tablet computers, smart phones, augmented reality head-mounted displays, personal computers, networked workstations, and other computer input/output devices known in the art, all of which are connected to the computer 520. The connection can be hardwired or wireless. The inspection devices 522 of a particular cytological specimen inspection system 510 can be the same or different from each other. The inspection devices 522 also include input devices 502, such as a keyboard, a computer mouse, a touch screen, and voice recognition hardware. Each inspection device 522 includes a display monitor 500 to facilitate interaction between the cytotechnician and the system 510. Each inspection device 522 also includes an input/output module 532, which is operatively connected to the input/output module 532 of the computer 520.

Although the computer 520 and inspection device 522 are depicted as separate devices in FIG5A , it should be appreciated that the personal computer 520, including all of the modules and memory described above, can reside in each inspection device 522. This system architecture can eliminate the need for a separate computer 520 connected to the imaging station 518. Accordingly, in some embodiments, each inspection device 522 can be directly connected to the imaging station 518.

For example, the embodiment depicted in Figure 5B includes only the imaging station 518 and the inspection device 522. The inspection device 522 includes an image processor 530, a memory 536, a display monitor 500, and an input device 502. The image processor 530 includes an input/output module 532, a processing module 534, and an analysis tool module 538. These modules are interconnected. The input/output module 532 of the inspection device 522 is directly connected to the imaging station 518. The other elements of the inspection device 522 perform the same functions as the corresponding elements in the embodiment depicted in Figure 5A. In further embodiments, the inspection device can operate independently of the imaging station and/or computer system. For example, the inspection device can be used to download specimen images for inspection. In a further example, any information generated on the inspection device can be uploaded to a central repository of specimen data.

The concepts of the disclosed embodiments can be applied to other cytology-related applications. For example, the inspection and analysis system (particularly the analysis tool) can be applied to cytotechnician training. A "proficiency test" can be a separate operating mode. The "proficiency test" mode is similar to regular digital image OOI inspection with the added function of a cytotechnician proficiency test, i.e., the cytotechnician can be scored on correct/incorrect classifications. Different proficiency levels can be tested by using cell classifications of different difficulty levels (i.e., pre-classified examples of abnormalities with clear outlines versus pre-classified but complex or ambiguous examples for more advanced students).

The inspection and analysis system (particularly the analysis tool) can be used to facilitate conversations about selected OOIs. As part of a training module, annotated instructional comments can be tied to the OOI. For example, in a training gallery of OOIs, an image of an ASCUS (Atypical Cells of Undetermined Significance) OOI can be double-clicked to cause a training explanation to be displayed. The explanation can contain relevant features and metrics for classifying it as ASCUS and can be displayed as text or highlighted in the image or explained along with previously extracted metrics. As part of a remotely monitored training module, a "I need help with this one" module can be constructed using selected thumbnails of specific OOIs. By digitally marking specific OOIs that students identify for remote inspection and commentary by the teacher, conversations can be facilitated through classification techniques and important features for classification.

Similarly, professional trainers can compile their own personal libraries and add comments such as "I think these images are good examples of ASCUS OOIs." By using some of the disclosed analysis tools, cytotech instructors can develop their own libraries for teaching purposes. The inspection and analysis system can also be used in an "I think the algorithm needs help" mode. To improve existing algorithms, the system can facilitate highlighting user-identified abnormal OOIs that the algorithm missed. The storage and transmission mechanism of the highlights, along with the annotated comments, will facilitate information exchange between users and algorithm developers for future modifications and improvements.

Inspection and analysis systems (particularly analysis tools) can be used to adjudicate clinical trials. In many clinical trials (or even in routine second-party opinion reviews or rereadings), the adjudication process is often used. For example, in a clinical trial, there may be three pathologists who simultaneously inspect and classify a group of patient samples. Through the analysis tools described herein, each examiner can create his/her own library of classified OOIs, with comments if necessary. The libraries of the resulting OOIs can be compared and adjudicated manually or automatically. Similarly, for second-party opinion reviews, the disclosed analysis tools can be used to facilitate the process of comparing and adjudicating the final diagnosis of the selected OOI.

In a further embodiment, the user may access a help feature on the system for diagnostic purposes. The user may access a "Help Me Quantify" function on the system. In response to the user's selection, the system may return information about how many OOIs are similar to the currently viewed OOI to facilitate classification/diagnosis. Additionally, the user interface may highlight similar OOIs that have already been classified and/or used as part of a diagnosis.

The inspection and analysis system can also be used to provide interactive grading of OOIs. In some embodiments, an inspector (e.g., a cytological technician, a pathologist, etc.) can specify the diagnostic category of the displayed OOI. Any input information can be associated with the OOI and stored for later use. For example, a subsequent inspector can access the information entered by the previous inspector. Classifications, annotations, and/or comments can be associated with specific OOIs and/or specimens as a whole. Subsequent inspectors gain the benefits of annotations, comments, and/or classifications displayed in context. According to another embodiment, digitally marked OOIs or specimens enhance the ability to capture data (e.g., by conventional methods of ink markings on slides). In a further embodiment, the inspector's classification of OOIs can be used for identification of OOIs based on an enhanced system. In one example, the system can implement a learning algorithm to identify OOIs. Inspector data can be used as training data to refine the algorithmic analysis of OOIs.

In a further embodiment, a user may annotate an OOI, specimen, etc. during an inspection. The user may then access the information they have annotated to facilitate further inspection. For example, the user may access other OOIs that they have inspected and/or graded to be displayed relative to the current OOI. In a further example, the user may input OOIs that were inspected, annotated, and/or graded in other specimens to provide one or more references for the current specimen inspection and/or inspection of the current OOI. According to one embodiment, the user may access previously inspected information in a "more like this" mode (e.g., FIG. 2 ).

According to some embodiments, the system generates an automatic identification of OOIs, which may include morphological features. Information generated during feature identification may be stored for later use by the system (e.g., in determining similar objects, etc.). In some examples, information about the OOIs may be stored and associated into one or more data records. The data records are organized into an intelligent database so that any information related to the object of interest can be accessed. In some embodiments, specimens may be taken from patients and various testing procedures may be performed. Image-based information (e.g., cytological analysis) may be combined with other test information so that the specimen and/or the OOIs within the specimen include information about other tests (e.g., a positive DNA test on a patient sample). For example, a database may store and index test information about a patient over time to provide historical information.

According to some embodiments, a database may include various records for storing data about OOIs. In some examples, the data records include at least specimen information, location information, and feature information for the OOI. In other examples, the data records may also include a calculated object score and/or user-entered information (e.g., classification labels, annotations, comments, etc.). In further examples, the data records may reflect patient information and/or include information about corresponding patient tests. In some examples, the data records include links for accessing patient test information.

In further embodiments, the database may include the calculated information. In one example, the system may calculate the distribution of characteristic features using one specimen, multiple specimens, etc. The calculated information may be incorporated into a user interface display. For example, the user interface may display a histogram of the distribution of features based on one specimen, multiple specimens, etc. The system may use the distribution information to determine whether one or more specific features provide any distinguishing information. Additionally, the database may track information about the examiner. The system may analyze the examiner's history and compare examiners based on the information stored in the database. This historical information may be used as feedback for the classification algorithm.

The various aspects and functions described herein can be implemented as specialized hardware or software components executed in one or more computer systems. There are many examples of computer systems currently in use. These examples include network applications, personal computers, workstations, mainframes, networked clients, servers, media servers, application servers, database servers, and web servers. Other examples of computer systems can include mobile computing devices (such as cellular phones and personal digital assistants) and network equipment (such as load balancers, routers, and switches). In addition, various aspects can be distributed on a single computer system, or between multiple computer systems connected to one or more communication networks.

For example, various aspects and functions may be distributed between one or more computer systems configured to provide services to one or more client computers or to perform an entire task as part of a distributed system. Alternatively, various aspects may be performed on a client server or a multi-layer system comprising components distributed between one or more server systems performing various functions. Therefore, the examples are not limited to performing on any particular system or set of systems. Furthermore, various aspects and functions may be implemented using software, hardware, or firmware, or a combination thereof. Thus, various aspects and functions may be implemented using a variety of hardware and software configurations within methods, actions, systems, system elements, and components, and the examples are not limited to any particular distributed architecture, network, or communication protocol.

6 , a block diagram of a distributed computer system 600 is illustrated, wherein various aspects and functions are implemented. As shown, the distributed computer system 600 includes one or more computer systems that exchange information. More specifically, the distributed computer system 600 includes computer systems 602, 604, and 606. As shown, the computer systems 602, 604, and 606 are interconnected via a communication network 608 and can exchange data via the communication network 608. The network 608 may include any communication network through which the computer systems can exchange data. In order to exchange data using the network 608, the computer systems 602, 604, and 606 and the network 608 may use various methods, protocols, and standards, including Fibre Channel, Token Ring, Ethernet, Wireless Ethernet, Bluetooth, IP, IPV6, TCP/IP, UDP, DTN, HTTP, FTP, SNMP, SMS, MMS, SS7, JSON, SOAP, CORBA, REST, and Web services, etc. To ensure that data transfer is secure, computer systems 602, 604, and 606 may use various security measures including, for example, TLS, SSL, or VPN to transmit data via network 608. Although distributed computer system 600 illustrates three networked computer systems, distributed computer system 600 is not so limited and may include any number of computer systems and computing devices networked using any media and protocol characteristics.

As illustrated in Figure 6, computer system 602 includes a processor 610, a memory 612, a bus 614, an interface 616, and a data store 618. To implement at least some of the aspects, functions, and processes disclosed herein, processor 610 executes a series of instructions that result in the manipulation of data. Processor 610 can be any type of processor, multiprocessor, or controller. Some exemplary processors include commercial processors such as Intel Xeon, Itanium, Core, Celeron, or Pentium processors, AMD Opteron processors, Sun UltraSPARC or IBM processors, and IBM mainframe chips. Processor 610 is connected to other system components, including one or more memory devices 612, via bus 614.

During operation of the computer system 602, the memory 612 stores programs and data. Thus, the memory 612 can be a relatively high-performance volatile random access memory, such as dynamic random access memory (DRAM) or static random access memory (SRAM). However, the memory 612 can include any device for storing data, such as a disk drive or other non-volatile storage device. Various examples can organize the memory 612 into a specific structure, and in some cases, the memory 612 can be organized into a unique structure to perform the functions disclosed herein. These data structures can be sized and organized to store specific data and values of various types of data.

The components of computer system 602 are coupled via interconnect elements, such as bus 614. Bus 614 may include one or more physical buses (e.g., buses between components integrated within the same machine), but may include any communication coupling between system elements, including specialized or standard computing bus technologies such as IDE, SCSI, PCI, and InfiniBand. Bus 614 enables communications, such as data and instructions, to be exchanged between system components of computer system 602.

The computer system 602 also includes one or more interface devices 616, such as input devices, output devices, and combined input/output devices. An interface device can receive input or provide output. More specifically, an output device can present information for external display. An input device can receive information from an external source. Examples of interface devices include a keyboard, a mouse device, a trackball, a microphone, a touch screen, a printing device, a display screen, a speaker, a network interface card, and the like. Interface devices allow the computer system 602 to exchange information and communicate with external entities (such as users and other systems).

Data storage 618 includes a computer-readable and writable non-volatile or non-transient data storage medium storing therein instructions that define a program or other object executed by processor 610. Data storage 618 may also include information recorded on or in the medium and processed by processor 610 during execution of the program. More specifically, the information may be stored in one or more data structures specifically configured to conserve storage space or increase data exchange performance. These instructions may be persistently stored as encoded signals, and these instructions may cause processor 610 to perform any of the functions described herein. The medium may be, for example, an optical disk, a magnetic disk, or flash memory. In operation, processor 610 or some other processor reads data from the non-volatile recording medium to another memory (such as memory 612), which allows processor 610 to access information more quickly than the storage medium included in data storage 618. The memory may be located in either data storage 618 or memory 612, however, processor 610 manipulates the data within the memory and then copies the data to the storage medium associated with data storage 618 after completing the processing. Various components may manage data movement between storage media and other memory elements, and the examples are not limited to specific data management components. Furthermore, the examples are not limited to specific memory systems or data storage systems.

While computer system 602 is shown by way of example as one type of computer system on which various aspects and functions may be practiced, the aspects and functions are not limited to being implemented on computer system 602 as shown in FIG6 . Various aspects and functions may be practiced on one or more computers having architectures or components different from those shown in FIG6 . For example, computer system 602 may include specially programmed, dedicated hardware, such as an application-specific integrated circuit (ASIC) tuned to perform the specific operations disclosed herein. However, another example may use a grid of several general-purpose computing devices running MAC OS System X with Motorola PowerPC processors and several specialized computing devices running proprietary hardware and operating systems to perform the same functions.

Computer system 602 may be a computer system that includes an operating system that manages at least a portion of the hardware elements included in computer system 602. In some examples, a processor or controller (such as processor 610) executes an operating system. Examples of specific executable operating systems include Windows-based operating systems available from Microsoft (such as Windows NT, Windows 2000 (Windows ME), Windows XP, Windows Vista, or Windows 7 operating systems), MAC OS System X available from Apple Computer, one of many Linux-based operating system distributions (e.g., Enterprise Linux operating system available from Red Hat Inc.), Solaris operating system available from Sun Microsystems, or UNIX operating systems available from various sources. Many other operating systems may be used, and the examples are not limited to any particular operating system.

The processor 610 and the operating system together define a computer platform for which applications are written in a high-level programming language. These component applications can be executable, intermediate bytecode or interpreted code that communicates over a communication network (e.g., the Internet) using a communication protocol (e.g., TCP/IP). Similarly, various aspects can be implemented using an object-oriented programming language such as .Net, SmallTalk, Java, C++, Ada, or C# (C-Sharp). Other object-oriented programming languages can also be used. Alternatively, functional, scripting, or logic programming languages can be used.

Additionally, various aspects and functions can be implemented in a non-programming environment (e.g., documents created in HTML, XML, or other formats) that can present various aspects of a graphical user interface or perform other functions when viewed in a window of a browser program. In addition, each example can be implemented as programming or non-programming elements or any combination thereof. For example, a web page can be implemented using HTML, while a data object called from within the web page can be written in C++. Thus, these examples are not limited to a specific programming language, and any suitable programming language can be used. Accordingly, the functional components disclosed herein may include various elements configured to perform the functions described herein, such as specialized hardware, executable code, data structures, or objects.

In some examples, the components disclosed herein can read parameters that affect the functions performed by these components. These parameters can be physically stored in any form of suitable memory, including volatile memory (such as RAM) or non-volatile memory (e.g., magnetic hard drive). In addition, these parameters can be logically stored in proprietary data structures (such as databases or files defined by user mode applications) or shared data structures (such as application registrations defined by the operating system). In addition, some examples provide both system and user interfaces that allow external entities to modify parameters and thereby configure component behavior.

Although particular embodiments of the disclosed invention are shown and described herein, those skilled in the art will understand that they are not intended to limit the invention, and it will be apparent to those skilled in the art that various changes and modifications (e.g., the dimensions of the various components) may be made without departing from the scope of the disclosed invention, which is defined solely by the appended claims and their equivalents. Accordingly, this specification and drawings are to be regarded as illustrative rather than restrictive. The various embodiments of the disclosed invention shown and described herein are intended to cover alternatives, modifications, and equivalents of the disclosed invention, which may be included within the scope of the appended claims.

Claims (69)

1. A computer-aided method for classifying images of cytological specimens, comprising the following actions: Analyze the first image of the cytological specimen to identify objects of interest within the cytological specimen; A second image of the identified object of interest is displayed to the user; In response to input from the user selecting an object of interest: Determine the primary characteristics of the selected object of interest; Receive a series of values from the user corresponding to a second characteristic of the selected object of interest; Evaluate one or more objects of interest to identify at least one other object of interest that has a property similar to the first property of the selected object of interest and has a second property measurement within the range of said values; and In response to identifying the at least one other object of interest, a third image of the selected object of interest and a fourth image of the at least one other object of interest are displayed to provide the user with a comparison of the objects of interest. 2. The method according to claim 1, wherein the third image and the fourth image are displayed in a comparison view. 3. The method according to claim 1 or 2, wherein the fourth image is provided from the same cytological specimen. 4. The method according to claim 1 or 2, characterized in that the fourth image is provided from a previously stored database of objects of interest. 5. The method according to claim 1 or 2, wherein the third image has a first staining agent, and the fourth image is the same selected object of interest having a second staining agent. 6. The method of claim 1 or 2, further comprising: using the third image to determine and store a classification of the selected object of interest, wherein the classification is determined by the user. 7. The method of claim 1 or 2, further comprising: using the third image to determine and store a classification of the selected object of interest, wherein the classification is determined by a processor. 8. The method according to claim 1 or 2, wherein the analysis of the first image is performed by a processor. 9. The method according to claim 1 or 2, further comprising: using the third image to create a database of categories for selected objects of interest. 10. The method of claim 1 or 2, further comprising: receiving an image of the cytology specimen from a remote workstation. 11. A computer-aided method for navigating images of cytological specimens, comprising the following actions: Analyze the first image of the cytological specimen to identify multiple objects of interest within the cytological specimen; Displaying multiple images, each image including one of a plurality of identified objects of interest within the cytological specimen and including a second image of at least one object of interest; and In response to receiving a user's selection of the second image of the at least one object of interest, the field of view of the at least one object of interest and neighboring objects of interest is displayed. The second image of the at least one object of interest has a first magnification, and The field of view of the at least one object of interest is displayed at a second magnification different from the first magnification. 12. The method according to claim 11, wherein the corresponding identified objects of interest are displayed to the user using a scroll bar. 13. The method of claim 11 or 12, further comprising: using the second image to determine and store a classification of the at least one object of interest, wherein the classification is determined by the user. 14. The method of claim 11 or 12, further comprising: using the second image to determine and store the classification of the at least one object of interest, wherein the classification is determined by a processor. 15. The method according to claim 11 or 12, wherein the analysis of the first image is performed by a processor. 16. The method of claim 11 or 12, further comprising: using the first image to create a database of classifications of the at least one object of interest. 17. The method of claim 11 or 12, further comprising: receiving an image of the cytology specimen from a remote workstation. 18. A system for navigation within an image of a cytological specimen, the system comprising: At least one processor operatively connected to the memory; User interface display; A user interface component executed by the at least one processor, the user interface component being configured to: Analyze the first image of the cytological specimen to identify objects of interest within the cytological specimen; Display a second image of the object of interest; Receive the user's selection of the object of interest; Determine the primary characteristics of the selected object of interest; Receive a series of values from the user corresponding to the second characteristic; Evaluate one or more objects of interest to determine at least one other object of interest that has a property similar to the first property of the selected object of interest and has a second property measurement within the series of values corresponding to the second property; and In response to identifying the at least one other object of interest, a third image of the object of interest and a fourth image of the at least one other object of interest are displayed. 19. The system of claim 18, wherein the user interface component is configured to display the selected object of interest and the at least one other object of interest in a comparison view. 20. The system according to claim 18 or 19, wherein the user interface component is configured to accept user selection of the object of interest to display the selected object of interest in a field of view including the selected object of interest and neighboring objects of interest. 21. The system according to claim 18 or 19, wherein the system is configured to accept user selection of the object of interest and display a menu of cellular properties of the object of interest. 22. The system according to claim 18 or 19, wherein the system is configured to accept selection of cell characteristics from a menu of the cell characteristics and identify additional objects of interest having the selected cell characteristics. 23. The system according to claim 18 or 19, wherein the cell characteristics include any one of morphological characteristics, staining agent, cell size, nucleus/cytoplasm ratio, optical density, contour regularity, color-based criteria, and nuclear density. 24. The system according to claim 18 or 19, wherein the system is configured to identify and display different images of a selected object of interest. 25. The system according to claim 18 or 19, characterized in that the system is configured to identify an object of interest having at least one of a conventional dye, an advanced dye, a color image, and a fluorescent dye, and is configured to accept user selection of the object of interest and display an image of the corresponding object of interest having such dye. 26. A system for facilitating the examination and analysis of cytological specimens, comprising: At least one machine, each of the at least one machine comprising a processor communicatively coupled to a storage device storing computer-executable instructions, the instructions, when executed by the processor, causing the processor to operate as a module of: A recording module is configured to request and permit input of cytological specimen image data, the image data including digital images of the cytological specimen; The image set module is configured to analyze the image data of the input cytology specimens and generate corresponding image sets and their analysis information to enable individual specimen examination based on the feature attributes of objects of interest in the cytology specimen images; and An analysis tool is configured to display the image set and analysis information generated by the image set module to the user, wherein the analysis tool is further configured to collect input and instructions from the user via one or more tools in the user interface, and to acquire and/or analyze additional specimen image data by the recording module and/or the image set module. 27. The system according to claim 26, wherein the input image data further includes one or both of the characteristic attributes and location of the object of interest in the cytological specimen image. 28. The system according to claim 26 or 27, wherein the image set module is configured to analyze the image data of the input cytology specimen and generate a corresponding image set and its analysis information for further individual specimen examination based on the location information of the object of interest in the cytology specimen image. 29. The system according to claim 26 or 27, wherein the computer-executable instructions, when executed by the processor, cause the processor to further operate as an image processing module, the image processing module being configured to extract one or both of the feature attributes and location of the object of interest in the cytological specimen image. 30. The system according to claim 26 or 27, wherein the image set module generates a corresponding image set in response to user input received by the analysis tool. 31. The system according to claim 26 or 27, further comprising: a graphical user interface. 32. The system according to claim 26 or 27, further comprising: a voice recognition user interface. 33. The system according to claim 26 or 27, wherein the image set module is configured to selectively enhance objects depicted in the images of the image set. 34. The system according to claim 26 or 27, wherein the image set module enhances the objects depicted in the images of the image set by varying the contrast and/or brightness of the depicted objects. 35. The system according to claim 26 or 27, wherein the analysis tool comprises one or more of a login module, a homepage module, a patient module, a patient dashboard, and an image browser. 36. The system according to claim 35, wherein the login module is configured to accept user identity credentials including username and password. 37. The system of claim 35, wherein the homepage module includes a user dashboard, a patient list, a search page, and a new patient module, and wherein the user dashboard includes a list of submitted analysis jobs and the status of the submitted analysis jobs. 38. The system of claim 35, wherein the patient dashboard includes cytological specimen images related to the patient identified by the dashboard, and/or relevant analysis results of the patient identified by the dashboard. 39. The system of claim 35, wherein the patient dashboard includes means for submitting new analyses of the patients identified by the dashboard. 40. A system for examining images of cytological specimens, the system comprising: At least one processor, which is operatively coupled to the memory; User interface display; and A user interface component, executed by the at least one processor, is configured to: Analyze the first image of the cytological specimen to identify multiple objects of interest within the cytological specimen; Display multiple images, each image including one of a plurality of identified objects of interest within the cytological specimen and including a second image of at least one object of interest; and In response to receiving a user's selection of the second image of the at least one object of interest, the field of view of the at least one object of interest and neighboring objects of interest is displayed. The second image of the at least one object of interest has a first magnification, and The field of view of the at least one object of interest has a second magnification different from the first magnification. 41. The system of claim 40, wherein the user interface component is further configured to display the second image of the at least one object of interest in a scroll bar. 42. The system according to claim 40, wherein the first magnification is suitable for cytological examination of the at least one object of interest. 43. The system according to any one of claims 40-42, further comprising: highlighting the at least one object of interest in the second image of at least a portion of the specimen. 44. The system according to claim 43, wherein the highlighting includes variations in the contrast and/or brightness of the depicted object. 45. The system according to claim 44, characterized in that a plurality of objects of interest and the second image are displayed in different areas of the display. 46. A method for navigating and examining cytological image data using a processor-controlled device, the image data comprising images of cytological specimens, the images including respective images of objects of interest therein, the method comprising: Analyze the first image of the cytological specimen to identify one or more objects of interest within the cytological specimen; Display one or more images, each image including one of one or more objects of interest within the cytological specimen and including a second image of at least one object of interest; In response to receiving a user's first selection of the at least one object of interest: Display the field of view of the at least one object of interest and neighboring objects of interest. The second image of the at least one object of interest has a first magnification, and The field of view of the at least one object of interest is displayed at a second magnification different from the first magnification. In response to receiving a second selection from the user for the at least one object of interest: Determine the primary characteristics of the selected object of interest; Receive a series of values from the user corresponding to a second characteristic of the selected object of interest; Evaluate at least one object of interest to determine at least one other object of interest that has a property similar to the first property of the selected object of interest and has a second property measurement within the range of said values; and In response to identifying the at least one other object of interest, a third image of the selected object of interest and a fourth image of the at least one other object of interest are displayed to provide the user with a comparison of the objects of interest. 47. The method according to claim 46, wherein the third image is obtained from a library of previously categorized cytological objects. 48. The method according to claim 46 or 47, wherein the third image depicts additional cytological objects in the specimen. 49. An automated method for navigating and examining cytological image data using a processor-controlled device, the image data comprising images of cytological specimens, the images including respective images of objects of interest therein, the device comprising an integrated or otherwise operatively associated user interface and display, the method comprising: In response to one or more user commands received through the user interface, a first plurality of images from the image data are displayed on the display, each of the first plurality of images depicting a corresponding object of interest in the specimen; The user selection of an image from the first plurality of images is detected through the user interface; and In response to the detected user selection, the following image is displayed on the monitor: (a) An image of at least a portion of the specimen, the at least portion of the specimen including a corresponding object of interest depicted in the selected image and neighboring objects in the specimen; and (b) A second plurality of images, each of which depicts a cytological object having (i) properties similar to those of a corresponding object of interest in a selected image from the first plurality of images, and (ii) calculated metrics that are the same as or similar to selected metrics of a corresponding object of interest in a selected image. 50. The method according to claim 49, characterized in that the second plurality of images are obtained from a library of previously categorized cytological objects. 51. The method according to claim 49, wherein the second plurality of images depict additional cytological objects in the specimen. 52. The method according to any one of claims 49-51, characterized in that at least a portion of the specimen is displayed at a magnification suitable for a cytological examination of the corresponding object of interest depicted in a selected image of the plurality of images. 53. The method of claim 52, further comprising: highlighting a corresponding object of interest depicted in a selected image of the plurality of images in an image of at least a portion of the specimen. 54. The method according to any one of claims 49-51, further comprising: Receive input representing a user-suggested classification of the displayed objects of interest in the first plurality of images; and Based on the previously determined classification of the selected object of interest, provide feedback on the user-suggested classification. 55. The method according to any one of claims 49-51, further comprising: Receive input representing a user-suggested classification of the displayed objects of interest in the first plurality of images; and Determine the percentage of user-suggested categories that match the corresponding previously determined categories for the objects of interest. 56. A system for navigating and verifying input cytological specimen image data, the input cytological specimen image data including images of cytological specimens, the system comprising: processor; A display integrated with or otherwise operatively associated with the processor; and The user interface is operatively coupled to the processor and the display. The processor is configured to: A first plurality of images from the image data are displayed on the display, each of the first plurality of images depicting a corresponding object of interest in the specimen; The user selection of an image from the first plurality of images is detected through the user interface. and In response to the detected user selection, the following image is displayed on the monitor: (a) An image of at least a portion of the specimen, the at least portion of the specimen including a corresponding object of interest depicted in the selected image and neighboring objects in the specimen; and (b) A second plurality of images, each of which depicts a cytological object having (i) properties similar to those of a corresponding object of interest in a selected image from the first plurality of images, and (ii) calculated measures that are the same as or similar to those of selected measures from a list. 57. The system according to claim 56, wherein the input cytological specimen image data further includes one or both of the characteristic attributes and location of the object of interest in the cytological specimen image. 58. The system of claim 56, wherein the processor is programmed or otherwise configured to extract one or both of the characteristic attributes and location of an object of interest in an input cytological specimen image. 59. The system according to any one of claims 56-58, wherein the processor is programmed or otherwise configured to generate the first plurality of images in response to user input via the user interface. 60. The system according to claim 59, wherein the user interface includes a voice recognition system. 61. The system of claim 60, wherein the processor is configured to selectively enhance objects depicted in the displayed image. 62. The system according to claim 61, characterized in that the objects depicted in the displayed image are enhanced by changes in contrast and/or brightness. 63. The system of claim 62, wherein the processor is programmed or otherwise configured to acquire the second plurality of images from a library of previously categorized cytological objects. 64. The system of claim 62, wherein the processor is programmed or otherwise configured to acquire the second plurality of images from a group of cytological objects in the specimen. 65. The system of claim 64, wherein the processor is programmed or otherwise configured to display an image of at least a portion of the specimen at a magnification suitable for a cytological examination of a corresponding object of interest depicted in a selected image of the plurality of images. 66. The system according to claim 65, wherein the display includes the user interface. 67. The system of claim 66, wherein the processor is programmed or otherwise configured to display the first plurality of images and images of at least a portion of the specimen, or to display a second plurality of images in different areas of the display. 68. The system according to claim 56, wherein the system is further configured to: Receive input representing a user-suggested classification of the displayed objects of interest in the first plurality of images; and Based on the previously determined classification of the selected object of interest, provide feedback on the user-suggested classification. 69. The system according to claim 68, wherein the system is further configured to: Receive input representing a user-suggested classification of the displayed objects of interest in the first plurality of images; and Determine the percentage of user-suggested categories that match the corresponding previously determined categories for the objects of interest.