US20190133514A1

US20190133514A1 - System and method for optical detection of skin disease

Info

Publication number: US20190133514A1
Application number: US16/179,490
Authority: US
Inventors: Daniel Gareau; Justin Martin
Original assignee: Rockefeller University
Current assignee: Rockefeller University
Priority date: 2013-07-22
Filing date: 2018-11-02
Publication date: 2019-05-09
Also published as: BR112016001399A2; CA2955917C; AU2014293317A1; MX2020002094A; BR112016001399A8; CN105578953A; CA3101279A1; US10182757B2; WO2015013288A2; AU2014293317B2; AU2019203346B2; CA2955917A1; US20190274619A1; WO2015013288A3; US20160166194A1; KR20160061978A; CN110742579A; JP6434016B2; US20150025343A1; JP2019058681A

Abstract

An optical system for the detection of skin disease, such as melanoma, acquires images of a lesion on a subject's skin at different wavelengths and utilizes a sweeping arm rotating about the lesion in a clock-like sweep to produce diagnostically relevant metrics and classifiers from the image data so as to enhance detection of the skin disease.

Description

This application claims the benefits of U.S. patent application Ser. No. 14/051,053, filed Oct. 10, 2013 and U.S. Provisional Application No. 61/857,143, filed Jul. 22, 2013, which are all incorporated by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The invention is directed to systems and methods for optical detection of skin disease and in particular apparatus and methods adapted to detect the presence of melanoma and to distinguish, for example, malignant melanoma from non-malignant dysplastic nevi and/or common nevi, using metrics and classifiers obtained from rotational analysis of image data obtained from a subject's skin lesion. The data obtained may be processed by one or more computer processors and output in a plurality of display modules.

Description of the Related Art

Melanoma, the most lethal skin cancer, incurs immense human and financial cost. Early detection is critical to prevent metastasis by removal of primary tumors. The early lateral growth phase is a vastly preferable detection window to the subsequent phase of metastatic initiation. Optical detection technologies for automated quantitative metrics of malignancy are needed to more accurately guide decisions regarding the need to biopsy and to make preoperative determination of adequate margins for surgical excision. After invasive biopsy or excision, diagnosis obtained by histopathologic evaluation is nearly 100% accurate; however deciding which lesions to biopsy is challenging. Only 3% to 25% of surgically-excised pigmented lesions are diagnosed as melanomas. Hence there is a need for noninvasive screening mechanisms that are both widespread and more accurate.
Dermoscopy is a common dermatological technique to evaluate skin lesions. The dermatoscope typically consists of a light emitting diode (LED) illuminator, a low magnification microscope, and a clear window surface to flatten the skin against. The use of polarization enables partial rejection of deeply penetrating light, which can enhance superficial features of particular diagnostic interest. A digital imaging camera may also be attached to the dermatoscope.
U.S. Pat. Nos. 7,006,223, 7,027,153, 7,267,243, and 7,167,244 describe handheld dermoscopic epiluminescence devices. As noted above, such handheld devices are available commercially which have been adapted for attachment to a cellular phone camera or portable camera.
Methods and apparatus for evaluating optical image data obtained from a skin lesion on a subject's body are taught in U.S. Pat. Nos. 6,208,749 and 7,894,651, assigned to Mela Sciences, Inc. One of the objects of the present invention is to employ algorithms that perform these evaluations with greater sensitivity, specificity and overall diagnostic accuracy, and which can be used to produce diagnostically relevant quantitative metrics in real time, in some cases without further per lesion evaluation.
Another object of the invention is to combine a dermatoscope, digital camera and automated screening by computer vision to bridge the diagnostic accuracy gap between invasive and noninvasive pathological analyses. Though the sophistication of the human brain may never be matched by computers, the present invention leverages three critical advantages over traditional dermatologist screening: standardization, quantification and the enhanced ability to perform brute-force calculations. As outlined in the following description and claims, objective analytical diagnostic technologies have the potential to dramatically improve the diagnostic accuracy of widespread melanoma screening.
Using rotational analysis of image data obtained from a skin lesion yields improved diagnostic accuracy compared to the prior art. The novel mathematical descriptors generated by the polar transformation of the image data may be trained on a set of skin lesions of known pathology to yield classifiers which provide a percent likelihood that a given lesion is malignant melanoma, paired with a percentage uncertainty for the prediction. The invention also provides enhanced opportunities to visualize the data obtained. In addition to a standard red-green-blue (RGB) image of the lesion, the present invention provides the user (doctor or patient) with a version of the image with suspicious regions highlighted, and the user may toggle between these display modes. The user may cycle through a set of gray scale images obtained at different wavelengths. The display may be toggled between x-y coordinates and a brightness map in polar coordinates (r, θ).

SUMMARY OF THE INVENTION

In one aspect, the invention is an apparatus for detecting skin disease in a lesion on a subject's skin, comprising: a mechanical fixture defining a distal plane against which a lesion on the subject's skin is pressed; a camera adapted to obtain image data from the lesion; a processor adapted to process the image data with a clock-like sweep algorithm to obtain metrics and/or classifiers defining the rotational symmetry of the pigmented lesion; and an output device to indicate a likelihood of the presence or absence of skin disease in the subject obtained from the processed image data.
The clock-like sweep algorithm evaluates the brightness of pixels on a line segment connecting the center of the lesion image with the lesion image border as the line segment rotates around the center of the lesion with one end of the line segment fixed at the center of the lesion image. Rotational symmetry refers to different information obtained on the line segment at different angular positions. Thus, a circle with uniform brightness throughout exhibits perfect rotational symmetry. However, if the distance from the border of the lesion to the center of the lesion is different at different angular positions; or if the brightness of pixels differs at different positions on the line segment, or at different angular positions of the line segment, then the lesion is not rotationally symmetric, The asymmetry may be quantified and used to produce diagnostically relevant metrics.
The camera is adapted to obtain multispectral images, as the skin lesion is illuminated with an array of LEDs that emit light of different spectral profiles (including, importantly, one or more LEDs that emit light in the non-visible UV range, such as 300 nm to 400 nm). The camera acquires M images, storing each pixel in the image as a set of M numbers that form a spectral measurement, which are then fixed as the weighted sum of N chromophores.
In another aspect, the invention is embodied as a method for obtaining an indication of a likelihood of the presence or absence of skin disease in a subject, comprising the steps of obtaining image data from a subject's skin with a camera; and processing the image data with a computer processor adapted to implement a clock-like sweep algorithm to obtain diagnostically relevant metrics and/or classifiers defining the rotational symmetry of a pigmented lesion on the subject's skin. In the method, at least one processor transforms the image data into diagnostically relevant metrics and/or classifiers defining the rotational distribution of [a] the spatial texture features or [b] the brightness features or [c] the features of the edge/border or [d] the color variation of a lesion on the subject's skin or [e] variations in the features of the pigmented network including the length, shape brightness and organization of the lesion segments or [f] the oxygen saturation of the tissue as defined by the amount and ratio of oxyhemoglobin and deoxyhemoglobin.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an exploded view of a clinical apparatus according to one embodiment of the invention.

FIG. 1A depicts the assembled apparatus of FIG. 1.

FIG. 2 depicts a cellular phone having an integrated processor and camera and an attached fixture for mounting the camera against a subject's skin according to another embodiment of the invention.

FIG. 3A and FIG. 3B depict the polar transformation of image data for a nevis.

FIG. 3C and 3D depict the polar transformation of image data for a malignant melanoma.

FIG. 4 is a topographical map of a lesion constructed from mask images according to a display module in one embodiment of the invention.

FIG. 5 is a schematic view of a solid false color display according to a display module in another embodiment of the invention.

FIG. 6 depicts asymmetry measurements obtained with respect to bisecting axes which are rotated with respect to the image.

FIG. 7 depicts another display module according to the invention.

FIG. 8 is a visualization of the discriminative power of metrics used in the method of the invention.

FIG. 9 is a Receiver Operator Curve (“ROC curve”) built using classifiers according to methods of the invention.

FIG. 10 is a flow chart depicting the operation of a server application controlling the image acquisition and data analysis process according to one embodiment of the invention.

FIG. 11 depicts a transformative process performed on image data from a skin lesion to obtain a metric relevant to the pigmented network regularity in a skin lesion.

DETAILED DESCRIPTION OF THE INVENTION

System and Apparatus

A system according to the invention comprises a camera, a mechanical fixture for mounting the camera in a fixed position with respect to the subject's skin, at least one processor adapted to perform the clock sweep algorithm, and at least one output device.
In the clinical embodiment depicted in FIG. 1, the camera is a circuit board level charge coupled device (CCD) detector imaging array mounted on a fixture that can be pressed against the subject's skin. In this embodiment, the fixture 100 defines a distal plane formed by a flat transparent plate 12 of glass, polycarbonate, polymethylmethacrylate (PMMA) or the like, that may be pressed against the subject's skin so that the lesion stays in one plane when the image is obtained. Plate 12 may be mounted on a spacer, such as nose cone 14, which protects the camera lens aperture and provides an optimal distance between the illuminating and imaging apparatus and the lesion.
An illumination apparatus, such as LED mounting ring 15, positions LEDs around the optical axis of the camera and may be located proximally of the distal plane which frames the skin lesion, but still forward of the imaging apparatus. The illumination apparatus includes a set of devices that emit light of different spectral profiles to illuminate the skin lesion with light at desired wavelengths. In FIG. 1, the LED mounting apparatus comprises a ring of light emitting diodes (LEDs) 16 each capable of emitting light at a specified wavelength in a range of 300 nm to 950 nm while the camera sequentially acquires images at the specified wavelengths. The apparatus may utilize commercially available LEDs which are inexpensive and widely available with various spectral characteristics. However, if more accurate and narrow spectra are desired, laser illumination elements may also be used.
The LED wavelengths are selected based on the methods used to extract relevant information from the image data to identify diagnostically relevant patterns in the lesion. For example, it is known in the art that blue light is absorbed by melanin (one of N chromophores in the skin). Thus, at least one of the LEDs in the array, and preferably a plurality, emit light in the violet-indigo-blue wavelength ranges, 400-500 nm. Blood absorbs in the green, so that at least one of the LEDs in the array and preferably a plurality, emit light centered at the 525 nm wavelength. Pigment at the deepest portion of a lesion, in a relatively deep lesion, has absorption shifted to the red, so that one or more LEDs emit light in the range of 600 nm to 750 nm, and even into the infrared (IR) (780 nm and above) which may be helpful to determine the deepest portion of a lesion to be excised, for example. Illumination in the non-visible ultraviolet (UV) range to obtain information about the skin lesion is another novel aspect of the invention. Thus at least one, and preferably a plurality of LEDs in the array, are adapted to illuminate the skin at a wavelength of 300 nm to 400 nm. At least one, and preferably a plurality of LEDs are adapted to illuminate the skin in accordance with the absorption profile of eu-melanin as distinct from the absorption profile of pheo-melanin. In this way, at each angular position, as the camera acquires M images at different wavelengths, each pixel in the image is stored as a set of M numbers that form a spectral measurement which may be fit as the weighted sum of N chromophores in the skin lesion.
In embodiments, particularly where off-the-shelf LEDs are used, the illumination system may comprise a set of LEDs having illumination spectra that overlap. In this case, correction may be made digitally, providing a processor adapted to remove overlapping regions of the spectra, thereby improving spectral resolution. For example, a set of LEDs may have spectra L₁, L₂, L₃. . . L_nthat overlap, and image data obtained at one illumination, I_L_i, may be used to correct the illumination at I_(L_i-1) by subtracting C*(I_L_i) from I_(L_i-1), where C is a constant related to the amount of overlap between the two spectra.
This correction may be programmed in advance based on the specifications from the manufacturer of the LED. Alternatively, the apparatus may be provided with a spectrometer to measure the emission spectra of the LED or other illumination device, and that measurement may be used to implement the correction for overlapping spectra. A fiber optic element located distally of the illumination devices provides the data from the actual emission spectra of each illumination device to the processor to perform the steps described above for resolving the overlapping spectra.
Housing 101 comprises one or more lenses 121 mounted in the mechanical fixture 122 between the camera 13 and the distal plane 12 to focus the camera on the distal plane, and may comprise one or more filters, such as a polarizing filter or chromatic filter to condition the light emitted by the LEDs or reflected by the lesion and captured by the camera. The lenses are designed to minimize optical aberrations and maximize optical transport throughput at the wavelengths of the illumination light.
The processing functions may be shared between first and second processors. The first processor is typically an onboard processor such as circuit board 11 adapted to drive the camera and illumination system to acquire the image data and provide a real time information display to the user. The first processor may transmit image data to a second processor adapted to perform data-intensive processing functions which cannot readily be provided as real time display. The second processor may deliver messages back to the first processor for display. The second processor, if present, is typically a remote processor. The second processor may create data files, image files, and the like, for later use.
In FIG. 1, the first processor is circuit board 11, adapted to drive the camera and the illumination of the LEDs, while the camera sequentially obtains M images at the selected wavelengths. The first processor may process the image data so obtained to produce a display on a liquid crystal display (“LCD”) view screen 19. Outer housing 191 encloses the LCD screen and circuit board. Cable 17 is used to attach a second processor and other components to the apparatus.
As shown in FIG. 1A, fixture 100 is provided with a lens holder 102 attached to the nose cone by a spacer 103, sized to provide a self-contained assembly that can be manipulated with one hand and positioned against a subject's skin lesion.
Provided sufficient image data are obtained at different wavelengths, diagnostically relevant areas of interest on the skin lesion may be identified and differentiated using a variety of display modules. Thus, colors correlating to blood vessels within the lesion border; colors correlating to blue and blue white structures in the lesion; colors correlating to pigmented networks which may be regular or irregular; colors correlating to negatively pigmented networks; patterns of oxygen saturation; and patterns of eumelanin and pheomelanin (which have different absorption profiles) all may be highlighted and separately displayed with the display modules described below.
The processor(s) is adapted to transform the image data into diagnostically relevant metrics and/or classifiers indicating the likelihood of disease or presence or absence of features such as those identified above by defining [a] the spatial texture features or [b] the brightness features or [c] the features of the edge/border or [d] the color variation of a lesion on the subject's skin or [e] variations in the features of the pigmented network including the length, shape, brightness and organization of the lesion segments or [f] the oxygen saturation of the tissue defined by the amount and ratio of oxyhemoglobin and deoxyhemoglobin, and these characteristics may be displayed in one or more display modules to render a version of the lesion image depicting the lesion, or segments of the lesion, with one or more of these features of interest highlighted on a display for the user. In one display module, depicted in FIG. 4, the N spectral images are processed to form a topographical map of the lesion pigment from mask images obtained at each of a plurality of specified wavelengths. A mask image is defined as an image having pixel brightness 1 inside the image border and 0 outside the image border. Image data obtained at each of the plurality of wavelengths will yield a different mask image. Adding mask images at different wavelengths permits the construction of a topographical map. FIG. 4 is a black and white rendering of an original color image. This display module, which approximates a three-dimensional display, may be useful to identify the appropriate excision borders for a skin lesion.
In another display module depicted in FIG. 5, the N sequential images obtained by the camera are processed to render a display of the lesion in which areas of interest in the lesion are shown in solid “false color.” The solid false colors in the display, for example, light brown, dark brown, red, black, blue/gray, and white, may be counted and the number of colors displayed, The solid false colors may correspond to detected regions of interest in the lesion, such as a region consisting of blood vessels within the lesion border; blue or blue-white skin structures a pigmented network that is labeled as regular or irregular; negative pigmented network (a pattern of lightly pigmented skin within the lesion borders); and an abnormal pattern of oxygen saturation as defined by spectral fitting using near-infrared wavelength illumination. The display may toggle between a color image created from the N spectral images and the same image with a region or regions of interest indicated at the selection of the user. The highlighted features R₁, R₂, R₃. . . R_nare depicted schematically in FIG. 5 as rectangles. In an actual embodiment, the shape of these highlighted features corresponds to the shape of the underlying feature in the skin lesion.
The display module of FIGS. 3A through 3D depicts the analytical advantage of the polar transformation of the visual data obtained from a skin lesion according to the present invention. FIG. 3A depicts conventional image data of a non-malignant skin lesion at a given wavelength. FIG. 3B depicts the polar transformation of the image data from FIG. 3A, in which the x-axis represents an angular position of the sweeping arm, and the y-axis represents the brightness values along the sweeping arm at each angular position. FIG. 3D depicts the same transformation of the image data from FIG. 3C, where the underlying image data is from a malignant melanoma. This display module provides a visual impression of the brightness variation in the malignant skin lesion, which is much greater than the variation in brightness of the non-malignant lesion. Even before quantitative evaluation, presentation of this image data in polar coordinates provides a new vantage to view a clinically relevant metric.
FIG. 11 depicts a series of data transformation steps which identify network nodes and branch segment lengths in skin lesion image data and provides a metric of network irregularity. In this context, “regular or irregular pigmented networks” refers to a measure of regularity defined by the branch segments of the network in terms of the length and width of each branch segment and the collective angular variation of those values. Important metrics are generated by identifying and characterizing pigmented networks from the image data, including the steps of: a) identifying branch segments; b) locating the coordinates of the centroid of the branch segment; c) determining the length and width of the branch segment and the ratio of length to width (or other mathematical combination of length and width); d) determining the brightness of the segment; e) determining the variation in brightness of the segment over different illumination wavelengths, I_L1, I_L2, I_L3 . . . I_Ln; f) determining the number of nodes (where two branch segments meet), the number of ends, and the ratio of the number of nodes to the number of ends (or other mathematical combination of the nodes and ends).
In identifying pigmented networks, especially to distinguish a pigmented network from a blood vessel structure, the variation in brightness across wavelengths is useful, because the blood vessel structure absorbs at different wavelengths than the pigmented reticular structure.
The ratio of length to width is used to differentiate globular pigment patterns (where the ratio is closer to 1), from reticular patterns (where the ratio is much greater than 1). Variation in the ratio across the angular sweep produced is another metric correlated with melanoma.
The ratio of the number of nodes to the number of ends produces a metric correlated with melanoma because a broken network (i.e., a lower node:end ratio) indicates melanoma.
In addition to LCD viewer 19, the apparatus may comprise additional display outputs, adapted to display the M black-and-white or color coded scale images taken at M wavelengths as views in sequence or in a selectable manner, which may be facilitated by a server application between a computer and the data acquisition device, Data analysis of the multispectral imaging described herein was performed in the Matlab environment. However, transferring these program elements to a different programming platform is within the skill of one having ordinary skill in the art and this transfer is contemplated for commercial applications.
The camera may also be controlled with a server application that facilitates the image acquisition process and which can be operated independently or controlled through any separate software system capable of file input and output and simulating keystrokes. The server application acts as a bridge between the data gathering process and the data analysis code, to power the LEDs that illuminate the sample, to send image acquisition triggers to the camera, and to receive image information from the camera for data analysis in an efficient manner. The server application works by waiting for keystrokes (real or simulated) using a Windows message loop, which it then interprets and uses to send different commands to the camera. Additional data transfer between the server application and third party programs is accomplished using standard file input/output (“I/O”) functions.
This server may be developed as a console application in C++ computer language, for example, with the ability to be re-implemented as a windows application, to handle image acquisition and changing resolution, exposure time and gains settings with the ability to add additional functionality as necessary. By enabling the server to be controlled by keystrokes, it can be used on its own to acquire images from the camera or in conjunction with third party applications that can simulate keystrokes. Total acquisition time for imaging 14 different wavelengths of light can be reduced to about 30 seconds or less (as opposed to around 60 seconds using software provided by the camera manufacturer). This server also enables a live feed display, enabling the user to position the assembly 100 around a suspicious lesion, for example, with a frame rate of at least 1.3 frames/second. Additional features may be included in the script to prevent accidental keyboard input from interfering with the server application while it is being controlled by a third-party application.
The functionality of the server application is enhanced by code that controls the lighting process, allowing for images to be taken at different wavelengths of light and with exposure times individually suited to each wavelength, as well as code that enables the images to be displayed as a live feed either on a monitor or on a small screen attached to the imaging device.
The flow chart of FIG. 10 depicts the flow of commands and data acquisition and analysis using the server application. A master script 400 running on a first computer provides actual or simulated keystrokes to the server application 420, which controls image acquisition triggers through camera serial input 480 and then accesses camera image buffer 482 to return image data to the server application. The server application powers the LED array 460 through microcontroller 430. Once obtained, image data is processed by the master script in a data analysis code module 410. The server application 420 provides data to drive LCD screen output 450 and provides the master script 400 with live feed data for display on the computer screen 440 of the first computer.
In the embodiment depicted in FIG. 2, the camera and processor are integrated in a cellular phone 20 (which includes “smart phones”). Many commercially available cellular phones have adequate camera capabilities and processing capabilities to implement the methods according to the invention. Cellular phones sold under the iPhone® and Android® brands, and many others, have the capability to download server applications to implement the methods described herein.
According to the embodiment of FIG. 2, a mechanical fixture 22 is attached to the cellular phone so that the camera can be securely mounted while the fixture is pressed against the subject's skin. The distal end of the fixture 22 resembles a dermatoscope, and defines a plane 221 against which the subject's skin lesion is pressed to obtain an image while the camera is held in a fixed position. The fixture 22 may include an illumination system in the distal portion 223 of the fixture including an array of LEDs similar to the CCD camera embodiment described above, and/or polarizing or chromatic filter to enable partial rejection of the illuminating wavelengths. In this case, the fixture may be adapted to disable the camera's built-in flash. Alternatively, the processor may be adapted to utilize the built-in flash system provided with the cellular phone.
As with the clinical apparatus, external server applications may be adapted to drive the camera provided with the cell phone and external illumination systems. The cellular phone or smart phone generally has a screen which serves as the output device which provides the user with an indication that a skin lesion is melanoma. The output may take the form of a percentage likelihood that a skin lesion is melanoma, together with a percentage uncertainty, or the program may provide the user with a qualitative message, such as “suspicious lesion: see your dermatologist.”

Methods, Metrics and Classifiers

The methods according to the invention may be described as a series of conceptual “steps.” As would be apparent to the person of ordinary skill in the art, the steps may be followed sequentially, or in an order different from the order stated; the steps may be done in parallel, done at the same time, or done iteratively, without departing from the scope of the invention. Describing a step as the “first step” or “next step” is for convenience only. The image data obtained from a subject's skin lesion may be manipulated by computer according to these steps and output to display modules.
The first step of the method consists of obtaining image data from a subject's skin with a camera. Generally, this means photographing a lesion on the skin. The resulting image data will comprise data from the lesion and the surrounding skin, and may include data which are not part of the lesion or surrounding skin, including hair, markings made by a dermatologist or other data elements that are not analyzed and simply need to be removed from the image. To complete this step, the processor may replace pixel brightness and color values of the hair-containing locations with pixel brightness and color values of the skin underlying or immediately adjacent the hair, for example.
The image data consists of pixel gray-scale or brightness information in M different color layers. As used herein, a “multispectral image” is an image obtained at a plurality of wavelengths or “layers,” so that each pixel in the image is associated with M numbers that form a spectral measurement, and each is a brightness or gray scale measurement at a different color layer. Thus, the image data consists of M images sequentially acquired by the camera while illuminating the skin at wavelengths that range from 300 nm to 950 nm. The spectral measurement is fit as the weighted sum of N chromophores, corresponding to the number M of images obtained. Typically, pixel brightness information is obtained at least in the red-green-blue (“RGB”) layers, but pixel brightness information is also preferably obtained for other spectral bands. Relevant information is obtained using illumination and detecting reflected light in the visible and non-visible range, including the blue and UV range at 300 nm to 500 nm, and even more particularly in the non-visible 300 nm to 400 nm UV range.
As used herein, “chromophores” refers to color components found in a skin lesion, such as melanin, oxygenated hemoglobin and deoxygenated hemoglobin. Generally, at least these three have distinct absorption profiles such that the spectral images can be analytically fit as the weighted sum of at least these three chromophores. However, skin contains water, which absorbs in the UV, bilirubin, which has a distinct absorption in the visible spectrum, and potentially could be found to contain other diagnostically relevant components, such that a measurement could be fit as a weighted sum of N chromophores, wherein N is 4, 5, 6, or more chromophores.
Once the image data is obtained, the shape and center of the lesion are identified. The first step in determining the shape is known as “segmenting” and various computer implemented techniques known in the art may be used to identify the shape and border of a lesion. Briefly, segmenting results in a mask being applied so that pixel brightness at a given wavelength is reduced to a mask image, in which pixels have brightness value of 1 inside the lesion and 0 outside the lesion, A “mask” as used herein is an image having a brightness value of 1 inside the image border and 0 outside the image border.
In a subsequent step, the center of the lesion (or dose approximation of the center) is determined. The center of the lesion may be calculated as the center of mass or geometric centroid of the mask image, such that each region of the lesion shape is treated as having identical density. Alternatively, the center of mass may take into account the variation of brightness in the shape. Unless stated otherwise, in the following examples, the center of mass is obtained from a mask image, such that the lesion is treated as having uniform brightness to determine the center of mass. As the image will have a different mask and therefore a different border at each wavelength, the image at each wavelength may be associated with a respective center, and the distance between the “centers” (“Δr”) may be used with other metrics. The variance (“var Δr”), range (“range Δr”) and mean (“mean Δr”) may also be combined into classifiers.
A sweeping arm is a line segment connecting the center of the lesion to the border. The “clock-like” sweep as used herein, means rotating the sweeping arm about the fixed center of the image in either a clockwise or counter-clockwise direction to obtain information about the pixels on the sweeping arm as a function of rotation angle. To obtain metrics from the image data, the sweeping arm rotates around the center with one end fixed at the center for 2 pi (2π) radians or 360° (one complete sweep). Data is sampled at regular intervals of radians or degrees. FIG. 7 depicts the clock sweep arm r at an angle θ with respect to the vertical. On the left hand side of FIG. 7, borders of the lesion at three different wavelengths are shown. On the right hand side of FIG. 7, the brightness of the pixels on the sweeping arm is plotted as a function of angular position. The data obtained in the sweep may be processed into a series of metrics and/or classifiers which cannot be obtained by evaluation of image data which have not been transformed into polar coordinates.
As used herein, “metrics” are values calculated from the image data which bear a correlation to disease states (melanoma in the preferred examples). Correlations of metrics to disease states may be obtained from a sample of lesions obtained from human subjects containing melanoma and nevi and applying two-sided unpaired t-tests. Table 1 below tabulates P-values for preferred metrics in two-sided unpaired t-tests applied using the methods of the invention to a sample including melanoma and non-cancerous nevi (n=115 samples).

TABLE 1

V1	Angular brightness range	0.00405135
V2	Mean standard deviation (S.D.) of brightness	0.00076328
V3	Range in S.D. of brightness	0.00216321
V4	Standard deviation (S.D.) of S.D. in radial	0.00010983
	brightness over all angles
V5	Mean absolute brightness shift between	3.96E−05
	successive angular positions
V6	S.D. of absolute brightness shifts	0.00065462
V7	Sum of the brightness shifts over full sweep	3.96E−05
V8	Maximum border asymmetry	0.25013811
V9	Border asymmetry evaluated at 90° with respect	0.81368149
	to the minimum asymmetry axis
V10	Lesion border length/lesion area	0.13076045
V11	Mean lesion demarcation (edge slope)	0.00954458
V12	S.D. of lesion demarcation	0.62700467
V13	Fractal dimension	0.08774601
V14	Lesion brightness variation over all lesion	0.19014081
V15	Mean demarcation (edge slope) fit error	0.00075154
V16	S.D. demarcation (edge slope) fit error	0.04725435
V17	Lesion brightness variation over all lesion	0.4408405
V18	Mean length/area of pigment segments	0.09845662
V19	S.D. length/area of pigment segments	0.10489129

Metrics V1 through V7 and V14 capture measurements and statistical information relating to the variation in brightness of pixels on the sweeping arm in relation to other pixels on the sweeping arm, and over different angular positions of the sweeping arm. Metrics V8 through V13 and V15 through V19 capture measurements and statistical information relating to the edge characteristics and presence of reticulated structures in the lesion.

One metric that may be obtained from the angularly sampled data is the angular brightness range (V1), defined as the maximum value of mean brightness on the sweeping arm minus the minimum value of mean brightness over the full rotational sweep. Thus, the mean brightness of the pixels on the sweeping arm is calculated at each angular sample position of the sweeping arm, and the minimum value calculated is subtracted from the maximum value to obtain V1.
The right hand side of FIG. 7 depicts the angular brightness range of pixels on the sweeping arm (V1) as a function of angular position. The mean standard deviation of brightness (V2) of pixels on the sweeping arm is depicted as the vertical line associated with each angular position. Large variations in V1 and a large range of V2 correlate to melanoma. Another metric that may be obtained is the range in standard deviation of brightness (V3). A standard deviation is obtained from all the values of brightness on the sweeping arm at each angular position and the range of these values over all angular positions is calculated to obtain (V3). Another metric is the standard deviation over all angles of the standard deviations at each angular position (V4).
Other metrics evaluate the brightness shift (absolute value) at successive angular positions (V5), the standard deviation of the absolute value of the brightness shift over all angular positions (V6), and the sum of the brightness shift (absolute value) over all angular positions (V7).
The person of ordinary skill in the art of computer-implemented diagnostic analysis of dermoscopic images will recognize that the angularly sampled spectral image data lend themselves to mathematical combination and statistical manipulation once the data is obtained, so that the foregoing list of metrics having correlation to disease states is not exhaustive.
The maximum border asymmetry (V8) is another useful metric, along with the border asymmetry perpendicular to the axis of most symmetry (V9). Border asymmetry was obtained by converting the lesion segment in the blue channel to a binary mask and flipping the binary lesion about a bisecting axis. Thereafter, the axis was rotated in 10 degree increments from zero to 180 degrees to obtain 18 samples of asymmetry as a function of analysis axis. The subtraction of the original mask from its flipped counterpart yielded a map where overlapping regions had a zero values (1−1=0), regions not occupied by either the original or flipped mask had zero values (0−0=0) and regions of mismatch had an absolute value of 1 (1−0=1 or 0−1=−1). The absolute value for a perfect circle would be zero everywhere and the sum would be zero, indicating perfect symmetry. Real lesions had mismatched areas, which lead to non-zero values in the subtraction map, which when summed and divided by the sum of just the original mask, equaled the fractional area of mismatch, and represented the asymmetry of the border of the lesion. The angle at which the minimum asymmetry factor occurred was designated as the axis of most symmetry. Then, the asymmetry of the lesion was evaluated at 90 degrees with respect to the symmetry axis. The individual asymmetry images are depicted in FIG. 6.
Some of the metrics obtained from scanning and analysis of the pixel brightness information are obtained for a given wavelength. Other metrics require a combination and/or comparison of image data obtained at different wavelengths. Regions of interest in a lesion may be associated with different colors, including blood vessels (red) within the lesion border, blue or blue-white skin structures, pigmented networks (associated with eumelanin (brown) or pheomelanin (red).
The border roughness metric (V10) is the length of the border of the lesion segment squared divided by the area of the lesion. For a circle, this would be the circumference squared divided by the area,
Initially, the clock sweep may be used to enhance the determination of the border. An edge fit algorithm runs during the clock sweep and utilizes the variation in pixel brightness at the edge of the lesion shape to iteratively determine a more accurate edge.
The “edge slope” metric (V11) is the mean gradient in brightness at the border during the transition from dark (inside the lesion) to light (outside the lesion) over the full sweeping arm rotational range. The standard deviation of edge slope over all angular positions produces the standard deviation of lesion demarcation (V12). An edge fit algorithm may be used to produce a function defining the border of the lesion from the edge slope which also produces edge slope fit error (V15). An edge slope fit error for the standard deviation of lesion demarcation (V16) may be similarly obtained. The fractal dimension (V13) is another measure of the border irregularity which may be calculated according to known methods. The length to area ratio of pigment segments (V18) and standard deviation of this ratio (V19) are also metrics which bear correlation to melanoma.
FIG. 8 depicts the discriminative power of each of the metrics V1 through V19. Each box in the chart represents a range of values for a given metric according to whether the skin lesion in the sample is a melanoma or nevus (as determined by pathologist screening); therefore, each metric is associated with two columns of data points and two boxes. The vertical lines in each column represent barriers for atypical data and the points above and below represent potential outliers. Thus, the larger the vertical displacement of the boxes for nevus and melanoma, the more discriminating the metric. A shorter overall height of the boxes in the chart represents less uncertainly in the prediction for that metric.
As used herein “classifiers” are combinations of metrics in functions built using multivariate methods to increase the predictive ability of the method to distinguish melanoma from nevi. Classifiers may be obtained and optimized according to known techniques by maximizing the performance of a set of classifiers in receiver operator curve (“ROC”) maximization. ROC maximization for classifiers distinguishing melanoma from nevi are reproduced in FIG. 9, which plots specificity (true negative rate) versus sensitivity (true positive rate), such that a maximum area under the curve in FIG. 9 represents an accurate classifier.
The output of a classifier is a percent likelihood that a lesion is melanoma, which may be coupled with a percent error or uncertainty for the classifier. This can be output for the user in any desired format. A dermatologist may want to see the underlying statistical information displayed as numbers and graphs, either on the device LCD screen, or on the screen of the computer communicating with the device. The ordinary patient may prefer an intuitive system of identifying potentially dangerous lesions, where the lesions most likely to be melanomas are identified with a red light and the least dangerous with a green light.
In order to develop the classifiers, a sample of nevi of known pathology was obtained and classifiers were developed using a “training” subset of the sample using both linear techniques (such as regressions and linear discriminant analysis) and nonlinear techniques (such as neural networks and decision tree algorithms). The following linear classifier is an example of a classifier developed from a training set having some predictive ability to discriminate between nevi and melanoma:
L=0.16*range−0.87*edge+0.68
where range and edge are metrics defined above and L represents a classifier that may be compared to a threshold to yield a classification of melanoma or nevus. “Training” was possible because the pathology of the lesions was known from prior pathologist screening, and the metrics and constants may be selected to maximize the area under the ROC curve. Subsequently, the “trained” classifiers are applied to lesions having unknown pathology. (In the experimental setting this means that the investigator was blind to the pathology of the lesions and did not adjust the classifiers; in the real world setting the device will typically be applied only to lesions having unknown pathology, and the thresholds and classifiers will be pre-programmed) As would be apparent to one of ordinary skill in the art, a larger training sample and slight variation of the metrics will likely yield improved classifiers, without departing from the scope of the invention. Once obtained, the classifiers are applied to the image data of lesions whose pathology is unknown. According to the invention, a selectivity/specificity of 86%/91% was obtained, with an overall diagnostic accuracy of 89%. This result is expected to improve with routine optimization.
The computer program reproduced in Table 2 describes and enables in detail the data processing steps described herein.
The foregoing description of the preferred embodiments is for illustration and is not to be deemed as limiting the invention defined by the following claims. The primary application of the invention is to detect melanoma in humans and to distinguish cancerous from non-cancerous lesions. However, in principle, the apparatus and methods have broad application in the detection and display of other skin diseases. Moreover, using the dock sweep method of analyzing multispectral image data according to the invention lends itself to the development of improved metrics and more discriminating classifiers for the detection of melanoma, without departing from the scope of the invention. The foregoing descriptions of a clinical apparatus and cellular phone apparatus enable the person of ordinary skill to practice variants thereof without departing from the scope of the invention.

Claims

What is claimed is:

1. An apparatus for detecting skin disease in a lesion on a subject's skin, comprising:

a mechanical fixture defining a distal plane against which a lesion on the subject's skin is pressed;

a camera adapted to obtain image data from the lesion;

a processor adapted to process the image data with a clock-like sweep algorithm to obtain metrics and/or classifiers defining the rotational symmetry of the pigmented lesion; and

an output device that indicates a likelihood of the presence or absence of skin disease in the subject from the processed image data.

2. The apparatus according to claim 1, further comprising one or more lenses mounted in the mechanical fixture between the camera and the distal plane to focus the camera on the distal plane.

3. The apparatus according to claim 1, further comprising an illumination system attached to the mechanical fixture comprising a set of devices that emit light of different spectral profiles in a wavelength range of 300 nm to 950 nm.

1. apparatus according to claim 1, wherein said illumination system includes at least one LED that emits light in a wavelength range of 300 nm to 400 nm.

5. The apparatus according to claim 2, wherein the camera has an optical axis and said illumination system comprises a plurality of LEDs arranged in a ring around the optical axis.

6. The apparatus according to claim 1, further comprising a server application that facilitates the image acquisition process that can be operated independently or controlled through a separate software system capable of file input and output and simulating keystrokes.

7. The apparatus according to claim 1, further comprising a second processor, adapted to receive image data from the first processor, transform the image data into enhanced image data, and transmit the enhanced image data to the first processor, or directly to a display device or a storage device.

8. The apparatus according to claim 1, further comprising a display adapted to display a topographical map of the lesion from mask images formed at each color channel.

9. The apparatus according to claim 1, wherein the processor and the camera are integrated on a cellular phone and the mechanical fixture is adapted to attach to the cellular phone.

10. apparatus according to claim 1, wherein the processor is adapted to transform the image data into diagnostically relevant metrics and/or classifiers defining the rotational distribution of [a] the spatial texture features or [b] the brightness features or [c] the features of the lesion edge/border, including the sharpness with which the lesion borders normal skin or [d] the color variation of a lesion on the subject's skin or [e] variations in the features of the pigmented network including the length, shape, brightness and organization of the lesion segments or [f] the oxygen saturation of the tissue as defined by the amount and ratio of oxyhemoglobin and deoxyhemoglobin.

11. The apparatus according to claim 1, wherein the processor is adapted to sequentially acquire M images, store each pixel in the image as a set of M numbers that form a spectral measurement, and fit the spectral measurement as the weighted sum of N chromophores.

12. The apparatus according to claim 11, further comprising a display adapted to display one or more regions of interest in the lesion in solid false color and to toggle between a red green blue color image of the lesion rendered from three or more of the M images and the same image with one or more regions of interest displayed.

13. The apparatus according to claim 12, wherein the region of interest is selected from the group consisting blood vessels within the lesion border; blue and/or blue white structures in the lesion; regular or irregular pigmented networks; negatively pigmented networks; patterns of oxygen saturation; and patterns of eumelanin and pheomelanin,

14. A system for detecting skin disease, comprising:

a cellular phone having an integrated processor and camera, wherein

the camera is provided with an external illumination system or provided with an integral flash adapted to obtain image data from a subject's skin, and

the processor is adapted to process the image data with a clock-like sweep algorithm to obtain metrics and/or classifiers defining the rotational symmetry of the pigmented lesion to produce a display depicting regions of interest in the subject's skin and/or an indication of a likelihood of the presence or absence of skin disease in the subject.

15. The system according to claim 14, further comprising a mechanical fixture mounting against the subject's skin and securing the camera in a fixed position, wherein the fixture defines a distal plane against which a lesion on the subject's skin is pressed.

16. The system according to claim 14, further comprising one or more lenses mounted in said mechanical fixture between the camera and the distal plane to focus the camera on the distal plane.

17. The system according to claim 14, further comprising an illumination system external to the camera attached to the mechanical fixture.

18. The system according to claim 14, wherein the illumination system comprises a set of M light emitting diodes that emit light having different spectral profiles.

19. The system according to claim 17, wherein the external illumination system disables any built in illumination system integral to the camera.

20. The system according to claim 14, further comprising at least one optical filter that (a) conditions light reflected from the skin detected by the camera; or (b) conditions light from a built-in flash or an external illumination system.

21.-42. (canceled)