JP3474883B2 - Spectral imaging device for moving objects based on interference - Google Patents

Description

FIELD OF THE INVENTION The present invention relates to an apparatus for spectral analysis of an object, namely spectral imaging, and in particular to interference for the purpose of spectral imaging for measuring the spectral brightness of each pixel of a movable object. A meter-based device. According to the present invention, the spectral brightness is emitted from a movable object (natural or induced fluorescence).
Or reflected or scattered by a moving object,
And / or the light transmitted through the movable object. All of these are referred to below as light emitted from an object.

Spectroscopy, in the fields of science and industry,
It is a well-known analytical tool that has been used for decades to characterize substances and their treatments based on the spectral characteristics of chemical constituents. The physical basis of spectroscopic methods lies in the interaction of light with matter. Traditionally, spectroscopic methods measure the brightness of light emitted, transmitted, scattered or reflected from an object as a function of wavelength, at high spectral resolution, but in the absence of stereoscopic information. Spectral imaging, on the other hand, is a combination of high resolution spectroscopy and high resolution imaging (ie stereoscopic information).

The spectrometer accepts light and disperses it into its constituent wavelengths,
And a device designed to measure the spectrum, which is the brightness of light as a function of wavelength. An imaging spectrometer (also referred to herein as a spectral imager) collects incident light from a field of view and measures the spectrum of each pixel (ie, pixel).

Spectral imaging is thus a technique that allows the measurement of emission spectra from all points (pixels) of an object. A spectrum imaging device is a measuring instrument capable of measuring the spectrum emitted from each point of an object in the field of view and storing the measurement data in memory for subsequent retrieval and analysis. A spectral image is a collection of the spectrum of an object measured by a spectral imager, usually as a luminance function defined in three dimensions with the image (x and y) in two dimensions and the spectral axis (λ) in one dimension. Can be assembled. therefore,
Spectral images are commonly referred to as "cubes" or "spectral cubes" of data.

There are three basic types of spectral imaging methods.
These are (i) spectral imaging based on the grating, (ii)
Filter-based spectral imaging and (iii) interferometer-based spectral imaging.

In a grating-based spectral imaging system, also known as a slit-type imaging spectrometer, for example,
DILOR system, etc. (see Barisa et al .: Valisa et.
al. (Sep. 1995) presentation at the SPIE Conference
European Medical Optics Week, BiOS Europe '95, Barc
elona, Spain), the single axis (space axis) of a CCD (charge coupled device) array detector provides the real image data, and the second (spectral) axis is the intensity of the light dispersed by the grating as a function of wavelength. Used for sampling. The system is also provided with a slit in the first focal plane, limiting the field of view at one point to one line of pixels. Therefore, the full image is only obtained after scanning the grating or incident light in a direction parallel to the spectral axis of the CCD, in what is known as line scanning. It is not possible to select the desired area in the field of view and / or optimize the system focus, exposure time, etc. prior to measurement, since the two-dimensional image cannot be obtained until the whole body is completed. is there. Grating-based spectral imagers are used for telemetry because planes (or satellites) flying over the surface of the earth naturally provide the system with a line scanning mechanism.

Although the front optics of the instrument collect all of the incident light at the same time, most of the pixels in a frame are not measured at one point, which is a major advantage of slit-type imaging spectrometers. It is a drawback. As a result, any
In order to obtain necessary information from the SN ratio (sensitivity), a relatively long measurement time is required, or conversely, the SN ratio considerably decreases for any measurement time. In addition, slits
Spectral imagers of the type require line scans to gather the required information across the field of view, which can lead to errors in the obtained measurement results.

The filter-based spectral imaging method further comprises:
It can be classified into a separation filter and a tunable filter. In these types of imaging spectrometers, at some point, a narrow band filter is continuously inserted in the optical path or an ultrasonic optical tunable filter (AOTF) or liquid crystal tunable filter (LCTF) is used. Electronically scanning the frequency band by passing light from all pixels in the field of view simultaneously through filters of different wavelengths to produce a spectral image (see below). As with the slit-type imaging spectrometer described above, the majority of the light is rejected at any time when using the filter-based spectral imaging method. In fact, all photons outside the instantaneous wavelength (band) of the measurement are rejected and do not reach the CCD, making it possible to measure the entire image at a particular wavelength.

Tunable filters such as AOTFS and LCTFS have no moving parts, but can be tuned to specific wavelengths within the spectral range of the instrument. One advantage of using tunable filters for spectral imaging is the ability to measure image intensity at multiple wavelengths in a desired sequence without arbitrary wavelength access, i.e., without using a filter wheel. It is in. But,
AOTFS and LCTFS have (i) limited spectral range (usually λmax = 2λmin) and need to block other light outside this spectral range, (ii) temperature sensitive, (iii)
There are drawbacks such as low transmission, (iv) polarization sensitive property, and (v) AOTFS having image shift during wavelength scanning.

Systems based on all these types of filters and tunable filters are limited in spectral resolution, insensitive, and difficult to use due to the lack of sophisticated algorithms for interpreting and displaying collected data. It has not been used extensively for spectral imaging for many years.

The advantage of interferometer-based spectroscopic methods being more sensitive than filters and grating methods is that in this technique:
Known as the multiplex or Fellgett advantage ("Interferometry Principles" Cham
berlain (1979) The principles of interferometric s
pectroscopy, John Wiley and Sons, pp.16-18 and p.26
See 3).

A spectral imaging method and apparatus having the above advantages is disclosed in U.S. Pat. No. 5,539,517 issued to Cabib et al. On July 23, 1996. Compared to conventional slit or filter type imaging spectrometers, all available information from the incident light from the image is used to significantly reduce the required frame time and / or significantly increase the signal to noise ratio. Reference is made to this patent for the purpose of providing a better used method and apparatus for spectral imaging.

According to the present invention, there is provided a method of analyzing an optical image in the field of view to measure the spectral intensity of each pixel. The method collects incident light from the field of view and passes this light through an interferometer that outputs modulated light that corresponds to a predetermined set of linear combinations of spectral luminance emitted from each pixel and is output from the interferometer. The light is focused on the detector array, and the optical path difference (OPD) that occurs in the interferometer for all pixels is scanned independently and simultaneously, and the output from the detector array is processed (interferogram of all pixels. Separately) and measure the spectral intensity of each pixel.

This method can be performed using various types of interferometers, and the OPD can be changed by moving the entire interferometer, the components in the interferometer, or the incident angle of the incident light to change the interferogram (interferogram). , But not limited to planes). In all of these cases, once the scanner has completed one interferometer scan, the interferogram for every pixel in the field of view is complete.

As described above, the device with these characteristics is different from the conventional slit-and-filter type imaging spectrometer in that an interferometer is used. The total throughput of the system is significantly improved without limiting the input wavelength with interference or tunable filters. Therefore,
Spectral imaging systems based on these interferometers make better use of all the information available from the incident light in the field of view to be analyzed, which significantly reduces the measurement time,
And / or the signal-to-noise ratio (ie sensitivity) is significantly improved.

FIG. 1 is a block diagram showing the main components of an interferometer-based imaging spectrometer in the prior art disclosed in US Pat. No. 5,539,517.

This imaging spectrometer has a high spectral resolution (about 4 to 16 nm depending on the wavelength) and spatial resolution (about 30 / Mμm, where M is the effective magnification of the microscope or front optics), It is most suitable to apply the device.

Therefore, the prior art imaging spectrometer shown in FIG.
Illumination optics, generally designated at 20, a one-dimensional scanner shown at block 22, an optical path difference (OPD) generator or interferometer shown at block 24, a one-dimensional, preferably at block 26, preferably It includes a two-dimensional detection array, and a signal processor and display, shown at block 28.

A key element in system 20 is an OPD generator or interferometer 24 that outputs a modulated light corresponding to a predetermined set of linear combinations of emission spectral intensities from each pixel in the analysis field of view. The output from the interferometer is focused on the detector array 26. So for every pixel in the field of view,
All the required optical phase differences are scanned simultaneously to obtain all the information needed to reconstruct the spectrum. Real time image analysis is possible because the spectra of all pixels in the field of view are collected simultaneously with the imaging information.

The device according to U.S. Pat. No. 5,539,517 can be configured in a wide variety, and in particular the interferometer may be combined with other mirrors such as those depicted in the relevant drawings of U.S. Pat. No. 5,539,517.

Thus, according to US Pat. No. 5,539,517, it is possible to use alternative types of interferometers. These include, but are not limited to:
(I) A movable interferometer that modulates light by changing the OPD, ie a Fabry-Perot interferometer with a scanning thickness,
(Ii) Michelson interferometer with a beam splitter that receives light from the optics and scanner and splits it into two optical paths, and (iii) an interferometer consisting of four mirrors and a beam splitter, etc. A Sagnac interferometer, which is optionally described in the U.S. patent (see, e.g., Figure 14 of that patent), optionally in combination with other optical means. In addition, in this interferometer, OPD changes according to the incident angle of incident light.

FIG. 2 shows an imaging spectrometer made according to US Pat. No. 5,539,517 with an interferometer whose OPD varies with the angle of incidence of the incident light. A beam entering the interferometer at a small angle to the optical axis produces an OPD that varies fairly linearly with this angle.

In the interferometer of FIG. 2, all light from all sources 30 at all pixels is aligned by the daylighting optics 31 and then scanned by the mechanical scanner 32. The light travels through the beam splitter 33 to the first reflecting mirror 34 and then to the second reflecting mirror 35, which is reflected by the beam splitter 33 and then through the focusing lens 36 to the detector. 37 arrays (eg CC
Incident on D). This beam is a beam splitter 3
3, interferes with the beam reflected by the second reflector 35, and finally the first reflector 34.

At the end of one scan, all pixels have been measured through all OPDs, so the spectrum of each pixel in the field of view can be reconstructed by a Fourier transform. The beam parallel to the optical axis is corrected, and the beam having an angle (θ) with respect to the optical axis produces an OPD that is a function of the thickness of the beam splitter 33, the refractive index, and the angle θ. At small angles, OPD is proportional to θ.

By applying the appropriate inverse transforms and taking careful records, the spectra of all pixels can be calculated.

In the configuration of FIG. 2, a ray incident on the beam splitter at an angle β (β = 45 ° in FIG. 2) has an OPD = 0.
A ray that passes through the interferometer at, but is incident at a typical angle β-θ causes OPD according to equation 1.

(1) OPD (β, θ, t, n) = t [(n ² −sin ² (β + θ)) ^0.5 − (n ² −sin ² (β−θ)) ^0.5 + 2sin βsinθ] In this case, β is the beam Is the ray incident angle on the splitter, θ is the angular distance of the ray from the optical axis, ie the interferometer rotation angle with respect to the center position, t is the thickness of the beam splitter, and n is the beam splitter's thickness. Is the refractive index.

From equation 1 it can be seen that it is possible to obtain a double-sided interferogram for all pixels by scanning both positive and negative angles with respect to the center position. This eliminates the phase error and gives a more accurate result from the Fourier transform calculation.

The scan amplitude determines the maximum achievable OPD for the measured spectral resolution. O depending on the size of the angle step
The PD step is determined, and the OPD step is defined by the shortest wavelength determined by the system sensitivity. In fact, the sampling theorem ("Principles of interferometry")
Chamberlain (1979) The principles of interferometr
According to ic spectroscopy, John Wiley and Sons, pp.53-55), this OPD step must be less than half the shortest wavelength in system sensitivity.

Another parameter to consider is the finite size of the detector elements in the matrix.
The detection element is connected to the OP in the interferometer via the focusing optical element.
Since the D is limited, the interferogram is affected by the rectangular function. As a result, system sensitivity is reduced at short wavelengths, dropping to zero for wavelengths below the limiting OPD. Therefore, we must ensure that the conditions of the modulation transfer function (MTF) are met, ie,
The OPD of the interferometer, which is determined by the detection element, must be smaller than the shortest wavelength of the instrument sensitivity.

Therefore, an imaging spectrometer made in accordance with the invention disclosed in U.S. Pat.No. 5,539,517 does not simply measure the brightness of the light coming from every pixel in the field of view, but at a predetermined wavelength range. The spectrum of each pixel is measured. It also takes advantage of all of the emission from each pixel in the field of view at a point in time, thus significantly reducing frame time and / or significantly improving the sensitivity of the spectrometer, as explained above. Such an imaging spectrometer can include various types of interferometers and daylighting and focusing optics, for example,
It can be used for various purposes such as medical diagnosis and treatment and biological research, and can also be used for remote measurement such as geological survey and agricultural survey.

The imaging spectrometer according to the invention disclosed in US Pat. No. 5,539,517 is an application of Applied Spectral Imaging Ltd., Industry of Israel.
Developed by al Park, Migdal Haemek, Israel) and sold under the trademark SpectraCube.

A SpectraCube system optically coupled to various optical devices is used in the device of the present invention. The SpectraCube system has the properties shown in Table 1 below.

However, an interferometer-based spectral imager must collect a few frames of the object in about 5 to 60 seconds, which is significantly longer than a camera or video camera snapshot to make a measurement. I won't.
This blurs the image of the object in the spectral imaging of the moving object and causes confusion in the algorithm that calculates the spectrum for each pixel.

In fact, when using the device disclosed in U.S. Pat. No. 5,539,517, it is necessary to keep the DUT approximately stationary for best results. This is also the case for many applications, and also when spectral imaging is used for color karyotyping and color banding of chromosomes as disclosed by Shreki et al. Type analysis ”Schroeck et al. (199
6) Multicolor spectral karyotyping of human chromo
See somes.Science 273: 494). However, in some other applications, spectral imaging of moving objects is required. This is the case, for example, when the object to be measured is an organ of a living body (for example, the human eye or a specific place or tissue).

Spectral images of organs and tissues of a living body provide important information on the composition of chemical substances in the organs or tissues, and thus, for example, information on metabolic functions of the living body can be obtained. This is because the physical basis of spectroscopic methods lies in the interaction of light with substances, which makes it possible to characterize objects based on the spectral characteristics of the chemical constituents.

For example, without limitation, various biological substances, such as hemoglobin, cytochrome, flavin, nicotinamide adenine dinucleotide, nicotinamide adenine dinucleotide phosphate, etc., each of which has a unique and distinguishable spectrum. . The oxidative level (ie, the ratio of oxidant to reductant) of such substances is often correlated with the amount of oxygen reaching the organ and the level of metabolism, and in some cases, changes in oxidative level. In order to suggest a pathological condition, and to measure and estimate the severity of the pathological condition, point spectroscopy has been used to measure the level of oxidation of such substances. As an example, a point spectrophotometer can be used for pessimistic measurements of the fundus ("Applied Optics" Delori (1994) Applied Optics 33: 7).
439-7452). But point spectroscopy is
Its application is limited because it provides spectral information that is unrelated to stereoscopic information such as that obtained by the method of spectral imaging.

Therefore, in the spectral image measurement of the biological organ, since the organ is not a still life, an artifact occurs,
Distorted, especially noisy, spectral image data results. If such an image is obtained using a spectral imager based on filters or gratings, a stereoscopic image positioning process is required for best results.
However, these spectral imagers have the previously mentioned limitations. On the other hand, obtaining such images with an interferometer-based spectral imager, which has a number of advantages over other spectral imaging systems, requires not only spatial positioning, but also spectral correction.

It is therefore an object of the present invention to provide an interferometer-based spectral imaging device capable of measuring spectral images even when the object is not stationary. Much of the improvement needed for this device lies in the mathematical algorithms used to process the data collected from the moving object, but in some cases the placement of the device relative to the measured moving object is also important. These improvements provide both spatial positioning and spectral correction.

SUMMARY OF THE INVENTION In accordance with the present invention, interferometric based spectral imaging of
An apparatus for spatial positioning and spectral correction is provided. This device can be used to obtain spectral images of moving objects.

Further, according to the following features in the embodiments of the present invention, (a) means for obtaining stereoscopic information and spectral information of a movable object by using a spectrum imaging device based on an interferometer, and (b) stereoscopic information. And means for correcting the spectrum information with respect to the movement of the movable object through the spatial positioning processing and the spectrum correction processing, and (c) the movable object is first. , The interferometer is a type in which the optical path difference in the frame changes along a single direction, and the spectrum imager with respect to the movable object, the optical path difference gradient in the first direction. There is provided a movable object spectral imaging device comprising means arranged at right angles to one another.

The means for obtaining the corrected three-dimensional information and spectral information collects the incident light from the movable object (a-1),
(A-2) The collected light is passed through an interferometer to be modulated light corresponding to the primary combination of the emission spectrum luminance from each pixel of the movable object, and emitted (a-3) output from the interferometer. The modulated light thus focused is focused on the detection array, and (a−
4) Scan the optical path difference created in the interferometer independently and simultaneously for all said pixels of the movable object, (a
-5) configured to process the output of the detector array to measure the spectrum of each of the pixels of the moveable object. Also,
Spatial positioning processing and spectrum correction processing by means of obtaining corrected three-dimensional information and spectral information select (b-1) the reference frame of the movable object, and (b-
2) This reference frame is used to calculate a transformation vector for another frame of the movable object, and (b-3) spatial transformation and spectrum correction processing is performed using these transformation vectors.

According to a feature in the preferred embodiment, the apparatus further comprises (d) means for displaying the corrected stereoscopic and spectral information as an image.

Further, according to a feature in the preferred embodiment, the display by this image is an RGB display of the corrected spectrum information according to the corrected stereoscopic information.

Furthermore, according to a feature in the preferred embodiment, the calculation of the transform vector is performed by superimposing the reference frame on one of the other frames and finding the position to produce the subtracted image for the luminance subtraction process.

The movable object is preferably a living body organ or a part thereof, and more preferably the living body organ is an eye.

Another spectral imaging apparatus according to the present invention is
(A) a means for aiming and focusing an interferometer-based spectral imager with respect to the moving object, and (b) capturing a continuous frame of the moving object by means of a detection array while scanning the interferometer at a constant speed, A means for storing the continuous optical path difference of the interferometer in a state in which the movable object does not become evenly spaced due to the movement of the movable object. At the same time as sending to the Gram function, using one of the consecutive frames as a reference frame, the three-dimensional transformation vector of each of the consecutive frames is calculated for this, and all pixels in each of the consecutive frames are calculated. And (d) applying a Fourier transform algorithm to each of the interferograms, by means of finding the actual optical path difference for , A means for calculating a Fourier transform of the movable object, (e) a means for calculating a spectrum for all pixels of the movable object, and (f) the movable object moving only along the first direction, and the interferometer, The optical path difference in the frame is of a type that changes along a single direction, and means for positioning the spectral imager with respect to the movable object so that the optical path difference gradient is perpendicular to the first direction. Become.

According to features in preferred embodiments, the apparatus further comprises means for displaying an image of the moving object.

Preferably, the image display is an RGB representation of the calculated spectrum.

More preferably, the calculation of the three-dimensional transformation vector of each of the successive frames relative to the one of the successive frames that is the reference frame is performed in order to find the position where the subtraction image for performing the luminance subtraction process is generated. And one of the other frames are overlapped.

The movable object is preferably a living body organ or a part thereof, and more preferably the living body organ is an eye.

Still another aspect of the present invention is a spectrum imaging apparatus, which includes (a) means for focusing and focusing an interferometer-based spectrum imaging apparatus on a movable object, and (b) scanning the interferometer at a constant speed. While the detection array captures a continuous frame of the movable object, and the continuous optical path difference of the interferometer is stored in a state where the movable object does not become equidistant due to the movement of the movable object. The collected data of each continuous frame for a pixel is sent to the interferogram function, and at the same time, one of the continuous frames is used as a reference frame, and the stereoscopic transformation vector of each continuous frame is set to this. A means for calculating and finding the actual optical path difference for every pixel of each successive frame, and (d) complementing the interferogram of each pixel of the moving object. Means for obtaining equally spaced optical path difference values to the means for calculating the Fourier transform for each of the (e) by applying a Fourier transform algorithm to each of the interferogram, the movable object pixels,
(F) a means for calculating the spectrum for all pixels of the moveable object, and (g) the moveable object moving only along the first direction, the interferometer measuring the optical path difference within the frame along a single direction. It is of a variable type and comprises means for positioning the spectral imager with respect to the movable object such that the optical path difference gradient is perpendicular to the first direction.

According to features in preferred embodiments, the apparatus further comprises means for displaying an image of the moving object.

According to a feature in another preferred embodiment, the representation of the image is an RGB representation of the calculated spectrum.

According to another feature in the preferred embodiment, the calculation of the three-dimensional conversion vector of each of the continuous frames with respect to one of the continuous frames which is the reference frame is performed by performing a luminance subtraction process to perform subtraction. This is done by overlaying the reference frame with one of the other frames to find the location where the image is generated.

The movable object is preferably a living body organ or a part thereof, and more preferably the living body organ is an eye.

BRIEF DESCRIPTION OF THE DRAWINGS The invention will now be described by way of example with reference to the accompanying drawings.

FIG. 1 is a block diagram showing the main components of an imaging spectrometer made according to US Pat. No. 5,539,517 (prior art).

FIG. 2 shows a Sagnac interferometer used in an imaging spectrometer according to US Pat. No. 5,539,517 (prior art).

FIG. 3a is a spectral image of the human right eye using the SpectraCube system.

FIG. 3b is a spectral image of the human right eye of FIG. 3a after spatial positioning and spectral correction using the apparatus of the present invention.

FIG. 4a shows a portion of the interferogram function for any pixel obtained from the spectral image of FIG. 3a.

FIG. 4b shows a portion of the same pixel interferogram function as in FIG. 4a taken from the spectral image of FIG. 3b.

FIG. 5a shows the spectrum of five adjacent pixels from the spectral image of FIG. 3a, where the position of each pixel is shown.

FIG. 5b shows the spectrum of five adjacent pixels from the spectral image of FIG. 3b, where the position of each pixel is shown.

6a to f show the operation of the interference fringe suppression algorithm according to the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention relates to an apparatus for spatial positioning and spectral correction of interferometric spectral imaging that can be used to obtain spectral images of moving objects. In particular, the invention can be used to obtain spectral images of living organs and / or tissues such as the eye.

The principle and operation of the device according to the invention can be understood from the drawings and their description.

US Pat. No. 5,539,517 and other publications (eg,
(I) "Multicolor spectral karyotype analysis of human chromosomes" Schr
oeck et al. (1996) Multicolor spectral karyotyping
of human chromosomes, Science 273: 494-497, (ii)
"Fourier transform multipixel spectroscopy for quantitative cytology" Malik et al. (1996) Fourier transform mult
ipixel spectroscopy for quantitative cyiology.J.of
Microscopy 182: 133-140, (iii) "Mediation of ALA in melanoma.
PDT: A Report on Photosensitizer Interactions Measured by a Novel Spectral Imaging System, Optical Methods for Tumor Treatment and Discovery: Mechanisms and Techniques of Photodynamic Therapy "Malik and Dishi (1995) ALA mediated PDT of m
elanoma tumors: Inght−sensitizer interactions dete
rmined by a novel spectral imaging system.Proceedi
ngs of optical methods for tumor treatment and det
ection: Mechanisms and techniques in photodynamic t
herapy IV, Feb.4-5,1995, San Jose, CA, SPIE Vol.2392,
pp.152-158, (iii) "New Spectral Imaging System for Biology by Combining Spectroscopic Analysis and Image Processing, Bulletin on Optical Imaging Technology in Biomedical Medicine" Malik et al. ( 1994) A novel spectral imaging s
ystem combining spectroscopy with imaging-applica
tion for biology.Proceedings of optical and imagin
g techniques in biomedicine, Sep.8-9,1994, Lille, Fr
ance, SPIE Vol.2329, pp.180-184, (iv) "Fourier Transform Multiplex Spectroscopy in Single Melanoma Cells and Spectral Imaging of Photoporphyrins, Photochemistry and Photobiology," Malik et al. (1996). ) Fourier transform mult
iplex spectroscopy and spectral imaging "of photopo
rphyrin in single melanoma cells.Photochemistry an
d photobiology 63: 608-614, (v) "Use of a novel bioimaging system as an imaging oximetry in the intact mouse brain, laser and optical spectroscopy for the diagnosis of tumors and other diseases. Bulletin on Progress in Soenksen
et al. (1996) Use of novel bio-imaging system as
an imaging oximeter in intact rat brain.Proceedin
gs of advances in laser and light spectroscopy to
diagnose cancer and other diseases III, Jan.29-30,
1996, San Jose CA, SPIE Vol.2679, pp.182-189)
Collects light from the surface of the object to be measured by an optical aperture or field lens and passes it through an interferometer to divide it into two coherent light beams, and then a focusing optical element is used to have a detection element surface. It teaches a spectral imaging device and imaging method in which the surface of the detector represents a real image of the surface of the object by focusing on a two-dimensional detection array device (eg CCD in the UV to visible range).

When the interferometer is scanned synchronously with the detector frame,
The signals from each and every detector element of the detector array obtained from many successive frames of the detector array are recorded.

At each position of the interferometer, the optical path difference (OPD) between the two separated beams for identifying the corresponding pixel (pixel) by the detection element changes to a known state, so that at the end of scanning, each pixel The signal collected from forms a function called the interferogram, which is the intensity of light as a function of the optical path difference (OPD) for that particular pixel. Since the interferometer speed is constant, the CCD frame time is constant and the OPD is proportional to the angular position of the interferometer.
The samples are evenly spaced.

According to the well-known technique of Fourier transform spectroscopy, the mathematical Fourier transform operation applied to this interferogram function calculates the spectrum, ie the brightness of the light at all wavelengths emitted from the pixel of interest. To do.

By knowing the interferogram function for every pixel on the surface of the object, by applying the Fourier transform to the total interferogram collected,
The spectra for all pixels can be calculated and revealed.

US Pat. No. 5,539,517 shows several embodiments of spectral imaging apparatus and methods. The type of interferometer used for each device is different,
Each device is capable of measuring a spectral image of the object.

In general, no matter how the interferometer is used, at any scan position by the interferometer, the OPD for rays on and off the axis is different, so the OPD in the same frame is different for each pixel. It is well known.

For example, ("The principle of interferometry""The principl
es of interferometric spectroscopy "by John Chamber
lain, John Wiley & Sons, 1979, page 220.Equations 8.
As explained in 3 and 8.4b), in Michelson interferometer, OPD changes according to the following equation (2).

In this case, λ is the wavelength of the light and α is the angle between the rays on and off the axis.

According to Equation 2, the dependence of OPD on a particular pixel is relatively low. In fact, since α in Eq. 2 is a small angle, condition (1-cosα) changes slowly as does α ₂ .

However, in a triangular interferometer as shown in FIG. 2, the OPD is much faster, ie, the projection of the ray incidence angle in the horizontal direction (center of the image), as shown in equation 31 of column 13 of US Pat. No. 5,539,517. Is equal to the horizontal projection of the distance from to the corresponding pixel).

This fact has two important consequences for interferometer-based spectral imagers.

First, at the end of the scan, the OPD trajectory for every pixel and every detector frame must be recorded so that the spectrum can be reconstructed via the Fourier transform algorithm. This is possible by knowing (i) the position of the interferometer for every frame and (ii) the pixel position in the image.

Second, if the DUT moves during the measurement, the spatial positioning of the various frames is lost, and the actual OPD of each pixel in each frame was obtained if the object was stationary. It will be different from what it would be. Therefore, if we ignore the movement during the measurement, calculate the spectral image of the object, use the collected data,
Or if the object is displayed via eg red, green, blue (RGB) functions defined over the whole spectral range, (i) the image loses its spatial positioning during measurement and becomes blurry. (Ii) the calculated spectrum does not represent the actual spectrum, which is noisy and inaccurate to the Fourier transform (ie,
Since it uses OPDS (which is not registered), it becomes inconsistent. These phenomena are further illustrated in the examples below.

Before starting the description of the device for the spatial positioning and spectral correction of interferometric spectral imaging according to the invention for obtaining a spectral image of a moving object, a prior art method for measuring stationary objects is described. explain.

  Measurement of a stationary object includes the following steps.

First, the spectral imager is positioned and focused on the DUT.

Second, the CCD captures and stores successive frames of the object while scanning the interferometer in evenly spaced OPD steps.

Third, send that data (eg, by software) to the interferogram function to get every pixel of the image of the object.

Fourth, it is preferable to carry out a well-known pretreatment step called windowing or apodization to organize the data ("Principles of Interferometry" Chamberlain (1979) The p.
rinciples of interferometric spectroscopy, John Wil
ey and Sons, pp.131 and following pages). By doing so, it is possible to use the measured data, which is a discrete finite data set, instead of using the theoretical continuous interferogram function.

Fifth, usually, by performing "fill zero" processing so that the number of data for each interferogram becomes a plurality of points equal to the square of the original number of data, interpolation points are inserted into the spectrum and the fast Fourier transform is performed. Using an algorithm ("Principles of interferometry" Chamberlain (197
9) The principles of interferometric spectroscopy,
See John Wiley and Sons, pp. 311 and following pages).

Sixth, applying a fast Fourier transform algorithm to each of the interferograms to compute a complex (real and imaginary) Fourier transform. Alternatively, but not preferred, the Fourier Transform algorithm may be applied directly without "filling to zero".

Seventh, calculate the spectrum of all pixels as a module (length) of the complex function so obtained. It should be noted that this function is defined on the discrete parameter of the conjugate parameter to the OPD, that is, the reciprocal of the wavelength (σ = 1 / λ) wave number σ.

The fast Fourier transform algorithm significantly reduces the computation time, but can only be used when the OPDs are evenly spaced and when the number of points forming the interferogram is equal to the square. Therefore, this simple Fourier transform algorithm is generally not used.

Now, an apparatus for the spatial positioning and spectral correction of interferometric based spectral imaging according to the invention which can be used to obtain a spectral image of a moving object is described.

The following description relates to objects that move in a rigid, linear, random or non-random manner in a plane that is fairly perpendicular to the line of sight of the imager. In other words, when viewed through a spectral imager, the object moves with all its parts retaining shape and size and relative distance.

Thus, for an object that moves firmly in a random direction without changing the plane (i.e., the object stays within the focal length without approaching or leaving the instrument). The measurement steps by are as follows.

First, the spectral imager is aimed and focused on the object to be measured.

Second, the interferometer is scanned at a constant speed, synchronized with the CCD frame, and the CCD captures and stores successive frames of the object. However, contrary to the description in the prior art above, the OPD steps are not essentially evenly spaced as in the above description because the object moves. The difference between consecutive OPDs is random. This is a result of the relative movement of the interferometer and the object, increasing or decreasing depending on the instantaneous position and velocity of the object relative to the position and velocity of the interferometer, and can even become negative (next Data points mean that OPD is reduced). Then, if the movement exceeds the field of view, or if the movement occurs with a sudden displacement greater than the field of view and immediately returns to the previous position, the data point disappears. In some areas of the OPD axis, data
The points are dense and in other areas are less dense.

Third, this data is converted (eg by software) into an interferogram function for every pixel of the image. However, the bookkeeping operation is rather complicated. To achieve this step, first of all, the stereoscopic transformation vector of all measurement frames must be found with respect to the frame taken as the reference. With this method, the actual OPD for all pixels in each frame can be found. This is a critical step of the method using the device according to the invention and is therefore explained in more detail below.

Fourth, some well-known methods called windowing or apodization so that data can be organized and measured data that is discrete data can be used in place of the theoretical continuous interferogram function. It is preferable to carry out a pretreatment step.

Fifth, here the method is divided into two alternative ways. According to the first, a simple Fourier transform algorithm is used to calculate the corresponding Fourier transform without further interpolating the measured interferogram of each pixel. On the other hand, according to the second method, the measured interferograms of each pixel are interpolated to obtain equidistant OPD values, and then the Fourier transform is calculated using the fast Fourier transform algorithm. Each alternative step has advantages and disadvantages. The latter is faster,
The former is more reliable because the error is introduced by the interpolation method.

Sixth, depending on the alternative choice in the fifth step, a simple Fourier transform algorithm or a fast Fourier transform algorithm is applied to each of the interferograms to generate a complex (imaginary and real) Fourier transform for each pixel. To calculate.

Seventh, calculate the spectrum of all pixels as a module (length) of the obtained complex function. This function is defined on the conjugate parameter to OPD, that is, the discrete value of wave number σ.

Details of the theory and mathematical steps of the Fourier transform for calculating the mathematical spectrum as an approximation to the true physical spectrum, starting from the measured interferogram, are also included in this description. Principles of Spectroscopy ”(Chamberlain (1979) The princip
les of interferometric spectroscopy, John Wiley and
It is recommended to refer to textbooks such as Sons).

Some of the highlights of Chamberlin (1979) chapters 2, 4, 5 and 6 explain the basics of the following Fourier transform process and related matters. The relationship of the Fourier integral between the function f (k) and its Fourier transform F (x) is shown on page 31. In principle, f (k) and F
(X) is a continuous function of each variable, but in practice it is always used for discrete values, so the Fourier integral is approximated by an infinite sum, as shown on page 55. This infinite sum is then replaced with a finite sum as shown on page 57. The perfect and practical interference function in the case of electromagnetic radiation is derived as shown on pages 96 and 104. The relationship between physical and mathematical spectra is shown on page 127, and the theory of sampling and correction of phase error is given in paragraphs 6.7 to 6.11. Finally, the Fast Fourier Transform algorithm is described in detail in Chapter 10 and it is shown that the operation is only possible when all the intervals are equal. Note that this calculation is faster than linear Fourier summation.

The third step of the method using the device of the present invention described above is
It will be apparent to those skilled in the art that many alternatives can be achieved. One of these options is as follows.

First, one of the frames is defined as the reference system. In principle, it does not matter which frame is chosen as a criterion, but in practice, the overall spatial overlap of the selected frame with all other frames is the largest,
It is good to choose a frame located approximately in the middle for the set of transform vectors. Choosing a reference system facilitates finding the transformation vector for each of the measurement frames.

Second, it displays the subtracted image, which is the difference in brightness between the first frame and the reference system.

Thirdly, the first frame is displayed in left and right and up and down directions in small steps while always displaying the brightness difference until the display subtraction image becomes almost zero in any part, that is, a position where almost no features are found. Move to.
In the ideal case where the movement is perfectly linear,
The subtracted image is equal to zero in all overlapping pixels, but in practice there is always a slight pattern, so the position where this pattern is minimal in brightness is the best position. Experience has shown that it is easy to find a near zero position with the unaided eye. This is because the slight lack of spatial positioning emphasizes the difference between the two frames and is therefore easy to detect. This process can be automated using various known algorithms (eg, "Basics of Digital Image Processing" Anil K. Jain (1989) Fundamentals of digital im).
age processing.Prentice-Hall International and s
ystem sciences science, pp. 400-402). However, since there are overlapping interference fringes in the frame, it is preferable to apply the interference fringe suppression algorithm before the automatic spatial positioning processing of the frame. For this, see Example 4 below.
Will be further described and illustrated in.

Fourth, record the transform vector for the first frame.

Fifth, this process is repeated for every measurement frame.

And finally, once the OPD dependence on position (specific dependence of each interferometer) is known, the OPD vector for every pixel in every frame is calculated and saved.

Most of the problems that occur during measurement and that affect the final result are related to the amplitude of movement of the object. In order for the method using the device of the present invention to be effective, it is preferable that the movement amplitude is not so large. Multiple problems can occur, including:

First, the entire area of the interferogram may be missing, making interpolation very difficult (in the case of interpolation).

Second, if the central part of the interferogram is completely missing, it is impossible to calculate the Fourier transform.

And finally, it is inaccurate if the actual OPD step (corrected for movement) is greater than the Nyquist condition (1/2 the lowest wavelength of the instrument's sensitivity) due to the movement of the DUT. Results may occur.

However, corrective action can be taken. Below is a list of some.

First, it is usually easy to find the center of the interferogram if the central part is present. In this case, if the interferogram is symmetric,
Data points on one side can be reflected on the other side to fill the void.

Second, it adopts the smallest OPD step suitable for the required spectral resolution and measurement time. This has the effect of leaving no large gaps in the interferogram.

Third, repeat the interferometer scan a few times and combine the data as if it had been obtained in one measurement. If the OPD is missing in one scan, it may be present in another (due to random movement).

Fourth, in an interferometer where the OPDs in one frame differ only in one direction (eg horizontally), if the motion of the object is only one direction (eg the human eye undergoes horizontal involuntary movements), then Rotate the instrument around the optical axis so that the OPD gradient is perpendicular to the direction of movement of the object. In this way, only the spatial positioning of the frame is affected by the movement, and the interferogram is hardly affected, so that the cause of the error can be considerably reduced.

Fifth, in the case of a featureless object, all pixels are equivalent, so it is expected that movement will have less effect on the result.

Finally, the following distinctions should be made. (I) An object that moves firmly and linearly on one plane. That is, the object moves while all parts retain their shape and size, and when viewed through a spectral imager, the relative distances of all parts are held constant. (Ii) An object that moves linearly on a plane. In this case, all parts retain their shape and size, but the relative distances of those parts change over time. Clearly, the former case is simpler than the latter. Furthermore, if an acceptable solution is found for the former, the latter can usually be solved by dividing the object into areas that move relative to each other. The former case solution can be applied to individual areas, even if each moves separately and firmly.

The above considerations are valid for certain types of movements, especially hard linear movements (random or non-random). Some of the above considerations can be generalized to other types of motion as well, eg, applicable to non-stiff and / or non-linear motions. In any case, the invention cannot, in principle, handle the case where the object rotates about an axis perpendicular to the instrument's line of sight. The reason is that each part of the object changes its shape and disappears from the field of view during the measurement, and it is obvious that the measurement becomes incomplete.
In addition, even if there is no movement, there is a chemical change in the object that has an effect on the spectrum during measurement,
The present invention is not mentioned.

It will be apparent to those skilled in the art that solving the problem of moving objects is the same as computing the Fourier transform of the interferogram defined for values that are not equidistant in nature. This problem is known in radiation astronomy ("Synthesis Imaging" (1986) Perle
y, Schwab and Bridle, Report of Summer School of the
National Radio Astronomy Observatory, p.72, Greenba
nk W. Virginia)). In this field, it is known that the aggregation of data in the low OPD range introduces a large wave in the image brightness, which makes it difficult to detect a weak point source.

Many applications are conceivable for a spectrum imaging device capable of measuring a spectrum image of a movable object. Previous publications (eg (i) "Multicolor spectral karyotype analysis of human chromosomes")
Schroeck et al. (1996) Multicolor spectral karyoty
ping of human chromosomes, Science 273: 494-497,
(Ii) "Fourier transform multi-pixel spectroscopy for quantitative cytology" Malik et al. (1996) Fourier transfor
m multipixel spectroscopy for quantitative cyiolog
yJof Microscopy 182: 133-140, (iii) "Melanoma A
LA-Mediated PDT: A Report on Photosensitizer Interactions Measured by a Novel Spectral Imaging System, Optical Methods for Tumor Treatment and Discovery: Mechanisms and Techniques of Photodynamic Therapy "Malik and Dishi (1995 ) ALA mediated PD
T of melanoma tumors: Iight−sensitizer interaction
s determined by a novel spectral imaging system.Pr
oceedings of optical methods for tumor treatment a
nd detection: Mechanisms and dechniques in photodyn
amic therapy IV, Feb.4-5,1995, San Jose, CA, SPIE Vo
l.2392, pp.152-158, (iii) "New Spectral Imaging System for Biology by Combination of Spectroscopy and Image Processing, Bulletin on Optical Imaging Technology in Biomedical Medicine" Malik et al. (1994) A novel spectral ima
ging system combining spectroscopy with imaging−a
pplication for biology.Proceedings of optical and
imaging techniques in biomedicine, Sep.8-9,1994, Li
lle, France, SPIE Vol.2329, pp.180-184, (iv) "Fourier transform multiplex spectroscopy in single melanoma cells and spectral imaging of photoporphyrin, photochemistry and photobiology" Malik et al. (1996) Fourier transfor
m multiplex spectroscopy and spectral imaging "of p
hotoporphyrin in single melanoma cells.Photochemis
try and photobiology 63: 608-614, (v) "Use of a novel bioimaging system as an imaging oximetry in the intact mouse brain, laser and optical spectroscopy for the diagnosis of tumors and other diseases. Newsletter on Progress in Law "So
enksen et al. (1996) Use of novel bio-imaging sys
tem as an imaging oximeter in intact rat brain.Pro
ceedings of advances in laser and light spectrosco
py to diagnose cancer and other diseases III, Jan.2
9-30, 1996, San Jose CA, SPIE Vol. 2679, pp. 182-189), some biomedical applications using spectral imaging are taught. This includes DNA analysis such as karyotype analysis, early diagnosis of retinal disease, and detection of tumors in living tissue whose surface can be observed by conventional imaging techniques.

Obviously, various applications in many other fields would also benefit from such a device. In fact, spectroscopic methods rely on the interaction of light with matter, so measuring the spectrum of light provides information about the composition of matter. Most medical imaging devices today are equipped with video camera attachments for real-time display of the tissue being analyzed or treated, and for storing and later retrieving the tissue image as a record of the patient. . These include
(I) A video camera attached to the endoscope or microscope of the operating table that displays to the surgeon the tissue to be treated, (ii)
Includes, but is not limited to, a video camera attached to the fundus camera that displays the patient's retina to the ophthalmologist.

The spectral imager according to U.S. Pat.No. 5,539,517 equates to a sophisticated video camera that adds spectral dimensions to the imaging, mechanically and optically to all these biomedical imaging instruments. Manufactured to fit. Therefore, it can be easily attached to any biomedical imaging instrument, and a spectral image of any tissue that can be displayed by the conventional imaging method can be measured.

Specific examples of applications include early detection of skin cancer and cancer of many internal organs such as lungs, stomach, colon, rectum, cervix, bladder, prostate, for example ischemia in the retina resulting from diabetes. Early detection, early detection of retinopathy, diabetic retinopathy, retinal dystrophy, glaucoma, macular degeneration, etc. In addition, since molecules related to metabolism such as hemoglobin, cytochrome, flavin, nicotinamide adenine dinucleotide, and nicotinamide adenine dinucleotide phosphate (without limitation) have a unique spectrum in the reduced form or the oxidized form, By measuring the concentration of various chemicals, it is also applicable to blood perfusion, and vitality and / or metabolic mapping and assessment of metabolic function of tissues.

Obviously, if not impossible, it is generally difficult in the case of living tissue to bring the analyzed tissue to a complete rest. This is due to breathing, heartbeat, unconscious movement of the patient, and the like. Even if the tissue itself is forced to rest by external mechanical means (for example, a special holder that holds the eye stationary during corneal surgery), the simple fact that blood is circulating in blood vessels These facts induce small movements in the diagnostic system. The movement of the analysis area is also magnified, especially when the object is magnified through the microscope.

Spectral imagers capable of providing such a large amount of information to the treating physician are severely limited if they cannot measure moving objects at all. Accordingly, the present invention provides a means for spectral imaging of moving objects.

Therefore, it is an object of the present invention that white light, ultraviolet light or laser-induced emission spectra from biological components such as oxygen and deoxygenated hemoglobin in retinal blood vessels and / or melanin pigmentation levels in the retina. To quantitatively map and identify tumors from healthy or diseased tissues or cells within a living organ. Therefore, the signals generated by the induced emission spectroscopy are used to characterize different components of living tissues or organs. In addition, this method uses proteins, sugars, NAD + and HADH, collagen,
It allows the identification and steric mapping of elastin and flavin, and various metabolic mediators within cells and / or tissues. By using the emission at a specific wavelength and analyzing by various algorithms, the alkalinity or acidity of a tissue (or cell), blood perfusion, and the presence of a neoplasm in a tissue can be investigated and mapped in vivo. And can be characterized. There are various experimental systems around the world that detect emitted light from biological samples. The method of analyzing the signals and images from these diagnostic detection systems has become one of the recent significant challenges in the medical field and the present invention addresses this issue.

Diabetic retinopathy is a disease that potentially destroys vision, but can often be suppressed by early laser treatment (Ferris (1993) (commentary) JAMA 269: 1290-129).
See 1). The American Academy of Ophthalmology proposes a screening schedule for patients with conditions that require treatment ("Diabetic Retinopathy: American Academy of Ophth").
almology Preferred Practice Patterns.San Francisc
o, Cal .: American Academy of Ophthalmology Quality o
f Care Committee Retinal Pane, American Academy of
Ophthalmology, 1989).

However, the proposed screening schedules are expensive and, at times, patients suffer from severe retinopathy by the time of the scheduled exam, which in some cases is currently expensive. Even screening is not enough. Nevertheless, this screening has been shown to be cost-effective (Ja
vitt et al. (1989) Ophthalmology 96: 255-64.). This study shows that higher-cost and lower-risk patients can be identified more efficiently, resulting in higher healthcare cost savings. Therefore, methods that increase the accuracy and reduce the cost of screening for diabetic retinopathy have great clinical value.

Currently, the recommended screening for diabetic retinopathy includes detailed measurements of the retina. In addition, color retinal photographs are taken for special cases ("Diabetic Retinopathy: A Treatment Recommended by the American Academy of Ophthalmology" Diabetic Retinopathy: Ame).
rican Academy of Ophthalmology Preferred Practice
Patterns.San Francisco, Cal.:American Academy of Op
hthalmology Quality of Care Committee Retinal Pan
e, American Academy of Ophthalmology, 1989).
Retinal fluorescein angiography, which is routine today, is also surgical, uncomfortable, and potentially fatal.
Moreover, the additional information obtained with fluorescein angiography does not help classify patients as to whether immediate laser therapy is effective or ineffective for patients (Ferris (19
93) (commentary) JAMA 269: 1290-1).

According to the invention, an interferometer-based spectral imager in combination with a specially developed algorithm uses spectroscopic data and image information at the same time when the diagnostic target organ or tissue moves during the measurement, Since it is possible to classify the symptoms of ischemia of the retina into stages, the clinician can classify diabetic patients into local anemia or non-local anemia, which proves to be a clinically useful means.

Oxygen supply to the retina is provided by both choroidal and retinal circulation. The choroid functions as a source of oxygen for photoreceptors in the avascular outer retina, and retinal circulation plays a critical role in maintaining oxygen supply to neural elements and fibers within the inner retina. Due to the large oxygen demand of the retina, changes in circulatory function such as diabetic retinopathy, hypertension, sickle cell disease, and vascular occlusion cause dysfunction and expand retinal tissue.

For the first time, Hickham et al. Proposed a pessimistic measurement of oxygen saturation of blood in the retinal vessels (“circulation” Hickham et al. (1963) Circulation 27: 3).
See 75). This is due to a photographic technique that uses two wavelengths (560 and 640 nm) for a retinal blood vessel (the area where the optic nerve connects to the retina) that intersects the optic disc.
Mr. Pittman and Duling
A more sophisticated approach, based on the method of using three wavelengths, is described in "Application of Optics" (Delori (1988) Appl.
ied Optics 27: 1113-1125).

An interferometer-based spectral imaging device according to U.S. Pat.No. 5,539,517, in combination with a device of the present invention that enables spatial positioning and spectral correction, in other words, based on the spectral information provided by the device, the object to be measured. Spatial and spectral corrections to motion not only allow a pessimistic measurement of the oxygen saturation level of hemoglobin in the retinal vessels, but also by the imaging information provided by the device. It also enables discovery and mapping of retinal ischemia. Further, by combining with a sophisticated spectrum analysis algorithm such as a principal component algorithm or a neural network algorithm (without limitation), it can be used for classification of the degree of retinopathy, and for treatment classification of diabetic patients, for example.

Since many chemicals in living tissues are involved in vascular function and metabolism, a major factor in retinal ischemia is oxygen, which can be measured through the concentration of hemoglobin in its oxy and deoxy forms, but NAD + Important information can also be obtained by measuring the concentrations of other components such as, NADH, flavins and cytochromes.

Considering many conventional techniques that describe the spectral detection of such chemical constituents of tissue, which correlates with absorption peaks in reflectance, fluorescence peaks in ultraviolet or blue light, and concentration of excitation by single or multiple wavelengths, the book The interferometer-based spectral imaging device according to US Pat. No. 5,539,517 in combination with the device of the invention can also be used to simultaneously map the concentration of one or more such components in a non-resting organ / tissue of a living body. . In this case, the type and amount of information obtained will depend on the particular hardware configuration using this video device.

For example, the simplest and simplest configuration is to attach the imager to the CCD port of the fundus camera. With this configuration, not only is the retina imaged, but the broadband white light source of the fundus camera can be used to measure the reflected light from the retina as before. In this case, the oxygen concentration is measured by applying Delori's algorithm or a similar algorithm to all pixels of the retina to be imaged (“Optical Application” Delori).
(1995) Appl.Optics 27: 1113-1188, and Appl Optics,
Vol.28,1061, and, Delori et al. (1980) Vision Resear
ch 20: 1099). A more complex system based on an interferometer-based spectral imager includes (i)
Autofluorescence spectral imaging, (ii) spectral imaging with ultraviolet or blue light fluorescence excitation, (iii) alone, simultaneously or sequentially, spectral imaging with laser excitation at the wavelengths listed below. is there. 65
0, 442, 378, 337, 325, 400, 448, 308, 378, 370, 35
5 or equivalent wavelengths at which similar information is obtained.

These configurations can be created in several ways, either individually or in multiple combinations within the same instrument. The instrument includes a computer and software that interprets the data and displays it in a manner useful to the ophthalmologist, a light source,
Formed from a fundus camera and a spectral imager.

In all cases in white light reflection, autofluorescence, single wavelength continuous wave laser excited fluorescence, or multiwavelength laser excited fluorescence, the sample is illuminated and the spectral image is measured.

In the case of pulsed laser irradiation, the method of operation of the spectral imager is slightly modified, necessitating some changes in the hardware, which are not fundamental or major, but important to the operation of the instrument. These changes are as follows:

For single-pulse laser-excited fluorescence spectral imaging, the laser pulse and imager CCD framing are synchronized to the interferometer scan angle. This causes the interferometer to take one step at each pulse and a new frame is collected by the computer (usually several pulses can be used for each frame as long as the number of pulses does not change from frame to frame). ). In this way, the interferogram value at each OPD value corresponds (but is different) to the same number of laser pulses. This is necessary to ensure that each frame is taken with the same total illumination intensity. Otherwise, each frame would measure fluorescence resulting from a different number of laser pulses and the interferogram would be distorted.

There are two methods of operation for fluorescence spectral imaging induced by multiple pulsed lasers. (I) Sequentially, collect the entire spectral cube for each laser as above. This means that only one laser is active during the measurement and at the end the measurement of one spectral cube for each laser wavelength is completed. (Ii) Pulse each laser in synchronism with the interferometer and framing so that all lasers are continuously replaced before taking the next step of the interferometer and the next frame. In this case, at the end, just one spectrum
You get a cube.

All information must be analyzed and interpreted. The most important algorithm is to compare intensities between different wavelengths and different pixels in the image. These algorithms should take into account the variation in brightness and the ratio between different regions and different wavelengths within the tissue. This method is very sensitive and provides a large amount of information,
Slit lamp image formation (white light or filtered light)
It replaces.

Other uses will be apparent to those skilled in the art. These are visual loss due to choroidal ischemia, ischemic optic neuropathy,
It includes corneal and iris problems, and many others that can be analyzed today by imaging techniques using white light or fluorescence from different light sources.

The spectral imager according to U.S. Pat.No. 5,539,517 can be attached to many imaging optics, including endoscopes and laparoscopes, to accurately define the diseased tissue to be removed, where to cut and where to stop. It can also be used before, during or after surgery to assist the surgeon in determining and determining whether all diseased tissue has been removed during surgery. These spectral imagers, by their nature, analyze the properties of tissues through chemical components related to their spectral properties and provide a (usually emphasized) visual aid that helps the user to understand, make decisions, and act. It is suitable for providing various maps.

The hardware configuration for cancer tissue detection in a living body and the types of analysis and display algorithms accompanying it are very similar to the above ophthalmological examples, and the difference lies in the daylighting optical system (for example, a fundus camera). Instead of various endoscopes). Some basic molecular components involved in this detection are general ones such as oxygen concentration, and others include genetic materials in the cell nucleus such as collagen and elastin, and DNA chromatin. The irradiation and synchronization conditions for multiple wavelengths or pulsed excitation are similar (Pitris et al., Paper presen.
ted at European Biomedical Optics Week by SPIE, 1 2
-16 September 1995, Barcelona Spain).

In all these examples spatial positioning and spectral correction are required, which are provided by the method using the device of the invention.

The purpose, advantages and novel features of the device according to the invention will be apparent to those skilled in the art upon understanding the following examples, which are not intended to limit the invention.

Example 1 Spatial Positioning and Spectral Correction-Implications for Images Reference is now made to Figures 3a and b. FIG. 3a shows a spectral image of the optic nerve disc of the right human retina of the right eye obtained by the SpectraCube system without the spatial positioning and spectral correction processes of the device according to the invention. On the other hand, FIG. 3b shows the same image after the spatial positioning process and the spectral correction process according to the invention.

In both images, the optic nerve in the middle of the image is shown with blood vessels (arterioles) that supply oxygen and other nutrients to the optic nerve, and blood vessels (veins) that remove waste products and carbon dioxide generated during metabolism. The disk looks bright. However, as is clear from a comparison of the two images, the image of Figure 3a is quite blurred due to eye movement during the measurement. However, the corrective action by the device of the invention, in which spatial positioning and spectral correction were applied, produced the clear image shown in Figure 3b.

Furthermore, the images of Figures 3a and 3b not only show the stereoscopic organization of the tissue, but also, if not directly, the spectral information. In other words, the different colors present in the image are RGB to the spectrum of each pixel in the image.
It is the result of applying the algorithm. Each pixel has a preselected RGB depending on its spectrum.
Depending on the function, it is shown in RGB color with corresponding brightness. Like the pixels in the image of FIG. 3a, due to the distorted spectrum, it is further demonstrated in Examples 2 and 3 below,
The RGB function gives different results when applied to either image.

This example, which in this case is the eye, emphasizes the importance of spatial positioning and spectral correction to obtain a clear and educational image of the moving object under test.

The following example demonstrates the importance of spectral correction to obtain meaningful spectral information that can be used, among other things, to obtain information about the metabolic status of tissues, etc., from selected areas of the DUT. It is a thing.

Example 2 Spectral Correction-Influence on Interferogram Reference is now made to Figures 4a and 4b. FIG. 4a shows a portion of the interferogram calculated for a single pixel (x = 112, y = 151) of the image shown in FIG. 3a. Therefore,
The spatial positioning process and the spectral correction process by the device of the present invention are not applied. On the other hand, FIG. 4b shows the corresponding part of the interferogram of the same pixel after the spatial positioning process and the spectral correction process according to the invention.

Examining the interferogram of Figure 4a, the left and center parts of the function (measured at evenly spaced times) resemble a typical interferogram, while the right part of the function shows It turns out to be quite anomalous.
The local maxima indicated by the arrows are due to sudden movements of the measured object (eg, intermittent eye movements). The increase in featureless signal is due to the fact that different points on the DUT suddenly appear to deviate from their original position, giving the interferogram function different values away from the stationary state of the object. Has become.

However, after the spatial registration process and the spectral correction process according to the present invention have been applied, the interferogram function of the same pixel becomes typical, as shown in FIG. 4b.

As can be seen from Figure 4b, the corrected interferogram is in place and there are no noticeable interruptions or quirks as in Figure 4a that characterize the uncorrected interferogram.

However, the corrected interferogram of Figure 4b is defined with non-uniform spacing. For example, around frame number 107, the data density is low, which means that the eye moves in the opposite direction to the scanning direction of the interferometer and
This indicates that the OPD interval has increased. On the other hand, around frame number 109.5, which is an unnatural frame number formed due to the large movement of the eye in the same direction as the scanning direction of the interferometer, the data density is relatively high, and the OPD around it is relatively high.
The intervals are decreasing.

Therefore, there are several ways to do a Fourier integral that approximates the physical spectrum of a particular pixel.
One method is a fast Fourier transform algorithm that approximates the physical spectrum of the pixel by forming a new interferogram with evenly spaced OPD values after interpolating between any number of OPD values. Enable use.
Alternatively, the Fourier integral can be calculated using equation (3) as the sum of the interferogram values with weights according to the intervals.

(3) f (σ) = 1 / K · ΣF (xi) Δie (iσxi) In this case, K is a constant, f (σ) is a spectrum value at the wavelength λ = 1 / σ, and Δi is OPD in xi
, And the OPD at xi + 1. Other methods of approximating the physical spectrum, eg "composite video"
(Synthesis Imaging (1986) Perley, Schwab and Bridl
e, Report of Summer School of the National Radio As
tronomy Observatory, p.72, Greenbank W. Virginia.) etc. are obvious to those skilled in the art.

Example 3 Spectral Correction-Influence on Spectra Reference is now made to Figures 5a and 5b. FIG. 5a shows the spectrum of five adjacent pixels extracted from the image of FIG. 3a without the spatial positioning and spectral correction processes of the inventive device. Four of these pixels are
The interferogram is located around the fifth pixel, which is the pixel shown in Figure 4a. On the other hand, FIG. 5b shows the spectrum of the same five pixels after the application of the spatial positioning process and the spectral correction process according to the invention. 57
The depression around 5 nm is characteristic of oxyhemoglobin absorption.

Two phenomena are noticeable when comparing the spectra of FIGS. 5a and 5b. First, compared to FIG. 5b, the corresponding spectrum in FIG. 5a is quite noisy. Second, as shown in Figure 5b, when the device of the invention is run from pixel to pixel, the spectrum changes in a uniform pattern, showing the expected smooth appearance over the entire spectral range, No such aspect is seen in the spectrum of Figure 5a.

Therefore, Examples 2 and 3 emphasize the importance of spectral correction to obtain meaningful interferograms and spectra from the measured object.

Example 4 Spatial Positioning of Frame Assisted by Interference Suppression Suppressing Raw Data of Randomly Moving Objects Measured by Interferometric Spectral Imager Before Performing Fourier Transform Operation on Interferogram of Pixels By doing so, the best final spectral cube can be obtained.

This is because in a spectral imager based on a Sagnac or similar interferometer, the instantaneous optical path difference (OPD) corresponding to the interferogram data point is simply a particular CCD frame, as described here. Due to the fact that the data points also depend on the particular pixel concerned.

As a result, if the object moves during the measurement, the pixels constrained to points on the object are different than if the object was stationary, so if no correction is made, the Fourier transform algorithm is , Will use the wrong OPD for that data point. If, by some means, the algorithm can be made to use the appropriate OPD instead of an incorrect value for each data point, the resulting spectral image cube can be significantly corrected. In order to find the appropriate OPD for each interferogram data point, one is based on (i) the spatial positioning of each measurement frame and its recording of the positioning vector, and (ii) the positioning vector and the position dependence of the OPD. Requires the calculation of the actual OPD for the data points.

However, when trying to do this positioning automatically,
There is one physical phenomenon, which is not impossible, but the appearance of interference fringes that make frame positioning quite difficult.
As shown in FIG. 6a, the interference fringes are brightness-modulated linear fringes overlapping on the frame, the positions of which change slightly for each frame according to the scanning position of the interferometer. The cause of the fringes is the constructive (light fringes) and destructive (dark fringes) interference of the rays as they pass through the interferometer. The fringe shape (vertical or horizontal line depending on the optical alignment) is such that every pixel on the vertical (or horizontal) line passes through the same OPD in each scan frame (for light of the same wavelength).
Due to the fact that the same amount of interference occurs. The position change from frame to frame is due to the fact that the constructive or destructive level of interference for a particular pixel changes during scanning depending on the position of the interferometer. When the scanned frame is aligned with the naked eye, one frame on top of the other, other features (such as the blood vessel pattern of the eye) are clearly visible in each frame, even with interference fringes. The fringes are less annoying and the appearance of the fringes does not interfere with the observer's best spatial positioning, i.e. frame superposition, but with the algorithm the non-uniformity overlaid on the features of the frame These streaks, which are changes in the intensity of various lights, make automatic processing difficult. As already mentioned, the interference fringes are vertical (or horizontal) fringes that move from frame to frame in a direction perpendicular to the fringes in response to rotation of the interferometer mirror.

The input of the interference fringe suppression algorithm is the cube of the frame of the interferogram with the interference fringes, and the output is the cube of the frame without the interference fringes. For this,
Further details will be described below.

Some assumptions are made regarding the operation of the interference fringe suppression algorithm. One of the assumptions is that the approximate value of the interference fringe "frequency" is known. In other words, assume the distance in pixels between adjacent fringes. This knowledge may be obtained from each frame of the interferogram cube itself for a particular type of sample or from the calibration process based on previous experience.

As is clear from FIG. 6a, the interference fringe information is very closely arranged in the frequency domain. Therefore, it is easy to find the center frequency of the interference fringes, and the width of the interference fringe information in the frequency domain is considered to be constant or almost constant in all scanning frames.

Therefore, the interference fringe suppression algorithm suppresses the interference fringes by artificially replacing the signal in the frequency range of the three-dimensional frequency domain where the interference fringe information is present with zero or interpolating and removing it for each scanning frame. To do.

Since the fringes are approximately parallel to one axis (eg, the x-axis), the frame is decomposed into vectors along an axis (eg, the y-axis) orthogonal to the fringes. FIG. 6b shows the intensity of 200 pixels, which is such a vector. Interference fringes clearly appear between the 100th pixel and the 150th pixel. Then, as shown in FIG. 6c, each vector is transformed into the frequency domain using, for example, a fast Fourier transform algorithm (FFT). In this case, there is a peak of interference fringe information in the range of about 0.15 to about 0.35 pixel-1. As shown in FIG. 6d, for each vector, the frequency domain in which the interference fringe information is located is zero-adjusted, and as shown in FIG. 6e, for example, the fast inverse Fourier transform algorithm (IFF
T) to convert back to the spatial domain. Performing this process on each of the vectors of each frame obtained by the spectral imager during scanning of the interferometer results in a frame with suppressed interference fringes, as shown in FIG. 6f.

If for some reason the interference fringes are angularly offset, that is, they are not aligned vertically or horizontally,
The frequency of the interference fringe information is slightly reduced. This problem can be solved by increasing the zero adjustment or interpolation region width of the signal in the three-dimensional frequency domain where the interference fringe information exists.

As is clear from FIG. 6c, most of the energy of the frame is located in the low frequency band of the frequency domain. Since the energy in the high frequency band is not attenuated, by using a band stop filter, it is possible to remove the blurring of the frame while maintaining the information of each scan frame, and also to maintain the edge information.

Haku transformation ("Methods and means for recog" by Paul VC, Hough, "Methods and means for recog"
It will be apparent to those skilled in the art that the frequency location of the interference fringe information can be derived and used for interference fringe suppression algorithms by using nizing complex patterns and US Pat. No. 3,069,654). It is also possible to find the direction of the interference fringes by Haku transform and make necessary adjustments.

In order to keep the signal after the IFFT as a real number, it is preferable to perform zero adjustment processing symmetrically with respect to the origin of the three-dimensional frequency axis (not shown in the figure, the frequency domain signal is positive and negative at the frequency f). Defined for both negative values, f
Is an even or symmetric function). IFFT is used to exclude using the absolute (or actual) part of the operation result.
In the latter signal, almost no interference fringes remain, as shown in Fig. 6e.

Returning to FIGS. 6b and 6e, instead of performing FFP, performing the zero adjustment and the IFFT process described above results in interference fringes, as indicated by the symbol I (intersection) in the interference fringe region of the plot of FIG. 6b. One can simply obtain an interpolated plot as the relative intensities across each of the intensity peaks in the. This will be very similar to that shown in Figure 6e.

Another option is to draw a straight line between the end points of the peaks instead of adjusting the zeros of the peaks (as shown in Figure 6e) for areas where interference fringe information is present in the stereoscopic frequency domain. it can.

The preferred interference fringe suppression algorithm of the present invention is mathematically represented below.

Let X (x, y) be the input frame (eg as shown in FIG. 6a) and Y (x, y) be the corresponding output frame (eg as shown in FIG. 6a). x and y are the intrinsic coordinates of one pixel in the frame, fCF is the center frequency of the interference fringe information, fLF is the low frequency of the interference fringe information, fHF is the high frequency of the interference fringe information, and Δf is The width of the interference fringe suppression frequency band, and u (f) is a step function.

By definition, fLF = fCF−0.5Δf (4) fHF = fCF + 0.5Δf (5) The “zero adjustment frequency band” function is defined as follows.

W (f) = {1- [u (f-fLF) -u (f-fHF)]-[u (f + fLF) -u (f + fHF)]} (6) W (f) ("zero adjustment frequency band" Function) can be defined as a function of frequency f, so when multiplied by another function of frequency f, there is no change for values of f lower than fLF and higher than fHF, higher than fLF, and fHF It changes to zero for lower values of f.

An output frame without interference fringes can be expressed as follows.

Y (x, y) = Re {IFFT {W (f) * FFT {x (:, y)}}} (7) By using the interference fringe suppression frame, the repeated pattern of interference fringes overlapping the frame is removed. , Automatic positioning process becomes easy.

Although the present invention has been described with respect to a limited number of embodiments,
Obviously, many variations, modifications and other applications of the invention can be made.

Continuation of the front page (56) Reference JP-A-8-251474 (JP, A) JP-A-7-301562 (JP, A) JP-A-5-231939 (JP, A) JP-A-5-14559 (JP , A) JP-A-4-339476 (JP, A) JP-A-5-507225 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G01J 3/00-3/52 G01N 21/00-21/61 A61B 3/00-3/16 G06T 1/00-9/40

Claims (19)

(57) [Claims] 1. A means for obtaining stereoscopic information and spectral information of a movable object by using a spectrum imaging device based on an interferometer, and (b) spatial positioning of the stereoscopic information and spectral information. Means for correcting the movement of the movable object through processing and spectrum correction processing to obtain corrected stereoscopic information and spectrum information, and means for obtaining the corrected stereoscopic information and spectrum information. (A-1) collects incident light from the movable object, and (a-2) allows the collected light to pass through the interferometer to linearly combine the emission spectrum luminance from each pixel of the movable object. (A-3) focus the modulated light output from the interferometer on a detection array, and (a-4) reduce an optical path difference generated in the interferometer. , In front of the movable object Scanning all pixels independently and simultaneously, and (a-5) processing the output of the detection array to measure the spectrum of each of the pixels of the movable object, The spatial positioning process and the spectral correction process by the means for obtaining physical information and spectral information are as follows: (b-1) selecting a reference frame of the movable object, and (b-2) using the reference frame. The transformation vector is calculated with respect to another frame of the animal, and (b-3) the spatial positioning and spectrum correction processing is performed using these transformation vectors. Moving only along the interferometer, the interferometer is of a type in which the optical path difference in the frame changes along a single direction, (c) The imaging device,
A spectral imaging device for a movable object, comprising means for arranging the optical path difference gradient so as to be perpendicular to the first direction. 2. The apparatus according to claim 1, further comprising (d) means for displaying the corrected stereoscopic information and spectral information as an image. 3. The RG of the corrected spectral information according to the corrected three-dimensional information is displayed by the image.
The device according to claim 2, which is a B display. 4. The calculation of the transform vector is performed by superimposing the reference frame on one of the other frames and finding a position for generating a subtraction image for performing luminance subtraction processing. apparatus. 5. The device according to claim 1, wherein the movable object is a living body organ or a part thereof. 6. The device according to claim 5, wherein the body organ is the eye. 7. The apparatus of claim 1, wherein the spatial positioning process is performed using an interference fringe suppression algorithm. 8. (a) means for aiming and focusing an interferometer-based spectral imager with respect to the movable object; and (b) scanning the interferometer at a constant speed while detecting the movable object by the detection array. Means for capturing successive frames and storing the successive optical path differences of the interferometer in a state where they are not equidistant due to movement of the movable object; (c) the continuation for all pixels of the movable object. The collected data of each successive frame to the interferogram function, and at the same time, using one of the successive frames as a reference frame, the three-dimensional conversion vector of each of the successive frames is calculated. Means for finding the actual optical path difference for all pixels in each of the successive frames; and (d) Fourier transforming each of the interferograms. Means for calculating a Fourier transform for each pixel of the moveable object by applying a conversion algorithm; (e) means for calculating a spectrum for all pixels of the moveable object; Moving only along a first direction, the interferometer is of the type in which the optical path difference within a frame changes along a single direction, the spectral imaging device relative to the movable object And a means for positioning the difference gradient so as to be perpendicular to the first direction. 9. The apparatus according to claim 8, further comprising means for displaying an image of the movable object. 10. The display of the image is of the calculated spectrum.
The device according to claim 9, which is an RGB display. 11. The position where the calculation of the three-dimensional conversion vector of each of the continuous frames with respect to one of the continuous frames, which is the reference frame, generates a subtraction image for performing luminance subtraction processing. 9. The apparatus of claim 8, which is done by superposing the reference frame and one of the other frames to find. 12. The device according to claim 8, wherein the movable object is a living body organ or a part thereof. 13. The device according to claim 12, wherein the body organ is the eye. 14. (a) A means for aiming and focusing an interferometer-based spectrum imaging device with respect to a movable object, and (b) the movable object by a detection array while scanning the interferometer at a constant speed. Means for capturing successive frames and storing the successive optical path differences of the interferometer in a state where they are not equidistant due to movement of the movable object; (c) the continuation for all pixels of the movable object. The collected data of each successive frame to the interferogram function, and at the same time, using one of the successive frames as a reference frame, the three-dimensional conversion vector of each of the successive frames is calculated. And (d) means for finding the actual optical path difference for every pixel of each of the successive frames, and (d) the interface of each of the pixels of the movable object. Means for interpolating the program to obtain evenly spaced optical path difference values; and (e) means for calculating a Fourier transform for each of the pixels of the movable object by applying a Fourier transform algorithm to each of the interferograms. (F) means for calculating a spectrum for all pixels of the moveable object, (g) the moveable object moving only along a first direction, and the interferometer measuring the optical path difference in the frame. A type that varies along a single direction, and means for positioning the spectral imaging device with respect to the movable object such that the optical path difference gradient is perpendicular to the first direction. Animal spectral imager. 15. The apparatus of claim 14, further comprising means for displaying an image of the moveable object. 16. The display of the image is of the calculated spectrum.
16. The device according to claim 15, which is an RGB display. 17. The calculation of the three-dimensional conversion vector of each of the continuous frames with respect to one of the continuous frames, which is the reference frame, performs luminance subtraction processing to generate a subtracted image. 15. The apparatus according to claim 14, which is performed by superposing the reference frame and one of the other frames to find a position to be used. 18. The device according to claim 14, wherein the movable object is a living body organ or a part thereof. 19. The device according to claim 18, wherein the body organ is the eye.