WO2000071980A9

WO2000071980A9 - Method and system for adaptive interpretation of spectrometric data combined with continual re-calibration

Info

Publication number: WO2000071980A9
Application number: PCT/CA2000/000583
Authority: WO
Inventors: Andrzej Barwicz; Michael P Wisniewski; Roman Z Morawski; Slima Mohamed Ben
Original assignee: Bookham Technology PLC
Current assignee: Lumentum Technology UK Ltd
Priority date: 1999-05-21
Filing date: 2000-05-23
Publication date: 2002-01-17
Anticipated expiration: 2001-11-21
Also published as: CN1361861A; AU4903000A; EP1183505A1; WO2000071980A1; JP2003500639A

Abstract

A method for adaptive reconstruction and interpretation of spectra combined with re-calibration of a device providing spectrometric data, consist in carrying out an automatic calibration using the external reference light spectrum and its corresponding digital reference data stored in the internal memory of the device. The continual re-calibration procedure allows for automatic adaptation of the values of the coefficients in a reconstruction sub-procedure as well as the estimation of the values of the coefficients in an interpretation sub-procedure on the bases of a current shape of spectrometric data.

Description

Method and system for adaptive interpretation of spectrometric data combined with continual re-calibration

Field of Invention

This invention relates generally to light-spectrum-measurement and more specifically to compact systems for light-spectrum-measurement for portability and for in situ applications.

Background of the invention

Spectroscopy is an analytic technique concerned with the measurement characterization of the interaction of radiant energy with matter, including the instruments designed for this purpose, called spectrometers, and corresponding means of the interpretation of the interaction both at the fundamental level and for practical analysis. The distribution of radiant energy, absorbed or emitted by a sample of a substance under study, is called its spectrum. Interpretation of spectra provides fundamental information at atomic and molecular energy levels, the distribution of species within those levels, the nature of processes involving change from one level to another, molecular geometries, chemical bonding, and interaction of molecules in solution. At the practical level, comparisons of spectra provide a basis for the determination of qualitative chemical composition and chemical structure, and for quantitative chemical analysis.

The spectra to be measured have two common features: they are non-negative, and they may be decomposed into relatively flat regions and peaks of various widths. The interpretation of the data representative of those spectra obtained by means of spectrometers, very frequently involves estimation of some parameters of the peaks, viz.: of their positions and magnitudes (i.e. areas or heights). The parameters of the peaks are used, in particular, for qualitative and quantitative analysis of complex chemical substances:

• the positions of peaks are used for identification of the compounds which are present in a sample of the substance under study; • the magnitudes of peaks are used for estimation of the concentrations of the identified compounds.

The estimation of the parameters of spectrometric peaks directly from the acquired spectrometric data is in many practically important cases problematic because of:

• instrumental imperfections of the spectrometer, and

• some natural phenomena in a sample of the substance under study.

The imperfections of the spectrometer produce blurring of peaks and noise-type disturbances in the data representative of the measured spectrum. On the other hand, some quantum phenomena in the substance enlarge considerably the width of peaks. Both instrumental blurring and natural enlargement of peaks may contribute to their overlapping and fusion, making impossible their proper identification. Different algorithms of deconvolution have been developed for reducing instrumental blurring; a review of these algorithms may be found in Jansson P. A. (Ed.) : Deconvolution with Applications in Spectroscopy. Academic Press, 1984 and Morawski R. Z.,

Szczecin ski L., Barwicz A.: "Deconvolution Algorithms for Instrumental Applications - A Comparative Study", J. Chemometrics, 1995, Vol. 9, pp. 3-20. These algorithms are designed and assessed using the criteria of the quality of deconvolution, which is not specific of spectrometry; e.g. the root-mean-square errors of approximation. Consequently, the estimates of the magnitudes determined on the basis of the results of deconvolution may turn out to be very poor. On the other hand, the curve-fitting algorithms may be very inefficient if not provided with a good initial guess of the sought- for estimates. These two observations inspired independently many researchers to subsequently use a deconvolution algorithm for estimation of the positions and a curve- fitting algorithm for estimation of magnitudes of spectral peaks.

Both the algorithms of spectrum reconstruction and the algorithms of spectrum parameter estimation require considerable amount of information on the mathematical model of the spectrometric data. This information is acquired during initial calibration of the spectrometer. The numerical algorithms necessary for calibration of the spectrometer, reconstruction of the spectrum and estimation of its parameters are available in many software libraries, e.g. in some MATLAB Toolboxes, including specialized libraries dedicated to processing of spectrometric data, e.g. GRAMS/32 of Galactic Industries. In the patent domain, there is only a partial methodology for spectrum deconvolution, e.g. the US Patent No.5 247 175, issued September 21, 1993, in the name of Alan E. Schoen et al, entitled Method and Apparatus for the Deconvolution of Unresolved Data as well the US Patent No.4 941 101, issued July 10, 1990, in the name of Paul B. Crilly, entitled Method for Analyzing Chromatograms specific to the domain of chromatography. On the other hand, there is a good example of a complete methodology of spectrometric data interpretation - c.f. the US Patent No. 5,991,023, issued November 23, 1999, in the name of R. Z Morawski et al., entitled Method of interpreting spectrometric data.

However, in all the above-listed references, none of them offers full support for the methodology of on-line continual re-calibration and adaptive spectrometric data interpretation, mainly due to the fact that there are no closed procedures of such re- calibration and adaptive interpretation.

The demand for an increased-resolution and a self calibrated spectrometric device comes from everyday practice. The US Patent No. 5040889, issued August 21,1991, in the name of Thomas J. Keane, entitled Spectrometer with Combined Visible and

Ultraviolet Sample Illumination, describes a spectrometer with an automatic calibration of the photodetectors, based on the use of a pivotable standard white sample positioned in the path of the light and controlled by a computer system. A calibration factor is computed and recalculated for each photodetector whenever the temperature changes within the spectrometer, and then used to multiply the output signal of the photodetectors. However, this is a partial static calibration, since it corrects only the effect of the photodetectors and doesn't take into account the effect of the dynamical errors introduced by the light diffraction element and the slits. Thus, there is a need for another dynamic calibration procedure to eliminate these instrumental errors and enhance the spectrum resolution. In the fields of telecommunications, especially for optical performance monitoring of networks, there is a need for a spectrometric device, which must be compact, operating under a broader range of environmental conditions and more particularly self-calibrating. In Miller C, Pelz L.: "Fabry-Perot Tunable Filters Improve Optical Channel Analyzer Performance", Lightwave, March 1999, Vol. 16, No. 3, a card mounted optical channel analyzer performance, based on the use of a fiber Fabry-Perot tunable filter, is described. This device uses an optical switch to alternately scan the light spectrum (coming for example from a DWDM system) and a known wavelengths reference source. This latter is used for the continual calibration of the device in order to correct the thermal drifts, the nonlinearities of the actuator and the voltage variations. A deconvolution algorithm is also used to remove the effects of the tunable filter spectral response in order to enhance the resolution of the measured light spectrum. The major drawback of this device is that it measures one wavelength or path length difference at a time, thus necessitating a long time to process all the optical channels. Thus, a need remains for a low-cost miniaturized spectrometric device, self-calibrated, with a spectral resolution comparable to that of conventional optical spectrum analyzer, and capable of determining the spectral signature of a wide variety of light spectra in situ.

In the domain of spectroscopy, the demand for an adaptive reconstruction and interpretation procedures combined with a continual re-calibration is generated by the need for a spectrometric device that can be compact, autonomous with a higher degree of reliability, self-calibrated, operating under a broader range of environmental conditions, and be cost-effective for wide deployment. The main objective of the invention is to respond to this demand.

Object of Invention

It is an object of this invention to provide a method for adaptive interpretation of spectra on the basis of spectrometric data.

It is an object of this invention to provide a method for calibration and continual re-calibration of integrated spectrometric devices allowing for miniaturization and low- cost mass production of micro-optical spectrum analyzers. Summary of the invention

It is an object of this invention to provide a procedure for interpretation of spectrometric data, representative of a sample of the substance under study, comprising the steps of: • initial calibration of the spectrometric device;

• continual re-calibration of the spectrometric device

• adaptive reconstruction of the spectrum under study; and

• adaptive interpretation of the spectrum under study.

The invention also provides a method for calibration and continual re-calibration of the spectrometric device, i.e. for identification of a numerical relation between the measurement results produced by this spectrometric device and established measurement standards of spectrum.

The invention also provides a method for continual re-calibration of the spectrometric device allowing for automatic adaptation of the method to possible variation of the instrumental imperfections and a shape of spectrometric data due to the aging of the device and/or thermal drifts and nonlinearities of the detector.

The invention also provides a method for numerical adaptive reconstruction of the spectrum under study, combined with the continual re-calibration of the spectrometric device, i.e. for partial elimination of the effects of noise and blur corrupting the spectrometric data.

The invention also provides a method for numerical adaptive interpretation, i.e. estimation of the positions and magnitudes of peaks the spectrum of the light is composed of.

According to the invention, the calibration of the spectrometer is accomplished using an auxiliary light whose spectrum is assumed to be known and representative of the spectra to be measured in the series of experiments following the calibration. The result of calibration has the form of two operators defining the following numerical algorithms: • an algorithm for simulation of the spectrometric data, given the spectrum to be measured (operator of projection ^);

• an algorithm for estimation of the spectrum, given the data (operator of reconstruction 3?).

The first algorithm is an algorithm of model identification including choice or estimation of the structural parameters of the model and estimation of its functional parameters. The second algorithm is an algorithm of deconvolution, or an algorithm of generalized deconvolution, or an algorithm for numerically solving the first-kind integral equations.

According to the invention, the continual re-calibration of the spectrometer is accomplished using an external reference light spectrum and the digital reference data representing this reference light, stored in a computer or an internal memory, when available, of the spectrometric device. The re-calibration of the device is aimed at updating the parameters of the above mentioned operators.

According to the invention, the processing of the data representative of the light under study is comprising two main operations:

• estimation of the unknown spectrum or its idealized version on the basis of the acquired spectrometric data, using the operator of reconstruction 5?;

• estimation of the parameters of this spectrum on the basis of the acquired spectrometric data, using the operator of projection and an algorithm of curve fitting.

The method of adaptive interpretation of spectrometric data is useful for providing the quality light-spectrum measurement especially in the case of miniature integrated intelligent spectrometric devices. When combined with automatic recalibration continually or at intervals the spectrometric device keeps optimum performance, in particular, for telecommunication applications.

The proposed method allows for automatic calibration of especially miniature integrated spectrometric devices during a manufacturing process and absent physical tuning of individual devices. The calibration will compensate for many forms of fabrication discrepancies without modification of the physical devices.

As the consequence, it contributes to a low-cost flexible quality light-spectrum- measurement solution, especially on-chip light spectrum measurement, which may be easily adapted to very diversified applications via customization of the specific set of algorithms.

Brief description of the drawings

Exemplary embodiment of the invention will now be described in conjunction with the drawings in which: Fig. 1. presents an exemplary measuring system;

Fig 2. presents the flow diagram of the procedures of adaptive reconstruction and interpretation a) the procedure AI-RC, b) the sub-procedure AI-RC_cal, c) the sub-procedure AI-RC_recal, d) the sub-procedure AI-RC_rec, e) the sub-procedure AI-RC_int;

Fig. 3. presents the exemplary measuring system; Fig. 4. presents the spectrum of the test sample: a) real spectrum x(λ) ,

Fig. 5. presents the spectrum of the sample used for calibration: a) real spectrum x^ca!( λ) ,

Fig. 6. presents the output of: a) the rational filter, b) the spline-based Kalman filter;

Fig. 7. presents the final result of spectrometric data interpretation; Fig. 8. presents the exemplary measuring system - MM microOSA; Fig. 9. shows possible applications of MM microOSA™ in Dense Wavelength Division Multiplexing (DWDM) system; Fig. 10. presents the spectrum of the test light: a) the idealized spectrum s(λ) of the light at the input of the DWDM transmitter,

Fig. 11. presents the emission spectrum of the laser; and,

Fig. 12. illustrates the final result of adaptive reconstruction and interpretation of telecommunication data.

Detailed Description of the Invention

The procedure of adaptive reconstruction combined with continual re-calibration of a device providing spectrometric data and adaptive inteφretation of spectra (AI-RC) is designed for a measuring system, for example the system shown in Fig.l, comprising:

• a spectrometric device in the form of a spectrometer or its essential part, in the form of a micro-optical spectrum analyzer converting an optical signal carrying the information on the measured spectrum into a digital code representing this spectrum;

• a processing means for processing the digital representation, in the form of a general- puφose computer, a microprocessor, a general-puφose digital signal processor, or an application-specific digital signal processor; and • other functional elements necessary for measuring the spectrum of light.

The following notation is used for the description of the procedure AI-RC: λ - wavelength; λ e[4_ni_n A_we] ;

N - number of data acquired by a spectrometric apparatus; Aλ - step of wavelength discretization; AΛ = ( l_max - l_min)/(N - 1) ; λ_n - n-th datum acquired by the spectrometric apparatus; λ_n = ^ + (n - \)Δλ for n = \,...,N ; x(λ) - real spectrum of a light under study; 1 - a vector of positions of peaks within the spectrum x(λ) is composed of;

^\ = [hh-lκ]^T\

I - an estimate of 1 ; a - a vector of magnitudes of peaks within the spectrum x(λ) is composed of;

a = [a_la₂...a_κ] ; a - an estimate of a; s(λ;l,&) - an idealized spectrum of light under study and, in the preferred embodiment, assumed to have the form:

A: s(λ;l,a) = ∑a_kv_s(λ,l_k) k=\ where v_s(λ,l) is an isolated, normalized peak in s(λ;\,a), whose maximum is located at λ = l;

[^v_s(λ,l)dλ = \ for / e[^_min, _max] ;

- spectrometric data representative of x(λ) , acquired by means of the spectrometric apparatus; {y_n}≡{y„\n = l,...,N} x^cαl(λ) - a real spectrum of light used for calibration of the spectrometric device; s(λ;\^cαl ,a^cαl ) - an idealized spectrum of light used for calibration of the spectrometric device;

(j'_l. J " ^me spectrometric data, representative of x^αl(λ) used for calibration of the spectrometric device;

{y¹}≡{y_n ^cαl\n=\,...,N^cαl);

x^recαl(λ) - the real spectrum of an external reference light used for the re-calibration of the spectrometric device; ⁱφj^recai j _ _{me S}p_ecrrometric data measured using the spectrometric device,

representative of x^r (λ) used for the re-calibration of this device; recal ^ecal)^≡ n =

s(λ;\^recaI ,a^r ) - an idealized spectrum of a light used for the re-calibration of the spectrometric device;

& - an operator (transform) of projection mapping the idealized spectrum s(λ; 1, a) into the space of the data:

where p_e is a vector or matrix of the parameters of the operator & , to be determined

during calibration of the spectrometric device; p₆ = \p_{& (I} p_{& 2} ...I or:

P& ,1,1 PG ,1,2

Ps PG ,2,1 PG ,2,2

R - an operator (transform) of reconstruction and inteφretation:

R ^r - an operator (transform) of reconstruction, in the form of a generalized deconvolution method, for transforming the data y_n } into an estimate {x_n } of x(λ) :

where p_{R r} = [p_R , _] p_R , ...f are parameters of the operator R ^r including regularization parameters for determination during calibration of the spectrometric device. R ' - an operator (transform) of inteφretation, in the form of a generalized deconvolution method, for transforming the data {x_n } into an estimate s(λ) of s(λ; I, a) :

where p_R ; = _R , , _R , ₂ ...J are parameters of the operator R ' including regularization parameters for determination during calibration.

2. Functional specification of a preferred embodiment of the invention 2.1. General specification of the procedure AI-RC

The main objective of the procedure AI-RC is estimation of the positions 1 and magnitudes a of the peaks contained in a spectrum of a light under study x(λ) on the basis of the acquired spectrometric data y_n j . The feasibility of this operation is critically conditioned by an auxiliary operation on the reference data | _n ^cα/ j and corresponding reference spectrum x^cal(λ) , referred to as calibration of the spectrometric device. This operation is aimed at the acquisition of information relating to a mathematical model of a relationship between spectrometric data and idealized spectrometric data known to be measured, which underlies the method chosen for estimation of the parameters 1 and a. Although calibration does not necessarily immediately precede processing of each sequence or sample of spectrometric data y_n , preferably valid calibration results are always available during spectral processing.

The main difficulty relating to estimation of the positions I and magnitudes a of spectrometric peaks within a sensed spectrum is implied by blurring of those peaks, caused by physical phenomena in a sample, and by blurring of their representations in the data [y_n J , caused by an imperfect spectrometric apparatus. This difficulty is overcome in the procedure AI-RC through application of a series of steps for adaptive reconstruction and adaptive inteφretation of an idealized - hypothetical - spectrum s(λ;l,a) in order to correct the sensed data to reduce both types of blurring; if s(λ;l,a) is assumed to be an approximation of x(λ), then only the instrumental blurring needs to be corrected.

In accordance with the above general functional specification, AI-RC is composed of the following broadly defined steps:

• initial calibration of the device providing the spectrometric data, i.e. of the spectrometric apparatus or of the spectrometric sensor or of the spectrometric transducer (the sub-procedure AI-RC_cal),

• continual re-calibration of that device on the basis of auxiliary reference data obtained by means of the device stimulated with the reference light (the sub-procedure AI- RC_recal), • adaptive reconstruction of the spectrum under study { C_Π }, i.e. estimation of this spectrum on the basis of the spectrometric data {y_n} representative of the spectrum {J „ }

(the sub-procedure AI-RC_rec), and

• adaptive inteφretation of the spectrum under study {x_n } in the form of adaptive estimation of the positions l_λ, l₂, ... and magnitudes a_λ, a₂, ... of peaks this spectrum is composed of- on the basis of an estimate {x_n} of the real spectrum, obtained by means of the sub-procedure AI-RC_rec (the sub-procedure AI-RC_int).

2.2. Detailed specification of the sub-procedure AI-RC cal

The sub-procedure AI-RC_cal comprises the following operations: a) choosing the method for idealization of the spectrum, i.e. the form of peaks of the idealized spectrum {s„} ; b) choosing the form of the operator R for reconstruction of the idealized spectrum: {?„}=R [{ >„}; PR ], where p_R is a vector of the parameters; c) choosing the rules for adaptation of the parameters p_R to the selected features F_x, F₂, ... of the spectrum under study {x_n} or of the data {y_n} - on the basis of the library of parameters jp ' | j = 1,...,J| to be determined during initial calibration, and updated during continual re-calibration; d) choosing reference light signals for the initial calibration: |^^α ( )j ^an^ related

acquiring the data ψ '\ representative of ^^α/(/)_j for j=l,...,J ; e) pre-processing of the data

including the elimination of outliers, baseline subtraction, smoothing, acquiring a priori information for the operation 2.g (e.g. a pre- estimate of the variance of errors in the calibration data), normalization, etc.; f) determination of the vectors of parameters pfc - on the basis of the triplets:

^„^a,{j)} and ^ (j)} foτ j=l,...,J ; g) approximation of the relationship between the parameters p_R and the features F_x , F₂ , ... controlling the adaptation.

2.3. Detailed specification of the sub-procedure AI-RC recal

The sub-procedure AI-RC_recal comprises the following operations: a) choosing the reference light signals for re-calibration:

and related

for /=1,...,J ; b) acquiring the data

for j=l,...,J ; c) pre-processing of the data γ ^{ca l}j including elimination of outliers, baseline subtraction, smoothing, acquiring a priori information for the operation 3.d (e.g. a pre- estimate of the variance of errors in the calibration data), normalization, etc.; d) updating of the vectors of parameters p_R ⁾ on the basis of triplets: „^eca'{j)), ^'O^')} and { ^a'(/)} for y = l,...,J; e) updating the approximation of the relationship between the parameters p_R and the features F_l, F₂, ... controlling the adaptation.

2.4. Detailed specification of the sub-procedure AI-RC rec

The sub-procedure AI-RC_rec comprises the following operations: a) acquiring data ly_n \ representative of the spectrum under study {x_n};

b) pre-processing of j5>„ } including elimination of outliers, baseline subtraction, smoothing, acquiring a priori information for the estimation of peak parameters in the form of a pre-estimate of the variance of errors in the data, acquiring a priori information for the reconstruction in the form of determination of apparatus fabrication imperfections on the basis of characteristics of measured spectrum

, ambient temperature sensed with a temperature sensor, and aging of working apparatus elements, normalization, etc.; c) estimation of the spectrum under study {x_n }on the basis of the data {y_n j , by means of the operator R ^r with the parameters p_{R r} being continuously adapted to the features of the data on the basis of the library of parameters, determined during initial calibration and updated during continual re-calibration.

2.5. Detailed specification of the sub-procedure AI-RC_int

The sub-procedure AI-RC_int comprises the following operations: a) adaptive smoothing of the result of estimation {x_n} in 2.4(c) of the spectrum under study; b) acquiring a priori information for inteφretation of {x_n } in the form of a number of absoφtion peaks and estimation of widths of absoφtion peaks; c) estimation of the idealized spectrum s(λ) on the basis of the data {x„}, by means of the operator £ with the parameters p_R , being continuously adapted to the shape of the data on the basis of the library of parameters determined during an inteφretation calibration; d) estimation of positions and magnitudes of peaks within the spectrum under study; e) estimation of the positions 1 ofthe peaks by means of a maximum-detection algorithm on the basis ofthe estimate s(λ) of s(λ;\,a), obtained in step 2.5(c); f) estimation ofthe magnitudes a ofthe peaks, by means of an adaptive algorithm of curve fitting using one ofthe following:

- the data

, v_s(λ, l) chosen in 2.2(a), the operator 3? chosen in 2.2(a) with parameters p_e determined in 2.2(f), and the estimate 1 resulting from 2.5(e);

- the estimate {x_n} obtained in 2.4(c), v_s(λ, l) chosen in 2.2(a), and the estimate ϊ resulting from 2.5(e). g) iterative correction ofthe estimates ofthe parameters of peaks, obtained in 2.5(e) and 2.5(f); h) adaptation ofthe results of parameter estimation to the requirements of a user, e.g. their transformation into some pre-defined parameters ofthe analyzed substance.

The flow-diagram ofthe whole procedure AI-RC is shown in Fig.2. 3. An exemplary embodiment ofthe invention

Example 1

A particular version ofthe procedure AI-RC has been designed for a measuring system shown in Fig. 3 and including: the absoφtion mini spectrophotometer - model S 1000 by Ocean Optics®; and a personal computer PC.

The following measurement parameters were selected both for calibration and for acquisition ofthe test data:

• the wavelength range: λ_min = 450nm, λ_max = 675nm ;

• the number of data acquired by the spectrophotometer: N = 1024 ; and, • the step of wavelength discretization: Aλ = (λ_max - λ_min )/(N - 1) = 0.22nm .

The test data were acquired for a standard holmium perchlorate solution sample; its real spectrum x(λ) is shown in Fig. 4a. The parameters of this spectrum are the following:

• the vector ofthe positions of peaks:

1 = [452.2 468.2 473.1 485 537.3 541.3 543.7 639.8 644.2 652.2 656.4

• the vector ofthe magnitudes of peaks: a= [l.l7 0.19 0.2 0.461 0.988 0.41 0.297 0.821 0.406 0.246 0.263J^r The idealized spectrum of a sample under study has been assumed to have the form:

s(λ;l,a) = ∑α_kv_s(λ,l_k) k=ϊ with the peaks defined by: v_s(λ, l) = δ(λ- l) for / e[,t_min ,Λ_max] The set of data representative of x(λ) , acquired by means ofthe spectrophotometer, {y„ }- {y_π \ ^{n =} 1,— ,1024}, is shown in Fig. 4b. Calibration data have been acquired for the standard holmium oxide filter sample; its real spectrum x^cal(λ) is shown in Fig. 5a. The parameters of this spectrum are the following:

• the vector ofthe positions of peaks:

l^cα/ = [450.7 454.5 460.4 463.5 473.9 483.9 488.8 536.4 547.5 633.4 636.5 648.4]⁷

• the vector ofthe magnitudes of peaks:

a^cal = [0.486 0.799 0.949 0.58 0.125 0.117 0.114 0.284 0.106 0.129 0.149 0.109]⁷ . The idealized spectrum of a sample used for calibration s(λ;\^cal ,a^ca ) has been assumed to have the form:

The set of data representative of x^cal(λ) , acquired by means ofthe spectrophotometer, {y„""}≡ jy„^cα' I n = 1 1024}, is shown in Fig. 5b.

The chosen operator (transform) of projection, mapping the idealized spectrum s(λ;l,a) into the space ofthe data:

{y_n} =& [s(λ;l,a);p₆ ] has been defined by the following operations:

x(λ) = exp °g_B{λ - λ')

dλ'

y{λ) = ^_∞S^χy(^λ - ^λ') ^λ') ^dλ' y_n = y(^λn) ^for n = ι,...,N

The function

has been assumed to have the form ofthe Gauss function:

Consequently, the vector ofthe parameters p_e ofthe operator & contains the discrete values of g_sx(λ) and the parameter σ .

The chosen operator (algorithm) of reconstruction, transforming the data

into an estimate s(λ) of s(λ;\,a),

*W=R [{?„}^;!* ] is defined by in the following way: a discrete estimate | _n] of x(λ) is found by means ofthe rational filter described by Szczecin ski L.: "Methodes non lineaires de reconstitution de signaux pour des applications spectrometriques", Ph. D. thesis, Universite du Quebec - Institut National de la Recherche Scientifique ((Telecommunications)), Montreal 1997. and applied to the data

• an estimate s(λ) of s(λ;\,a) is computed by means ofthe adaptive rational filter as described in Wisniewski M. P., Morawski R. Z., Barwicz A., 1999, "An Adaptive Rational Filter For Inteφretation of Spectrometric Data", to be published in IEEE Trans. Instrum. & Meas. applied to {*„ j .

The vector p_R , = p_R , p_R , ...f ofthe parameters ofthe operator R ^r contains the coefficients ofthe rational filter, as well as the discrete values ofthe function g_sx( ) as described in Ben Slima M., Szczecinski L., Massicotte D., Morawski R. Z., Barwicz A.: "Algorithmic Specification of a Specialized Processor for Spectrometric Applictions", Proc. IEEE Instrum. & Meas. Technology Conf. (Ottawa, Canada, May 19- 21, 1997), pp. 90-95 and in Ben Slima M., Morawski R. Z., Barwicz A.: "Kalman-filter- based Algorithms of Spectrophotometric Data Correction - Part II: Use of Splines for Approximation of Spectra", IEEE Trans. Instrum. & Meas., Vol. 46, No. 3, June 1997, pp. 685-689.

The following operations are accomplished during the initial calibration: • identification ofthe function g iλ) , using an iterative algorithm as described by Jansson;

• estimation ofthe parameter σ^ of the function g^iλ) on the basis of a real spectrum x^cal(λ) using an optimization algorithm; and • estimation of coefficients ofthe rational filter using an optimization algorithm.

The following operations are accomplished during the continual re-calibration:

• updating coefficients ofthe rational filter using an optimization algorithm; and

• estimating coefficients ofthe adaptive rational filter using an optimization method and a shape of spectrometric data.

The exemplary results of spectrophotometric data inteφretation, obtained by means ofthe described preferred embodiment are shown in Fig. 6a, Fig. 6b and Fig. 7.

The following errors are defined to evaluate the exactitude ofthe procedure AI- RC: ^■ relative root-mean-square error (RRMSE)

normalized maximum error (NME)

The errors of estimation of positions and heights of peaks, are: d = 0.0015 , d = 0.0024 ; d° = 0.20 , δ_∞ ^a = 0.23 ;

Example 2

An embodiment of AI-RC was designed for a measuring system which is represented by the integrated MM microOSA (Optical Spectrum Analyser) as shown in Fig. 8. The analyser is shown used for optical channel monitoring in DWDM networks in Fig. 9. The following DWDM system parameters have been selected both for calibration and for acquisition of test data:

• number of optical channels : 81 ;

• wavelength range: λ_mm = \530.77nm, Λ_max = 1562.68«/κ ; • channel spacing : Δλ = 0.4 nm (50 GHz);

• laser sources with a known FWHM = 0.2 nm, where FWHM denotes Full Width at Half Maximum

The test data were acquired for a telecommunication bandwidth; its real spectrum x(λ) is shown in Fig. 10a. The parameters of this spectrum are the following: • the vector of the positions of peaks (carrier frequencies): /_k = 1530.28 + (k-1) Δλ + d/ (k=l,..., 81), where d _k are realizations of identical independent random variables uniformly distributed in the interval [-0.05 nm;0.05 nm], i.e. dl_k oc U (-0.05, 0.05) ;

• the vector a ofthe heights of peaks, that are assumed to be realizations of identical independent random variables uniformly distributed in the interval [0.01 ; 1], i.e. α_k ∞ U (0.01, 1.0) .

The adaptation ofthe MM μOSA for DWDM applications means that the model ofthe data takes on the form:

where g_χy{λ) is the optical response ofthe spectrometric transducer,

81 s(λ;l,a) = α_kδ{λ -l_k ) is the so-called idealized spectrum of input light, and {η_n } is a k=\ sequence modeling the random errors in the data and the amplified spontaneous emission (ASE) noise caused by the laser sources. This is an adequate model for the DWDM measurements if the function g_sx{λ) is an approximation ofthe shape ofthe laser light emission spectrum.

The α priori information about emission light of a laser is used for reconstruction ofthe idealized light spectrum s{λ;\,a). The continual re-calibration sub-procedure allows the measurement ofthe spectrum of laser emission in Is time intervals to continually update parameters p_R of reconstruction sub-procedure.

The chosen operator of projection, mapping the idealized spectrum s(λ;l,a) into the space ofthe data:

has been defined by the following operations:

y_n = y(λ_n) for n = \,...,N

The function g_™( ) has been assumed to have the form ofthe Gauss function:

Consequently, the vector ofthe parameters p_e of the operator G contains the discrete values of g_sx{λ) , shown in Fig. 11 and the parameter σ , equal to the spectral bandwidth of the MM μOS A.

The chosen operator of reconstruction, transforming the data |J _nj into an estimate s(λ) of s(λ;\,a), :

is defined in the following way: • a discrete estimate { x_n ) of x(λ) is found by means ofthe rational filter applied to

• an estimate s(λ) of s(Λ;l,a) is computed by means ofthe spline-based Kalman filter applied to \x„} . The vector p_R = \ p_R , p_{k 2} ••• ofthe parameters ofthe operator R contains the coefficients ofthe rational filter, as well as the discrete values ofthe function g_sx(λ) and a regularization parameters required by the spline-based Kalman filter as described in in Ben Slima M., Morawski R. Z., Barwicz A.: "Kalman-filter-based Algorithms of Spectrophotometric Data Correction - Part II: Use of Splines for Approximation of Spectra", IEEE Trans. Instrum. & Meas., Vol. 46, No. 3, June 1997, pp. 685-689.

The following operations are accomplished during the initial calibration: • estimation ofthe coefficients ofthe rational filter using an optimization algorithm; and, • estimation ofthe regularization parameter ofthe spline-based Kalman filter, using an optimization algorithm.

The update of coefficients ofthe rational filter is achieved during the continual re- calibration on the basis ofthe measured light emission spectrum of a laser.

Results of adaptive reconstruction and inteφretation of telecommunication data obtained according to the present exemplary embodiment are shown in Fig. 12.

The following errors are defined to evaluate the exactitude ofthe present exemplary method: ^■ relative root-mean-square error (RRMSE)

^■ maximum absolute error (MAE)

The errors of estimation of positions and heights of peaks, are: ε, =2Λ0^'3 % , Δ_; =0.05 nm ; ε_a =\3% , Δ, =0.5 dBm ; 4. Applicability and scope ofthe invention

The proposed procedure AI-RC may be applied in various light spectrum measuring instruments and systems, especially in miniaturized integrated spectrometers for in situ applications such as environmental probes, optical spectrum analyzers and optical performance monitors in telecommunication etc. The motivation for its application in a given measurement situation is founded on expected gains in performance and resolution such as the following examples:

• increased accuracy of spectrometric analyses accomplished by a given spectrometric system, resulting from correction of instrumental imperfections and reduced uncertainty of estimation of the parameters of measured spectra;

• possibility of automatic calibration of especially miniature integrated spectrometric devices during manufacturing process, taking into account their fabrication discrepancies;

• miniaturization and integration of light-spectrum-measurement related tools, resulting from the software compensation of their hardware imperfections and software replacement of some their functions; and

• reduced costs of spectrometric analyses, given the required accuracy, by replacing a high-resolution spectrometric devices with a functionally equivalent but low-resolution instrument.

As is evident to those of skill in the art, a known drawback to miniaturisation of spectral devices is that manufacturing at very small scales and light propagation through devices at those scales often highlights even small imperfections in the manufacturing processes and magnifies errors in manufacture. As such, a method of calibrating each device to correct for said imperfections would be extremely advantageous in some situations.

The exemplary embodiment ofthe invention, presented in section 3 is not intended to limit the applicability of AI-RC to absoφtion spectrophotometry and the monitoring of optical telecommunication channels. Neither is it intended to limit the variety of algorithms that may be used to embody the operations the procedure is composed of. On the contrary, the invention is intended to cover alternatives, modifications and. Some practical options for methods embodying the operations of AI- RC are briefly characterized.

4.1. Optional forms ofthe operators of projection and reconstruction

The following mathematical models ofthe spectrometric data may be used for defining the operator G : a) the stationary linear model:

b) the non-stationary linear model:

c) the non-linear model, e.g.: y(λ) = _∞°g{λ,λ') F_s[_S{λ';\,a)] dλ'

where g{λ) and g{λ,λ') are the apparatus functions ofthe spectrometric apparatus;^ and F_y are non-linear functions.

The corresponding operators G may have the following forms: a) the operator corresponding to the stationary linear model:

where _{p& y} = _g[^±^ ^ ;

b) the operator corresponding to the non-stationary linear model:

where p_{& μ v} = g[ λ_n, ^v+λ^ ^vj Aλ ;

c) the operator corresponding to the exemplary non-linear models:

where p_{G p,v} = Aλ .

4.2. Optional methods of signal reconstruction and inteφretation

The following methods of signal reconstruction - deconvolution or generalized deconvolution - may be used for defining the operator R : a) the original domain, numerical differentiation-based method defined in Morawski R. Z., Sokol owski P.: "Application of Numerical Differentiation for Measurand Reconstruction", Proc. 7th IMEKO-TC4 Int. Symp. Modern Electrical & Magnetic Measurements (Prague, Sept. 13-14, 1995), pp. 230-234; b) the iterative methods of Jansson and Gold; c) the spectrum-domain, Tikhonov-regularization-based method; d) the cepstrum-domain, Tikhonov-regularization-based method defined in Kalinowska A., Morawski R. Z., Lubianka T.: "Incoφora- tion ofthe Positivity Constraint into a Cepstral Method of Measurand Reconstruction", Proc. Xlll-th IMEKO World Congress, (Torino, Italy, Sept. 5-9, 1994), pp. 429-434; e) the original-domain, Tikhonov-regularization-based method with the positivity constraint imposed on the solution; f) the Kalman-filter-based method with with the positivity constraint imposed on the solution; g) the Kalman-filter-based method with spline-approximation ofthe solution; h) the adjoint-operator method; i) the entropy-based variational method; j) the Volterra-series-based methods; k) the rational-filter-based method; and,

1) the adaptive-rational-filter-based method. Moreover, many other methods developed in the domain of chemometrics as described, for example, in Brown S. D., Bear Jr. R. S., Blank T. B.: "Chemometrics", Anal. Chem., Vol. 64, No. 12, 1992, pp.22R-49R, and Brown S. D., Sum S. T., Despagne F.: "Chemometrics", Anal. Chem. Vol. 68, No. 12, 1995, pp. 21R-61R; telecommunications as described, for example, in Abreu E., Mitra S. K., Marchesani R.: "Non-minimum Phase Channel Equalization Using Non-causal Filters", IEEE Trans. Signal Processing, Vol. 45, No. 1, Jan. 1997, pp. 1-13 or seismology as described, for example, in Berkhout A. J.: Seismic Migration, Elsevier 1985 and image processing as described, for example, in numerous publications such as Gonsalves R. A., Nisenson P.:"HST Image Processing: An Overview of Algorithms for Image Restoration", Proc. SPIE, Vol. 1567, 1991, pp. 294-307 and Zerwakis M. E., Kwon T. M., "On the Application of Robust Functionals in Regularized Image Restoration ",Proc IEEE Int. Conf. Acoustics, Speech & Signal Process. - ICASSP'93 (Minneapolis MN, USA, April 27-30, 1993),Vol. 5, pp. 289-292 may be applied for this puφose.

The following methods are useful for determining the regularization parameters ofthe operator R : a) the discrepancy principle with a pre-estimate ofthe variance of measurement errors in the data as described in Tikhonov A. N., Goncharsky A. V., Stepanov V. V., Yagola A. G.: Numerical Methods for the Solution of Ill-Posed Problems, Kluwer 1995; b) the method ofthe L-curve as described in Hansen P. C, O'Leary D. P.: "The Use of the L-curve in the Regularization ofthe Discrete Ill-posed Problems", SIAMJ. Sci. Comput.,Vol. 14, No. 6, 1993, pp. 1487-1503; c) the method of additional set of calibration data as described in Szczecin ski L., Morawski R. Z., Barwicz A.: "Numerical Correction of Spectrometric Data Using a

Bilinear Operator of Measurand Reconstruction", Proc. IEEE Instrum. & Meas. Technol. Conf. - IMTC'95 (Boston, MA, April 24-26, 1995), pp. 488-491.

4.3. Methods of calibration

It is often desirable to assume that the isolated peak v_s(λ,l) has the following forms: a) the Dirac distribution δ(λ) for all values of / ; b) a triangle whose width is constant or varying versus / ; c) a rectangle whose width is constant or varying versus / ; d) a Gauss function whose width is constant or varying versus / ; and e) a Lorenz function whose width is constant or varying versus / .

The following methods are useful in estimation ofthe apparatus function g(λ) : a) smoothing approximation applied directly to the data { >„ j if the isolated peak v_s(λ,l) is assumed to have the form the Dirac distribution δ(λ) ; b) deconvolution of the data

,a^cαl ) ; and c) subsequent use of deconvolution and smoothing approximation.

The following methods may be used for determining other parameters ofthe operator R : a) a direct transformation ofthe parameters ofthe operator G ; b) the minimization of any norm ofthe solution |p_R || under constraints imposed on

another norm ofthe disrepancy ( l;r^α , a^cαl) -R y_n ^cαl };p_R J ; and

c) the minimization of any norm ofthe disrepancy Λ;r^α ,a^co/)- PR under

constraints imposed on another norm ofthe solution |p_R || .

4.4 Methods for estimation ofthe parameters of peaks

The following methods are useful for estimation ofthe magnitudes a ofthe peaks, given the estimates 1 of their positions 1 :

a = arg_a inf| |{ ?„} -e [s(/l;ϊ,a);p_s I a eA

or

a = arg_a inf s(λ)- s(λ;l,a) a eA with A - being a set of feasible solutions; options: q=2 and A c R ^k ; q=∞ and A c R* ; q=2 and A c R * ; q=∞ and A c R * . Some examples of algorithmic solutions are given in Morawski R. Z., Miς kina A., Barwicz A.: "Combined Use of Tikhonov Deconvolution and Curve Fitting for Spectrogram Inteφretation", Instrum. Science & Technology, Vol. 24, No. 3, August 1996, pp. 155-167 and Morawski R. Z., Mi? kina A., Barwicz A.: "The Use of Deconvolution and Iterative Optimization for Spectrogram Inteφretation", IEEE Trans. Instrum. & Meas., Vol. 47, No. 5, Oct.1997. A particularly effective solution ofthe above optimization problem may be based on a non-stationary Kalman filter or an adaptive LMS algorithm is given in Ben Slima M., Morawski R. Z., Barwicz A.: "Kalman-filter-based Algorithms of Spectrophotometric Data Correction - Part II: Use of Splines for Approximation of Spectra", IEEE Trans. Instrum. & Meas., Vol. 46, No. 3, June 1997, pp. 685-689.

Optionally, the methods for estimation ofthe magnitudes a are used for iterative correction ofthe estimates ofthe magnitudes a and positions 1 ofthe peaks interchangeably with the following methods:

Ϊ = argι inf| |{5/_Il} -S [s(λ;I,a);p₆ ]|| | l l_

or ϊ = arg, in μ)-5( ;l,a)[ 1 e L J

with L being a set of feasible solutions; options: q=2 and L c R * ; q=∞ and L c R * ; q=2 and L c R * ; q=∞ and L c R * .

4.5. Methods for pre-processing ofthe spectrometric data

The following methods are useful for normalization ofthe data: a) the linear or nonlinear transformation ofthe λ -axis, aimed at diminishing the non- stationarity effects in the data; b) the linear or nonlinear transformation of the y-axis, aimed at diminishing the non- linearity effects in the data; and c) the linear or nonlinear transformation ofthe A-axis and^-axis, aimed at diminishing the non-stationarity and non-linearity effects in the data.

The following methods are useful for smoothing the data: a) the linear, FIR-type or IIR-type, filtering; b) the median filtering; c) the smoothing approximation by cubic splines; and d) the deconvolution with respect to an identity operator.

Standard methods for baseline correction useful with the present invention are described in Brame E. G., Grasselli J., Infrared and Raman Spectroscopy, Marcel Dekker 1976 and in ASTM 1987. (American Society for Testing and Materials): Annual Book of ASTM Standards 1987.

Numerous other embodiments may be envisaged without departing from the spirit or scope ofthe invention.

Claims

What is claimed is:

1. A method of adaptive inteφretation of spectrometric data combined with re-calibration for use with a spectrometric apparatus including transducer elements and comprising the steps of: a) initially calibrating the spectrometric apparatus providing spectrometric data; b) at intervals automatically re-calibrating the apparatus absent physical tuning of individual transducer elements in isolation by the steps of:

• providing light from a reference light source to the apparatus,

• acquiring auxiliary reference data relating to the detected light from the reference light source, and

• computing updated results of calibration based on the auxiliary reference data; c) sensing a spectrum of light {x_n} with the spectrometric apparatus to provide spectrometric data {y_n} representative ofthe spectrum of light {x_n}; d) adaptively determining an estimate {s_n} of an idealized spectrum {s_n} related to the spectrum of light {x_n } based on the spectrometric data {y_n } and updated results of calibration; and, e) adaptively estimating positions /, , /₂, ... and magnitudes a_], a₂, ... of peaks composing the spectrum of light {x_n } on the basis ofthe estimate {s_n} .

2. The method of claim 1 wherein the step (b) of recalibrating involves updating the results of a previous calibration absent physical modification and adjustment ofthe sensor elements.

3. The method of claim 1 wherein the step (a) comprises the steps of: al) selecting a form of peaks within the idealized spectrum {s_n} ; a2) choosing the form ofthe operator R for reconstruction ofthe idealized spectrum: {f„} = R [{y_n}; p_R ], where p_R is a vector ofthe parameters, the form selected in dependence upon the selected form ofthe peaks; a3) choosing rules for adaptation ofthe parameters p_R to selected features F_X, F₂, ... of the data y_π} on the basis ofthe library of parameters ψg' | j = 1,...,J) forming part of the calibration data determined during the step of initially calibrating and for updating during the step of re-calibrating; a4) choosing a reference light signal for the step of initially calibrating: {x ^o/(y^')} and related {y ( )} for j=\,...,J ; a5) acquiring data | ^α ' j representative of

; a6) pre-processing ofthe data |%^α/); a7) determination ofthe vectors of parameters p > - on the basis ofthe triplets:

a8) approximating a relationship between the parameters p_R and the features F_x , F₂ , ... controlling adaptation.

4. The method of claim 3 wherein the step (a6) comprises the step of: acquiring α priori information relating to a pre-estimate ofthe variance of errors in the calibration data.

5. The method of claim 4 wherein the step (a6) comprises the steps of: performing elimination of outliers, performing baseline subtraction, performing smoothing, and normalizing the data.

6. The method of claim 3 wherein the step (al) comprises the step of: selecting a method of idealizing a spectrum indicative of a form of the peaks.

7. The method of claim 1 wherein the step (b) comprises the steps: bl) choosing a reference light signals for re-calibration:

and related

for y^'=l,...,J ; b2) acquiring data

; b3) pre-processing the acquired data ( ^ecα/ j; b4) updating ofthe vectors of parameters p_R ^y) on the basis of triplets: „^eca'{j)),

b5) updating the approximation ofthe relationship between the parameters p_R and the features F_X, F₂, ... controlling the adaptation.

8. The method of claim 7 wherein the step (b3) comprises the step of: acquiring α priori information relating to a pre-estimate ofthe variance of errors in the acquired data.

9. The method of claim 8 wherein the step (b3) comprises the steps of: performing elimination of outliers, performing baseline subtraction, performing smoothing, and normalizing the data.

10. The method of claim 1 wherein the step (d) comprises the steps of: dl) pre-processing of

| ; d2) computing an estimate ofthe idealized spectrum {s_n} related to the spectrum under study {x_n } on the basis ofthe data y„ j , by means ofthe operator R with the parameters p_R adapted, at each consecutive step of data processing, to the features F_x, F₂, ... - on the basis ofthe library of parameters, determined during initial calibration and updated during continual re-calibration.

11. The method of claim 10 wherein the step (dl) comprises the step of: acquiring a priori information relating to accurate estimation of peak parameters.

12. The method of claim 11 wherein the step of acquiring a priori information comprises the step of: acquiring a priori information relating to a pre-estimate ofthe variance of errors in the spectrometric data.

13. The method of claim 12 wherein the step (dl) comprises the steps of: performing elimination of outliers, performing baseline subtraction, performing smoothing, and normalizing the data.

14. The method of claim 1 wherein the step (e) comprises the steps of: el) pre-processing ofthe estimate {s_n} obtained in step (c); e2) computing estimates ofthe positions and the magnitudes ofthe peaks within the first spectrum {x_n}.

15. The method of claim 14 wherein the estimated positions and magnitudes relate to positions and magnitudes within an idealised first spectrum based on the acquired spectrum, the updated calibration data, and operators derived therefrom.

16. The method of claim 15 wherein the step of (el) comprises the step of: acquiring a priori information relating to a number of peaks within the spectrometric data and an estimate of their widths.

17. The method of claim 16 wherein the step (el) comprises the steps of: performing elimination of outliers, performing baseline subtraction, performing smoothing, and normalizing the data.

18. A spectrometric apparatus comprising: transducer elements for sensing a spectrum of light {x_n } with the spectrometric apparatus to provide spectrometric data y_n } representative ofthe spectrum of light {x_n} ; means for initially calibrating the spectrometric apparatus in response to a provided reference light to provide initial results of calibration; means for automatically re-calibrating the apparatus in response to a provided reference light absent physical tuning ofthe transducer elements; and, a processor for determining an estimate {s_n} of an idealized spectrum {s_n} related to the first spectrum of light {x_n} based on the spectrometric data y_n} and updated results of calibration and for estimating positions l_x, l₂, ... and magnitudes a_x, a₂, ... of peaks within the idealized spectrum {s_n} based on the estimate {?„} .

19. A spectrometric apparatus comprising: transducer elements for sensing a spectrum of light {x_n} with the spectrometric apparatus to provide spectrometric data y_n} representative ofthe spectrum of light {x_n}; and, means for initially calibrating the spectrometric apparatus in response to a provided reference light to provide initial results of calibration; characterised by: means for automatically re-calibrating the apparatus in response to a provided reference light absent physical tuning ofthe transducer elements wherein during calibration ofthe spectrometric apparatus determination of parameters for use in reconstructing an idealised spectrum from data relating to light provided to the apparatus is performed.