lmfit-general Mailing List for lmfit - least squares fitting

Hi

I would like to know if the lmfit might solve a problem of mine.

I have the equations

x' = Ax + By + C
y' = Dx + Ey + F

where I need to estimate coefficients A - F for a given data set (x, y) -> (x', y')

My knowledge in math could be better, hence the question.

Tanks in advance
/Niklas

_________________________________________________________________
Vi vet vem du passar ihop med! Klicka här för att få veta!
http://dejting.se.msn.com/channel/index.aspx?trackingid=1002952

Dear Scott,

like other fitting algorithms, Levenberg-Marquardt performs local optimisa=
tion.
Its ability to find a global optimum depends on the model. Generally, with=

increasing number of parameters it becomes more likely that the fit gets
trapped in a local optimum which may be far off the global optimum. Typica=
l
applications of lmfit have, say, one to five parameters. Fifteen is really=
 a lot.
There are situations where I have used 10 or 12 parameters - but typical
these cases involved kind of a Taylor or Fourier series so that each resid=
ue
depended linearly on the individual parameters. In less favorable cases I
would not expect that a 15-parameter model converges independently of
starting values. It is hardly possible to give more specific advice withou=
t
knowing more about your applications. Just some ideas:
- perhaps least-squares fitting isn't the adequate approach =3F
- perhaps your model is over-ambitious =3F
- perhaps you can start with a rough 5-parameter fit, then refine your
  model successively.

Good luck, Joachim

> Hi Joachim,
>=20
> I've been using lmfit for a while now & find it incredibly useful, thank=
s.
>=20
> I'm currently using it to fit a 15-parameter model & I notice that the
> solution I get is highly-dependant upon the initial parameter values.
> Even a small change in the value of 1 parameter can lead to a completely=

> different solution (with much larger or smaller residuals).
>=20
> I was just wondering if you had any general advice about how to
> choose initial parameter values=3F
>=20
> Scott. :)

=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Viren-Scan f=FCr Ihren PC! Jetzt f=FCr jeden. Sofort, online und kostenlos.
Gleich testen! http://www.pc-sicherheit.web.de/freescan/=3Fmc=3D022222

Hi Joachim,

I've been using lmfit for a while now & find it incredibly useful, thanks.

I'm currently using it to fit a 15-parameter model & I notice that the
solution I get is highly-dependant upon the initial parameter values.
Even a small change in the value of 1 parameter can lead to a completely
different solution (with much larger or smaller residuals).

I was just wondering if you had any general advice about how to
choose initial parameter values?

Scott. :)

Changes to the code seem okay;
the fit results also indicate that you got a basically correct program.

>From my own experience I can confirm that changing control parameters
usually does not help.

For debbuging, only a few suggestions:
- make sure array lengths are passed correctly (no confusion n vs n-1,
or in your case n/3)
- shift your data away from 0,0,0, then repeat the fit (lmfit kind of uses
a multiplicative scale for p; therefore the parameter value 0 is potentially
dangerous)
- try to fit self-made data before you address the real problem
- inspect your data graphically; perhaps lmfit is plain right

Good luck, Joachim

> The problem I had yesterday has been partially solved (thanks for your
> help). However one issue remains, and that is convergence tolerance.
> The computation converges clearly in the right direction, only to stop
> after 132. iteration with relatively huge error, which renders results
> unusable.
> 
> I tried setting parameters inside the control structure in lmmin.c to
> other values without much success.
> Here are results of my computation:
> 
> terminated after 132 evaluations
>  par:  -1.81519e-009 1.25936e-005   -0.0574769      6.00498 => norm:
>    16.8893
> 
> which is only partially correct, the correct ones should be closer to
> 0, 0, 0 and 6.
> 
> The lines changed in lm_test.c are:
> 
> double my_fit_function( double *coord, double *p )
> {
>        double x = coord[0] - p[0];
>        double y = coord[1] - p[1];
>        double z = coord[2] - p[2];
>        double f = sqrt( x*x + y*y + z*z ) - p[3];
> 
>        return ( f*f );
> }
> 
> and in lm_eval.c:
> 
> double a[3];
> for (i=0; i<m_dat; i++)
> {
>        a[0] = mydata->user_x[i];
>        a[1] = mydata->user_y[i];
>        a[2] = mydata->user_z[i];
> 
>        fvec[i] = mydata->user_func( a, par );
> }
> 
> Almir

_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=100071&distributionid=000000000066

The problem I had yesterday has been partially solved (thanks for your
help). However one issue remains, and that is convergence tolerance.
The computation converges clearly in the right direction, only to stop
after 132. iteration with relatively huge error, which renders results
unusable.

I tried setting parameters inside the control structure in lmmin.c to
other values without much success.
Here are results of my computation:

terminated after 132 evaluations
 par:  -1.81519e-009 1.25936e-005   -0.0574769      6.00498 => norm:
   16.8893

which is only partially correct, the correct ones should be closer to
0, 0, 0 and 6.

The lines changed in lm_test.c are:

double my_fit_function( double *coord, double *p )
{
       double x = coord[0] - p[0];
       double y = coord[1] - p[1];
       double z = coord[2] - p[2];
       double f = sqrt( x*x + y*y + z*z ) - p[3];

       return ( f*f );
}

and in lm_eval.c:

double a[3];
for (i=0; i<m_dat; i++)
{
       a[0] = mydata->user_x[i];
       a[1] = mydata->user_y[i];
       a[2] = mydata->user_z[i];

       fvec[i] = mydata->user_func( a, par );
}

Almir

Hi Almir:

> My question is: is it possible to minimize multivariable functions
> with levmar, and if so, what is the way to do this=3F
>=20
> For example my objective function for fitting a sphere to data is functi=
on:
>=20
> f =3D sqrt( (x-x0)*(x-x0) + (y-y0)*(y-y0) + (z-z0)*(z-z0) ) - r*r
>=20
> where x0, y0, and z0 correspond to the center of sphere and r to it's
> radius, all to be found.
> I understand x0, y0, z0, r are contained in p array (initial guess),
> but how to pass x, y and z which correspond to coordinates of current
> point=3F

You understand correctly how the parameter array p is used.
The remaining problem is mainly one of terminology:
you want to fit a curve f(x,y,z|p) to some data d(x,y,z).
Beware of confusion: in my sample application, the data are called y(t).
I used y, because I only thought of univariate functions; and I used
t instead of x because some people use x for what I call p.

Technically, the solution to your problem is simple:
in lm=5Feval.h, replace the array user=5Ft by three arrays, representing
your x,y, and z values, and user=5Ft=5Fpoint by user=5Ft=5Ftriple[3], and
adapt the few code lines where theses entities are used.

Good luck, Joachim
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Viren-Scan f=FCr Ihren PC! Jetzt f=FCr jeden. Sofort, online und kostenlos.
Gleich testen! http://www.pc-sicherheit.web.de/freescan/=3Fmc=3D022222

Hello,

I am writing a program for least squares fitting of objects to sets of
measured points, as used in computational metrology. For this purpose
I intend to use lmfit. However I am new to the field of least squates
fitting and Levenbereg-Marquardt algorithm, so I hope you will be able
to help me.

My question is: is it possible to minimize multivariable functions
with levmar, and if so, what is the way to do this?

For example my objective function for fitting a sphere to data is function:

f = sqrt( (x-x0)*(x-x0) + (y-y0)*(y-y0) + (z-z0)*(z-z0) ) - r*r

where x0, y0, and z0 correspond to the center of sphere and r to it's
radius, all to be found.
I understand x0, y0, z0, r are contained in p array (initial guess),
but how to pass x, y and z which correspond to coordinates of current
point?

Regards
Almir

> what would be considered a "good" starting value=20
I have no other criterion to offer than common sense & experience: plot yo=
ur
data and your starting curve; if their qualitative features (extreme point=
s,
bending points, asymptotes - just the =BBKurvendiskussion=AB stuff) agree, the=
re
is a good chance that the curve will converge towards the data points

> and how could such a value be generated in a generalized manner
for instance, if you are fitting a Gaussian distribution: determine the
center-of-gravity, the width and the height of the distribution from weigh=
ted
sums over your data points.

> I tried fabs(p[1])
good idea

> and sqr(p[1])
even a better idea, provided you really mean sqr and not sqrt
(but why do you use the ugly pow(..,2.0) if you have a sqr function=3F)

> FitFunction4( double t, double* p)
> {
>=20
> double deltax =3D (t-p[0])*(t-p[0]);
>=20
> double dFWHM =3D dHalbwertsbreite;
>=20
> double omega =3D pow(dFWHM/TWOTIMESROOTLN2 , 2.0);
>=20
> return ( p[1]* exp(-deltax/omega) + p[2] ); // Ansatz =FCber Position Gaus=
s
>=20
> }

why don't you leave your Halbwertsbreite free -> yet another fit parameter=


Hope this helps. Otherwise: you may also hire me for consultancy.
Two hours discussion with a data analysis expert may save you
days of deadlock.

Kind regards, Joachim
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
SMS schreiben mit WEB.DE FreeMail - einfach, schnell und
kostenguenstig. Jetzt gleich testen! http://f.web.de/=3Fmc=3D021192

Dear Joachim,

what would be considered a "good" starting value  - and how could such a
value be generated in a generalized manner ?
At the moment I generate three such values which lie more or less around the
required optimum for a gauss function but the
implementation sometimes seems to generate negative amplitudes for a
function of the form specified below. I plotted the initial
points as well as the fit results and found that the deviation was way too
large - it couldn't possiby have approximated the required
curve in the least squares sense. Is there any way of restricting the search
to positiove amplitudes ? I tried fabs(p[1]) and sqr(p[1])
but that didn't seem to work for my case.


FitFunction4( double t, double* p)
{

double deltax = (t-p[0])*(t-p[0]);

double dFWHM = dHalbwertsbreite;

double omega = pow(dFWHM/TWOTIMESROOTLN2 , 2.0);

return ( p[1]* exp(-deltax/omega) + p[2] ); // Ansatz über Position Gauss

}

Any suggestions ?


Regards,

Nandini



----- Original Message -----
From: "Joachim Wuttke" <wu...@we...>
To: <lmf...@li...>; "Nandini Hengen"
<nh...@he...>
Sent: Thursday, May 18, 2006 8:57 AM
Subject: Re: [Lmfit-general] Fitting Periodic Functions using
Levenberg-Marquardt's method


> Dear Nandini,
>
> there is no reason why a fit function ought to be periodic.
>
> Try to track down your problem to one of the following:
> - bad starting values -> did you visualise the initial function before
starting to fit?
> - overflow or underflow in exp
>
> Good luck, Joachim
> _______________________________________________________________
> SMS schreiben mit WEB.DE FreeMail - einfach, schnell und
> kostenguenstig. Jetzt gleich testen! http://f.web.de/?mc=021192
>

Dear Nandini,

there is no reason why a fit function ought to be periodic. 

Try to track down your problem to one of the following:
- bad starting values -> did you visualise the initial function before starting to fit?
- overflow or underflow in exp

Good luck, Joachim
_______________________________________________________________
SMS schreiben mit WEB.DE FreeMail - einfach, schnell und
kostenguenstig. Jetzt gleich testen! http://f.web.de/?mc=021192

Hi,
   the implementation seems to work for nonlinear functions of the form =
A*sine(B) + C, cosine etc. but fails for functions like exp etc. which =
are nonperiodic. Do the relevant functions have to be specified as =
periodic (repetitive) form to ensure convergence ?

Any help is appreciated.

Regards,
Nandini

> Hi Joachim,
> I was wondering whether this code support non linear least squares
> fitting
> 
> Or not.
> 
> Thanks,
> 
> Sri

the fit function must not be linear - Joachim

______________________________________________________________
Verschicken Sie romantische, coole und witzige Bilder per SMS!
Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193

Hi Joachim,

=20

I was wondering whether this code support non linear least squares
fitting

Or not.

=20

Thanks,

Sri

Hi Giorgio,

you should not modify lm_dif.

You solve your problem on a higher level, outside lmfit, and pass
only the parameters you want to fit to lm_minimize.

Good luck, Joachim

Gio...@ph... schrieb am 08.03.06 18:40:19:
> 
> Hi all,
>  I want to modify lmfit so I can tell from a list of parameter which I want to
> fit, like Numerical Recipes does. I have a simulation program of my
> experimental data and I want to be able to tell which parameters should be
> fitted and which ones should be kept fix. 
>  Is there any other code available for that?
>  For doing this, Is there any other way than modifying lm_lmdif? 
>  If I modify lm_lmdif, should I modify others functions? 
> 
> Thanks,
> 
> Giorgio
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> Lmfit-general mailing list
> Lmf...@li...
> https://lists.sourceforge.net/lists/listinfo/lmfit-general


______________________________________________________________
Verschicken Sie romantische, coole und witzige Bilder per SMS!
Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193

Hi all,
 I want to modify lmfit so I can tell from a list of parameter which I want to
fit, like Numerical Recipes does. I have a simulation program of my
experimental data and I want to be able to tell which parameters should be
fitted and which ones should be kept fix. 
 Is there any other code available for that?
 For doing this, Is there any other way than modifying lm_lmdif? 
 If I modify lm_lmdif, should I modify others functions? 

Thanks,

Giorgio

Hi Joachim,

> Please have a look into lm_eval.c:
> 
>     for (i=0; i<m_dat; i++)
>             fvec[i] = mydata->user_y[i] 
>                 - mydata->user_func( mydata->user_t[i], par);
> 
> The one input the fit routine really needs is the vector fvec = your_data - your_theory.

Thank-you very much. Your reply halped me work out what I need to do
& now it's working just great.

Thanks for writing such a useful program!

SCoTT. :)

Hi Scott,

   there is no other documentation than the comments in the sources.

Please have a look into lm_eval.c:

    for (i=0; i<m_dat; i++)
            fvec[i] = mydata->user_y[i] 
                - mydata->user_func( mydata->user_t[i], par);

The one input the fit routine really needs is the vector fvec = your_data - your_theory.
If your data have a more complicated structure than assumed in the above code
fragment, then you may modify the struct lm_data_type.

In your question you say nothing about how your equations are related to each other.
Do you want to simultaneously fit them? With one common set of coefficients A,B,C ?
Then, I guess, you need to replace user_t by two vectors user_t_x and user_t_y.

Good luck, Joachim

> Hi all,
> 
> Apologies in advance for what may sound a little cryptic. I'm not
> familiar with the correct nomenclature but I know exactly what I
> want to do.
> 
> I've been using lmfit for simple fits & have found it incredibly
> useful. Now I'm trying to do more complex fitting but I'm not sure
> if lmfit is capable of providing the functionality I need.
> 
> In short, I'd like to know how to use lmfit to fit data to the
> following form:
> 
> x' = A + Bx + Cy
> 
> Obviously A, B & C are the coefficients I'd like to fit, but I have
> 2 variables - x & y.
> 
> I have a number of equations I want to fit to & many of them have
> 2 variables.
> 
> Any info/pointers would be muchly appreciated.
> 
> SCoTT. :)
> PS Is there any documentation for lmfit?
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Lmfit-general mailing list
> Lmf...@li...
> https://lists.sourceforge.net/lists/listinfo/lmfit-general


______________________________________________________________
Verschicken Sie romantische, coole und witzige Bilder per SMS!
Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193

Hi all,

Apologies in advance for what may sound a little cryptic. I'm not
familiar with the correct nomenclature but I know exactly what I
want to do.

I've been using lmfit for simple fits & have found it incredibly
useful. Now I'm trying to do more complex fitting but I'm not sure
if lmfit is capable of providing the functionality I need.

In short, I'd like to know how to use lmfit to fit data to the
following form:

x' = A + Bx + Cy

Obviously A, B & C are the coefficients I'd like to fit, but I have
2 variables - x & y.

I have a number of equations I want to fit to & many of them have
2 variables.

Any info/pointers would be muchly appreciated.

SCoTT. :)
PS Is there any documentation for lmfit?

Thank you, Olivier.
You are absolutely right.
Your checking the source is highly welcome.
I just released version 2.1 to fix the bug.
Joachim

"oliwif" <ol...@us...> schrieb am 23.05.05 17:16:39:
> 
> Message body follows:
> 
> Hi
> 
> I am not sure but I think there could be a typo in lmmin.c on 
> line 987 :
>          if (kmax != j) goto pivot_ok;
> 
> I think it should be :
>         if (kmax == j) goto pivot_ok;
> 
> Olivier



______________________________________________________________
Verschicken Sie romantische, coole und witzige Bilder per SMS!
Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193

Dear subscribers,

   an improved version of lmfit has been released.
For better readablility, several files and subroutines
have been renamed; subroutine parameter lists have
also changed. To indicate the incompatibility of the
user interface, the major release number has been
incremented. Nevertheless, the overall design of the
package has remained unchanged.

   Fit progress can now be monitored by a routine
that is passed to lm_minimize and down to lm_lmdif
as a function parameter in the same way as the
evaluation routine. A default implementation is
provided by lm_eval.c (former lmuse.c).

   The internal struct status could be removed.

   While lmfit_1.0 was work in progress, I have currently
no plans for further modifications.

   With best wishes for 2005 - Joachim
__________________________________________________________
Mit WEB.DE FreePhone mit hoechster Qualitaet ab 0 Ct./Min.
weltweit telefonieren! http://freephone.web.de/?mc=021201