ecell-devel Mailing List for E-Cell System

Hi,


I've finally finished my thesis defense, and am now quite motivated
about a number of things, including the development of E-Cell Version
3.1.101 :)


Here goes the list of items that I currently think of as release
criteria of 3.1.101:

Bugs:
882127 DAEStepper doesn't support Algebraic processes  kaizu
864898 FixedODE1Stepper should restore stepsize after interruption kaizu
878912 SBML Importer can't convert "Rule" and "Events" etc..  tishida


Below is the list of features those should be included in the version:

Feature requests:
755136 Stop simulation at an exact point in time* shafi
848785 DMInfo frontend interface shafi


We have decided to adopt release criteria-driven release schemd
abandoning schedule-driven one.   However, I roughly estimate 
the release date around the end of February.



FYI, currently I can see the following items on SF marked to be included
in version 3.2:

Bug:
861065 empy escape@ is not ignored inside a comment line tsakurada

FR:
860231 ODEStepper kaizu
715861 faster EML loading bgabor
748987 upgrade to empy-3.1 tsakurada

-sha

> what do you mean by step counting logging interval?

I mean pushing a DataPoint to the storage once in N calls of
log() method of a Logger.     I personally think this was one of
those nasty features of ecell1, and dropped from the specification of
ecell3.   I mean the procedure is not well-grounded in terms of
numerical analysis, although it is somewhat commonly used in statistic
data processing.   Actually, it is ok if the trajectory is very smooth
OR it is in a steady state OR it has a steady stochastic fluctuation.
Otherwise it can be problematic.   Since in general cases it is
impossible for users to predict what kind of behavior the trajectory
shows before running simulation and logging the data, I think this
feature should be non-default, unless the user explicitly orders so.

> I will add the interval setting feature to osogo soon.

Thanks.


-sha

> > > >  > Other way to make files smaller is reducing precision from double to
> > > single
> > >  > float.
> > >
> > > I see. How can I do this?
> > >
> > We have to change the code. Shafi do you have any comments on this?
> >
> >I disagree with the single-precision solution.   We need to support
> >more types (float and double), and we get at most 2 fold disk space.
> >And, precision is the most important thing for scientific software.
> >I also disagree with the use of stream compression libraries such 
> >as zlib.   
> >
> >I think this problem should be solved by some smart logging policy and
> >making it easily accessible to users.    Adding support for
> >the step-counting logging interval might constitute the solution.
> >Also, support of that functionality by frontend components is
> >very important.  The system must provide easy ways of setting and
> >changing the logging policy and interval.
> >For example, is the logging interval settable from
> >osogo's logger window?   
> >
> >-sha
> >
> >
> >
> >-------------------------------------------------------
> >The SF.Net email is sponsored by EclipseCon 2004
> >Premiere Conference on Open Tools Development and Integration
> >See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> ><a Target='_new' Href='http://talk21.btopenworld.com/redirect.html?http://www.eclipsecon.org/osdn'>http://www.eclipsecon.org/osdn</a>
> >_______________________________________________
> >Ecell-devel mailing list
> >Ece...@li...
> >https://lists.sourceforge.net/lists/listinfo/ecell-devel
> 
> 
> 
> --------------------
> talk21 your FREE portable and private address on the net at http://www.talk21.com
> 
-- 
Kouichi Takahashi,
Laboratory for Bioinformatics,
Institute for Advanced Biosciences,
Keio University, Japan.
http://www.sfc.keio.ac.jp/~shafi

Hi

what=20do=20you=20mean=20by=20step=20counting=20logging=20interval=3F
I=20will=20add=20the=20interval=20setting=20feature=20to=20osogo=20soon.
Gabor


>=20>=20>=20=20>=20Other=20way=20to=20make=20files=20smaller=20is=20reducing=20precision=20from=20double=20to
>=20>=20single
>=20>=20=20>=20float.
>=20>
>=20>=20I=20see.=20How=20can=20I=20do=20this=3F
>=20>
>=20We=20have=20to=20change=20the=20code.=20Shafi=20do=20you=20have=20any=20comments=20on=20this=3F
>
>I=20disagree=20with=20the=20single-precision=20solution.=20=20=20We=20need=20to=20support
>more=20types=20(float=20and=20double),=20and=20we=20get=20at=20most=202=20fold=20disk=20space.
>And,=20precision=20is=20the=20most=20important=20thing=20for=20scientific=20software.
>I=20also=20disagree=20with=20the=20use=20of=20stream=20compression=20libraries=20such=20
>as=20zlib.=20=20=20
>
>I=20think=20this=20problem=20should=20be=20solved=20by=20some=20smart=20logging=20policy=20and
>making=20it=20easily=20accessible=20to=20users.=20=20=20=20Adding=20support=20for
>the=20step-counting=20logging=20interval=20might=20constitute=20the=20solution.
>Also,=20support=20of=20that=20functionality=20by=20frontend=20components=20is
>very=20important.=20=20The=20system=20must=20provide=20easy=20ways=20of=20setting=20and
>changing=20the=20logging=20policy=20and=20interval.
>For=20example,=20is=20the=20logging=20interval=20settable=20from
>osogo's=20logger=20window=3F=20=20=20
>
>-sha
>
>
>
>-------------------------------------------------------
>The=20SF.Net=20email=20is=20sponsored=20by=20EclipseCon=202004
>Premiere=20Conference=20on=20Open=20Tools=20Development=20and=20Integration
>See=20the=20breadth=20of=20Eclipse=20activity.=20February=203-5=20in=20Anaheim,=20CA.
><a=20Target=3D'_new'=20Href=3D'http://talk21.btopenworld.com/redirect.html=3Fhttp://www.eclipsecon.org/osdn'>http://www.eclipsecon.org/osdn</a>
>_______________________________________________
>Ecell-devel=20mailing=20list
>Ece...@li...
>https://lists.sourceforge.net/lists/listinfo/ecell-devel



--------------------
talk21=20your=20FREE=20portable=20and=20private=20address=20on=20the=20net=20at=20http://www.talk21.com

> >  > Other way to make files smaller is reducing precision from double to
> > single
> >  > float.
> >
> > I see. How can I do this?
> >
> We have to change the code. Shafi do you have any comments on this?

I disagree with the single-precision solution.   We need to support
more types (float and double), and we get at most 2 fold disk space.
And, precision is the most important thing for scientific software.
I also disagree with the use of stream compression libraries such 
as zlib.   

I think this problem should be solved by some smart logging policy and
making it easily accessible to users.    Adding support for
the step-counting logging interval might constitute the solution.
Also, support of that functionality by frontend components is
very important.  The system must provide easy ways of setting and
changing the logging policy and interval.
For example, is the logging interval settable from
osogo's logger window?   

-sha

Hi,

>  > Other way to make files smaller is reducing precision from double to
> single
>  > float.
>
> I see. How can I do this?
>
We have to change the code. Shafi do you have any comments on this?

>  > patch applied.
>  >
>  > yoyo, could you test it now?
>
> I've just tested it with the latest CVS version, but
> unfortunately, the vvector files weren't deleted. Hmm...

I have tested it by setting up quota for my user and it worked. I couldnt
test what happens when disk is physically full.
Can you mail me the error message you get when files remain?

Gabor




--------------------
talk21 your FREE portable and private address on the net at http://www.talk21.com

Hi. I'm very sorry for the late reply.

 > btw do you use LoggerStub.setMinimumInterval command?

No I didn't use the following command because
I really didn't need to have small logging intervals.
The main source of the problem was not small logging intervals,
but a large number of loggers (in some cases 100).
I temporarily solved this problem by dividing the original
.ess file into a few smaller files (with less loggers).

 > Other way to make files smaller is reducing precision from double to 
single
 > float.

I see. How can I do this?

 > patch applied.
 >
 > yoyo, could you test it now?

I've just tested it with the latest CVS version, but
unfortunately, the vvector files weren't deleted. Hmm...

I would like to address another possible problem related to vvectors.
If I recall correctly, Shafi told me the other day that the total
size of all the vvector files are independent of the step interval.
That is, they depend only on the logging time and logging interval.
However, I've had models that have filled up /tmp/ with vvector files
and those that haven't, depending on the size of the step interval.
Is this a bug?

Yohei

Hi,

I tried zipping a vvector file and I found that the file can be compressed
to 2/3 of its original size, so it is quite compact. However zipping files
runtime for 30% extra storage space may not be a good deal.
I think the question is whether you are happy with obtaining sparse data
 with longer logging intervaks), and just got bored of always adjusting
logger intervals, or dislike logging intgervals and you really want all the
data comming out.
In the first case maybe the system could guess the necessary logging
interval, or instead of logging intervals it could just log every nth
element.
Other way to make files smaller is reducing precision from double to single
float.
Gabor

----- Original Message -----
From: "Yohei Yamada" <yo...@sf...>
To: "Kouichi Takahashi" <sh...@sf...>; "Bereczki Gabor"
<gab...@ax...>
Cc: "Yohei Yamada" <yo...@sf...>
Sent: Friday, January 16, 2004 9:37 AM
Subject: Re: automatic deletion of vvector-* from /tmp/


> Thank you, shafi and gabor,
> for the correspondence.
>
> Another related comment that I would like to make from
> a modeler's perspective is that because the vvector
> files seem to overfill the /tmp/ directory, even for
> simulation runs of relatively normal scale, it would be
> helpful if the files were somehow made smaller. I've talked
> to a few other modelers that had trouble with estimation
> and/or analysis because of this problem.
>
> This problem can be temporarily avoided, at the modeling level,
> by creating less loggers, making the logging periods shorter,
> or making the logging intervals longer. However, I thought
> that there should be a more fundamental solution, one at
> the system level.
>
> Kouichi Takahashi wrote:
>
> >
> >
> >>The situation is that vvector files are always deleted after simulation
run
> >>ends even after the most serious segmentation errors but except when the
> >>errot happens in vvector itself. This is bulletproof because vvector
hooks
> >>atexit(), so whatever exception happens it can handle it. However if the
> >>error is in vvector ( like the ones Yoyo had mentioned ), then the
abort()
> >>is called, which is probably able to dodge atexit().
> >>
> >>
> >
> >Ok I now remember that.   That explains why I didn't hook it in
> >PyEcsSignalHandler.
> >
> >
> >
> >>But I thought it was intentional, so that the user be able to retain
vvector
> >>files after an assumedly serious file error had occured. That is why I
ask
> >>Shafi whether to retain this functionality or not.
> >>
> >>
> >
> >Our experience so far with ecell1 and 3 tells that the behavior to keep
> >vvector files when it crashes is not very useful as expected.
> >I think we can change it to always clean up the files.
> >
> >
> >
> >>If not, I suppose it can be easily worked around by calling exit(1)
instead
> >>of abort() when an error in vvector happens, or directly calling the
mopup
> >>routines before abort.
> >>
> >>
> >
> >Ok, go ahead, this should work.
> >
> >
> >
> >
> >>BTW is it possible to trap signals from Python?
> >>Try this:
> >>ecell3-session
> >>theSimulator.createEntity('MichaelisUniUniFluxProcess', 'Process:/:P0')
> >>getEntityProperty( 'Process:/:P0:StepperID')
> >>
> >>
> >
> >
> >
> >>I am not able to trap this signal from python.
> >>
> >>
> >
> >This is trapped by PyEcsSignalHandler.
> >This is a bug in C++ code indeed, and should halt the program without
> >going back to the user Python code.
> >
> >I'll fix this bug.
> >
> >-sha
> >
> >
> >
> >
> >>Gabor
> >>
> >>----- Original Message -----
> >>From: "Kouichi Takahashi" <sh...@sf...>
> >>To: "Bereczki Gabor" <gab...@ax...>
> >>Cc: <sh...@sf...>; "Yohei Yamada" <yo...@sf...>;
> >><ga...@e-...>
> >>Sent: Wednesday, January 14, 2004 6:50 AM
> >>Subject: Re: automatic deletion of vvector-* from /tmp/
> >>
> >>
> >>
> >>
> >>>Call vvector's cleanup function from
> >>>
> >>>static void PyEcsSignalHandler( int aSignal )
> >>>
> >>>in PyEcs.cpp.
> >>>
> >>>This should work.
> >>>
> >>>If it turns out to be not enough, an even robust way is:
> >>>
> >>>1. Let ecell3-python create a temporary dir under /tmp (such as
> >>>  /tmp/ecell3-python-$$/)
> >>>2. Pass this dir to ecell3 via envvar (ECELL3_TMP_DIR?)
> >>>3. When ecell3-python exits, rm -rf $ECELL3_TMP_DIR.
> >>>   Trap all signals.
> >>>
> >>>This deletes whole thing even if the system failed to remove tmp files.
> >>>
> >>>-sha
> >>>
> >>>
> >>>
> >>>
> >>>>How is it going?
> >>>>I think the modifications you require can be done.
> >>>>If Shafi has no objections I start to look into it.
> >>>>Can you raise a bug report or feature request?
> >>>>thanks,
> >>>>Gabor
> >>>>----- Original Message -----
> >>>>From: "Yohei Yamada" <yo...@sf...>
> >>>>To: <ga...@e-...>
> >>>>Cc: "Yohei Yamada" <yo...@sf...>; "Kouichi Takahashi"
> >>>><sh...@sf...>
> >>>>Sent: Sunday, January 11, 2004 9:12 PM
> >>>>Subject: automatic deletion of vvector-* from /tmp/
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Hi, Gabor.
> >>>>>
> >>>>>Is it possible to have all the vvector-* files under
> >>>>>/tmp/ be automatically deleted after simulation is
> >>>>>halted due to some error?(I believe they are
> >>>>>always deleted after simulation ends normally)
> >>>>>
> >>>>>In cases where a lot of loggers are created and used
> >>>>>to log data for a long time with small intervals,
> >>>>>the /tmp/ directory becomes full with vvector-* files
> >>>>>and gives error messages such as "error in vvector"
> >>>>>and "vvector disk full".
> >>>>>
> >>>>>It would be nice if the vvector-* files were deleted
> >>>>>automatically is such cases also, especially when SGE
> >>>>>is used because it's a bit troublesome to delete the
> >>>>>files individually for each node.
> >>>>>
> >>>>>Thanks!
> >>>>>
> >>>>>yoyo
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>--
> >>>Kouichi Takahashi,
> >>>Laboratory for Bioinformatics,
> >>>Institute for Advanced Biosciences,
> >>>Keio University, Japan.
> >>>http://www.sfc.keio.ac.jp/~shafi
> >>>
> >>>
>
>
>
>

yes that is how it's designed.
'saving' varrefs returns just '_'s, while by 'getting' them you will see
the internal unique names.

So in future when your ME is integrated with the core, you can just use
getEntityProperty() when manipulating the model, and use
saveEntityProperty() to save the model file.

-sha

> Or maybe it is enough if Simulator::getEntityProperty can return the
> variablereferences with these automatically given names, no need to save
> them for me?
> Will it be possible?
> Gabor
> ----- Original Message -----
> From: "Bereczki Gabor" <gab...@ax...>
> To: "Kouichi Takahashi" <sh...@sf...>; "ecell-devel"
> <ece...@li...>
> Cc: <sh...@sf...>
> Sent: Thursday, January 08, 2004 8:36 PM
> Subject: Re: [Ecell-devel] varref name omission
> 
> 
> > Hi Shafi,
> > Would it be possible to identify variablereferences uniquely for model
> > editing purposes?
> > Currently I am using variablereference name as a unique identifier when
> > deleting , modifying variablereferences.
> > I think it would be nice if these auto generated variablereference names
> > were saved.
> > Gabor
> >
> >
> > ----- Original Message -----
> > From: "Kouichi Takahashi" <sh...@sf...>
> > To: "ecell-devel" <ece...@li...>
> > Cc: <sh...@sf...>
> > Sent: Thursday, January 08, 2004 4:28 PM
> > Subject: [Ecell-devel] varref name omission
> >
> >
> > >
> > >
> > > Variable reference names in VariableReferenceList property of Process
> > > can now be omitted if the Process does not require that information.
> > >
> > > The ellipsis is '_'.   The system internally give it sequential
> > > names to avoid duplication. (current mangling scheme is like '___000',
> > > '___001', ...)      These are converted back to '_' when the model is
> > > saved.
> > >
> > >
> > > Sample usage:
> > >
> > >         Process MassActionFluxProcess( E_AB )
> > >         {
> > >                 VariableReferenceList   [ _ :.:A -1 ]
> > >                                         [ _ :.:B 1 ];
> > >                 k     0.01;
> > >         }
> > >
> > >
> > > -sha
> > >
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.net email is sponsored by: Perforce Software.
> > > Perforce is the Fast Software Configuration Management System offering
> > > advanced branching capabilities and atomic changes on 50+ platforms.
> > > Free Eval! http://www.perforce.com/perforce/loadprog.html
> > > _______________________________________________
> > > Ecell-devel mailing list
> > > Ece...@li...
> > > https://lists.sourceforge.net/lists/listinfo/ecell-devel
> > >
> > >
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: Perforce Software.
> > Perforce is the Fast Software Configuration Management System offering
> > advanced branching capabilities and atomic changes on 50+ platforms.
> > Free Eval! http://www.perforce.com/perforce/loadprog.html
> > _______________________________________________
> > Ecell-devel mailing list
> > Ece...@li...
> > https://lists.sourceforge.net/lists/listinfo/ecell-devel
> >
> >
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Perforce Software.
> Perforce is the Fast Software Configuration Management System offering
> advanced branching capabilities and atomic changes on 50+ platforms.
> Free Eval! http://www.perforce.com/perforce/loadprog.html
> _______________________________________________
> Ecell-devel mailing list
> Ece...@li...
> https://lists.sourceforge.net/lists/listinfo/ecell-devel
-- 
Kouichi Takahashi,
Laboratory for Bioinformatics,
Institute for Advanced Biosciences,
Keio University, Japan.
http://www.sfc.keio.ac.jp/~shafi

Or maybe it is enough if Simulator::getEntityProperty can return the
variablereferences with these automatically given names, no need to save
them for me?
Will it be possible?
Gabor
----- Original Message -----
From: "Bereczki Gabor" <gab...@ax...>
To: "Kouichi Takahashi" <sh...@sf...>; "ecell-devel"
<ece...@li...>
Cc: <sh...@sf...>
Sent: Thursday, January 08, 2004 8:36 PM
Subject: Re: [Ecell-devel] varref name omission


> Hi Shafi,
> Would it be possible to identify variablereferences uniquely for model
> editing purposes?
> Currently I am using variablereference name as a unique identifier when
> deleting , modifying variablereferences.
> I think it would be nice if these auto generated variablereference names
> were saved.
> Gabor
>
>
> ----- Original Message -----
> From: "Kouichi Takahashi" <sh...@sf...>
> To: "ecell-devel" <ece...@li...>
> Cc: <sh...@sf...>
> Sent: Thursday, January 08, 2004 4:28 PM
> Subject: [Ecell-devel] varref name omission
>
>
> >
> >
> > Variable reference names in VariableReferenceList property of Process
> > can now be omitted if the Process does not require that information.
> >
> > The ellipsis is '_'.   The system internally give it sequential
> > names to avoid duplication. (current mangling scheme is like '___000',
> > '___001', ...)      These are converted back to '_' when the model is
> > saved.
> >
> >
> > Sample usage:
> >
> >         Process MassActionFluxProcess( E_AB )
> >         {
> >                 VariableReferenceList   [ _ :.:A -1 ]
> >                                         [ _ :.:B 1 ];
> >                 k     0.01;
> >         }
> >
> >
> > -sha
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: Perforce Software.
> > Perforce is the Fast Software Configuration Management System offering
> > advanced branching capabilities and atomic changes on 50+ platforms.
> > Free Eval! http://www.perforce.com/perforce/loadprog.html
> > _______________________________________________
> > Ecell-devel mailing list
> > Ece...@li...
> > https://lists.sourceforge.net/lists/listinfo/ecell-devel
> >
> >
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Perforce Software.
> Perforce is the Fast Software Configuration Management System offering
> advanced branching capabilities and atomic changes on 50+ platforms.
> Free Eval! http://www.perforce.com/perforce/loadprog.html
> _______________________________________________
> Ecell-devel mailing list
> Ece...@li...
> https://lists.sourceforge.net/lists/listinfo/ecell-devel
>
>

Hi Shafi,
Would it be possible to identify variablereferences uniquely for model
editing purposes?
Currently I am using variablereference name as a unique identifier when
deleting , modifying variablereferences.
I think it would be nice if these auto generated variablereference names
were saved.
Gabor


----- Original Message -----
From: "Kouichi Takahashi" <sh...@sf...>
To: "ecell-devel" <ece...@li...>
Cc: <sh...@sf...>
Sent: Thursday, January 08, 2004 4:28 PM
Subject: [Ecell-devel] varref name omission


>
>
> Variable reference names in VariableReferenceList property of Process
> can now be omitted if the Process does not require that information.
>
> The ellipsis is '_'.   The system internally give it sequential
> names to avoid duplication. (current mangling scheme is like '___000',
> '___001', ...)      These are converted back to '_' when the model is
> saved.
>
>
> Sample usage:
>
>         Process MassActionFluxProcess( E_AB )
>         {
>                 VariableReferenceList   [ _ :.:A -1 ]
>                                         [ _ :.:B 1 ];
>                 k     0.01;
>         }
>
>
> -sha
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Perforce Software.
> Perforce is the Fast Software Configuration Management System offering
> advanced branching capabilities and atomic changes on 50+ platforms.
> Free Eval! http://www.perforce.com/perforce/loadprog.html
> _______________________________________________
> Ecell-devel mailing list
> Ece...@li...
> https://lists.sourceforge.net/lists/listinfo/ecell-devel
>
>

Variable reference names in VariableReferenceList property of Process
can now be omitted if the Process does not require that information.

The ellipsis is '_'.   The system internally give it sequential 
names to avoid duplication. (current mangling scheme is like '___000',
'___001', ...)      These are converted back to '_' when the model is
saved.


Sample usage:

        Process MassActionFluxProcess( E_AB )
        {
                VariableReferenceList   [ _ :.:A -1 ]
                                        [ _ :.:B 1 ];
                k     0.01;
        }


-sha