vis5d-users Mailing List for Vis5d+

When installing the vis5d-5.1, the directory was created as "vis5d-
5.1". can that be changed without problems with running the linux 
compiled version?

Right now I am downloading the linux version for 5.2beta. The 
question above is more for when I install the new beta version. Some 
software in linux has files that have the directory inside and have to 
be changed before it will work. I am trying to change the directories 
for all of my software to when they don't have version numbers after 
them.KD5GIV
Storm spotter Tarrant County, Texas, USA
(Fort Worth)
Future Storm Prediction Center Forecaster

Michael,

A few things are done using GLlists but none of the important things are.   I'll
recently experimented with
using GLlists for all the graphics objects in vis5d+, but on my system I didn't
see any real performance improvement and it causes several other vis5d features
to be shortcircuted.  I recently merged the vis5d+ code with the NCAR 5.2 code
(which must be what you are using), in doing so I threw out the GLlist experiment
rather than trying to merge it.  I still have the code in CVS if you want to
experiment with it.

Jim

http://vis5d.sf.net


Michael Redmond wrote:

> We put the Vis 5D 5.2 version on an HP Visualize Center (uses HPUX 11) with
> success, including stereo viewing. We are now working with HP to get it
> working on an SV6 and we are trying to push the limits of that system (16+
> cpu parallel rendering engine). I'd love to get my hands on some 2G vis5d
> files for testing.
>
> One issue we run into with parallel rendering is whether OpenGL is rendered
> in immediate mode or if everything is in display lists that can be
> decomposed. To get performance, the Visualize Center and the SV6 work best
> with display lists. Bill or ?...which method is used in Vis5d 5.2? We would
> find out quickly with a large data set, but so far we are only using the
> typical LAMPS scale datasets that run pretty well.
>
> HP may demonstrate Vis5d on an SV6 at SC 2001 if they can present something
> compelling. If someone is willing to let them (and us) test a large
> dataset, please contact me.
>
> Thanks
> Mike Redmond
> Associate Director, eMedia Center
>
> ---
>
> At 10:47 PM 10/3/2001 -0600, Don Middleton wrote:
> >We routinely create and visualize >2GB files with Vis5D. Long ago, I added
> >explicit code to the file handling module to accomodate this. Since then,
> >Jeff Boote has integrated our stereo3D, largefile, and VRML support into the
> >5.2 release. I'm pretty sure that large files are handled just fine in here
> >(differently, using Posix contstructs instead of SGI extensions, I think),
> >but I'm cc'ing Jeff on this 'cause I haven't tried it myself. The software
> >is available at:
> >
> >             http://www.scd.ucar.edu/vg/SoftwareSystems.html
> >
> >We have a primarily SGI complex, dunno about special Linux configs.
> >
> >cheers - don
> >
> >---
> >Don Middleton
> >Head, Visualization & Enabling Technologies
> >Scientific Computing Division
> >National Center for Atmospheric Research
> >http://www.scd.ucar.edu/vets
> >PO Box 3000; Boulder, Co. 80307-3000
> >Voice:303-497-1250 Cell:303-589-5865 FAX:303-497-1286
> >
> >----- Original Message -----
> >From: "Dan McCormick" <Mo...@ho...>
> >To: "Leigh Orf" <or...@ma...>
> >Cc: <vis...@ss...>
> >Sent: Wednesday, October 03, 2001 9:01 PM
> >Subject: Re: 2 GB file size limit
> >
> >
> > > Leigh,
> > >
> > >    The routines in question are not integral to Vis5d - they are generic
> >FORTRAN
> > > (or C, depending on which you are using).  Therefore, this is either a
> >problem
> > > with the OS, the hardware, the file system, or the compiler you're using.
> >Even
> > > though your file system, hardware, and OS may support 64-bit operations,
> >your
> > > compiler's i/o routines may be limited to 32-bit file pointer values.
> > >
> > >    You may want to write several files (maybe one for every 30-minute
> >period, or
> > > one for each variable or two), instead of writing one large file.
> > >
> > >    I hope this helps.
> > >
> > > Regards,
> > > Dan McCormick
> > >
> > >
> > >
> > > Leigh Orf wrote:
> > >
> > > > OK, I should have provided more information.
> > > >
> > > > This occurs with both Linux, kernel 2.4.10, using the filesystem
> > > > Reiserfs (which allows for terrabyte sized files) and IRIX64 6.5 (the
> > > > SGI Origin (modi4) at NCSA). This is not a filesystem problem.
> > > >
> > > > What I am calling modern hardware is what I just got for myself - a
> > > > 1.4 GHz Athlon with 1 GB of memory. I do numerical modeling and lots
> > > > of visualization/rendering - have been using vis5d since 1990 when I
> > > > had to reserve time on a Stardent Stellar machine at SSEC where it was
> > > > developed. A two-hour simulation with 6 variables, data every minute, on
> > > > a ~ 100x100x60 grid can get big - and I am using a compression factor of
> > > > 4 for accuracy.
> > > >
> > > > Can anyone tell me that they have created vis5d files which are larger
> > > > than 2 GB successfully?
> > > >
> > > > The errors that go to stderr are of the "perhaps disk is full?" variety.
> > > >
> > > > Leigh Orf
> > > >
> > > > Glenn Carver wrote:
> > > >
> > > > |   Leigh,
> > > > |
> > > > |   You have probably hit a file size limit on the operating
> > > > |   system.  Various unixes have a 2Gb limit on a single file
> > > > |   because this represents the maximum integer for a 32bit
> > > > |   operating system. This limitation is removed in 64bit
> > > > |   operating systems such as Sun's Solaris 8.
> > > > |
> > > > |     Glenn
> > > >
> > > >
> > > > Janko Hauser wrote:
> > > >
> > > > |   Leigh Orf writes:
> > > > |
> > > > |    > I've run into a problem creating large vis5d files, namely
> > > > |    > once a vis5d file hits 2 GB the routines to write it fail.
> > > > |    >
> > > > |    > Is this an integer overflow problem, or an inherent
> > > > |    > problem with the vis5d file format? Can it be easily
> > > > |    > fixed? With modern hardware I can easily visualize really
> > > > |    > big datasets and the 2 GB file size limit will become a
> > > > |    > real problem.
> > > > |
> > > > |   Not only to help with your problem it would be quite
> > > > |   interesting, what you call modern hardware and also qhich OS
> > > > |   you are using. Or is everywhere hardware with more than 2Gb
> > > > |   of RAM in common use :-). If this is the case I have some
> > > > |   more arguments to get new hardware for the department.
> > > > |
> > > > |   __Janko
> > > >
> > > >
> > > >
> > >
> > >
> > >

Sheri,
We run Vis5D on a 3 screen HP Visualize Center with success. HP is now 
working with it on an HP SV6, though I am not sure if they have gone to 
tiled screen systems yet.

We have a Panoram GVR-120 operating at 3200 X 1024 resolution overall and 3 
HP J5600 CPUs to run the system. The images are pretty compelling with the 
scale (6' X 16') and curved screen. We also operate in stereographic mode 
with the NCAR extensions.

Any other questions...please let me know...
Thanks
Mike Redmond
Associate Director, eMedia Center
---
At 02:00 PM 8/27/2001 -0500, Sheri Mickelson wrote:
>I was just wondering if anyone out there has done any work with Vis5D on
>tiled displays (Powerwalls, Active Murals, etc)
>
>
>Sheri A. Mickelson
>Mathematics & Computer Science
>Argonne National Laboratory
>9700 South Cass Ave, Argonne, Illinois 60439
>Tel: 630-252-7526

We put the Vis 5D 5.2 version on an HP Visualize Center (uses HPUX 11) with 
success, including stereo viewing. We are now working with HP to get it 
working on an SV6 and we are trying to push the limits of that system (16+ 
cpu parallel rendering engine). I'd love to get my hands on some 2G vis5d 
files for testing.

One issue we run into with parallel rendering is whether OpenGL is rendered 
in immediate mode or if everything is in display lists that can be 
decomposed. To get performance, the Visualize Center and the SV6 work best 
with display lists. Bill or ?...which method is used in Vis5d 5.2? We would 
find out quickly with a large data set, but so far we are only using the 
typical LAMPS scale datasets that run pretty well.

HP may demonstrate Vis5d on an SV6 at SC 2001 if they can present something 
compelling. If someone is willing to let them (and us) test a large 
dataset, please contact me.

Thanks
Mike Redmond
Associate Director, eMedia Center

---

At 10:47 PM 10/3/2001 -0600, Don Middleton wrote:
>We routinely create and visualize >2GB files with Vis5D. Long ago, I added
>explicit code to the file handling module to accomodate this. Since then,
>Jeff Boote has integrated our stereo3D, largefile, and VRML support into the
>5.2 release. I'm pretty sure that large files are handled just fine in here
>(differently, using Posix contstructs instead of SGI extensions, I think),
>but I'm cc'ing Jeff on this 'cause I haven't tried it myself. The software
>is available at:
>
>             http://www.scd.ucar.edu/vg/SoftwareSystems.html
>
>We have a primarily SGI complex, dunno about special Linux configs.
>
>cheers - don
>
>---
>Don Middleton
>Head, Visualization & Enabling Technologies
>Scientific Computing Division
>National Center for Atmospheric Research
>http://www.scd.ucar.edu/vets
>PO Box 3000; Boulder, Co. 80307-3000
>Voice:303-497-1250 Cell:303-589-5865 FAX:303-497-1286
>
>----- Original Message -----
>From: "Dan McCormick" <Mo...@ho...>
>To: "Leigh Orf" <or...@ma...>
>Cc: <vis...@ss...>
>Sent: Wednesday, October 03, 2001 9:01 PM
>Subject: Re: 2 GB file size limit
>
>
> > Leigh,
> >
> >    The routines in question are not integral to Vis5d - they are generic
>FORTRAN
> > (or C, depending on which you are using).  Therefore, this is either a
>problem
> > with the OS, the hardware, the file system, or the compiler you're using.
>Even
> > though your file system, hardware, and OS may support 64-bit operations,
>your
> > compiler's i/o routines may be limited to 32-bit file pointer values.
> >
> >    You may want to write several files (maybe one for every 30-minute
>period, or
> > one for each variable or two), instead of writing one large file.
> >
> >    I hope this helps.
> >
> > Regards,
> > Dan McCormick
> >
> >
> >
> > Leigh Orf wrote:
> >
> > > OK, I should have provided more information.
> > >
> > > This occurs with both Linux, kernel 2.4.10, using the filesystem
> > > Reiserfs (which allows for terrabyte sized files) and IRIX64 6.5 (the
> > > SGI Origin (modi4) at NCSA). This is not a filesystem problem.
> > >
> > > What I am calling modern hardware is what I just got for myself - a
> > > 1.4 GHz Athlon with 1 GB of memory. I do numerical modeling and lots
> > > of visualization/rendering - have been using vis5d since 1990 when I
> > > had to reserve time on a Stardent Stellar machine at SSEC where it was
> > > developed. A two-hour simulation with 6 variables, data every minute, on
> > > a ~ 100x100x60 grid can get big - and I am using a compression factor of
> > > 4 for accuracy.
> > >
> > > Can anyone tell me that they have created vis5d files which are larger
> > > than 2 GB successfully?
> > >
> > > The errors that go to stderr are of the "perhaps disk is full?" variety.
> > >
> > > Leigh Orf
> > >
> > > Glenn Carver wrote:
> > >
> > > |   Leigh,
> > > |
> > > |   You have probably hit a file size limit on the operating
> > > |   system.  Various unixes have a 2Gb limit on a single file
> > > |   because this represents the maximum integer for a 32bit
> > > |   operating system. This limitation is removed in 64bit
> > > |   operating systems such as Sun's Solaris 8.
> > > |
> > > |     Glenn
> > >
> > >
> > > Janko Hauser wrote:
> > >
> > > |   Leigh Orf writes:
> > > |
> > > |    > I've run into a problem creating large vis5d files, namely
> > > |    > once a vis5d file hits 2 GB the routines to write it fail.
> > > |    >
> > > |    > Is this an integer overflow problem, or an inherent
> > > |    > problem with the vis5d file format? Can it be easily
> > > |    > fixed? With modern hardware I can easily visualize really
> > > |    > big datasets and the 2 GB file size limit will become a
> > > |    > real problem.
> > > |
> > > |   Not only to help with your problem it would be quite
> > > |   interesting, what you call modern hardware and also qhich OS
> > > |   you are using. Or is everywhere hardware with more than 2Gb
> > > |   of RAM in common use :-). If this is the case I have some
> > > |   more arguments to get new hardware for the department.
> > > |
> > > |   __Janko
> > >
> > >
> > >
> >
> >
> >

Hello,

I would  like to produce animations automatically with Vis5d 5.2 by =
running a tcl script in=20
offscreen mode. I use vis5d on two platforms : SGI Origin3200 =
(production=20
machine) and a PC Linux (interactive visualization machine). In =
interactive=20
mode, it runs fine on both machines. Now I have two problems :

1. When I save an image as PPM for instance, its colors don't correspond =
on=20
what I see on the screen (from blueish to redish in my case...) Do you =
know=20
how to fix it ?

2. The offscreen mode doesn't seem to work on the Irix machine. It gets =
stuck=20
at the line "vis5d_save_window $name $format" and take 99% of the cpu =
even=20
for a quasi empty image... (Under Linux, it is extremely slow but =
finally=20
save a redish ppm image :-)
Are you aware of problems in offscreen mode ? (I compiled it with mesa)

If you only have time for one question, please, could you tell me how to =
fix=20
the color of the saved images ?

Thank you for you help,

Bruno.

--
Bruno Jobard   ---   bj...@cs...  --  http://www.cscs.ch/~bjobard
CSCS, Swiss Center for Scientific Computing | Tel: +41 (91) 610.82.60
Via Cantonale, 6928 Manno, Switzerland      | Fax: +41 (91) 610.82.82

At 11:58 AM -0500 10/22/1, Marcus Buker wrote:

>or reading in and plotting a flight track? (just a fixed polyline  along
>all the points)

I'd also be interested in this capability as well.  Any ideas or 
communication that didn't get sent via the list?

Thanks,
Mike

-- 
Mike Zulauf
maz...@me...

Hi,

Does anyone know how to get irregular (SYNOP/METAR) data into vis5d
without going through the irregular importer?

If anyone on the list is regularly using irregular data in vis5d I would
like to here from you how you are using it and what you think are its
strengths and weaknesses.

Jim


PS:  I am a developer of the vis5d+ project at http://vis5d.sf.net -
look for a new release announcement on this list in the coming few
weeks.

Is there any existing code for putting in latitude longitude lines on a
map,

or reading in and plotting a flight track? (just a fixed polyline  along
all the points)

or constructing a hodograph?

Thanks for any help.

--
Marcus Buker  ><+>
ma...@da...
http://dave1.meteor.wisc.edu/

> I am using Vis5D 5.1 in Linux kernel 2.2.16. Sometimes when I click on a
> variable button after I already have something displayed and the former
> control bar is replaced by the new variable, so then I can no long change
> settings on the first variable I used. I am not sure if it is a bug but sometimes
> it doesn't do that. I am also running 96MB RAM on a PC. The data file is
> only 6.6MB so I know it isn't a memory problem. Does anybody know of a
> way to fix the problem or should this be a bug report? It's minor problem and
> I haven't had any other problems with Vis5D since I got alot more RAM.

If you want to see the control bar for a variable that's
already displayed, click the center mouse button on that
variable's button. If you don't have a center button, try
clicking the variable off and on with the left button.

Good luck, Bill
----------------------------------------------------------
Bill Hibbard, SSEC, 1225 W. Dayton St., Madison, WI  53706
hi...@fa...  608-263-4427  fax: 608-263-6738
http://www.ssec.wisc.edu/~billh/vis.html

Hello,
I am using Vis5D 5.1 in Linux kernel 2.2.16. Sometimes when I click on a 
variable button after I already have something displayed and the former 
control bar is replaced by the new variable, so then I can no long change 
settings on the first variable I used. I am not sure if it is a bug but sometimes 
it doesn't do that. I am also running 96MB RAM on a PC. The data file is 
only 6.6MB so I know it isn't a memory problem. Does anybody know of a 
way to fix the problem or should this be a bug report? It's minor problem and 
I haven't had any other problems with Vis5D since I got alot more RAM. 


Michael Maxwell
KD5GIV
Fort Worth, Texas, USA
Storm Spotter Tarrant County
Future Storm Prediction Center Forecaster

hi vis5d+ mailing list,

I read greate think about vis5d+ we can link vis5d+ to www
i want to know more about  how can i make interface betwen  vis5d+ and
www
and can i use vis5d+ to load segy files and vis. it ( have you any
example ).

can you replay me on my email : mar...@ya...

thanks,

Well, I spoke too soon. I did create a > 2 GB vis5d file after tweaking
the code, but much of the animation is garbage, starting at the point
at which I'd assume the 2 GB mark is hit. The values of the variable
being viewed are reasonable but the isosurfaces are very strange and
jump around from frame to frame.

I'm kind of at a loss as of what to do at this point.

Could someone who has created > 2 GB vis5d files on the SGI platform
(I'm on the NCSA Origin) please let me know what compiler options to
pass to make this work.  The utility I use to create the vis5d files
borrows v5d.c, binio.c, config.h, binio.h and v5d.h from vis5d, creates
object code from them, and links at the end. Does the vis5d application
also need any special compiler options to view these large files?

Also, to LAM SAI-LAP, who had luck with creating > 2GB vis5d files under
Linux using gcc, I'm wondering what use the -D_FILE_OFFSET_BITS=64
and -D_LARGEFILE_SOURCE options are since none of the vis5d 5.2 code
contains those preprocessor variables!

Leigh Orf

Leigh Orf wrote:

|   I finally managed to write vis5d files greater than 2 GB
|   in size on the SGI Origin (IRIX64 modi4 6.5 01101245 IP27)
|   but only by hacking the code... I'm wondering if there is a
|   simpler way.

I am a new Vis5D user seeking some help.

Is there an expert out there who can tell me the parameters 
that I need to include in my configuration file in order 
to display an entire 3D volume (about 1GB) instead of 
slides that I keep getting?

Thanks for the help!

Luc Ikelle
 

I finally managed to write vis5d files greater than 2 GB in size on the
SGI Origin (IRIX64 modi4 6.5 01101245 IP27) but only by hacking the
code... I'm wondering if there is a simpler way.

Here's what was happening: in v5d.c, the following exception was
occurring, from line ~2466 or so:

   /* move to position in file */
   pos = grid_position( v, time, var );
   if (lseek( v->FileDesc, pos, SEEK_SET )<0) {
      /* lseek failed, return error */
      printf("Error in v5dWrite[Compressed]Grid: seek failed, disk full?\n");
      return 0;
   }

pos is an int, and grid_position returns an int. The problem
is grid_position was getting larger than INT_MAX (from
/usr/include/limits.h, INT_MAX = 2147483647) and when it did, returned a
large negative integer, and lseek barfed.

So I changed pos, grid_position etc. to long ints instead of ints, did
some (long int) castings on some int variables, and got it to work. Oh,
I also put

#define lseek lseek64

in v5dconfig.h

My question now is, is there a more correct way to do this? Are there
compiler options I can pass to the SGI C compiler to basically tell it
to treat all ints as long ints? Passing the -64 option doesn't do the
trick.

BTW I am using the  vis5d+1.1 source code.

I have not yet tried to get this to work with Linux.

Leigh Orf

"From: Bill Hibbard <hi...@fa...>" <bi...@ss...> writes:
 > > Sorry for the sarcasm in my last mail, but I thought, that the hole
 > > vis5d-file is loaded into memory if it gets visualized. Isn't this
 > > true? And if the OS starts swapping isn't the rendering considerably
 > > slowed down? Just wondering, haven't tried it yet.
 > 
 > Not at all. At start up, Vis5D compyytes the ration of
 > file size to available memory size and adopts one of
 > three caching startegies, depending on the ratio.
 > 
 > Bill
 > 

Thanks for all the hints, time for me to test it for my data-sets.

With regards,
__Janko

> Sorry for the sarcasm in my last mail, but I thought, that the hole
> vis5d-file is loaded into memory if it gets visualized. Isn't this
> true? And if the OS starts swapping isn't the rendering considerably
> slowed down? Just wondering, haven't tried it yet.

Not at all. At start up, Vis5D compyytes the ration of
file size to available memory size and adopts one of
three caching startegies, depending on the ratio.

Bill

Subject: Re: 2 GB file size limit


> Don Middleton writes:
>  > We routinely create and visualize >2GB files with Vis5D. Long ago, I
added
>  > explicit code to the file handling module to accomodate this. Since
then,
>  > Jeff Boote has integrated our stereo3D, largefile, and VRML support
into the
>  > 5.2 release. I'm pretty sure that large files are handled just fine in
here
>  > (differently, using Posix contstructs instead of SGI extensions, I
think),
>  > but I'm cc'ing Jeff on this 'cause I haven't tried it myself. The
software
>  > is available at:
>  >
>  >             http://www.scd.ucar.edu/vg/SoftwareSystems.html
>  >
>
> Sorry for the sarcasm in my last mail, but I thought, that the hole
> vis5d-file is loaded into memory if it gets visualized. Isn't this
> true? And if the OS starts swapping isn't the rendering considerably
> slowed down? Just wondering, haven't tried it yet.

Our SGI systems have 7-8GB of physical memory and in the past I've
configured swap for 50GB or so, as I recall. If you swap things of course
slow down. If you traverse your dataset linearly it can be livable. If you
let a "make_isosurface" run on multiple processors on a large dataset it can
be horrific. I always run Vis5D in "verylarge" mode and, more often than
not, with an option to turn off multi-processing. I don't know if Vis5D
actually loads the entire dataset at startup time or not - it may just
allocate the memory. Even with very large datasets, it starts up quite fast.

don

Leigh Orf wrote:

> I've run into a problem creating large vis5d files, namely once a
> vis5d file hits 2 GB the routines to write it fail.
>
> Is this an integer overflow problem, or an inherent problem with the
> vis5d file format? Can it be easily fixed? With modern hardware I can
> easily visualize really big datasets and the 2 GB file size limit will
> become a real problem.
>
> Leigh Orf

Perhaps, you may try to recompile vis5d with the following compiler
options for gcc in the relevant makefile:

   gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE ...

We have used these compiler options to create file > 2Gb on RedHat 7.1.

Regards,
Lam.

Don Middleton writes:
 > We routinely create and visualize >2GB files with Vis5D. Long ago, I added
 > explicit code to the file handling module to accomodate this. Since then,
 > Jeff Boote has integrated our stereo3D, largefile, and VRML support into the
 > 5.2 release. I'm pretty sure that large files are handled just fine in here
 > (differently, using Posix contstructs instead of SGI extensions, I think),
 > but I'm cc'ing Jeff on this 'cause I haven't tried it myself. The software
 > is available at:
 > 
 >             http://www.scd.ucar.edu/vg/SoftwareSystems.html
 > 

Sorry for the sarcasm in my last mail, but I thought, that the hole
vis5d-file is loaded into memory if it gets visualized. Isn't this
true? And if the OS starts swapping isn't the rendering considerably
slowed down? Just wondering, haven't tried it yet.

__Janko

We routinely create and visualize >2GB files with Vis5D. Long ago, I added
explicit code to the file handling module to accomodate this. Since then,
Jeff Boote has integrated our stereo3D, largefile, and VRML support into the
5.2 release. I'm pretty sure that large files are handled just fine in here
(differently, using Posix contstructs instead of SGI extensions, I think),
but I'm cc'ing Jeff on this 'cause I haven't tried it myself. The software
is available at:

            http://www.scd.ucar.edu/vg/SoftwareSystems.html

We have a primarily SGI complex, dunno about special Linux configs.

cheers - don

---
Don Middleton
Head, Visualization & Enabling Technologies
Scientific Computing Division
National Center for Atmospheric Research
http://www.scd.ucar.edu/vets
PO Box 3000; Boulder, Co. 80307-3000
Voice:303-497-1250 Cell:303-589-5865 FAX:303-497-1286

----- Original Message -----
From: "Dan McCormick" <Mo...@ho...>
To: "Leigh Orf" <or...@ma...>
Cc: <vis...@ss...>
Sent: Wednesday, October 03, 2001 9:01 PM
Subject: Re: 2 GB file size limit


> Leigh,
>
>    The routines in question are not integral to Vis5d - they are generic
FORTRAN
> (or C, depending on which you are using).  Therefore, this is either a
problem
> with the OS, the hardware, the file system, or the compiler you're using.
Even
> though your file system, hardware, and OS may support 64-bit operations,
your
> compiler's i/o routines may be limited to 32-bit file pointer values.
>
>    You may want to write several files (maybe one for every 30-minute
period, or
> one for each variable or two), instead of writing one large file.
>
>    I hope this helps.
>
> Regards,
> Dan McCormick
>
>
>
> Leigh Orf wrote:
>
> > OK, I should have provided more information.
> >
> > This occurs with both Linux, kernel 2.4.10, using the filesystem
> > Reiserfs (which allows for terrabyte sized files) and IRIX64 6.5 (the
> > SGI Origin (modi4) at NCSA). This is not a filesystem problem.
> >
> > What I am calling modern hardware is what I just got for myself - a
> > 1.4 GHz Athlon with 1 GB of memory. I do numerical modeling and lots
> > of visualization/rendering - have been using vis5d since 1990 when I
> > had to reserve time on a Stardent Stellar machine at SSEC where it was
> > developed. A two-hour simulation with 6 variables, data every minute, on
> > a ~ 100x100x60 grid can get big - and I am using a compression factor of
> > 4 for accuracy.
> >
> > Can anyone tell me that they have created vis5d files which are larger
> > than 2 GB successfully?
> >
> > The errors that go to stderr are of the "perhaps disk is full?" variety.
> >
> > Leigh Orf
> >
> > Glenn Carver wrote:
> >
> > |   Leigh,
> > |
> > |   You have probably hit a file size limit on the operating
> > |   system.  Various unixes have a 2Gb limit on a single file
> > |   because this represents the maximum integer for a 32bit
> > |   operating system. This limitation is removed in 64bit
> > |   operating systems such as Sun's Solaris 8.
> > |
> > |     Glenn
> >
> >
> > Janko Hauser wrote:
> >
> > |   Leigh Orf writes:
> > |
> > |    > I've run into a problem creating large vis5d files, namely
> > |    > once a vis5d file hits 2 GB the routines to write it fail.
> > |    >
> > |    > Is this an integer overflow problem, or an inherent
> > |    > problem with the vis5d file format? Can it be easily
> > |    > fixed? With modern hardware I can easily visualize really
> > |    > big datasets and the 2 GB file size limit will become a
> > |    > real problem.
> > |
> > |   Not only to help with your problem it would be quite
> > |   interesting, what you call modern hardware and also qhich OS
> > |   you are using. Or is everywhere hardware with more than 2Gb
> > |   of RAM in common use :-). If this is the case I have some
> > |   more arguments to get new hardware for the department.
> > |
> > |   __Janko
> >
> >
> >
>
>
>

Leigh,

   The routines in question are not integral to Vis5d - they are generic FORTRAN 
(or C, depending on which you are using).  Therefore, this is either a problem 
with the OS, the hardware, the file system, or the compiler you're using.  Even 
though your file system, hardware, and OS may support 64-bit operations, your 
compiler's i/o routines may be limited to 32-bit file pointer values.

   You may want to write several files (maybe one for every 30-minute period, or 
one for each variable or two), instead of writing one large file.

   I hope this helps.

Regards,
Dan McCormick



Leigh Orf wrote:

> OK, I should have provided more information.
> 
> This occurs with both Linux, kernel 2.4.10, using the filesystem
> Reiserfs (which allows for terrabyte sized files) and IRIX64 6.5 (the
> SGI Origin (modi4) at NCSA). This is not a filesystem problem.
> 
> What I am calling modern hardware is what I just got for myself - a
> 1.4 GHz Athlon with 1 GB of memory. I do numerical modeling and lots
> of visualization/rendering - have been using vis5d since 1990 when I
> had to reserve time on a Stardent Stellar machine at SSEC where it was
> developed. A two-hour simulation with 6 variables, data every minute, on
> a ~ 100x100x60 grid can get big - and I am using a compression factor of
> 4 for accuracy.
> 
> Can anyone tell me that they have created vis5d files which are larger
> than 2 GB successfully?
> 
> The errors that go to stderr are of the "perhaps disk is full?" variety.
> 
> Leigh Orf
> 
> Glenn Carver wrote:
> 
> |   Leigh,
> |
> |   You have probably hit a file size limit on the operating
> |   system.  Various unixes have a 2Gb limit on a single file
> |   because this represents the maximum integer for a 32bit
> |   operating system. This limitation is removed in 64bit
> |   operating systems such as Sun's Solaris 8.
> |
> |     Glenn
> 
> 
> Janko Hauser wrote:
> 
> |   Leigh Orf writes:
> |
> |    > I've run into a problem creating large vis5d files, namely
> |    > once a vis5d file hits 2 GB the routines to write it fail.
> |    >
> |    > Is this an integer overflow problem, or an inherent
> |    > problem with the vis5d file format? Can it be easily
> |    > fixed? With modern hardware I can easily visualize really
> |    > big datasets and the 2 GB file size limit will become a
> |    > real problem.
> |
> |   Not only to help with your problem it would be quite
> |   interesting, what you call modern hardware and also qhich OS
> |   you are using. Or is everywhere hardware with more than 2Gb
> |   of RAM in common use :-). If this is the case I have some
> |   more arguments to get new hardware for the department.
> |
> |   __Janko
> 
> 
> 

OK, I should have provided more information.

This occurs with both Linux, kernel 2.4.10, using the filesystem
Reiserfs (which allows for terrabyte sized files) and IRIX64 6.5 (the
SGI Origin (modi4) at NCSA). This is not a filesystem problem.

What I am calling modern hardware is what I just got for myself - a
1.4 GHz Athlon with 1 GB of memory. I do numerical modeling and lots
of visualization/rendering - have been using vis5d since 1990 when I
had to reserve time on a Stardent Stellar machine at SSEC where it was
developed. A two-hour simulation with 6 variables, data every minute, on
a ~ 100x100x60 grid can get big - and I am using a compression factor of
4 for accuracy.

Can anyone tell me that they have created vis5d files which are larger
than 2 GB successfully?

The errors that go to stderr are of the "perhaps disk is full?" variety.

Leigh Orf

Glenn Carver wrote:

|   Leigh,
|
|   You have probably hit a file size limit on the operating
|   system.  Various unixes have a 2Gb limit on a single file
|   because this represents the maximum integer for a 32bit
|   operating system. This limitation is removed in 64bit
|   operating systems such as Sun's Solaris 8.
|
|     Glenn


Janko Hauser wrote:

|   Leigh Orf writes:
|
|    > I've run into a problem creating large vis5d files, namely
|    > once a vis5d file hits 2 GB the routines to write it fail.
|    >
|    > Is this an integer overflow problem, or an inherent
|    > problem with the vis5d file format? Can it be easily
|    > fixed? With modern hardware I can easily visualize really
|    > big datasets and the 2 GB file size limit will become a
|    > real problem.
|
|   Not only to help with your problem it would be quite
|   interesting, what you call modern hardware and also qhich OS
|   you are using. Or is everywhere hardware with more than 2Gb
|   of RAM in common use :-). If this is the case I have some
|   more arguments to get new hardware for the department.
|
|   __Janko

Leigh Orf writes:
 > 
 > I've run into a problem creating large vis5d files, namely once a
 > vis5d file hits 2 GB the routines to write it fail.
 > 
 > Is this an integer overflow problem, or an inherent problem with the
 > vis5d file format? Can it be easily fixed? With modern hardware I can
 > easily visualize really big datasets and the 2 GB file size limit will
 > become a real problem.
 > 
Not only to help with your problem it would be quite interesting, what
you call modern hardware and also qhich OS you are using. Or is
everywhere hardware with more than 2Gb of RAM in common use :-). If
this is the case I have some more arguments to get new hardware for
the department.

__Janko

I've run into a problem creating large vis5d files, namely once a
vis5d file hits 2 GB the routines to write it fail.

Is this an integer overflow problem, or an inherent problem with the
vis5d file format? Can it be easily fixed? With modern hardware I can
easily visualize really big datasets and the 2 GB file size limit will
become a real problem.

Leigh Orf