[Vis5d-users] Re: 3G files

Vaidya,

Some background:

Out-of-the box, vis5d does not support files which are > 2 GB in
a 32 bit operating system with 32 bit integers. Some operating
system/compiler combinations can make "int" be 64 bits and it will work
fine, but I could not find a way to tell gcc on Linux to treat ints as
64 bits.

The v5dstruct structure has the following integers which are used to
navigate through a vis5d file:

  int CurPos;              /* current position of file pointer */
  int FirstGridPos;        /* position of first grid in file */
  int GridSize[MAXVARS];   /* size of each grid */
  int SumGridSizes;        /* sum of GridSize[0..NumVars-1] */

When the size of the vis5d file exceeds 2^31 bytes (2 GB), there is an
overflow in CurPos and probably SumGridSizes. I found this out while
trying to write a large vis5d file.

My patch does the following: changes "int" to "int64_t" in variables
which refer to positioning within a vis5d file.

How to compile using my patch:

Take the patch which was attached to the file you quoted and put it in
the src directory. Type

patch < patch64

and it will modify some files.

You must then set the following envrinment variables which will be
passed to gcc during compilation. If you are using the bash shell the
command is:

export CFLAGS="-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE"

If you are in some csh derivative it's

setenv CFLAGS "-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE"

Then, type configure and make.

You can then link with v5d.o and binio.o in your program which creates
vis5d files if you want to create > 2 GB files, and of course the vis5d
binary you created can read them.

I'm pretty sure that the vis5d binary that you create using my method
will be able to read vis5d files which are made using the default source
code, but I can't be 100% sure. I know it doesn't work the other way
around! What will happen if you run the default vis5d with one of my
large files is once you get 2 GB into the file all the isosurfaces and
contours get all messed up.

I have sent my patch to the vis5d list but I don't think there is any
development going on right now. There may be a "better way" to make
vis5d work with these large files, but I think my way is sound.

Incidentally, just to make sure, you have to be using a filesystem that
supports > 2 GB files, such as ext3 or reiserfs... ext2 does not. Most
modern Linux distros come with these filesystems, and I would strongly
recommend that you use either on on all of your main partitoins.

Leigh

vaidyasubramanian chandrasekhar wrote:

|   Leigh orf,
|   Iam Vaidya from EVL @uic working on the vis5d and getting it on the geo
|   wall. I received your mail for runnning huge datasets > 2G on vis5d, I am
|   not too sure where the patches need to be added and then what to excatly
|   do,.
|   
|   Can you please tell me where the patch needs to be added and where the
|   changes need to be done .?
|   
|   The smaller vis5d data sets load and are animated in the geowall and hence
|   I persume the problem as you had suggested is due to the large data sets.
|   
|   regards
|   
|   vaidya