Subject: Summary - G98/Linux and huge files

From: "Dr. Alexander Hofmann" <[email-masked]>
Date: Tue, 12 Nov 2002 09:23:21 +0100

Hi,

here is the summary of all responses to my question, Thanks a lot to all contributers!

The question was:

Are there possibilities to circumvent the 2 GB limit with Gaussian 98 under Linux (Suse 8.0 2.4.x Kernel). I know the rwf-splitting. My aim is to write really huge files (like 100 GB or so). I haven't succeeded to write more than 16 Gb in total. (8x2 Gb), although I've tried.

Answer:

Shortly, the answer is no within quotation marks. There are possible attempts to circumvent it by using new gcc-libs (>3.1) to reach 16 GB in one file. But anyway, you can't go beyond because of 32bit addressing. 
Douglas J. Fox suggested to build a 64bit exe on the 32bit system, if there is no way to reach a real 64bit machine. The performance is expected to be very bad. 

Thanks again.

Alex

PS: The answers of all contributors

Gaussian is really messy, changing something usually means breaking
something else.

--
Groeten, David.
________________________________________________________________________
Dr. David van der Spoel,        Biomedical center, Dept. of Biochemistry
Husargatan 3, Box 576,          75123 Uppsala, Sweden
phone:  46 18 471 4205          fax: 46 18 511 755
[email-masked]    [email-masked]   http://zorn.bmc.uu.se/~spoel
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Yes, it is possible to cross 2GB file limit under Linux. You need to
install new GNU GCC version (3.1 or latest) to enable large files
support with g77 Fortran compiler. I have not tryed it with G98, but
GCC 3.1 recompiled GAMESS works fine.

But I'm insure that you really could run such a big tasks on 32-bit
platform due to other (say, maximum array index, etc) restrictions.

--
Best regards,
Gregory Shamov,
Dept. Chemistry
Kazan State University                          mailto:[email-masked]

Dear, Alexander,

  We recently build LINDA parallel version of g98 with large file support(LFS).

Now we can make scratch file of >2GB without file splitting.

You will get information on how to build g98 with LFS at our page
  http://ruby.ch.wani.osaka-u.ac.jp/QC.html

But, there is still 16GB limit because this is due to limitation of 32 bit integ
er used
in g98. We do not intend to challenge this problem at present because it is time
-consuming
job and the resulting build must be well-tested.

----------------------------------------------------
   Koichi Nozaki, Dr.
Department of Chemistry, Graduate School of Science, Osaka university
TEL: 06-6850-5777 / FAX: 06-6850-5785
<[email-masked]>

Hi,

I have a patch that will enable 16GB file segments for g98/Linda on Linux.
At first you might think there's no advantage to it, since it's possible
to split the RWF into eight 2GB segments. However, it's not possible to
split the file used by the Linda workers. So the patch at least enables
16GB for the worker processes. I can send it to you if you like. It used
to be posted on a Web site (not mine) until Gaussian, Inc.  requested that
it be removed. Perhaps because the method was not certified to produce
correct results.

As far as using files >16GB, that will require a more thorough conversion
of integers to 64-bit. I haven't yet been successful yet in getting this
to work on Linux. I've also been told that even if it did work, the effect
on the performance would be devastating, perhaps a factor of 10.

Anyway, if you're successful in using large files >16GB, please let me
know...

Bryan
--
Bryan Putnam, IT Research Computing Services
   Mathematical Sciences Building            Email: [email-masked]
   150 North University Street               Phone: 765.496.8225
   West Lafayette, IN 47907-2068               Fax: 765.494.0566
   http://www.purdue.edu/ITaP   http://www-rcd.cc.purdue.edu/~bfp/

  Dr. Hofmann,

   The short answer is no, there is no option in G98 to do this.

   The situation is that because the Intel Pentium of processors are 32 bit
processors, g98 is configured as a 32 bit application.  Using larger
word lengths for disk addresses would require promoting all integers in
G98 to be 64 bits.  We have not done the experiment but for systems with
64 bit hardware where we previously had g98 configured as a 32 bit application
we see a 10-15% performance decrease on going to 64 bits.  Doing all of the
integer manipulations in Gaussian, as would be required to make it a 64 bit
application on a 32 bit platform, would be catastrophic to performance.  Perhaps
not a bad trade for small applications but assuming you want 100GB of disk you
need every ounce of performance you can get.

   If you are doing large, conventional post-HF methods you will need a platform
with 64 bit arithmetic throughout.  At this point the Alpha Linux systems are an
option.  We also have the first generation Itanium but price performance is not
so great.  We hope to have Itanium2 early next year with our next release.

--

  Douglas J. Fox
  Technical Support
  Gaussian, Inc.
  [email-masked]

-- 

Dr. Alexander Hofmann

Humboldt-Universitaet zu Berlin
Institut fuer Chemie
Arbeitsgruppe Quantenchemie

Brook-Taylor-Strasse 2

12489 Berlin

[email-masked]

Tel.: +49-30-2093-7138
Fax.: +49-30-2093-7136

http://www.chemie.hu-berlin.de/ag_sauer/index.html

PGP-Key: wwwkeys.de.pgp.net ID: D9D62D35