Summary - G98/Linux and huge files
- From: "Dr. Alexander Hofmann" <ah at.at
chemie.hu-berlin.de>
- Subject: Summary - G98/Linux and huge files
- Date: Tue, 12 Nov 2002 09:23:21 +0100
Hi,
here is the summary of all responses to my question, Thanks a lot to all
contributers!
The question was:
Are there possibilities to circumvent the 2 GB limit with Gaussian 98 under
Linux (Suse 8.0 2.4.x Kernel). I know the rwf-splitting. My aim is to write
really huge files (like 100 GB or so). I haven't succeeded to write more than 16
Gb in total. (8x2 Gb), although I've tried.
Answer:
Shortly, the answer is no within quotation marks. There are possible attempts to
circumvent it by using new gcc-libs (>3.1) to reach 16 GB in one file. But
anyway, you can't go beyond because of 32bit addressing.
Douglas J. Fox suggested to build a 64bit exe on the 32bit system, if there is
no way to reach a real 64bit machine. The performance is expected to be very
bad.
Thanks again.
Alex
PS: The answers of all contributors
Gaussian is really messy, changing something usually means breaking
something else.
--
Groeten, David.
________________________________________________________________________
Dr. David van der Spoel, Biomedical center, Dept. of Biochemistry
Husargatan 3, Box 576, 75123 Uppsala, Sweden
phone: 46 18 471 4205 fax: 46 18 511 755
spoel at.at xray.bmc.uu.se spoel at.at gromacs.org http://zorn.bmc.uu.se/~spoel
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Yes, it is possible to cross 2GB file limit under Linux. You need to
install new GNU GCC version (3.1 or latest) to enable large files
support with g77 Fortran compiler. I have not tryed it with G98, but
GCC 3.1 recompiled GAMESS works fine.
But I'm insure that you really could run such a big tasks on 32-bit
platform due to other (say, maximum array index, etc) restrictions.
--
Best regards,
Gregory Shamov,
Dept. Chemistry
Kazan State University mailto:gas5x at.at
bancorp.ru
Dear, Alexander,
We recently build LINDA parallel version of g98 with large file support(LFS).
Now we can make scratch file of >2GB without file splitting.
You will get information on how to build g98 with LFS at our page
http://ruby.ch.wani.osaka-u.ac.jp/QC.html
But, there is still 16GB limit because this is due to limitation of 32 bit integ
er used
in g98. We do not intend to challenge this problem at present because it is time
-consuming
job and the resulting build must be well-tested.
----------------------------------------------------
Koichi Nozaki, Dr.
Department of Chemistry, Graduate School of Science, Osaka university
TEL: 06-6850-5777 / FAX: 06-6850-5785
<nozaki at.at ch.wani.osaka-u.ac.jp>
Hi,
I have a patch that will enable 16GB file segments for g98/Linda on Linux.
At first you might think there's no advantage to it, since it's possible
to split the RWF into eight 2GB segments. However, it's not possible to
split the file used by the Linda workers. So the patch at least enables
16GB for the worker processes. I can send it to you if you like. It used
to be posted on a Web site (not mine) until Gaussian, Inc. requested that
it be removed. Perhaps because the method was not certified to produce
correct results.
As far as using files >16GB, that will require a more thorough conversion
of integers to 64-bit. I haven't yet been successful yet in getting this
to work on Linux. I've also been told that even if it did work, the effect
on the performance would be devastating, perhaps a factor of 10.
Anyway, if you're successful in using large files >16GB, please let me
know...
Bryan
--
Bryan Putnam, IT Research Computing Services
Mathematical Sciences Building Email: bfp at.at purdue.edu
150 North University Street Phone: 765.496.8225
West Lafayette, IN 47907-2068 Fax: 765.494.0566
http://www.purdue.edu/ITaP http://www-rcd.cc.purdue.edu/~bfp/
Dr. Hofmann,
The short answer is no, there is no option in G98 to do this.
The situation is that because the Intel Pentium of processors are 32 bit
processors, g98 is configured as a 32 bit application. Using larger
word lengths for disk addresses would require promoting all integers in
G98 to be 64 bits. We have not done the experiment but for systems with
64 bit hardware where we previously had g98 configured as a 32 bit application
we see a 10-15% performance decrease on going to 64 bits. Doing all of the
integer manipulations in Gaussian, as would be required to make it a 64 bit
application on a 32 bit platform, would be catastrophic to performance. Perhaps
not a bad trade for small applications but assuming you want 100GB of disk you
need every ounce of performance you can get.
If you are doing large, conventional post-HF methods you will need a platform
with 64 bit arithmetic throughout. At this point the Alpha Linux systems are an
option. We also have the first generation Itanium but price performance is not
so great. We hope to have Itanium2 early next year with our next release.
--
Douglas J. Fox
Technical Support
Gaussian, Inc.
help at.at gaussian.com
--
Dr. Alexander Hofmann
Humboldt-Universitaet zu Berlin
Institut fuer Chemie
Arbeitsgruppe Quantenchemie
Brook-Taylor-Strasse 2
12489 Berlin
ah at.at chemie.hu-berlin.de
Tel.: +49-30-2093-7138
Fax.: +49-30-2093-7136
http://www.chemie.hu-berlin.de/ag_sauer/index.html
PGP-Key: wwwkeys.de.pgp.net ID: D9D62D35