Summary - G98/Linux and huge files



Hi,
 here is the summary of all responses to my question, Thanks a lot to all
 contributers!
 The question was:
 Are there possibilities to circumvent the 2 GB limit with Gaussian 98 under
 Linux (Suse 8.0 2.4.x Kernel). I know the rwf-splitting. My aim is to write
 really huge files (like 100 GB or so). I haven't succeeded to write more than 16
 Gb in total. (8x2 Gb), although I've tried.
 Answer:
 Shortly, the answer is no within quotation marks. There are possible attempts to
 circumvent it by using new gcc-libs (>3.1) to reach 16 GB in one file. But
 anyway, you can't go beyond because of 32bit addressing.
 Douglas J. Fox suggested to build a 64bit exe on the 32bit system, if there is
 no way to reach a real 64bit machine. The performance is expected to be very
 bad.
 Thanks again.
 Alex
 PS: The answers of all contributors
 Gaussian is really messy, changing something usually means breaking
 something else.
 --
 Groeten, David.
 ________________________________________________________________________
 Dr. David van der Spoel,        Biomedical center, Dept. of Biochemistry
 Husargatan 3, Box 576,          75123 Uppsala, Sweden
 phone:  46 18 471 4205          fax: 46 18 511 755
 spoel at.at xray.bmc.uu.se    spoel at.at gromacs.org   http://zorn.bmc.uu.se/~spoel
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Yes, it is possible to cross 2GB file limit under Linux. You need to
 install new GNU GCC version (3.1 or latest) to enable large files
 support with g77 Fortran compiler. I have not tryed it with G98, but
 GCC 3.1 recompiled GAMESS works fine.
 But I'm insure that you really could run such a big tasks on 32-bit
 platform due to other (say, maximum array index, etc) restrictions.
 --
 Best regards,
 Gregory Shamov,
 Dept. Chemistry
 Kazan State University                          mailto:gas5x at.at
 bancorp.ru
 Dear, Alexander,
   We recently build LINDA parallel version of g98 with large file support(LFS).
 Now we can make scratch file of >2GB without file splitting.
 You will get information on how to build g98 with LFS at our page
   http://ruby.ch.wani.osaka-u.ac.jp/QC.html
 But, there is still 16GB limit because this is due to limitation of 32 bit integ
 er used
 in g98. We do not intend to challenge this problem at present because it is time
 -consuming
 job and the resulting build must be well-tested.
 ----------------------------------------------------
    Koichi Nozaki, Dr.
 Department of Chemistry, Graduate School of Science, Osaka university
 TEL: 06-6850-5777 / FAX: 06-6850-5785
 <nozaki at.at ch.wani.osaka-u.ac.jp>
 Hi,
 I have a patch that will enable 16GB file segments for g98/Linda on Linux.
 At first you might think there's no advantage to it, since it's possible
 to split the RWF into eight 2GB segments. However, it's not possible to
 split the file used by the Linda workers. So the patch at least enables
 16GB for the worker processes. I can send it to you if you like. It used
 to be posted on a Web site (not mine) until Gaussian, Inc.  requested that
 it be removed. Perhaps because the method was not certified to produce
 correct results.
 As far as using files >16GB, that will require a more thorough conversion
 of integers to 64-bit. I haven't yet been successful yet in getting this
 to work on Linux. I've also been told that even if it did work, the effect
 on the performance would be devastating, perhaps a factor of 10.
 Anyway, if you're successful in using large files >16GB, please let me
 know...
 Bryan
 --
 Bryan Putnam, IT Research Computing Services
    Mathematical Sciences Building            Email: bfp at.at purdue.edu
    150 North University Street               Phone: 765.496.8225
    West Lafayette, IN 47907-2068               Fax: 765.494.0566
    http://www.purdue.edu/ITaP   http://www-rcd.cc.purdue.edu/~bfp/
   Dr. Hofmann,
    The short answer is no, there is no option in G98 to do this.
    The situation is that because the Intel Pentium of processors are 32 bit
 processors, g98 is configured as a 32 bit application.  Using larger
 word lengths for disk addresses would require promoting all integers in
 G98 to be 64 bits.  We have not done the experiment but for systems with
 64 bit hardware where we previously had g98 configured as a 32 bit application
 we see a 10-15% performance decrease on going to 64 bits.  Doing all of the
 integer manipulations in Gaussian, as would be required to make it a 64 bit
 application on a 32 bit platform, would be catastrophic to performance.  Perhaps
 not a bad trade for small applications but assuming you want 100GB of disk you
 need every ounce of performance you can get.
    If you are doing large, conventional post-HF methods you will need a platform
 with 64 bit arithmetic throughout.  At this point the Alpha Linux systems are an
 option.  We also have the first generation Itanium but price performance is not
 so great.  We hope to have Itanium2 early next year with our next release.
 --
   Douglas J. Fox
   Technical Support
   Gaussian, Inc.
   help at.at gaussian.com
 --
 Dr. Alexander Hofmann
 Humboldt-Universitaet zu Berlin
 Institut fuer Chemie
 Arbeitsgruppe Quantenchemie
 Brook-Taylor-Strasse 2
 12489 Berlin
 ah at.at chemie.hu-berlin.de
 Tel.: +49-30-2093-7138
 Fax.: +49-30-2093-7136
 http://www.chemie.hu-berlin.de/ag_sauer/index.html
 PGP-Key: wwwkeys.de.pgp.net ID: D9D62D35