Subject: Complete Summary) CCL:"%mem" trouble of Gaussian

From: "Telkuni" <telkuni:at:venus.dti.ne.jp>
Date: Sat, 29 Nov 2003 01:29:23 +0900

Hello, CCLers.
The summary mail which I've sent a few minutes before was not
complete version. Sorry!

<Complete Summary>
I had sent a question of "%mem=xxx" on Gaussian98W a few days ago.

---------- Original Question -------------
>I've met curios phenomenon with G98W_ver.A7. However I modified the
>PC's memory "%mem=128MB" to "%mem=512MB" on same calculation,
>there was very little calc time difference between them(57min and 64 min!)
>
>If anyone met same phenomenon, please let me know.
>All responses, I will appreciate. And I will summarize them.

In general, each calculation type has each suitable memory size. If the memory
is excess or insufficent on optimization, spreed-up is unexpectable(but for excited
states calc and frequency calc).

Here are replies that I have received.
Thanks a lot!

--- 1) from  Lorant Janosi ----
Hi!
I know what you mean. I noticed that gaussian tends to use up all
physical memory given as parameter in the %Mem section. This may
prolongue your time execution (like in your case) if the calculation
does not require that much amount of memory. I would recommend a few
test runs to see what would be the optimimum memory required. There is
also a formula given by the manual for the memory requirement, but,
honestely, I don't really trust it. But it's a good starting point for
test runs.

All the best,
Lori.

Lorant Janosi
Department of Physics and Astronomy
University of Missouri-Columbia

email: lj8q5:at:mizzou.edu

--- 2) from  Matt Zwier ----
Telkuni,

I ran a similar test several days ago.  The impact of available memory
depends strongly on the size of the job you're running.  If you have
enough RAM to store all AO integrals for SCF in 128MB, then 512MB will
likely not run faster, if at all.  If you can fit 90% of the AO
integrals in 128MB, then you'll see a greater, but not amazing, speed
gain when going up in RAM.  If you can only fit 30% of your AO integrals
in 128MB, you'll see a BIG speed increase when you get to 512MB.

Matt Zwier
Hope College

--- 3) from  Igor Avilov ----
Hello, Telkuni!

You must always try to allocate as much memory as Gaussian needs, no more. If you will allocate much more memory than it needs, the
calculation will slow down. I do not know why, but the fact is - it will slow down. In Linux (Unix) and Windows NT you can always
see how much memory Gaussian is actually using. For geometry optimization you don't need much memory. On the contrary, for excited
states calculation and frequency calculation you will need lots of memory. Its amount depends, of cause, on the size of your system.

Best regards,
Igor Avilov.

--- 4) from  Laurence Cuffe ----
I've not used g98W however my experience with g94 and g98 rev
A7 and rev A11 would be similar to yours.  In general once a
calculation has adequate RAM adding more via a %mem directive
rarely makes much difference and can (perversely) slow down a
calculation where the operating system has to throw out other
processes to get a large RAM image into memory.  There are
exceptions for instance if you can get an entire calculation into ram
using a large %Mem and the directive scf=(incore,nosave) then the
calculation can be scary fast! however this opportunity occurs
rarely.  On a related subject very occasionally calculations have
failed with a "malloc" error indicating that they have not got
sufficient ram.  I have found that this error is often resolved by
either increasing or decreasing the %Mem amount by a small
amount, indicating that the program occasionally miscalculates the
amount of memory available by a small amount and a small
change in the memory allocation can fit it in.
Hope this helps
All the best
Laurence cuffe

--- 5) from Pierre Vitorge ----
I had similar problems. I run tests with different values for %mem, it seems
there is an optimum %mem value to obtain the shortest CPU time, this optimum
value is usually far from the maximum reasonable value (i.e the amount of
physical memory minus 150to300Mb for the system). This probably depends
(among other reasons) on the size of the exchange file memory, and whether
this file is at the begining of the disk. I asked help to Gaussian, but did
not get any answer.

Pierre Vitorge
CEA DEN Saclay DPC/SECR/LSRM & UMR 8587 (CEA-Universite d'Evry-CNRS)
Bat.391 Pe.40a
91191 Gif sur Yvette cedex
France
tel.(+33) 169-08-32-65, secr.: (+33)169-08-32-50, fax:(+33)169-08-32-42
http://perso.club-internet.fr/vitorgen/pierre

--- 6) from David Shobe  ----
Telkuni,

If the program cannot advantageously use the additional memory, then increasing %mem has no effect.  However, if the additional
memory allows Gaussian to store an array in memory that would otherwise have been stored on disk, increasing %mem will significantly
shorten the time needed for the calculation.  Also, some jobs may not be able to run at all if insufficient memory is allocated with
%mem.

Hope this helps,

--David Shobe, Ph.D., M.L.S.
S|d-Chemie, Inc.
phone (502) 634-7409
fax (502) 634-7724

--- 7) from Sigismondo Boschi  ----
Hi,

what kind of computation? In SCF memory is not used over a certain
limit, and using more is just an overhead. For post-scf computations
more memory makes a huge difference (in the sense of improving the
performance).

Regards,

    Sigismondo Boschi

>>from Telkuni
Hello, Dr.Boschi.
Thank you for your reply.

It  is Opt --> Freq (# B3LYP/6-31G Opt Freq) calculation applying to
hydrogen bonded two molecles.
CPU times are about (Opt by 512MB)35min and (Opt by 128MB)39min,
(both Freq)18 min.

As you wrote, memory differnce doesn't act on SCF(opting, and freq?).

--- 8) from Toomas Tamm ----
On Tue, Nov 25, 2003 at 03:55:44PM +0100, VITORGE Pierre 094605 wrote:
> I had similar problems. I run tests with different values for %mem, it seems
> there is an optimum %mem value to obtain the shortest CPU time, this optimum
> value is usually far from the maximum reasonable value (i.e the amount of
> physical memory minus 150to300Mb for the system). This probably depends
> (among other reasons) on the size of the exchange file memory, and whether
> this file is at the begining of the disk. I asked help to Gaussian, but did
> not get any answer.

There exists a hypothesis that with today's fast CPU's it is better to
re-calculate some integrals in the CPU rather than store and read them
> from memory. Integrals are accessed rather randomly and the CPU cannot
cache them effectively. A cache miss can cost several tens of CPU
cycles while the value is fetched from RAM. It may be faster to
re-calculate a simple integral. Thus the calculation is faster if
there is less memory available, since this forces the program to
re-calculate integrals rather than store them.

Can someone with intricate knowledge of modern CPUs and integral
calculation algorithms tell if this is true?

--
Toomas Tamm                                 e-mail: tt-ccl:at:kky.ttu.ee
Chair of Inorganic Chemistry                voice:  INT+372-620-2810
Tallinn Technical University                fax:    INT+372-620-2020
Ehitajate tee 5, EE-19086 Tallinn, Estonia  http://www.kk.ttu.ee/toomas/

----------------------------------------------------
      Telkuni Tsuru     telkuni:at:venus.dti.ne.jp