Subject: G98.memory - summary

From: Ulrike Salzner <[email-masked]>
Date: Thu, 13 Jun 2002 13:52:08 +0300

Dear all,

thanks to everybody who responded to my request. Please find the 
original answers attached below.
To summarize the essentials: First of all I interpreted the information 
of the "free" commond incorrectly. Linux uses most of the memory until 
another process requests it. The proper place to look for memory 
information is the second line starting with: -/+ buffers/chache.
About using Gaussian: the most efficient procedure seems to be to 
estimate the memory a job requires and specify not much more than this 
amount using the "%mem" command. If too much memory is requested, even 
if it is less than the computer has, the performance will decrease. With 
more than one processors, the specified memory is per processor.

Best regards,
Ulrike Salzner

Original message:

Dear Compouter Experts,

I am trying to run a big job (1260 basis functions) on a dual 933MHz PC.
The operating system is RedHat 7.2, and there is 1 GB of memory. I have
not specified memory in the default route since test jobs showed that the
performance decreased considerably even if much less memory was requested
than the machine has.  
Now the number of processors is reduced to 1 by G98 due of lack of memory.
Checking with the top command, G98 is supposed to use only 5.8% of the
memory. Gtop (memory usage resident) shows that there are three main
users of memory: l502 (59756k), httpd (54312k), and nautilus (54780k). Sum
of resident sizes is 260752k. In contrast the command "free" gives:
Mem total: 1027928, used: 1022784, free 5144, shared 47864, buffers 100320
cached 771980
I do not understand where the memory is. Is there a way to tune the
system better? Would it be better to not use a graphical interface? Or am
I doing everything wrong? Any suggestions wil
l be greatly appreciated.
Sincerely,
Ulrike Salzner

> from Patrik Johansson:

Dear Ulrike

without knowing to many details there's the possibility that the Linux
OS uses some considerable amount of the memory to speed up processes
(like an extra cache in some way). This, however, will only happen if
you do not specify the G98 job to use an extensive amount (like 850 if
you really only need 450). So more memory doesn't always improve timing.
If the run can be made with let's say 200Mb I usually specify like 350Mb
and let the OS use the rest - this works for me.

best regards
/Patrik

> from Laurence Cuffe:

1) I'm no expert!
2) performance should improve if you drop the graphic interface.
3) try using %Mem=250MB, this should allow a two processor job 
with both parts fitting into main memory, where they take 250MB 
each thus giving a total of 500 Mb.
G98's memory usage can confuse red hat 7.2's memory 
management routines, running a 4 processor system with 4 g98 
jobs each of which required less than 1/4 of the ava
ilable ram I 
found that swap and memory management routines were taking 
60% of one processors time, the solution was to provide more 
conservative memory allocations for each job.
Another thing to note is that the memory requirement for a job with 
the following:
%Nproc=4
%Mem=100MB
is 100x4MB =400MB not 100MB in total as I had first supposed.
I hope this helps
All the best
Laurence Cuffe

> from Andreas Goetz:

Dear Ulrike,

I am not a computer expert nor am I using gaussian often.
So I cannot tell much about gaussian.

If you invoke "man httpd" you can inform yourself about httpd - Apache
hypertext transfer protocol server. I don't know what else than
calculations you are using the computer for, but I believe you don't want
to shut down httpd.

As far as I am informed, the memory which is given as buffers and cached
evoking the command "free" is memory which is reserved by the system for
speeding up certain pro
cesses. This memory does not have to be used and
can be freed in favour of a program which might request it.
In other words: Linux will always occupy almost all the memory to cache
instructions in the memory which might be required by programs. If a
program like gaussian will need more physical memory, the system will
reduce the cache memory and let gaussian use this part of the memory.
The numbers which really should concern you are those in the line
"-/+ buffers/cache" of the output from the "free" command.

As a comfortable tool for monitoring system ressources you might also want
to use "xosview".

Kind regards,
	Andreas

> from Fabien Fontaine:

I guess most of the memory is cached, which mean that it is considered
as used but actually it is like a buffer for future use.
If you really want to know if you have memory problem, check if the swap
memory is used.

Regards,
Fabien

> from Serguei Patchkovskii:

Ulrike,

Try 'ipcs -ma' command - G
aussian stores most of its data in a shared
memory region. If the system has been up for a while, Gaussian tends to
leave memory regions behind, making memory inaccessible. If you see any
regions with 'nattch' of zero, it is usually safe to remove them with
'ipcrm'. This should free up some memory.

Serguei

> from Doug Fox:

  Prof. Salzner,

   Ignoring the question about tuning the system let me describe the
way G98 allocates and uses memory.  

   Gaussian uses dynamic allocation within a large block of memory which
is allocated/released by each link.  By default this block of memory is
6MW (48MB) long which is in reasonable agreement with your gtop output,
some additional memory is used for code and local data.  Within this block
of memory some arrarys are fixed by the problem size, others can be 
adjusted to allow the calculation to succeed, sometimes at a cost of
additional CPU due to recomputing integrals, but in each case there is a
maximum about of disk wh
ich is useful and a minimum below which it 
cannot be done.

   In the case of shared memory and a two processor system if the amount
of memory is small enough that two+ passes would be needed to process
all of the integrals the code drops down to 1 processor because that allows
for a single long inner loop which is more efficient than the shorter
loop over two processors.  Remember that with two processors it would 
need to setup the loops twice, i.e. it nearly destroys the time saved by
running on two processors.  

   The fact that any given problem has a maximum effective memory is the
reason for your observation that giving G98 an arbitrary large memory 
will slow down small problems.  It takes time to process the allocation
and deallocation of the memory so if it is not used it simply shows up as
overhead.  On the other hand giving the program more memory on a problem
by problem basis, i.e. adding %mem=nnnmw at the top of the input, lets
you increase memory for cases wh
ich need it.  There is a rough formula
in chapter 4 of the G98 User's Reference which tells you the memory
for a single processor taking account of problem size and the nature of
the basis set and method.  For a dual processor system you don't need to 
double all of this, just double the portion dependent on basis set and
method.  This is a very rough guide but it should be an over estimate
and will let you use both processors.

> from Cory Allen Rice:

ULRIKE SALZNER ;

YOU CAN USE THE %mem=XXXMb IN G98. THIS WILL GIVE YOU MORE MEMORY GO TO
THE RUNNING OF A JOB ON G98.

Corey A Rice
Southern Illinois University at Edwardsville
Chemistry Department
[email-masked]
http://www.siue.edu/~corrice

> from Bern Kallies:

Hallo Ulrike,
G98 will be compiled by default for a usage of 4mw (1 word = 8 byte,
mega = 
1024 kilo) of scratch memory. I think this is what you see in
your memory usage statistics (59MB includes the size of the exe, its
statically allocated memory and the dynamically allocated scratch memory
of 32MB).

Regarding your problem to get g98 running on more than 1 processor with
your system, I'll try to explain the strategy of gaussian:
When running parallel gaussian jobs (without LINDA), the master process
of a gaussian link allocates scratch space (4mw by default, or as
specified in the %mem card), and then it forks children which use this
allocated scratch space as shared memory. In l502 scratch memory will be
used mainly for handling integrals. When there is no possibility to hold
a substantial amount of integrals in memory, they will be recomputed as
needed. But some information about integrals must be stored in memory in
any event. The size required for that depends on the number of basis
functions, the maximum l value, the packing scheme of integrals and
other t
hings. There is no easy way to find out this amount. But what
counts in your case is that every process stores these data private in
the scratch area. Thatswhy more scratch space will be needed when
running g98 in parallel. There exists an empirical rule for the amount
of memory needed:

mem(nproc) = mem(1proc) * (1 + 0.75*(nproc-1))

where nproc is the number of processors you want to use, and mem(1proc)
is the minimal amount required to run a particular job on 1 processor.
Gaussian checks if there is enough memory available when running in
parallel. If it guesses that it will not be enough, then it reduces the
number of processes until they fit in the available amount of memory, as
in your case. So you should specify something greater than 4mw in your
%mem card.
How much? Is an interesting question. To my experience, gaussian becomes
slower when it gets an amount of memory, which is larger than needed for
a direct SCF, but which is not enough for incore SCF. When using abou
t
120 basis functions, about 30mw are enough for incore. That means, using
8mw is slower than 4mw, with 16mw is slower than with 8mw, with 32mw is
faster than with 4mw (switches to incore), with 64mw is as fast as with
16mw and so on. When using about 350 basis functions, one would need
2.7gw (about 20GB) for incore. Allocating only 2.6gw in the latter case
would have the only effect that the job runs terribly long, longer than
with 8mw. So in your case I would succesively try it with 8mw, 16mw,
32mw, until the job runs. When it runs already with 4mw on one
processor, then 8mw would be enough according to the formula above.

Hope this helps,
best regards, Bernd

> from Demetrio de Silva Filho:

Hello Ulrike,

There is something you must remember. The memory you specify

is not what Gaussian will always use but the maximum memory

Gaussian will use during the job. If you follow what is happening while

you are running, you will see these numbers change,
 depending on the

link.

The problem with gaussian is : there are few parts of the program written in

old fortran 77 that cannot alocate memory dynamicaly, i.e. as the process

needs. So, you must specify in advance how much memory you have.

Hope it helps!

Demetrio FIlho