Subject: Summary: gaussian calculation speed

From: "Elena Jakubikova" <[email-masked]>
Date: Mon, 17 Dec 2001 19:54:53 -0700

Dear CCLers,

I would like to thank very much to everybody who responded to my question. 
Your answers were very helpful in solving my problem.
I have run hdparm -tT on my hard drive and found out that the operating 
system loded on millenia was indeed not using correct DMA and PIO modes. I 
fixed the problem by using hdparm:

hdparm -X66 -d1 -u1 -m16 -c3 /dev/hda

and I have also put this line in the /etc/rc.d/rc.local file, so the command 
get automatically executed if I reboot the computer. Now the job cpu times 
on beaver and on millenia are almost the same.
Best regards,

Elena Jakubikova
graduate student
Colorado State University

------------
Dear CCLers,

I have run one CISD calculation with a relatively large
basis set on CH4 molecule on 2 different computers.
Computer one is Dell 550MHz Pentium III, 512MB RAM
(100MHz SDRAM), 18 GB SCSI #1 Hard Drive, and Red Hat
Linux 6.0 operating system. Computer two is Millennia
733MHz Pentium III, 770MB RAM (133MHz SDRAM), 27GB IDE
ATA-66 hard drive, and Red Hat Linux 7.0 operating
system.
To my big surprise the calculation was much faster on
Dell (job cpu time = 24 minutes 1.8 sec) than on
Millennia (job cpu time = 1 hour 4 minutes 18.6
seconds). The output files I got from these computers
were completely identical, except the time gaussian spent
executing each link. Millennia computer took less time in
almost all cases (as one would expect from computer
specifications), except Link 401 (forms initial MO
guess), Link 804 (integral transformation), and Link 913
(calculates post SCF energies and gradient terms), where
Dell computer was much faster.
I tried to run top while running only this one
calculation and found out that during the extensive I/O
phase of calculation (l804,l913), there is a lot of CPU
used by the system on Millennia computer (at certain
moments its more than 90%), whereas in the case of Dell
computer almost all cpu is used by the user.
I am very puzzled by this results - Millennia computer
appears to be better suited for calculations, but it is
about 3 times slower. I wonder if this result is a
software issue that can be easily fixed or it is some
hardware problem. I would be very grateful to anybody who
has any ideas what is causing this result.

Thank you

Elena Jakubikova
graduate student
Colorado State University

--------------
Hi, It is because you have the SCSI harddisk in Dell whereas the IDE in 
Millennia. As you stated, the CISD calculation involves extensive I/O. Hence 
the performance of the disk (i.e. the read/write rate) will significantly 
affect the total performance of the calculation. The similiar case is 
observed in G3 method, it is also I/O intensive job. Cheers, Kurtz Dr. Kurtz 
Chiu Department of Chemistry The Chinese University of Hong Kong Hong Kong

---------------
I would imagine the I/O subsystem to be a serious determining performance 
factor with SCF calculations, unless you're doing direct SCF. I've had 
similar results between a Dell PIII 450 with a SCSI disk, and a Dell PIII 
733 with ATA. I understand that with Linux systems, I/O speed can vary 
greatly on how you've setup the OS, but I'm not intimately familiar with Red 
Hat 6 or 7, so I can't offer any real suggestions there. I'm sure if you 
replaced the ATA drive with a single SCSI drive, or even a SCSI RAID array, 
coupled with a good SCSI adapter, you'd have a very fast system, utilizing 
more of the faster processing speed. Hope that helps! Jason Douglas

---------------
Dear Elena, Isn't that the so-called memory problem? I recall that for 
certain calculations (HF, DFT energies) the execution times were shorter (on 
the same machine) if you gave Gaussian smaller amount of memory (via %Mem= 
directive). Couldn't you try your calculations with the same %Mem setting 
and see what happends? Best regards Peeter

---------------
Seems like you are using PIO transfer mode on Millenia. What does the output 
of hdparm say? Because of the disk I/O required for the job the (always 
operating in DMA mode) SCSI system is faster. Regards, Jochen

----------------
Dear Elena, Jason suggested that it may be the I/O-Subsystem. If it's the 
case it can be tuned:
When you type (as superuser)
hdparm -tT /dev/hda (or whatever the device of the HD is)
you should get something like
/dev/hda:
Timing buffer-cache reads: 128 MB in 1.26 seconds =101.59 MB/sec
Timing buffered disk reads: 64 MB in 3.56 seconds = 17.98 MB/sec
if your HD is set up using the correct DMA and PIO modes.
If not, you get values like
30 MB/sec (buffer-cache reads) and
3 MB/sec (buffered disk reads)
for example. The values above show the HD in my RH 7 System, and its not one 
of the fastest.
To set up the drive with hdparm look at the options with hdparm -h. And be 
careful. I just switched on DMA and 32-bit I/O-Support.
hdparm /dev/hda shows you how the HD is configured.
"Newer" 2.4 kernels (and late 2.2? not sure) can be set up to use DMA if 
available (I think it is the default setting). If you have an older kernel 
that does not support 32 bit I/O-Support you should get a new one.
Good luck,
Lars

-----------------
Indeed, several distributions of Linux do not turn on the ATA-66 (or 
ATA-100) support by default. One can make a quick test of disk speed (as 
root) with "hdparm -t /dev/hda" (or /dev/sda for SCSI). Results in 4-5 
MB/sec range indicate that the ATA support is off. A modern disk should 
deliver around 20-30 MB/sec with ATA-66 or ATA-100. For serious 
benchmarking, a more advanced tool, e.g. "bonnie" is recommended. One can 
enable UDMA (ATA-66 / ATA-100) on IDE disks with "hdparm -d1 /dev/hda" or 
similar. Other flags such as -c1 or -m16 to hdparm may benefit as well, but 
may also lead to data loss, so proceed very carefully. Avoid the -X flag to 
hdparm: I have personally had a case of data loss with such setup. -- Toomas 
Tamm

------------------------
Most likely, running: /sbin/hdparm -d 1 /dev/hdc # Or whatever your scratch 
disk # device is will improve results. You need to run this command as root, 
so ask your system administrator to do it for you (and to add the command to 
an appropriate startup file). To check how much difference it makes for I/O 
transfers, run /dev/hdparm -t /dev/hdc # Again, whatever your scratch disk 
is before and after turning on DMA. It is not uncommon to see an order of 
magnitude improvement in transfer speed, once DMA is turned on. Regards, Dr. 
Serguei Patchkovskii
-------------------------
Don't know what the answer is, but I would be very interested in knowing if 
you figure this out. One question. Did you also measure the wall time (i.e. 
actual execution time). I have had a few gaussian jobs where the CPU time is 
very much shorter than the wall time. It might be that RH6 and RH7 report 
cpu time differently, so that the Dell computer looks like it is much faster 
because some large amount of time (for example, used for disk I/O) is not 
being counted as cpu time. Prof. Scott L. Anderson
--------------------------
>One can enable UDMA (ATA-66 / ATA-100) on IDE disks with "hdparm -d1 > 
>/dev/hda" or similar. Other flags such as -c1 or -m16 to hdparm may > 
>benefit as well, but may also lead to data loss, so proceed very > 
>carefully. Avoid the -X flag to hdparm: I have personally had a case > of 
>data loss with such setup. But without the -X flag you only enable to use 
>dma-1 --> with a maximum transferrate of 16 MB/s. Using the -X flag you 
>enable Ultra-DMA (ATA-33 -- -X34, ATA-66 -- -X66, etc.). Bernd
--------------------------
On Wed, 31 Oct 2001, Bernd Schubert wrote: > > Avoid the -X flag to hdparm: 
I have personally had a case > > of data loss with such setup. > > But 
without the -X flag you only enable to use dma-1 --> with a maximum > 
transferrate of 16 MB/s. Using the -X flag you enable Ultra-DMA (ATA-33 -- > 
-X34, ATA-66 -- -X66, etc.). Some types of disk's, notably those made by 
Western Digital, and some controllers, notably the older one made by Via, 
don't behave quite correctly with UDMA, and thus if you have a WD disk, or a 
VIA motherboard controller, avoid the -X flag. Otherwise, it aught to be Ok.
---------------------------
>Some types of disk's, notably those made by Western Digital, and some 
>controllers, notably the older one made by Via, don't behave quite 
>correctly with UDMA, and thus if you have a WD disk, or a VIA motherboard 
>controller, avoid the -X flag. Otherwise, it aught to be Ok. What 
>documentation I've seen (it's pretty thin) suggests the problem is confined 
>to the VIA Apollo VP2 chipset. Check your /proc/pci to see what you have, 
>e.g $ less /proc/pci [...] Bus 0, device 1, function 0: PCI bridge: VIA 
>Technologies VT 82C598 Apollo MVP3 AGP (rev 0). Medium devsel. Master 
>Capable. No bursts. Min Gnt=12. [...] Recent VIA chipsets should be fine. 
>My machine uses the VIA Apollo MVP3 chipset (quite common on low-end PIII 
>machines) and I've seen no problems in six months or so using -X. I'm 
>running kernel version 2.2.19, and I did need to enable an 'experimental' 
>configuration flag (CONFIG_BLK_DEV_VIA82C586) to make this work. However, I 
>believe it should work out of the box for the 2.4 series. Here's a few 
>webpages with some further information: 
>http://www.linuxdoc.org/HOWTO/mini/Ultra-DMA.html 
>http://lhd.datapower.com/db/dispdriver.php3?DISP?310  
>http://www.linux-ide.org/[unfortunately the information in the Ultra-DMA 
>mini-howto is a couple of years old, and partially out of date] Misbehaving 
>drives should be automatically detected and blacklisted by the kernel. A 
>(possibly out-of-date) list is included in the Ultra-DMA mini-HOWTO listed 
>above. I have also seen patches to the 2.2 and 2.4 series for additional 
>chipset and feature support (e.g. UDMA100) but I haven't looked into this. 
>See http://www.lwn.net/2000/1102/a/ide.php3
cheers, Malcolm
------------------
Elena, sometimes benchmarks on a computer behave strange, because a 
screensaver is avtivated. They start quite fine, but when the sceensaver 
sets on, the performance drops significantly and you could have 50+% spend 
on this. Thomas
----------------

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com