Subject: Re: CCL:Linux 2.4.0 and 2 GB promlem

From: Christoph van Wüllen <[email-masked]>
Date: Wed, 7 Feb 2001 12:07:47 +0100

Files larger than 2GB on 32-bit unix systems need new system calls,
e.g. read64(), lseek64() etc., to handle large files. In these system calls,
"long long" parameters instead of the standard "int" are used for passing
the offset, and for passing return values.

The calling (FORTRAN) program must access the file in blocks (if it is
direct access) and use I/O procedures written in C, then you can use the new
system calls. If a FORTRAN integer has to specify a *byte* offset in the file,
the new system calls are of no use.
-- 
+---------------------------------+-------------------------------------+
| Prof. Christoph van Wullen      | Tele-Phone (+49) (0)30 314 27870    |
| Technische Universitat Sekr. C3 | Tele-Fax   (+49) (0)30 314 23727    |
| Strasse des 17. Juni 135        | eMail                               |
| D-10623 Berlin, Germany         | [email-masked]    |
+---------------------------------+-------------------------------------+

---

From: Jonas Juselius <[email-masked]>
Date: Wed, 7 Feb 2001 10:11:47 +0200

The 2GB limit is a thing of the past in 2.4.1, but you must use ReiserFS
(or XFS from SGI) to get past the limit. AFAIK ext2 does still not handle
files larger than 2GB. I have been using ReiserFS together with LVM
(www.sistina.com) for some time now, and I'm extermely happy with the setup.
If I do a one finger boot, the filesystem check on my 25 GB work partition
(ReiserFS) takes about 2-3 seconds :-)

-jonas-

On Tue, Feb 06, 2001 at 12:45:35PM +0100, Matthias Mann wrote:
> Hallo,
> 
> > Dear CCL'ers,
> > I've heard that new Linux kernel 2.4.0 can handle files larger than 2 GB.
> > Have anybody tried to test 2.4.0 kernel with quantum-chemical programs?
> 
> it seems, nobody has experience with this until now ?
> Because I'm also very interested to solve this problem, I asked Gaussian
> according the LFS support in G98. Here is the answer:
> 
> > The limit is partly relieved in Rev. A.7.  We have further
> > changes expected in the next minor revision which should 
> > remove it entirely for machines with native 64 bit integers.
> > For A.7 the limit is around 20GB due to a limit in the bin
> > sort required for post-MP2 methods. 
> >
> > This does not extend to Linux on Intel platforms because here
> > 32 bit integers are still used to maintain good performance.
> > We have not yet tested with the 2.4 kernels but since we see
> > as bad as 30% penalty for using 64 bit integers on machines
> > like the IBM Power3 when using 64 bit integers we don't plan on
> > releasing a version for Linux on Intel.
> 
> This is not totally clear, because it is possible to support large
> files also on 32 bit systems. For example, large files are supported
> with 32bit IBM AIX 4.2 on RS/6000 (without reducing performance).
> 
> Because the 2 GB limit is one of the remaining disadvantages in
> comparison of Linux systems with commercial workstations,
> we should try to solve this for quantum chemical programs ?!
> 
> Greetings,
> Matthias
> 
> --
> Dr. Matthias Mann 
> Institut fuer Organische Chemie, TU Dresden
> D-01069 Dresden, Mommsenstr. 13
> Tel.: +49 (351) 463-4286 / Fax: -3162
> Email: [email-masked]
> 
> 
> -= This is automatically added to each message by mailing script =-
> [email-masked] -- To Everybody  | [email-masked] -- To Admins
> [email-masked] -- HELP CHEMISTRY or HELP SEARCH
> [email-masked] -- archive search    |    Gopher: gopher.ccl.net 70
> Ftp: ftp.ccl.net  |  WWW: http://www.ccl.net/chemistry/   | Jan: [email-masked]
> 
> 
> 
> 

-- 
	And what is good, Phaedrus,
	And what is not---
	Need we ask anyone to tell us these things?

[ PGP public key: http://www.iki.fi/jonas/pubkey.asc ]

---

From: Herbert Fruchtl <[email-masked]>
Date: Wed, 07 Feb 2001 16:36:30 +0000

> (ReiserFS) takes about 2-3 seconds :-)

Does anybody have performance information about this? What's the penalty 
compared to ext2? Does it run on any old EIDE, or are there hardware
requirements? How does compiler support look like (you probably have to 
have 64-bit file pointers)?

Thanks in advance,

  Herbert

---

From: Wolf-Dietrich Ihlenfeldt <[email-masked]>
Date: Wed, 07 Feb 2001 18:18:47 +0100

> (ReiserFS) takes about 2-3 seconds :-)

This statement is not correct. We operate a 3.5 GB 
single file database successfully under Linux2.4 
on an ext2 filesystem.

However, there are a number of issues which we observed
in our SuSE7.0 distribution setup (some of these
problems may be resolved in newer releases, SuSE 7.0 contains
a prerelease 2.4 kernel)

a) quota support is broken
b) regardless of user limit settings, only root can actually write
files larger than 2 Gb. Otherwise, a file size exceeded
signal will be generated and no bytes are written.
c) ftell64() will return negative position values if the
file pointer is beyond 2Gb, regardless of compilation flags.
Add 4Gb to compensate if the value is negative (and of course
use int64_t/off64_t vars for this kind of arithmetic). I don't
know what happens if your file is larger than 4Gb.
d) Accroding to our measurements, 
the performance of mmap64()ed memory from
extended segments of a big file is atrocious 
and leads to severe paging, even if only
a small section of the mapped area is actually touched.
The vmem memory page swapping mechanisms desparately require 
improvements for this kind of application 
(IRIX does much better with the same software)

-- 
Dr. Wolf-D. Ihlenfeldt
Computer Chemistry Center, Institute for Organic Chemistry,
University of Erlangen-Nuremberg
Naegelsbachstrasse 25, D-91052 Erlangen (Germany)
Tel (+49)-(0)9131-85-26579  Fax (+49)-(0)9131-85-26566
http://www2.ccc.uni-erlangen.de/wdi/

---

From: Serguei Patchkovskii <[email-masked]>
Date: Wed, 7 Feb 2001 13:43:30 -0700 (MST)

On Wed, 7 Feb 2001, Jonas Juselius wrote:
> files larger than 2GB. 

This is most certainly false. ext2 does support files larger than 2Gbytes;
moreover, such files have been supported since at least 2.2 kernels:

# dd if=/dev/zero of=xxx seek=4096k bs=4k count=1
1+0 records in
1+0 records out
# ls -l xxx
-rw-r--r--    1 ps       ftp      17179873280 Feb  7 13:58 xxx
# uname -a
Linux CeNsOrEd 2.2.14-6.0 #1 Tue Mar 28 16:56:56 EST 2000 alpha unknown
# df -T .
Filesystem    Type   1k-blocks      Used Available Use% Mounted on
/dev/sda4     ext2     2080939        51   1973317   0% /usr2

/Serge.P

---
Home page: http://www.cobalt.chem.ucalgary.ca/ps/

---

From: Hannes Loeffler <[email-masked]>
Date: 07 Feb 2001 22:17:03 +0100

> Linux CeNsOrEd 2.2.14-6.0 #1 Tue Mar 28 16:56:56 EST 2000 alpha unknown
                                                            ^^^^^
Yes, it works on Linux/Alpha but not Linux/i386.

> /dev/sda4     ext2     2080939        51   1973317   0% /usr2

Hannes.