Subject: Summary addendum :: Uploading PDB files to a CGI script

From: "Robert Fraczkiewicz" <[email-masked]>
Date: Wed, 1 Jul 1998 12:07:44 -0500

Dear Netters,

It seems that Malcolm Gillies has found a perfect explanation of
the "CGI bug" reported to CCL a few days ago. With his permission
and for the sake of summary completeness, I am including his recent
e-mail below.

Robert Fraczkiewicz

--- Forwarded mail from Malcolm Gillies <[email-masked]>

>> I just did a quick test connecting to the httpd via telnet and feeding
>> in text with copy-and-paste, and I can confirm that the cgi process
>> deadlocks in the same way, and in the same place.
>>
>> So I'm fairly confident it's not a problem with the browser.
>
> Hm... You are right, it must be the server. But that's very interesting,
> because you use server from NCSA, and I use one from Netscape (unless
> one is based on the other ;-)

I had another look at the NCSA httpd server source code yesterday, and
as I suspected, it's a design "feature" of the cgi interface. While I
would have to check the CGI spec to see if this is purely an implementation
detail, or an actual requirement, the circumstances are fairly predictable.
The Netscape server is very indirectly descended from the NCSA sever,
I think, so it would also be interesting to see whether this is any
different on a server with a different provenance, such as CERN, or one
of the W3C reference implementations.

As you probably know, communication between the cgi process and the httpd
is done with standard unix pipes. The NCSA httpd performs the cgi call by
first forking off the cgi process, then reading all the POST content
off the network socket, then writing it all to the pipe attached to the cgi
process, and only then, reading the output of the cgi off the other
pipe and writing it to the network socket.

Given that pipes only have a certain amount of buffering, this means
that interleaving input and output in the cgi process will result in
the buffer of the output pipe filling up (as the httpd process is
not reading from it -- it's still busy at the stage of writing the
POST data to the cgi's input pipe), and therefore the 'write' system
call in the cgi (e.g. called by putchar) blocking. Once the write
blocks, then the cgi is stuck, and things don't go any further, until
the server times out and tries to kill the cgi process.

I can envisage alternative server architectures where I/O to the
cgi is multiplexed using the select system call, or where the server
forks off separate processes for input and output to the CGI, but
given the extra overhead and effort, I have my doubts as to whether
these have been implemented.

So, the upshot appears to be: it's best to read all input before
generating any output, at least if you want to be compatible with
the NCSA and Netscape httpds (and quite probably a number of others,
I would suspect).

cheers,

Malcolm

---End of forwarded mail from Malcolm Gillies <[email-masked]>