rsync over ssh - lockups with pipe()

[Ben Lindstrom]

[Removed tridge at linuxcare.com, and thanked him for a bit of help with
it on NeXT]

Has anyone tried this with OpenSSH?  I know I ran it for about 5 - 10
minutes and it never deadlocked.  Wondering if anyone else had any
problems.

On Sat, 29 Jan 2000 tridge at linuxcare.com wrote:

> For a couple of years people have been reporting intermittent problems
> with rsync over ssh freezing during a large transfer (typically
> several GB). I have now gotten to the bottom of these problems, and
> have written a small test program which demonstrates a deadlock in
> sshd when doing large bi-directional transfers. Luckily the problem is
> easily solved by modifying sshd to use socketpair() instead of
> pipe(). Just remove "#define USE_PIPES 1" from near the bottom of
> includes.h does the trick.
> 
> The test program, along with instructions for use and an explanation
> of some of the pipe() bugs in various OSes that make this change
> necessary is available from ftp://samba.org/pub/tridge/misc/sshtest.c
> 
> In past releases of rsync I added various buffering tricks in rsync
> which reduced the likelyhood of this problem occuring, but didn't
> eliminate it completely. I have removed those tricks in rsync 2.4.0 so
> you may find an increase in the number of bug reports regarding rsync
> over ssh. I removed the tricks as they caused excessive memory usage
> (to the point of memory starvation) in some cases and are completely
> unnecessary once sshd is modified to use socketpair().
> 
> I think the combination of pipe/select bugs on various OSes means that
> you can't work around this problem by playing with non-blocking pipes
> or the size of writes (except by dropping the size by 1). 
> 
> I'm also amazed at the number of pipe/select bugs in the various OSes
> I tested - Linux 2.2 was particularly bad (fixed in 2.3). I really
> expected pipes to be pretty well defined and bug-free by now!
> 
> Cheers, Tridge
> 
> PS: I am not on this list, I'm just using this address as noted in the
> README.
>