SFTP and outstanding requests

[Damien Miller]

On Thu, 26 Apr 2007, Chris Rapier wrote:

> Damien,
> 
> Well, currently the default shouldn't be a problem in most any situation. As
> far as I can tell it pretty much acts like a flow control buffer with (by
> default) a 512K window (16x32k). So when you layer that on top of SSH's flow
> control and on top of TCP's as long as its not the minimum value it shouldn't
> act as a bottleneck.
> 
> It does end up being problematic with the HPN patch (boosting it to 256 gave
> me 32MB/s while before it was maxing out at 4MB/s) but I can address that,
> probably, with user education. There is another, more complicated, method that
> wouldn't tax low memory systems, but I'm not even going to think about that
> until I get back.

Did you measure the memory impact of increasing it? I'm not averse to
cranking it up if it improves performance and doesn't have too much of an
effect on memory use.

> What I am curious about, and maybe you can help point me to the right
> portion of the code, is what happens when transferring multiple files
> in SFTP (SCP as well). If you look at outstanding data graphs (derived
> by tcptrace from a tcpdump) it seems that between each file there is
> something happening that causes the network to drain completely and
> then there is a 2RTT pause before the next file gets sent out. I can
> put a copy of the data somewhere if you want to look at it. If I can
> get a better understanding of what is happening there I can at least
> explain to my users why they should do a tar pipe if they have many
> small files.

There is a pipeline stall between each file because of the current client
implementation. To fix this, sftp-client.c:do_(up|down)load really
needs to be modified to accept a vector of files rather than a single
file at a time.

The other point where we waste round-trips in sftp is globbing. There
are lots of round-trips there and (worse) the glob implementation we
use throws away the Attrib data, which we then have to refetch (another
round-trip per file).

A while back, I posted these as part of a list of TODOs for sftp. It's
a pity I wasn't more organised earlier in the year because they would
have been excellent projects for Google's Summer of Code.

-d