[EXTERNAL] Re: Controlling SO_RCVBUF

[Robinson, Herbie]

>On Thu, Dec 12, 2019 at 12:00:40AM +0000, Robinson, Herbie wrote:

> > I have a customer who is complaining about slow SFTP transfers over a 
> > long haul connection.  The current transfer rate is limited by the TCP 
> > window size and the RTT.  I looked at HPN-SSH, but that won't work 
> > because we don't control what software the peer is using.  I was 
> > thinking about coding a much more modest enhancement that just does 
> > SO_RCVBUF for specific subsystems.  In the interest of adding 
> > something that the OpenSSH community would take back into the source 
> > base, what do people think is a better fit in the configuration file?

David Wilson replied:

> It is possible to simply tweak the kernel default instead, at least on Linux (tcp_rmem). Is this not workable in your case?

It's not Linux, but there is a mechanism and we have used it to get them a small performance boost.  Unfortunately, the customer only wants the large window size for the sftp upload, not all connections.  They are looking at window sizes somewhere between 250K and 500K to get the speed they really want.  They would overload the long haul line if they did that for all connections.  


>On Thu, Dec 12, 2019 at 12:00:40AM +0000, Robinson, Herbie wrote:

> > A single keyword for specifying connection options:
> >
> > SubsystemOptions <subsystem-name> [command-line-option...]
> >
> > The subsystem name would be scp, sftp-server, etc.  The command line options would be -r <n> or --rcvbuf <n> to allow one to specify the number passed to SO_RCVBUF.  This would allow one to easily add other options (like SO_SNDBUF support).
> >
> > An alternative would be to add the following option:
> >
> > SubsystemRcvbuf <subsystem-name> <n>
> >
> > This would be much easier to implement, but doesn't allow for more options to be added without another keyword. I guess this is the way I'm leaning, but I can be swayed.

Darren Tucker Replied:

> Sending, receiving or both?  

Server (sftp-server) receiving.

> What's the speed and RTT on the link?

The RTT is between 40 and 50 msec.  The window when the customer measured was 32K.  The throughput they saw was 6mbit/sec (which implies an RTT of 44 msec).  We worked with them to adjust kernel parameters so they get 65K windows and that's kept them happy in the short term, but they also indicated they were hoping for 100mb; so, I'm trying to get ahead of them before they come back for more.

> If your limiting factor is really the TCP BDP that's between the ssh and sshd process, and that's independent of subsystem.

Correct.  The actual TCP socket is not available to the subsystem.  If it was, I could just add options to the sftp-server code.  Given that upping the window size to a really big number is not a good idea for all connections, I wanted to have sshd decide what to use based on the subsystem.  Perhaps it might also be useful to select the window size based on input and/or output subnet masks, too.  Opinions?

> There is no scp subsystem.

I guess I'm taking a little poetic license there.  If it turns out to be difficult to include scp, as a selector, I would just drop it.

> If you're talking about the client side you could implement whatever you want in a ProxyCommand dialer then possibly pass the open fd back via ProxyUseFdpass.

I am looking to address a server issue.