High Performance SSH/SCP - HPN-SSH when?

[Darren Tucker]

Chris Rapier wrote:
> 
> Darren Tucker wrote:
> 
>> I think this is another symptom of the HPN patch letting the buffers get
>> way too big under some conditions, then ssh spents a disproportionate
>> amount of time compacting those buffers.
> 
> Assuming this is true it doesn't address the performance issue with an 
> unmodified OpenSSH under cygwin.

That's true, I just didn't want people spending time analyzing (what I
think is) the wrong thing.

[...]
> I will try find out exactly how big the buffer is actually getting in 
> this. I just need to identify exactly which buffer I need to be paying 
> attention to :) I'll also just drop MAX_HPN_LEN to 10MB and see what 
> happens.

The existing buffer limit is 10MB so unless you have paths with a
BDP/TCPWINSZ of 10MB or higher (do you?) then I think you'll see your
throughput go up a little or your CPU and memory utilization go down a
little.

>> The solution is to not write more than, say, 1-2 x TCP window size to
>> the output buffer.  If you're IO bound on the source, you'll never write
>> more than 1 x TCPWINSZ to the socket and have plenty of CPU to fill it
>> up again.  If you're CPU bound, you won't spend extra time compacting
>> the buffers so it will be a small performance improvement.
> 
> I think I'm understanding what you are saying here better now. My 
> apologies for not getting it quicker. What about in situations where the 
> tcp window size is not a constant - like under Linux 2.6 and Vista?

At the moment, you do this in channel.c:
-limit = MIN(limit, (BUFFER_MAX_LEN - BUFFER_MAX_CHUNK - CHAN_RBUF));
+limit = MIN(limit, (BUFFER_MAX_HPN_LEN-BUFFER_MAX_CHUNK-CHAN_RBUF));

Instead, I suggest leaving the existing line and adding something like:
	limit = MIN(limit, gettcpwinsz());
or
	limit = MIN(limit, 2 * gettcpwinsz());

That way the buffer utilization would scale with your window size.

BTW hpn12 doesn't seem to be on your web page?

To Corinna's question about buffer sizes: I personally would have no
objection to making it 64k, possibly with a #define for making it
smaller for folks targeting old/embedded systems and such.  I suspect it
would be a minor performance gain on most platforms.

If a similar bottleneck applies to pipe/socketpairs, it may also be work
increasing the IO buffer in scp and seeing if that helps your
performance (the HPN patch bumps it from 2k to 16k, you may want to bump
it to 64k).

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.