High Performance SSH/SCP - HPN-SSH when?

[Chris Rapier]

Darren Tucker wrote:

> I think this is another symptom of the HPN patch letting the buffers get
> way too big under some conditions, then ssh spents a disproportionate
> amount of time compacting those buffers.

Assuming this is true it doesn't address the performance issue with an 
unmodified OpenSSH under cygwin. This was the first issue brought up and 
as such doesn't have anything to do with the HPN patches. I'm not trying 
to deflect things here just trying to make sure the original issue 
doesn't fall by the way side.

> Assume the the 8k writes are relatively slow on Cygwin (which appears to
> be the case[1]).  ssh will be emptying the output buffer relatively
> slowly, but the CPU can encrypt much faster than the IO rate.  Normally
> the buffer would peak at 5-10 MB under these conditions, but the
> BUFFER_MAX_HPN_LEN change means that they can grow really big (up to
> 2^29 bytes).  The buffer gets compacted at 1MB consumed, so the process
> becomes "read 1MB in 8k chunks then memmove (2^29 - 1M).

I will try find out exactly how big the buffer is actually getting in 
this. I just need to identify exactly which buffer I need to be paying 
attention to :) I'll also just drop MAX_HPN_LEN to 10MB and see what 
happens.

> The solution is to not write more than, say, 1-2 x TCP window size to
> the output buffer.  If you're IO bound on the source, you'll never write
> more than 1 x TCPWINSZ to the socket and have plenty of CPU to fill it
> up again.  If you're CPU bound, you won't spend extra time compacting
> the buffers so it will be a small performance improvement.

I think I'm understanding what you are saying here better now. My 
apologies for not getting it quicker. What about in situations where the 
tcp window size is not a constant - like under Linux 2.6 and Vista?