High Performance SSH/SCP - HPN-SSH when?

[Corinna Vinschen]

On Apr  5 18:23, Chris Rapier wrote:
> Darren Tucker wrote:
> > The existing buffer limit is 10MB so unless you have paths with a
> > BDP/TCPWINSZ of 10MB or higher (do you?) then I think you'll see your
> > throughput go up a little or your CPU and memory utilization go down a
> > little.
> 
> I've changed this to 32MB. This should handle up to 3.2Gbps at 80ms RTT 
> - which is a little low for some applications. However, I think I have a 
> way to handle this later on.
> 
> > At the moment, you do this in channel.c:
> > -limit = MIN(limit, (BUFFER_MAX_LEN - BUFFER_MAX_CHUNK - CHAN_RBUF));
> > +limit = MIN(limit, (BUFFER_MAX_HPN_LEN-BUFFER_MAX_CHUNK-CHAN_RBUF));
> > 
> > Instead, I suggest leaving the existing line and adding something like:
> > 	limit = MIN(limit, gettcpwinsz());
> > or
> > 	limit = MIN(limit, 2 * gettcpwinsz());
> 
> I made this change. It only checks multiple times if the option to do 
> tcp buffer polling is enabled. It stores the channels associated 
> connections window size in c->tcpwinsz now.
> 
> I also cleaned up some other stuff in channel.c for the sake of clarity 
> and cleaned up some logic errors in other locations.
> 
> This latest version of the patch can be found at
> http://www.psc.edu/networking/projects/hpn-ssh/hpn12v5.diff.gz
> Usage notes can be found at
> http://www.psc.edu/networking/projects/hpn-ssh/ssh-hpn12.notes
> 
> Let me know if anyone finds anything out vis a vis performance. I didn't 
> have the time to test it for anything except functionality today. Its 
> been an annoyingly busy day.

I tested your patch and I don't see a difference in relation to the
previous hpn version as far as performance is concerned.

The performance for the slowest copy:

  cygwin> scp linux:file .

is still slower than without the hpn patch,

  file   100%  118MB   1.0MB/s   01:58

vs.

  file   100%  118MB   1.3MB/s   01:31

The curious thing here is that the CPU usage is very low in both cases,
it only takes about 2% CPU.

But I also have good news for the hpn patch.  In my tests last week I
forgot to test the hpn patch together with raised buffer sizes in
clientloop.c.  

I raised the buffer sizes in clientloop.c from 8192 to 87380 bytes.  As
I already noted, the vanilla version of OpenSSH then behaves much better,
for instance:

  file   100%  118MB   5.9MB/s   00:20

However, with 87380 bytes buffers the hpn patch suddenly behaves much
better than with 8192 bytes buffers.  It is now roughly 80-90% faster
than the vanilla version with 87380 bytes buffers:

  file   100%  118MB   9.9MB/s   00:12

I also noticed that in this configuration the CPU usage climbs to an
average of 30%, while without the hpn patch it sticks at 2%.

However, I also installed a hpn sshd on my Cygwin box and tried the copy
initiated from the other direction:

  linux> scp file cygwin:

The result for the vanilla sshd:

  file   100%  118MB   9.1MB/s   00:13

The result for sshd with the hpn patch:

  file   100%  118MB   2.0MB/s   00:59

Looks like another buffer might be in the way now, but I didn't look
into this deeper so far.

Note that all tests are using a vanilla OpenSSH on the Linux side.
I didn't want to add another variable to the test configuration.


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat