ssh / scp slow on 10GBE

[Iain Morgan]

On Wed, Jan 02, 2013 at 16:29:14 -0600, Stefan Priebe wrote:
> Hello list,
> 
> right now SSH Tunnel / scp is reaches just around 76Mb/s on my E5 Xeon 
> using AES-NI but openssl reaches around 600-700Mb/s using 128aes-cbc cipher.
> 
> As far as i understand http://www.psc.edu/index.php/hpn-ssh this is due 
> to very small buffers in ssh / scp.
> 
> Is there any work on this? Like autotuning the buffer size? Are there 
> plans to integrate the hpn patches?
> 
> Greets,
> Stefan

I think you have a typo above. You presumably mean 600-700 MB/s in the
case of OpenSSL using AES-NI, but do you mean MB/s or Mb/s in the scp
case?

How are you measuring the ssh/scp performance? Is this over a LAN or WAN
connection?

The issue which the HPN patch addresses is the sizing of the channel
buffer. This can be helpful in the case of high-latency (strictly, high
BDP) connections. If you are truly getting 76 Mb/s (i.e. 9-10 MB/s) and
the transfer is over a high-latency (WAN) connection, then the HPN patch
may be of benefit.

On the other hand, if you are getting 76 MB/s, then you may have some
other bottlenecck to address. You might try doing a dd through ssh to
factor out scp and disk I/O. For example,

% dd if=/dev/zero bs=1048576 count=4096 | ssh somehost dd of=/dev/null

As to adding something like the HPN functionality to stock OpenSSH, I'll
leave that to the OpenSSH developers to comment on. I believe that
OpenBSD added TCP autotuning (or something roughly analogous) to their
kernel a few releases ago. If that is the case, it would seem to
increase the likelihood of something similar to the HPN dynamic
windowing patch being added to OpenSSH. 

-- 
Iain Morgan