ssh / scp slow on 10GBE
imorgan at nas.nasa.gov
Fri Jan 4 06:19:07 EST 2013
On Wed, Jan 02, 2013 at 16:29:14 -0600, Stefan Priebe wrote:
> Hello list,
> right now SSH Tunnel / scp is reaches just around 76Mb/s on my E5 Xeon
> using AES-NI but openssl reaches around 600-700Mb/s using 128aes-cbc cipher.
> As far as i understand http://www.psc.edu/index.php/hpn-ssh this is due
> to very small buffers in ssh / scp.
> Is there any work on this? Like autotuning the buffer size? Are there
> plans to integrate the hpn patches?
I think you have a typo above. You presumably mean 600-700 MB/s in the
case of OpenSSL using AES-NI, but do you mean MB/s or Mb/s in the scp
How are you measuring the ssh/scp performance? Is this over a LAN or WAN
The issue which the HPN patch addresses is the sizing of the channel
buffer. This can be helpful in the case of high-latency (strictly, high
BDP) connections. If you are truly getting 76 Mb/s (i.e. 9-10 MB/s) and
the transfer is over a high-latency (WAN) connection, then the HPN patch
may be of benefit.
On the other hand, if you are getting 76 MB/s, then you may have some
other bottlenecck to address. You might try doing a dd through ssh to
factor out scp and disk I/O. For example,
% dd if=/dev/zero bs=1048576 count=4096 | ssh somehost dd of=/dev/null
As to adding something like the HPN functionality to stock OpenSSH, I'll
leave that to the OpenSSH developers to comment on. I believe that
OpenBSD added TCP autotuning (or something roughly analogous) to their
kernel a few releases ago. If that is the case, it would seem to
increase the likelihood of something similar to the HPN dynamic
windowing patch being added to OpenSSH.
More information about the openssh-unix-dev