ssh / scp slow on 10GBE

[Iain Morgan]

On Fri, Jan 04, 2013 at 05:44:46 -0600, Stefan Priebe wrote:
> Hi,
> 
> Am 03.01.2013 20:19, schrieb Iain Morgan:
> > I think you have a typo above. You presumably mean 600-700 MB/s in the
> > case of OpenSSL using AES-NI, but do you mean MB/s or Mb/s in the scp
> > case?
> uups i mean MB in both cases.
> 
> > How are you measuring the ssh/scp performance? Is this over a LAN or WAN
> > connection?
> LAN
> 
> > On the other hand, if you are getting 76 MB/s, then you may have some
> > other bottlenecck to address. You might try doing a dd through ssh to
> > factor out scp and disk I/O. For example,
> >
> > % dd if=/dev/zero bs=1048576 count=4096 | ssh somehost dd of=/dev/null
> #~ dd if=/dev/zero bs=1048576 count=4096 | ssh -c aes128-cbc 10.255.0.24 
> dd of=/dev/null
> 4096+0 records in
> 4096+0 records out
> 4294967296 bytes (4,3 GB) copied, 19,3355 s, 222 MB/s
> 8388608+0 records in
> 8388608+0 records out
> 4294967296 bytes (4,3 GB) copied, 18,6107 s, 231 MB/s
> 
> But:
> # iperf -c 10.255.0.24 -t 30 -d
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> ------------------------------------------------------------
> Client connecting to 10.255.0.24, TCP port 5001
> TCP window size:   990 KByte (default)
> ------------------------------------------------------------
> [  4] local 10.255.0.22 port 44006 connected with 10.255.0.24 port 5001
> [  5] local 10.255.0.22 port 5001 connected with 10.255.0.24 port 37393
> [ ID] Interval       Transfer     Bandwidth
> [  4]  0.0-30.0 sec  34.6 GBytes  9.90 Gbits/sec
> [  5]  0.0-30.0 sec  33.9 GBytes  9.70 Gbits/sec
> 
> and
> openssl aes-128-cbc speed:
> 
> The 'numbers' are in 1000s of bytes per second processed.
> type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 
> bytes
> aes-128-cbc     648664.33k   688924.90k   695855.45k   700784.64k 704027.06k
> 
> Greets,
> Stefan
> 

As Tomas noted, the choice of MAC also impacts performance. The default
is hmac-md5, but umac-64 at openssh.com can provide significantly better
performance.

I'm assuming that the number you quoted in your original email was from
an scp transfer. Be aware that it uses fairly coarse time granularity in
its calculations, so transfers of small files can report deceptive
values. You may also want to check the disk I/O performance on both ends
of the transfer.

-- 
Iain Morgan