Parallel transfers with sftp (call for testing / advice)

[Darren Tucker]

On Thu, 9 Apr 2020 at 01:34, Cyril Servant <cyril.servant at gmail.com> wrote:
[...]
> Each of our front
> nodes has an outgoing bandwidth limit (let's say 1Gb/s each, generally more
> limited by the CPU than by the network bandwidth),

You might also want to experiment with the Ciphers and MACs since
these can make a significant difference in CPU utilization and, if
that's the bottleneck, your throughput.  Which one is best will vary
depending on your hardware, but it's likely to be either AES GCM if
the hardware has AES instructions or chacha20-poly1305 if not.

In the first example below the bottleneck is the source's relatively
elderly 2.66GHz Intel CPU.  In the second it's the gigabit network
between them.

$ scp -c aes256-ctr -o macs=hmac-sha2-512
ubuntu-18.10-desktop-amd64.iso.bz2 nuc:/tmp/
ubuntu-18.10-desktop-amd64.iso.bz2            100% 1899MB  63.5MB/s   00:29

$ scp -c chacha20-poly1305 at openssh.com
ubuntu-18.10-desktop-amd64.iso.bz2 nuc:/tmp/
ubuntu-18.10-desktop-amd64.iso.bz2            100% 1899MB 112.1MB/s   00:16

-- 
Darren Tucker (dtucker at dtucker.net)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA (new)
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.