Extensions for long fat networks?

[Dan Kaminsky]

Paul Hyder wrote:

>Before I get too far in my attempts...
>
>Has anyone already implemented support in scp for larger
>buffers/windows that would take advantage of wscaled TCP
>windows?
>	Paul Hyder
>	NOAA Forecast Systems Lab
>	Boulder, CO
>
>FYI:  Linux 2.4.20, 30-80ms RTT, data rates 100-1000Mbps,
>and a need to fill TCP windows of 2-8MBytes.
>(Existing limits appear to be about 256KB.)
>
>_______________________________________________
>openssh-unix-dev mailing list
>openssh-unix-dev at mindrot.org
>http://www.mindrot.org/mailman/listinfo/openssh-unix-dev
>  
>
Paul--

    scp isn't particularly optimized for speed...you're likely to get 
better performance using a wrapper around tar/star and ssh.  This is 
usually implemented like so:

    tar cf - files directories | ssh user at host tar xf -

    This seems to provide the fastest transfers over SSH, though the TCP 
forwarding code is pretty speedy as well.  Just to pre-answer a 
question, ssh decapsulates and re-encapsulates TCP, so you don't have 
classic TCP-over-TCP issues.

    For pure speedy transfers, you should examine udpcast.  On a 100mbit 
LAN, udpcast is able to *reliably multicast* at *93.65Mbit*.  There's 
presently no crypto implemented within it, but I've been looking at 
integrating gpg support.  (One can already simply add gpg to the 
pipeline, but some buffering issue drops transfer speed dramatically.)  
udpcast abandons tcp entirely and moves error handling into userspace -- 
with astonishingly effective results.  It would require your WAN to 
support multicast traffic, though, and it's notably still too immature 
to even really handle two udpcasted streams in the same network.

    Of course, if you'd like to make ssh run as fast as its crypto 
algorithms could let it -- it'd be most helpful :-)

Yours Truly,

    Dan Kaminsky
    DoxPara Research
    http://www.doxpara.com