behaviour of ssh/scp over flakey links - timeout/retry?

[Dan Kaminsky]

Carl:

    I've honestly never had the best of luck with scp.  In my experience,
command forwarding the tar command is the fastest, most cross-platform(even
on Windows, using the Cygwin OpenSSH daemon) method of moving files.

    Try this:

    # For unpacked files on the backup host
    alicehost$  ssh alice at bobhost "tar -cf - /path" | tar -xf -
    # To get the tarball itself
    alicehost$  ssh alice at bobhost "tar -cf - /path" > /path/bobhost.tar
    # Slight variant -- send a tarball somewhere else
    bobhost$  tar -cf - /path | ssh bob at alicehost "cat > /path/bobhost.tar"

    Now, that being said, it sounds like the real problem is that the TCP
session dies on occasion.  There are a couple solutions to this, some of
which I'm still working on:

    1) Use a transfer system that can handle incremental(and thus resumable)
updates.  Rsync comes to mind.  Make sure Keepalives are enabled("ssh -o
KeepAlive Yes", or modify your /etc/ssh.conf) so timeouts will happen
quicker, then have your script go back and rsync again if rsync returns an
error code.  (It won't upon a successful syncing.)  Do something like:

    alicehost$  rsync -e "ssh -o KeepAlive yes" alice at bobhost:/path
/path/bobhost/
    or
    bobhost$  rysnc -e "ssh -o KeepAlive yes" /path
bob at alicehost:/path/bobhost

    2) Add a TCP Reliability Layer.  ROCKS, available at
http://www.cs.wisc.edu/~zandy/rocks/ ,handles links that are...ah..."less
than stable".  Quoting the description:  "Rock-enabled programs continue to
run after any of these events; their broken connections recover
automatically, without loss of in-flight data, when connectivity returns.
Rocks work transparently with most applications, including SSH clients,
X-windows applications, and network service daemons."  You'll almost
certainly need to disable SSH keepalives for this to work, but the
reliability layer will almost certainly handle even extended network
outages.

    I haven't tested ROCKS at *all*, but I'll be doing so shortly.

    You might find yourself missing, well, the status updates that you get
with scp.  cpipe, available at http://wsd.iitb.fhg.de/~kir/cpipehome/ , is a
nice general purpose tool for monitoring the speed of a pipe.

    Lemme know if any of this helps; I'm working on stuff related to this
right now.

Yours Truly,

    Dan Kaminsky
    DoxPara Research
    http://www.doxpara.com