spc restarts

[Salvador Fandino]

On 02/26/2012 06:09 PM, Ben Lindstrom wrote:
> 
> On Feb 26, 2012, at 9:02 AM, Mark Anderson wrote:
> 
>> I was wondering why scp didn't have a restart-in-the-middle
>> option when transferring a large file.  Isn't that something
>> that is supported by the underlying ssh protocol?  Is there
>> some other program that can give me this functionality ?
> 
> Scp came from rcp.. Which is a stupid simple(ish) protocol
> more like doing:  tar -cvf | ssh tar -xvf.  So it really has nothing
> to do with the ssh protocol itself as it just needs a streaming
> socket to throw data at.
> 
> I suspect you're thinking sftp service, and that could be easily 
> extended to support it.  I did a cheap proof of concept that 
> required adding an sha1 hash request extension to the server 
> so I could determine where I needed to pick up in the transfer.  
> It really was an attempt to implement a simplified rsync over sftp.

You don't really need a hashing extension in order to support transfer
resuming. You can make a good heuristic just comparing remote and local
timestamps and file sizes. For instance, for "put":

  if local_ts > remote_ts:
    do full transfer
  else:
    if local_size <= remote_size: done
    else: resume transfer from remote_size

That works reliably as far as the remote files are not modified in other
ways.

SFTP clients supporting that that I know are lftp and my perl module
Net::SFTP::Foreign.