sftp reget/reput

[Dmitry Lohansky]

Hello openssh@

I thought about sftp's reget/reput commands.

Several days ago, Damien Miller write to tech at openbsd.org (it was
reply for my letter):

> Herein lies a problem which is not easy to detect or solve. For
> performance reasons, the sftp client does pipelined reads/writes when
> transferring files. The protocol spec allows for a server to process
> these requests out of order. For example:

> client                     server
> ------                     ------
> open file                  your file handle is "blah"
> gimme bytes 0-8191
> gimme bytes 8192-16383
> gimme bytes 16384-24575
> gimme bytes 24576-32767    here are bytes 24576-32767
> close file                 here are bytes 16384-24575
>                            here are bytes 8192-16383
>                            here are bytes 0-8191
>                            close successful

> If the client writes the bytes our in the order they are received (which
> it probably should, to avoid buffering large amounts of data) then an
> interruption will leave a full-length, but "holey" file on disk. There
> is no general way to determine how to do resume such a transfer.

> The best the client can do to make transfers resumable is ftruncate()
> the file at the highest contiguous byte received. This will stop the
> potential corruption on resume.

This is good method, but if client crash, we also may get a "hole".
What your think about next way?

Storing extra-data at the end of file, for example:

<---orig-part-><-extra->
[*][ ][*][ ][*][*******]
<---------file--------->

where [*] - already loaded data, [ ] - not yet

In extra part, we may store which block was already loaded and it
offset and size. After download, extra part will be removed.

Comments?
-- 
 Dmitry Lohansky