sftp reget/reput

Dmitry Lohansky sq at oganer.net
Wed Sep 17 13:12:36 EST 2003

Hello openssh@

I thought about sftp's reget/reput commands.

Several days ago, Damien Miller write to tech at openbsd.org (it was
reply for my letter):

> Herein lies a problem which is not easy to detect or solve. For
> performance reasons, the sftp client does pipelined reads/writes when
> transferring files. The protocol spec allows for a server to process
> these requests out of order. For example:

> client                     server
> ------                     ------
> open file                  your file handle is "blah"
> gimme bytes 0-8191
> gimme bytes 8192-16383
> gimme bytes 16384-24575
> gimme bytes 24576-32767    here are bytes 24576-32767
> close file                 here are bytes 16384-24575
>                            here are bytes 8192-16383
>                            here are bytes 0-8191
>                            close successful

> If the client writes the bytes our in the order they are received (which
> it probably should, to avoid buffering large amounts of data) then an
> interruption will leave a full-length, but "holey" file on disk. There
> is no general way to determine how to do resume such a transfer.

> The best the client can do to make transfers resumable is ftruncate()
> the file at the highest contiguous byte received. This will stop the
> potential corruption on resume.

This is good method, but if client crash, we also may get a "hole".
What your think about next way?

Storing extra-data at the end of file, for example:

[*][ ][*][ ][*][*******]

where [*] - already loaded data, [ ] - not yet

In extra part, we may store which block was already loaded and it
offset and size. After download, extra part will be removed.

 Dmitry Lohansky

More information about the openssh-unix-dev mailing list