sq at oganer.net
Wed Sep 17 13:12:36 EST 2003
I thought about sftp's reget/reput commands.
Several days ago, Damien Miller write to tech at openbsd.org (it was
reply for my letter):
> Herein lies a problem which is not easy to detect or solve. For
> performance reasons, the sftp client does pipelined reads/writes when
> transferring files. The protocol spec allows for a server to process
> these requests out of order. For example:
> client server
> ------ ------
> open file your file handle is "blah"
> gimme bytes 0-8191
> gimme bytes 8192-16383
> gimme bytes 16384-24575
> gimme bytes 24576-32767 here are bytes 24576-32767
> close file here are bytes 16384-24575
> here are bytes 8192-16383
> here are bytes 0-8191
> close successful
> If the client writes the bytes our in the order they are received (which
> it probably should, to avoid buffering large amounts of data) then an
> interruption will leave a full-length, but "holey" file on disk. There
> is no general way to determine how to do resume such a transfer.
> The best the client can do to make transfers resumable is ftruncate()
> the file at the highest contiguous byte received. This will stop the
> potential corruption on resume.
This is good method, but if client crash, we also may get a "hole".
What your think about next way?
Storing extra-data at the end of file, for example:
[*][ ][*][ ][*][*******]
where [*] - already loaded data, [ ] - not yet
In extra part, we may store which block was already loaded and it
offset and size. After download, extra part will be removed.
More information about the openssh-unix-dev