OpenSSH GSoC Project

[Pavel Labath]

Iain Morgan wrote:
> While lftp supports resuming, I'm not sure that it addresses the issue
> that the SFTP protocol allows file chunks to arrive out of order.
> Addressing this issue for get's is straightforward (Damien previously
> suggested a patch to sftp to address this) but addressing the issue for
> put's would (presumably) require server-side mods.
If the file for upload is opened in append mode (which isn't possible
yet, but...), then the client would have to send (and the server
process) chunks in sequence. The server can reorder requests, only if
the overall effect remains the same (although I find the wording in the
draft a bit ambiguous). Then it would be up to the client to deliver the
requests in proper order, thereby ensuring that the file doesn't have
"holes" even in case of a system failure. Performance shouldn't be
affected since the current server processes the requests serially anyway
(and I can't imagine, why it would be faster to do them out of order).

Speaking of holes... What about sparse files? I don't know how many
people actually transfer sparse files, but in case the protocol is used
as a filesystem it could happen. Optimizing puts is easy. Gets would
need an extension -- a flag telling that the chunk is empty. Or maybe
just drop it and let ssh's compression deal with it?

Also, I had an idea about decreasing latency on small file transfers.
Right now a small (< chunk size) file needs 3 message-response pairs to
transfer -- open, read/write, close (although the last could be, in
theory, sent even before the second completes). However, if we add a
message "open file X, read/write Y bytes and (possibly) close it" then
it could be done in one round-trip. And one could send a bunch of these
messages together. This works best for upload since you know the size of
the file, but download could benefit as well. On the other hand, it
complicates the protocol (what should be done if open succeeds, but
write fails? ...) so I'm not sure if it's worth it, I'm just
brainstorming here. I suppose testing would be needed.

-- 
Pavel Labath