OpenSSH GSoC Project

[Salvador Fandino]

Jim Knoble wrote:
> On 2009-03-23 16:05, Pavel Labath wrote:
> 
> : So, what are the features we want a polished sftp to have?
> 
> The best interactive terminal-based SFTP client i've used so far is LFTP
> <http://lftp.yar.ru/>, by opening an URL like 'sftp://remote-host'.

lftp is also my preferred SFTP (and FTP) client. One of these utilities 
I install on any machine I have to work with (if I can).

> : The only thing mentioned so far is the recursive up/down load, which
> : looks more like a thing for a two day hackfest than a all-summer-long
> : $4.5k operation.
> 
> Perhaps i misunderstand, but i think the intent of the GSoC description
> is to make sftp(1) a potential replacement for scp(1), insofar as
> non-interactive invocation from the shell prompt is concerned.  Example:
> 
>     scp -p -r ./some_dir remote-host:
> 
> Currently, sftp(1) understands neither '-p' nor '-r'.  Obviously,
> sftp(1) currently has some conflicting options, and it will be necessary
> to balance compatibility with scp(1) against compatibilty with prior
> versions of sftp(1).
> 
> It may be that the best way to do some of this would be the same way
> F-Secure's SSH does it:  use scp2(1), which would eventually completely
> replace scp(1).

SFTP also has some drawbacks:

- it is a talky protocol and so it doesn't perform too well on networks 
with (relatively) high latencies (though, it seems that the packet queue 
and packet size has been increased in latest OpenSSH versions, so this 
may not be the case anymore).

- it is complex, there is no RFC, and implementations are based on the 
currently available and not completely compatible drafts. 
Interoperability is sometimes difficult.

On the other hand...

- even if there is no RFC for SCP, there is a reference implementation 
(rcp), and the protocol is so simple that it can be inferred just 
inspecting a couple of sessions.

- It is very efficient when transferring large files. After some 
handshake, it just sends the file data over the ssh channel without any 
further protocol overhead or talk.

- It is not unusual for administrators to disallow the sftp server as it 
gives too much power to the client, and allow remote file copy via scp only.

Summarizing, IMO there is place for both protocols: SCP being simple, 
efficient and very interoperable and SFTP, allowing for more complex 
operations but with all the drawbacks that the higher complexity introduces.

> That said, i often use either rsync(1) or combinations of ssh(1) and
> tar(1) to do transfers from a shell command-line.

Something I would find very useful would be some SFTP extension to allow 
implementation of some kind of rsync over SFTP.

> : Features i would consider useful are:
> : - tab completion (both commands and filenames) - very useful, not difficult
> 
> Tab completion is nice.  LFTP provides both local and remote filename
> completion, but remote completion can sometimes take awhile, so beware.
> 
> : - transfer resume (thanks Iain Morgan, I almost forgot) - very useful,
> : not difficult
> 
> Look at LFTP's interface for resuming ("continuing") a transfer.
> 
> : - background transfer (possibly threads, or just fancy select() calls) -
> : useful, not very difficult
> 
> Again, look at LFTP for ideas.

as LFTP already seems to do everything we need (and much more) and it is 
freely available, does really makes sense to reimplement its 
functionality inside sftp? why not just recommend it on the docs?

I think that there are other more interesting areas to work on related 
to SFTP, for instance:

- remove the max packet size limitation for read and write operations 
that forces SFTP clients to send/get data in chunks. File data could be 
moved directly between the network (actually from the slave SSH process 
pipes) and the on disk file without going to a memory buffer first.

- implement an extension to allow rsync over SFTP (ssync :-)

- implement fine grained access control for the SFTP server, limiting 
which SFTP operations are available (for instance, forbidding directory 
reading).

- implement file locking

- implement an extension to tell the client the encoding used on the 
server file system.

- implement an extensible SFTP server in a high level language as Perl 
or Python.

- etc.

Cheers,

  - Salva