[PATCH] Improving sftp client performance

[Dan Kaminsky]

> Doing multiple ftp streams is what you are refering to.  That could be
> done, but it does not improve a single file transfer.  I think I would
> perfer to get this working first and then look at if we want to improve
> the "mget *" or "mput *" case.
>
> Because that code would be at a higher level in some respects.

The direct equivalent to what happens in the FTP realm is for multiple ssh
sessions to be established with multiple sftp-server's, each taking
individual files as fast as possible.

SFTP shouldn't require such inelegance.  Instead of:

1) Choose file
2) Send a windowed series of blocks from that one file, until there are no
blocks remaining
3) Choose next file

we should be able to:

1) Choose file
2) Send a windowed series of blocks from that one file
3) If there are no further blocks available in this file, choose next file
and begin sending blocks from that.  Repeat ad infinitum until window size
is exhausted

There's a real elegance to this solution:  For large files, we remain
completely sequential, avoiding the useless and counterproductive
interleaving that multi-thread FTP tends to create.  But for small files, we
have no downtime between each transfer.

I definitely grant that this is of less import than simply getting
individual file transfers to go at a reasonable speed.  But it's not useless
to be able to transfer many individual files quickly -- not in the least.

--Dan