File Offsets for SCP (patch revised yet again)

[Philipp Marek]

On Tuesday 07 December 2010, rapier wrote:
> The second is that the messaging window size is a limitation for the
> users I'm supporting (major research institutions usually connected via
> multiple gig or 10Ge connections). Anything that ends up impeding
> maximum performance is going to be problematic (that is why I developed
> the HPN-SSH patch). At some point it may make sense to dynamically grow
> the message window size (because expecting users to set it correctly is
> just not going to work (thats why we did the work on the auto-tuning
> receive buffers)). I may get the funding to do that but I just don't
> know at this point. So right now I'm going with the quick and dirty fix
> :)
> 
> Basically, network performance is where my focus is rather than SSH in
> and of itself.
...
> >> Again, I don't know if this will be useful to anyone else. We're going
> >> to be using this to do parallelized transfers of large (250GB+) files
> >> (using the HPN patch set as well). We're basically trying to avoid
> >> the slow start penalties on very fast throughputs by splitting the
> >> transfer up.
Reading all this I just don't know whether this is just the wrong 
solution... If you want faster startup, how about using a different TCP 
scheduling algorithm?

	http://www-iepm.slac.stanford.edu/bw/tcp-eval/


Having a config option to choose the scheduler might be useful to more 
people, I think - especially as it would do faster transfers automatically, 
without messing around with partial transmissions.
BTW that would lower fragmentation on the destination, too.


Regards,

Phil