New Set of High Performance Networking Patches Available

[Darren Tucker]

Chris Rapier wrote:
> Darren Tucker wrote:
[hpn patch]
>> I've been testing with tunbridge[1] on OpenBSD to add latency.  I've 
>> seen an improvement of around 50% throughput on scp with 100ms of 
>> latency (each way, ie 200ms rtt) simulated link with Linux endpoints.
> 
> In the real word test environments we have set up we're commonly seeing 
> improvements of 10 to 30x greater throughput. 25MBytes/s+ where before 
> we were seeing less than 1Mbyte/s.
> 
>> Using -w doesn't seem to make any difference (or sometimes it's a net 
>> loss) although it's quite possible something in my test environment is 
>> responsible for that.  (Yes, I did the stack tuning, both netstat and 
>> getsockopt show the buffers are 1MB or more.)
> 
> It will only make a difference when the client is acting as the data 
> sink. Obviously the max buffer size has to be large enough to handle the 
> user defined setting.

I can see 1+ MB sitting in the server's TCP send queue.  I suspect it's 
some local problem limiting TCP throughput in the high-BDP configuration 
(they're not super beefy hosts but a direct connect gives me ~6MB/s so 
they're capable of more than the ~ 500-600 KB/s I'm seeing with the 
latency).

> In our test environments it has been shown to 
> make a difference - some of our users have reported good results with it 
> as well. If its possible maybe we could get you on one of our test 
> machines to try it out. Shoot me a note if you are interested and I'll 
> see what our policies on this are.
> 
>>> 2) HPN client can now set the local tcp receive buffer on a per 
>>> connection basis. Using the -w option allows the client to override 
>>> the local tcp receive window settings up to the maximum tcp buffer 
>>> size. This is just a setsockopt() call really.
>>
>> I think this should be a ssh_config(5) option (maybe 
>> "TCPReceiveBuffer" ?) rather than a command-line switch (ssh already 
>> has enough switches...)
> 
> Well, the problem is that you only want to set a large buffer size when 
> you really need it and even then, its often best to tune the buffer to 
> the specific path. Providing this option on the command line, in our 
> view, allows for the greatest flexibility. I have no objection to there 
> being a default in the ssh_config but I think its important for the user 
> to be able to override it.

You can specify it in the config file on a per-host basis and/or as a 
default so I think it's OK.  In fact, it's probably more convenient for 
end users (as opposed to people testing ssh mods :-) since they can just 
enable it for the appropriate hosts then forget about it.

>> This would allow it to be set either per-connection or globally, and 
>> may be passed through from the scp command line with the "-o" option.
> 
> Okay, so thats basically the same thing. I'd suggest using a shorter 
> name though but thats not a mahjor point to get hung up on.

There's an existing config TCP option (TCPKeepAlive), I picked it to be 
consistent with that.

>> The latter would also mean that scp would need less modification (and 
>> scp's code is mostly shared with rcp, so that's also a plus).
> 
> I honestly think changing scp's code isn't a bad thing. :)

Yeah but you don't have to maintain it :-)

[patch]
> Mike Stevens and I will take a look at next week I think. We're working 
> on some other stuff based on SSH (basically a more advanced port 
> forwarding system) that we're trying to nail down. That should be done 
> by the end of this week though.

Thanks.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
     Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.