New Set of High Performance Networking Patches Available

[Chris Rapier]

Darren Tucker wrote:
> Chris Rapier wrote:
> 
>> http://www.psc.edu/networking/projects/hpn-ssh/
> 
> 
> Looking at these has been on my to-do list for a while and I finally 
> took a look.

Excellent.

>> 1) HPN performance even without both sides of the connection being HPN 
>> enabled. As long as the bulk data flow is in the direction of the HPN 
>> side you should see improved performance. I've measure 200Mb/s to an 
>> HPN server from a non HPN client and vice versa.
> 
> 
> I've been testing with tunbridge[1] on OpenBSD to add latency.  I've 
> seen an improvement of around 50% throughput on scp with 100ms of 
> latency (each way, ie 200ms rtt) simulated link with Linux endpoints.

In the real word test environments we have set up we're commonly seeing 
improvements of 10 to 30x greater throughput. 25MBytes/s+ where before 
we were seeing less than 1Mbyte/s.

> Using -w doesn't seem to make any difference (or sometimes it's a net 
> loss) although it's quite possible something in my test environment is 
> responsible for that.  (Yes, I did the stack tuning, both netstat and 
> getsockopt show the buffers are 1MB or more.)

It will only make a difference when the client is acting as the data 
sink. Obviously the max buffer size has to be large enough to handle the 
  user defined setting. In our test environments it has been shown to 
make a difference - some of our users have reported good results with it 
as well. If its possible maybe we could get you on one of our test 
machines to try it out. Shoot me a note if you are interested and I'll 
see what our policies on this are.

>> 2) HPN client can now set the local tcp receive buffer on a per 
>> connection basis. Using the -w option allows the client to override 
>> the local tcp receive window settings up to the maximum tcp buffer 
>> size. This is just a setsockopt() call really.
> 
> 
> I think this should be a ssh_config(5) option (maybe "TCPReceiveBuffer" 
> ?) rather than a command-line switch (ssh already has enough switches...)

Well, the problem is that you only want to set a large buffer size when 
you really need it and even then, its often best to tune the buffer to 
the specific path. Providing this option on the command line, in our 
view, allows for the greatest flexibility. I have no objection to there 
being a default in the ssh_config but I think its important for the user 
to be able to override it.

> This would allow it to be set either per-connection or globally, and may 
> be passed through from the scp command line with the "-o" option.

Okay, so thats basically the same thing. I'd suggest using a shorter 
name though but thats not a mahjor point to get hung up on.

> The latter would also mean that scp would need less modification (and 
> scp's code is mostly shared with rcp, so that's also a plus).

I honestly think changing scp's code isn't a bad thing. :)

> Attached is a diff relative to openssh-4.1p1-hpn11.diff with a couple of 
> proposed changes:
> * move the sshconnect.c setsockopt code into its own function
> * make that function style(9) compliant
> * fix a bug where strerror was used on the non-error path
> * make BUFFER_MAX_HPN_LEN an unsigned to placate gcc -Wsign-compare
> * replace magic numbers in channels.h with symbolic names
> 
> I don't think I changed any functionality (but I could have missed 
> something...)
> 
> [1] http://www.iijlab.net/~kjc/software/dist/tunbridge-0.1.tar.gz

Mike Stevens and I will take a look at next week I think. We're working 
on some other stuff based on SSH (basically a more advanced port 
forwarding system) that we're trying to nail down. That should be done 
by the end of this week though.

Chris