cygwin performance problem

[Chris Rapier]

> So, well, maybe setting the buffer size to 20480 would be a useful
> compromise for all OSes?!?

I only looked at 3 buffer sizes. 8k, 20k and 65k.
In each case I ran 20 iterations (I should run 100 really).

The format for the results is low/high/average/median

OS X -> OS X (same subnet)
RAM disk to /dev/null, 200MB file, no compression
100bT network, full duplex, rtt 0.293ms
64k - 8.7/10.5/9.9/10.0
20k - 9.5/10.5/10.2/10.0
8k  - 8.3/10.5/9.8/10.0

Linux -> OS X (different subnets)
disk to /dev/null, 200MB file, no compression
1000bT -> 100bT network, full duplex, rtt 1.2ms

64k - 8.3/10.5/10.1/10.5
20k - 9.1/10.5/10.2/10.5
8k  - 9.1/10.5/10.3/10.5

Linux -> Linux (different subnets (and different than the above))
RAM disk to /dev/null, 200MB file, no compression
1000bT network, full duplex, rtt 0.868ms
64k - 12.5/15.4/13.5/13.3
20k - 15.4/18.2/16.6/16.7
8k  - 18.2/20.0/19.0/18.2

Okay, so I didn't trust that last round of tests so I re-ran them.
64k - 13.3/18.2/14.46/14.3
20k - 15.4/16.7/15.53/15.4
8k  - 18.2/20.0/18.47/18.2

This seemed odd to me so I tried it with another system a bit further 
away. Unfortunately, to get past the impact of the congestion control 
buffer on high BDP links I had to use the HPN patch. So that *may* 
influence the change in clientloop.c somewhat.

Linux -> Linux (pittsburgh to colorado)
RAM disk to /dev/null, 200MB file, no compression
155mbps network via Internet2/Abilene, rtt 58.8ms
64k -  5.6/12.5/10.56/10.5
20k -  7.7/12.5/10.60/10.5
8k  - 11.1/11.8/11.59/11.8
      ( 9.5/12.5/10.83/11.1)

I reran the 8k sample to make sure the results for that weren't time 
dependent as the 64k and 20k samples were run around 90 minutes after 
the 8k sample. The rerun sample is in parentheses. At this point I'm 
going to guess that there is statistically no or minimal impact on 
throughput between the different buffer sizes on this network path.

So my conclusions are that switching the buffer in 
client_process_net_input might have, in some conditions with midrange 
rtts on fast networks, end up having an overall negative impact. I'm not 
going to speculate as to why at this point.

If anyone cares I'll rerun the tests with increase sample sizes, some 
more buffer sizes, and get the standard deviation.

Chris