how to test the performance of modified openssh

[Darren Tucker]

Chris Rapier wrote:
> Ben Lindstrom wrote:
>> The only performance testing I've ever seen was a "max transfer rate" 
>> test. 
> 
> This is the tests we most commonly use when developing HPN-SSH. We're 
> actually in the process of developing some methods to test application 
> layer latency though.

For testing raw throughput I use "dd if=/dev/zero bs=1024 | ssh server 
ssh dd of=/dev/null bs=1k" (making sure compression is off) which takes 
any disk IO out of the picture but still does IO in chunks.

If I care about disk IO (eg fiddling with scp) then I'll just grab a 
random large file and use it for the test data.

>> Which lead to deciding the encryption order and discussions about 
>> v2 vs v1 preformance differences.  Along with Internet2 testing to find a 
>> better way of gaining preformance on massive pipes.
>>
>> In both cases I believe the test was pretty much a "dd if=/dev/urandom .. 
>> | ssh '| cat /dev/null'" style testing or transferring of real data. 
>> Timing the time it takes to move XX amount of data then doing the math.
> 
> I also use Iperf over a SSH tunnel to do tests. It can give you some 
> useful options (mutliple streams, different RWINs, MSS, and multiple 
> streasm, disable nagle, etc) and statistics (1sec avgs, etc). 
> Unfortunately, it uncovered an odd situation with the HPN-SSH code 
> though that I'm still trying to resolve (the window doesn't grow. :\).

Chris: make sure you have clientloop.c rev 1.171.  There's a latent bug 
that sets the max packet size to nonsensically large values for remote 
port forwards, which may or may not be affecting you.

>> I don't believe anyone has done any testing on the cost of 
>> authentication.  Truth of the matter is if it is near instance I doubt 
>> anyone really cares (I sure never have =).
> 
> I care! :) Not a whole lot because we generally see the authentication 
> process as sunk cost. It matters a lot on low bandwidth long delay 
> connections involving small transfers. Its more difficult to amortize 
> the cost of authentication in those instances.

I've also done the "time ssh server true" thing Ben suggested when 
looking at things that potentially change the connection establishment 
time (not that common, but things like Diffie-Hellman tweaks, random 
reseeding or the recent protocol 1 ephemeral server key thing).

The catch is that if your authentication require user interaction then 
you have a massive extra variable which you would need to account for.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
     Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.