how to speed up OpenSSH command execution (and a speed analysis)

[Christoph Anton Mitterer]

Hi Alan, et all.


On Sun, 2012-03-25 at 19:32 +0200, Alan Barrett wrote:
> "ControlMaster auto" is supposed to do that.
This alone wouldn't help (in my scenario[0], right?
I'm already using it and when the connection that initialises the master
is closed the then it goes full away.

But...


> "ControlPersist 24h" is supposed to do that.
... this is as far as I can see what solves both,...

Thanks for the hint, I haven't seen it before, as my ssh on which I
tested it was too old.

I'll try later and see whether it does exactly what I want.



But the main problem is the speed thingy,... and I have found something
really strange (but also interesting):

I played around with -t. Still having no ControlPersist set.
I always do:
1) killing any old control masters and starting a new one with:
# ssh -f -N nagtest at host.example.org
2) measuring the time of the remote execution of the check script
# time ssh nagtest at host.example.org <command and parameters for my nagios check>


a) Tried around with -a -x -k -C:
They seem to change nothing with respect to speed


b) Then I played around with -t, both with and without the options from
(a) above:
Here comes the weird part:
- It does basically NOT matter, whether I set -t on creating the control
master (step (1) above).

In step (2) (the actual command executions):
- When the FIRST time, I execute a command, -t is used,... then speed
get much better, and I get times around 0.006 (s) which is the same that
NRPE (even without the fake-SSL)  required. GREAT
- For subsequent tries (with the same Control Master)... it doesn't
matter whether -t is used or not,.. it stays between 0.014-0.010s.

BUT

- When the FIRST time, I execute a command, -t is NOT used,... then
speed stays at the boring 0.045s.
- For subsequent tries (with the same Control Master)... it doesn't
matter whether -t is used or not,.. it stays at around 0.045s.


CONCLUSIONS:
I) Something seems to be fishy there, given that it
   * depends on the FIRST command executed using -t
   * while most other options get "fix" by starting the ControlMaster
     the -t option seem to be not "set" by it.
II) With -t (and ControlMaster) you can speed up SSH dramatically
(effectively making NRPE absolutely useless).


PROBLEMS LEFT:
Apart from the strange point (I) above... (shall I open a bug on this?)
there is one "big" problem left:
Whenever I use "-t" in the command execution step (that was (2) above) I
first get the output from the script, e.g.:

   # time /usr/lib/nagios/plugins/check_nrpe  -H host.example.org -c check_load
   OK - load average: 0.00, 0.02, 0.00|load1=0.000;15.000;30.000;0; load5=0.020;10.000;25.000;0; load15=0.000;5.000;20.000;0;

but directly afterwards, ssh prints the following on stderr:
   Shared connection to host.example.org closed.

   real    0m0.006s        user    0m0.004s        sys     0m0.000s

This is (in my case quite annoying, as it seems to be useless (or does
anyone know what it means?) and it disturbs Nagios.
Of course I could just redirect stderr to /dev/null or write another
wrapper that filters this.
But the former would hide real errors, too, and the later would just
cost time again.


Another thing:
Perhaps disabling the pseudo-tty allocation is something which I
shouldn't want with my Nagios checks and I just don't see some
problems?!
(I'm really no expert with respect to the pseudo tty stuff.)

So if some experts have more knowledge on -t and the pseudo-ttys... just
tell me.



Cheers,
Chris.

[0] Nagios schedules the checks typically not at the same time,.. so
multiple SSH connections wouldn't get opened concurrently, but
subsequentially.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5677 bytes
Desc: not available
URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20120325/5047c580/attachment.bin>