[PATCH] Fix oom_adj on Linux after sshd reload

[Matthew Ife]

Currently, on linux sshd attempts to remove itself from the influence of
oom-killer by modifying the oom_adj parameter for itself in proc to -17.

This is controlled via two functions; oom_adjust_setup() and
oom_adjust_restore().

Setup saves the old score (typically zero on initialization) and sets sshd
to -17 whilst oom_adjust_restore places the saved value from initialization
back into the running sshd process.

The principle here being that whilst we want the main sshd listener process
never to be oom-killed, we want any subsequent children that sshd spawns to
be oom-killed if necessary.

The problem that occurs however is on a SIGHUP, sshd re-execs itself and
inherits the oom_adj score from its protege. When sshd goes through its
initialization steps it calls oom_adjust_setup() and saves the now -17
value. On oom_adjust_restore() sshd then restores -17 to all child
processes sshd spawns.

This has the effect of after performing a reload causing all sshd processes
(and by the nature of inheritance) all descendant pids having -17 as their
oom_adj value which causes all sshd sessions after the reload to never be
candidates for oom-killer.

Clearly this is not desirable as it can lead to unrecoverable out of memory
conditions on linux.

This is very simple to test on linux. Restart sshd on a linux host. Login
to it via ssh and then run cat /proc/self/oom_adj. How reload ssh, relogin
again via ssh and recheck cat /proc/self/oom_adj.

I have created a patch, which I believe is honourable to the design
philosophy being utilized by openssh.

This patch adds an additional function to platform.h called
platform_pre_restart() that (if defined) will perform oom_adjust_restore().
This is then called inside of sighup_restart() prior to performing the
re-exec of ssh.