port-linux.c bug with oom_adjust_restore() - causes real bad oom_adj - which can cause DoS conditions.

[Cal Leeming [Simplicity Media Ltd]]

So I modified the code to try and repair this oom_adj problem...

port-linux.c:
line 235: //static int oom_adj_save = INT_MIN;
line 236: static int oom_adj_save = 0;
line 277: verbose("Set %s to %d - sleepycal", OOM_ADJ_PATH, oom_adj_save);


I then ran compiled the package, ran SSHd, and yet we still have -17 in 
oom_adj_save. Wtf? Now, I'm not much of a C coder, but this is weird 
even in my books...

May 30 22:18:19 vicky sshd[12825]: Set /proc/self/oom_adj to -17 - sleepycal

So, I went all out crazy, and did the following patch:

         static int sleepycal_oom_adj_save = 0;
         verbose("sleepycal_oom_adj_save=%d", sleepycal_oom_adj_save);

         if (fprintf(fp, "%d\n", sleepycal_oom_adj_save) <= 0)
                 verbose("error writing %s: %s", OOM_ADJ_PATH, 
strerror(errno));
         else
                 verbose("Set %s to %d - sleepycal", OOM_ADJ_PATH, 
sleepycal_oom_adj_save);

And it worked!!! :)

May 30 22:27:12 vicky sshd[2532]: sleepycal_oom_adj_save=0
May 30 22:27:12 vicky sshd[2532]: Set /proc/self/oom_adj to 0 - sleepycal

root at vicky:~/openssh-5.5p1# cat /proc/2532/oom_adj
0

So, it turns out that it is actually OpenSSH which is broken, after 
almost 3 days of frustrating digging through millions of lines of code 
lol. Anyways, would appreciate if someone could get this merged into 
master (obv rename the vars if you want).

Attached is the appropriate patch file as of openssh-5.5p1

Cal

On 30/05/2011 21:56, Cal Leeming [Simplicity Media Ltd] wrote:
>  Just did some testing..
>
> root at vicky:~# cat /var/log/auth.log | grep "Set"
> May 30 21:41:05 vicky sshd[1568]: Set /proc/self/oom_adj from -17 to -17
> May 30 21:41:07 vicky sshd[1574]: Set /proc/self/oom_adj to -17
>
> root at vicky:~# ps faux | grep 1574
> root      1574  0.0  0.0  70488  3404 ?        Ss   21:41   0:00  \_ 
> sshd: root at pts/1
>
> root at vicky:~# ps faux | grep "1568"
> root      1568  0.0  0.0  49168  1152 ?        Ss   21:41   0:00 
> /usr/sbin/sshd
>
> In sshd.c there seems to be:
> static int oom_adj_save = INT_MIN;
>
> root at courtney:~/openssh-5.5p1# grep -R "Set %s to %d" .
> ./openbsd-compat/port-linux.c:          verbose("Set %s to %d", 
> OOM_ADJ_PATH, oom_adj_save);
>
> Then I tried on a server with different network card hardware (as 
> shown below), and got this from the logs:
>
> root at courtney:~/openssh-5.5p1# cat /var/log/auth.log  | grep "Set"
> May 30 21:50:15 courtney sshd[4821]: Set /proc/self/oom_adj from 0 to -17
> May 30 21:50:26 courtney sshd[4848]: Set /proc/self/oom_adj to 0
>
> root at courtney:~/openssh-5.5p1# ps faux | grep "4848"
> root      4848  0.0  0.0  70488  3372 ?        Ss   21:50   0:00  \_ 
> sshd: root at pts/1
>
> root at courtney:~/openssh-5.5p1# ps faux | grep "4821"
> root      4821  0.0  0.0  49168  1160 ?        Ss   21:50   0:00 
> /usr/sbin/sshd
>
> root at courtney:~/openssh-5.5p1# cat /var/log/auth.log  | grep -e "Set" 
> -e "oom_adjust_restore"
> May 30 21:50:15 courtney sshd[4821]: Set /proc/self/oom_adj from 0 to -17
> May 30 21:50:26 courtney sshd[4848]: debug3: oom_adjust_restore
> May 30 21:50:26 courtney sshd[4848]: Set /proc/self/oom_adj to 0
>
>
>
>
> On 30/05/2011 21:30, Cal Leeming [Simplicity Media Ltd] wrote:
>> Hi all,
>>
>> Please find below a complete transcript of the emails between 
>> debian/kernel-mm mailing lists.
>>
>> I've had a response back from someone on the deb mailing list stating:
>>
>> ====================================
>> The bug seems to be that sshd does not reset the OOM adjustment before
>> running the login shell (or other program).  Therefore, please report a
>> bug against openssh-server.
>> ====================================
>>
>> Therefore, I am submitting this bug to you also.. If someone would be 
>> kind enough to have a flick thru all the below debug/logs, it'd be 
>> very much appreciated.
>>
>> Cal
>

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: oom_patch_for_openssh-5.5p1_by_sleepycal.patch
URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20110530/3a257e54/attachment.ksh>