Fwd: Re: Bus Error with OpenSSH 3.7.1p2 on Solaris 8, SPARC 64-bit

[aphor at speakeasy.net]

The story of this problem, AFAIK, is that Solaris 8 YASSP and JASS and vigilant/paranoid sysadmins have been known to set a restricitve umask in /etc/default/login. OpenSSH compatibility for Solaris 8 has been spotty at times for sparcv9 targets. This time, when a sparcv9 binary tries to sscanf(3C) the numeric umask as a long octal and put it in a mode_t, SIGBUS happens.

What this looks like in real life is: You are running a 64 bit OpenSSH_3.7.1p2
sshd on Solaris 8, and you have enforced UMASK in /etc/default/login. You try to log into this sshd, but after all the authentication and channel setup, just before you get your shell/command executed *POOF*. The daemon is getting a SIGBUS trying to handle the UMASK from /etc/default/login with sscanf(3C);

It just so happens that I'm using gcc-3.3.1, and I can't say this is definitely an OS or libC or compiler issue. However, the 32 bit binary works, and the 64 bit binary SIGBUSes. Here is a demo that will work whether you are YASSP/JASS hardenened or whatever (because the umask is hard-coded instead of read from /etc/default/login).

---cut---
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>

int main (){
 char var[4] = "777\0";
 mode_t mask;
 printf("Test case: UMASK=%s in /etc/default/login.\n",var);
 printf("Reading var for a regular octal value: \n");
 sscanf(var, "%5o", &mask);
 printf("\tUMASK=%o\n",mask);
 printf("Reading var for a long octal value: \n");
/*** expect a SIGBUS here ***/
 sscanf(var, "%5lo", &mask);
 printf("\tUMASK=%o\n",mask);
}
---cut---

This bug is in session.c. It is only exposed AKAIK on 64 bit binaries running on hardened (default umask is set) Solaris boxes. I'm not sure why session.c needs to sscanf a long octal. Can someone try this with Forte compilers? Can we get by with sscanf(var, "%5o", &mask)? What is the *right* thing to do here? 

---
Jeremy