pam_open_session w/o tty on Solaris

[David Butts]

Hello, all-

Apparently, under Solaris (I can personally confirm SunOS 5.7 and 5.8),
pam_open_session will generate a segfault if PAM_TTY is not set.  The
obvious symptom of this is that OpenSSH 2.9.9p2 will segfault on any
operation that does not request a tty (do_exec_no_pty).

Based on a quick google search, this seems to have been encountered
by others, though the specific symptoms seem to have changed a bit.

(eg http://www.castaglia.org/proftpd/doc/devel-guide/src/modules/mod_pam.c.html
contains a reference to this problem -- the first instance of PAM_TTY)

I wasn't able to find any other reference to Sun bugid 4250887 that
was mentioned in the comment, and an empty string worked for me as
well.

In any case, the following change appears to address the problem:

diff -ru openssh-2.9.9p2_orig/auth-pam.c openssh-2.9.9p2/auth-pam.c
--- openssh-2.9.9p2_orig/auth-pam.c	Mon Apr 23 14:38:37 2001
+++ openssh-2.9.9p2/auth-pam.c	Mon Oct 29 15:32:08 2001
@@ -272,6 +272,12 @@
 
 	do_pam_set_conv(&conv);
 
+#ifdef PAM_SUN_CODEBASE
+	if (ttyname == NULL) {
+		ttyname = "";
+	}
+#endif /* PAM_SUN_CODEBASE */
+
 	if (ttyname != NULL) {
 		debug("PAM setting tty to \"%.200s\"", ttyname);
 		pam_retval = pam_set_item(__pamh, PAM_TTY, ttyname);

Obviously that expands the meaning of PAM_SUN_CODEBASE a bit from its
current definition, but it seemed a fairly reasonable thing to use,
since this appears to be another misbehavior of PAM under Solaris.

I don't honestly know enough about the inner workings of PAM to know
whether it would be better to use an empty string, or a real but
useless file like /dev/null, though.

Is there a better way to deal with this?

Thanks,
David