Change request For OpenSSH 3.8p1

[Antoine Verheijen]

NOTE: This patch requires a previously sent patch fixing a small problem in
      OpenSSH PAM support when POSIX threads are used.

This is a small patch to the OpenSSH portable configuration process that
I'd like to have considered for inclusion in the distributed version. It
will set the use of (native) POSIX threads in Solaris if the header and
library files are present on the system. At present, this will only affect
PAM support on that OS.

Here's my problem. We use AFS at the University of Alberta. On Solaris,
authentication to AFS at login time is done via a PAM module. In versions
of OpenSSH up to at least 3.4p1, this worked fine. However, sometime before
version 3.8p1, the AFS PAM module stopped working properly with OpenSSH.

I have tracked this down to the use of "threads" in the PAM support
(auth-pam.c). With the approach now taken, a new "thread" is begun in the
task to call pam_authenticate(). This "thread" terminates on completion
of that call and the remaining PAM calls are done from the original "thread".
Seems reasonable. (The reason for the quotes around the word thread is
because of the next paragraph.)

Now, if a system does not have POSIX thread support, it is simulated using
processes (fork()). This works okay for the most part. Unfortunately, in the
AFS PAM module, the pam_authenticate() routine saves some critical
module-specific data (via the pam_set_data() routine) for use by the
pam_setcred() routine later on. This is perfectly acceptable (in fact, it's
provided for) in the PAM framework. When fork() is used to simulate threads,
the data saved by pam_authenticate() is associated with the new process and
is not available to the old process. Thus, the pam_setcred() call will always
fail. When true POSIX threads are used, this is not an issue because all heap
storage (which is where the saved data is placed) is accessible to all threads
and the PAM module works.

Solaris 8 and beyond (at least) has full POSIX thread support. OpenSSH never
tries to use it, however, unless you do some magic with environment variables
prior to the configure. The simple patch below checks for the existence of
the POSIX threads header file and library on Solaris and includes them, along
with the setting of USE_POSIX_THREADS, if they exist. The build of OpenSSH
then uses true POSIX threads on Solaris and the problem goes away.

Please give serious consideration to including this (or something similar) in
the distributed version of portable OpenSSH.

Note that this patch ONLY affects Solaris systems. A slightly different
approach could, of course, be used for a more general solution (using
something like HAVE_PTHREAD_H instead of USE_POSIX_THREADS) along with a
possible flag such as --with-threads. If you prefer, I could put in this type
of solution but this would require testing on various other platforms that I
don't have access to (although that would only matter if the flag was set).

diff -r -c old/configure.ac new/configure.ac
*** old/configure.ac	Mon Feb 23 22:47:04 2004
--- new/configure.ac	Thu Feb 26 17:23:55 2004
***************
*** 273,278 ****
--- 273,285 ----
  	AC_DEFINE(LOGIN_NEEDS_TERM)
  	AC_DEFINE(PAM_TTY_KLUDGE)
  	AC_DEFINE(LOCKED_PASSWD_STRING, "*LK*")
+ 	# Check for existence of POSIX threads.
+ 	AC_CHECK_HEADER(pthread.h, [
+ 		AC_CHECK_LIB(pthread, pthread_create, [
+ 			CPPFLAGS="$CPPFLAGS -DUSE_POSIX_THREADS"
+ 		 	LIBS="-lpthread $LIBS"
+ 		])
+ 	])
  	# Pushing STREAMS modules will cause sshd to acquire a controlling tty.
  	AC_DEFINE(SSHD_ACQUIRES_CTTY)
  	external_path_file=/etc/default/login

-----------------------------------------------------------------------
Antoine Verheijen                  Email: antoine.verheijen at ualberta.ca
CNS Network Services               Phone: (780) 492-9312
University of Alberta              Fax:   (780) 492-1729