openssh 3.7.1p2 fault on solaris 9 for sparc when built as 64-bit

[Alex Kiernan]

"Thomas A. Kyle" <tkyle at jinx.umsl.edu> writes:

> I built OpenSSH as a 64-bit binary on Solaris 9, using gcc 3.3.2,
> OpenSSL 0.9.7c and zlib 1.2.1.  sshd starts up normally, and will begin
> the login session, however, some time after it reads /etc/default/login,
> it faults and kills the connection.  There are no error messages logged
> to syslog (with LogLevel set to DEBUG) or with the -ddd option.
> 
> Here's a truss of the failure:
> 
> 8132:   open("/etc/nologin", O_RDONLY)                  Err#2 ENOENT
> 8132:   getuid()                                        = 1000 [1000]
> 8132:   getuid()                                        = 1000 [1000]
> 8132:   getuid()                                        = 1000 [1000]
> 8132:   getuid()                                        = 1000 [1000]
> 8132:   open("/etc/default/login", O_RDONLY)            = 7
> 8132:   fstat(7, 0xFFFFFFFF7FFFD060)                    = 0
> 8132:   fstat(7, 0xFFFFFFFF7FFFCF30)                    = 0
> 8132:   ioctl(7, TCGETA, 0xFFFFFFFF7FFFCF9C)            Err#25 ENOTTY
> 8132:   read(7, " # i d e n t\t " @ ( # )".., 8192)     = 2010
> 8132:   read(7, 0x100287C14, 8192)                      = 0
> 8132:   lseek(7, 0, SEEK_CUR)                           = 2010
> 8132:   close(7)                                        = 0
> 8132:       Incurred fault #5, FLTACCESS  %pc = 0xFFFFFFFF7EC991C8
> 8132:         siginfo: SIGBUS BUS_ADRALN addr=0xFFFFFFFF7FFFE334
> 8132:       Received signal #10, SIGBUS [default]
> 8132:         siginfo: SIGBUS BUS_ADRALN addr=0xFFFFFFFF7FFFE334
> 8128:       Received signal #18, SIGCLD [caught]
> 8128:         siginfo: SIGCLD CLD_KILLED pid=8132 status=0x000A

I'd guess this might fix it (I'm guessing w/o a stack trace) - its
completely untested:

--- session.c.orig	2004-01-07 21:55:40.647497013 +0000
+++ session.c	2004-01-07 21:56:25.357777123 +0000
@@ -915,7 +915,7 @@
 {
 	char **tmpenv = NULL, *var;
 	u_int i, tmpenvsize = 0;
-	mode_t mask;
+	long mask;
 
 	/*
 	 * We don't want to copy the whole file to the child's environment,
@@ -936,7 +936,7 @@
 	
 	if ((var = child_get_env(tmpenv, "UMASK")) != NULL)
 		if (sscanf(var, "%5lo", &mask) == 1)
-			umask(mask);
+			umask((mode_t)mask);
 	
 	for (i = 0; tmpenv[i] != NULL; i++)
 		xfree(tmpenv[i]);

mode_t is uint_t when you're in a 64 bit Solaris userland, so %lo is
too wide to fit.

-- 
Alex Kiernan, Principal Engineer, Development, THUS plc