ssh2 key passphrase problems in 2.9.9 on Linux

[Phil Stracchino]

On Thu, Sep 27, 2001 at 02:26:14PM -0700, Phil Stracchino wrote:
> I've just compiled and installed openssh-2.9.9p2 (compiled against 
> openssl-0.9.6b using gcc-3.0.0) on a Slackware 7-based Linux machine 
> (kernel 2.4.6ac2).  The previously installed version was 2.9p2, compiled 
> against openssl-0.9.6a, also with gcc-3.0.0, but with a different build of 
> gcc-3.0.0.
> 
> Everything seems to work fine except for one problem:  passphrase matching 
> for ssh2 keys *always* fails.  I've run ssh-add under gdb several times 
> trying to see what's going wrong, so far without learning anything 
> particularly enlightening.


I have finally managed to isolate this down to the following:  For SSH2 
DSA and RSA keys, the OpenSSL PEM_read_PrivateKey() macro, called from 
authfile.c line 448:

	pk = PEM_read_PrivateKey(fp, NULL, NULL, (char *)passphrase);

is consistently failing and always returns NULL, whereas it should be
returning a EVP_PKEY struct with pk->type containing either EVP_PKEY_RSA
or EVP_PKEY_DSA.  As far as I can see from the OpenSSL code, this means
that BIO_new(BIO_s_file()) has to be returning NULL, but that's as far as
I can figure it out; the internals of OpenSSL are utterly impenetrable to
me.

Any suggestions, anyone?  I think I've taken this problem about as far as 
I can diagnose it myself.


OpenSSL was configured using the following options:  --prefix=/usr shared 
threads -D_REENTRANT.  Recompiling with no-threads out of constructive 
paranoia made no difference.

OpenSSH was configured with:  configure --prefix=/usr --with-tcp-wrappers 
--with-md5-passwords --with-ipv4-default --sysconfdir=/etc

The same key files are handled correctly on a Solaris 2.8 machine with the
same OpenSSL and OpenSSH versions, configured identically except for no
--with-md5-passwords because Solaris still doesn't support md5crypt.  Bad
Sun, bad.  No donut.



-- 
   Linux Now!   .........Because friends don't let friends use Microsoft.
  phil stracchino   ::   alaric at babcom.com   ::   halmayne at sourceforge.net
    unix ronin     ::::   renaissance man   ::::   mystic zen biker geek
     2000 CBR929RR, 1991 VFR750F3 (foully murdered), 1986 VF500F (sold)