ssh_rsa_verify: RSA_verify failed: error:

[Kim Lewall]

Host based authentication does not seem to be working for us after
upgrading to openssh-3.4p1 (we were at openssh-3.1p1) (openssl is at
0.96d).  Any time we try to connect from another unix box also running
openssh-3.4p1, we get the following error (on the server side) and host
based auth fails (it falls back to password prompt).

sshd[15038]: error: ssh_rsa_verify: RSA_verify failed:
error:04077068:lib(4):func(119):reason(104)

We are running on AIX 4.3.3 using the IBM VAC C compiler.

                     User binaries: /usr/local/bin
                   System binaries: /usr/local/sbin
               Configuration files: /usr/local/etc
                   Askpass program: /usr/local/libexec/ssh-askpass
                      Manual pages: /usr/local/man/manX
                          PID file: /usr/local/etc
  Privilege separation chroot path: /var/empty
            sshd default user PATH: /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
                    Manpage format: man
                       PAM support: no
                KerberosIV support: no
                 KerberosV support: no
                 Smartcard support: no
                       AFS support: no
                     S/KEY support: no
              TCP Wrappers support: yes
              MD5 password support: no
       IP address in $DISPLAY hack: no
          Use IPv4 by default hack: no
           Translate v4 in v6 hack: no
                  BSD Auth support: no
              Random number source: ssh-rand-helper
     ssh-rand-helper collects from: Command hashing (timeout 200)

              Host: rs6000-ibm-aix4.3.3.0
          Compiler: cc
    Compiler flags: -g
Preprocessor flags: -I/usr/local/ssl/include  -I/usr/local/include
      Linker flags: -L/usr/local/ssl/lib  -L/usr/local/lib -blibpath:/usr/lib:/lib:/usr/local/lib
         Libraries: -lwrap  -lz  -lcrypto

Changing UsePrivilegeSeperation to no has no effect.  Removing and
creating new keys on both sides has no effect.  The only changes to the
sshd_config file are:

PermitRootLogin no
IgnoreRhosts no
HostbasedAuthentication yes

The only changes to the ssh_config file are:

Host *
   ForwardX11 yes
   HostbasedAuthentication yes
   RhostsRSAAuthentication yes

The relevant part of sshd -ddd output seems to be:

debug3: mm_send_debug: Sending debug: Accepted by .rhosts.
debug3: mm_send_debug: Sending debug: Accepted host bmx.comp.uvic.ca ip
142.104.16.101 client_user klewall server_user klewall
debug3: mm_key_verify entering
debug3: mm_request_send entering: type 22
debug3: monitor_read: checking request 22
ssh_rsa_verify: RSA_verify failed:
error:04077068:lib(4):func(119):reason(104)
debug1: ssh_rsa_verify: signature incorrect
debug3: mm_answer_keyverify: key 2003b5e8 signature unverified
debug3: mm_request_send entering: type 23
Failed hostbased for klewall from 142.104.16.101 port 36574 ssh2
debug3: mm_request_receive entering
debug3: mm_key_verify: waiting for MONITOR_ANS_KEYVERIFY
debug3: mm_request_receive_expect entering: type 23
debug3: mm_request_receive entering
debug2: userauth_hostbased: authenticated 0
Failed hostbased for klewall from 142.104.16.101 port 36574 ssh2

Any ideas?  Thanks.

--------------------
Kim Lewall                                       tel 250/721-7650
Systems Programmer          klewall at uvic.ca      cel 250/213-7887
University of Victoria      Cle D039             fax 250/721-8778