Problem with interaction between commercial and openssh

[Ashton Trey Belew]

	Hello all,
	Earlier this week we disabled protocol 1 upon our machines while
installing commercial ssh 3.2.0.  Suddenly I discovered that the AIX
systems running Openssh were not able to connect.  I upgraded to the newly
minted 3.4p1 and discovered the same problem.
	My limited poking around has shown the following:
<16:59:38>atb at ursus:>ssh -vv atb at host
<snip>
debug1: bits set: 503/1024
debug1: ssh_dss_verify: signature incorrect
key_verify failed for server_host_key
debug1: Calling cleanup 0x2000ca44(0x0)

Putting an #if 0  ... #endif around the key_verify call in kexdh.c solves
the problem, but in a most unsatisfactory fashion.  I double checked my
compiles for linux and solaris and they are working perfectly.

	Looking in ssh-dss.c at ssh_dss_verify it appears that all is well
right up until the end when the following occurs:

        ret = DSA_do_verify(digest, dlen, sig, key->dsa);
        memset(digest, 'd', sizeof(digest));

        DSA_SIG_free(sig);

        debug("ssh_dss_verify: signature %s",
            ret == 1 ? "correct" : ret == 0 ? "incorrect" : "error");
        return ret;

	If I am not mistaken, DSA_do_verify comes out of openssl.  I am
poking around in it now to see what I can learn; but I figured I would
write here and see if anyone has any suggestions about where I should
look.
	Have a nice day,
	-Trey

--
Ashton Trey Belew
802 656 1260
atb at zoo.uvm.edu

Recall Larry's 2nd Law of Language Redesign:
       Larry gets the colon.