openssh-1.2.2 with openssl-0.9.5 rsa problem

[Christopher McCrory]

Hello...

to openssh-unix-dev at mindrot.org
cc djm at ibs.com.au

devel platform: linux intel redhat 6.2beta
http://violet.ibs.com.au/openssh/files/openssh-1.2.2.tar.gz from rpm at
same.

I am NOT subscribed to the openssh-unix-dev list.


	I ran into a problem using openssh-1.2.2 with openssl-0.9.5.  With
openssl 0.9.5 you can now compile shared libraries without idea, rc5 and
rsa.

openssl compiled with:
./config no-rc5 no-idea -DRSA_NULL results in libraries without rc5 and
idea and (AFAIK) with the patented parts of rsa removed.  compiled and
installed this way openssh compiles and  installs fine.  When run there
are errors though.

specifically:
rsa_public_encrypt() failed
on the client side
or
rsa_private_decrypt() failed
on the server side


------------------actual sample-------------
( squid is known working version from violet rpms, wednesday is new test
version)

[squid at chrismcc]$ slogin -v wednesday 
SSH Version OpenSSH-1.2.2, protocol version 1.5.
Compiled with SSL.
debug: Reading configuration data /etc/ssh/ssh_config
debug: ssh_connect: getuid 114 geteuid 0 anon 0
debug: Connecting to wednesday.netus.com [209.95.208.9] port 22.
debug: Allocated local port 621.
debug: Connection established.
debug: Remote protocol version 1.5, remote software version
OpenSSH-1.2.2
debug: Waiting for server public key.
debug: Received server public key (768 bits) and host key (1024 bits).
debug: Host 'wednesday.netus.com' is known and matches the host key.
debug: Encryption type: 3des
debug: Sent encrypted session key.
Connection closed by 209.95.208.9
debug: Calling cleanup 0x80560b0(0x0)
user chrismcc Mon Feb 28 on squid in ~
------------------------
server syslog says:
Feb 28 17:02:04 wednesday sshd[15363]: fatal: rsa_private_decrypt()
failed


[squid at chrismcc]$ rpm -qa | grep openss
openssh-1.2.2-1
openssh-askpass-1.2.2-1
openssh-clients-1.2.2-1
openssh-server-1.2.2-1
openssl-0.9.4-3

( rpms are from violet ) 


----another

[wednesday at chrismcc]$ slogin wednesday
rsa_public_encrypt() failed
user chrismcc Mon Feb 28 on wednesday in ~
[wednesday at chrismcc]$ 
server says:
Feb 28 17:05:19 wednesday sshd[17036]: Connection closed by 209.95.208.9



after looking at the openssh source code (I am not a programer though)
it seems that an the keyexchange point if rsa fails it should, IMHO,
fall through to dsa(?) or something else.  

-- 

Christopher McCrory
Lead Bithead, Netus Inc.
chrismcc at netus.com
admin at netus.com

"Linux: Because rebooting is for adding new hardware"