sshd problems openssh3.4p1 on 6.5.17

[David Steiner]

Greetings,

I have run into a rather bizarre problem that is beyond my meager 
skills to debug so I am turning to the collective wisdom of this list.


Here are the facts:

2 SGI machines (alpha running IRIX 6.5.15 & beta running 6.5.14) 
running openssh-3.4p1. ssh was built on each machine using the same 
configuration and were working fine.

I upgraded the OS on alpha to 6.5.17 and ran into no problems with sshd.

After upgrading beta to 6.5.17, we found that machines with hostnames 
with the pattern, name.something.dartmouth.edu were denied access by 
the tcp-wrapper code in sshd (determined by running sshd in debug 
mode). It appears to think that these connections come from 
255.255.255.255 (determined by syslog entries). If I add 
255.255.255.255 to the hosts.allow, connections are allowed.

Machines with hostnames like name.dartmouth.edu do not have any 
problems connecting.

This is what I have done so far:

As stated above, sshd on beta was originally compiled under 6.5.14, 
on alpha under 6.5.15. Everything else about the build is the same 
except the binary on beta was not stripped (this does not seem to be 
a factor as you will see).

I moved the sshd binary compiled on alpha to beta and named it sshd.a

I tested sshd.a in debug mode on port 2222 and found that the problem 
disappears.

Thinking that the workaround would be to use alpha's binary on beta, 
I copied sshd (original beta binary) to sshd.b and sshd.a to sshd and 
restarted the daemon. I was unable to login from a machine with a 
problematic hostname (somehost.somewhere.dartmouth.edu).

Since it now looks like the problem is related to being either in 
debug vs. daemon or port 22 vs 2222, I ran test for the various 
combinations:

	    |	sshd.b   |    sshd.a
-----------------------------------------
debug/2222  |  works	 |   works
-----------------------------------------
daemon/2222 |  works 	 |   works
-----------------------------------------
debug/22    |  works	 |   works
-----------------------------------------
daemon/22   |  works	 |   works
-----------------------------------------

In the off chance that I got confused somewhere, I then copied the 
same sshd.a binary used in the tests above to sshd and started it up. 
It does not work.

Conclusion:

There is something in the 3.4p1 code in conjunction with the upgrade 
to 6.5.17 that causes sshd to break based on the name that it runs 
under.

As a workaround, I have started the daemon on beta using the binary 
named sshd.a.

I would appreciate any help anyone can offer. If you need additional 
data, just let me know.

TIA.
-- 
David R. Steiner                               david.r.steiner at dartmouth.edu
UNIX System Manager                            Phone:  603.646.3127
Dartmouth College                              Fax:     603.646.1041