sshd hanging after multiple successive logons

[douglas.manton at uk.ibm.com]

Folks,

I use OpenSSH to poll a number of remote servers once every five minutes
and obtain a number of attributes.  This is done using ssh as "sexec":

        ssh stats at remotehost getstats

This returns the output of the getstats program which is parsed, etc...

The problem is that after so many connections, the parent sshd hangs and
does not accept any more connections.  I have reproduced the problem using
a simple shell script on my local machine:

        while sleep 1
        do
                ssh me at localhost whoami
        done

This iterates about 4000 times before sshd hangs.  I can see that sshd is
waiting in _pthread_waitlock when it has hung.  It actually decreases the
CPU time when in this state.  I can conenct to the daemon on port 22 but
it does not present the version string.

I realise that 4000 connections is not bad, but at five minute intervals
over 100 machines this is happening every couple of days.

I am running OpenSSH 2.3.0p1 under AIX 4.3.3.0-ML6, compiled with IBM VAC
v5.

Any ideas?  Has this been seen.  I can reproduce it every time.  When I
get a chance I will test the build on my Ultra 10.

Many thanks,
--------------------------------------------------------
 Doug Manton, AT&T EMEA Firewall and Security Solutions

                   demanton at att.com
--------------------------------------------------------
"If privacy is outlawed, only outlaws will have privacy"