Multiple (multiplexed) simultaneous ssh connections - Cygwin bug?

[Darren Tucker]

Goldburt, Dan wrote:
> Hello,
> 
> I need to make many (>50) ssh connections from linux to cygwin at the
> same time. Using Windows 2000 Server (OpenSSH_4.3p2, OpenSSL 0.9.8b
> and updated cygwin) and Linux RHEL4 (OpenSSH_3.9p1, OpenSSL 0.9.7a).

It's not pretty but you could run multiple sshd's on several ports.

> It's been difficult to optimize many simultaneous connections. Here
> were some issues:
> 1.       On Windows XP/Professional, Microsoft
> intentionally cripples the TCP/IP stack. Official word
> (http://support.microsoft.com/kb/Q127144) is that the backlog queue
> limit on a listen socket is 5 (200 when Server), so you can't
> accept() more than 5 new connections concurrently.

> 2. Using a master connection that is shared, the sshd_config variable
>  MaxStartups has no effect. This is because we are not opening lots
> of ssh connections, but are opening multiple sessions within a single
>  connection. The parameter that needs to be changed is MAX_SESSIONS, 
> which is hardcoded in sessions.c at 10. Request: add "MAX_SESSIONS" 
> as a configuration parameter in sshd_config.

Maybe.  It's certainly too late for the upcoming release, though.

> (also, you should 
> mention in INSTALL documentation that by default, compiled binaries 
> are quite a bit larger than usual. Do you use strip -strip-debug?

We use whatever "install -s" uses on your platform.  If you're using the 
bundled install-sh script then it just calls "strip".

> Finally, I'm able to make many connections most of the time. But
> then, sshd errors:
> 
> fcntl(223, F_GETFL, 0): Bad file descriptor and sometimes: [sig] bash
> 3720 _cygtls::handle_threadlist_exception and then loops spitting out
> "select: Bad file descriptor" and taking up 100% CPU. I have not done
> a stack trace or increased sshd debug output because the error comes
> up when about 100 connections are made, so it would be difficult to
> track down. If this isn't enough information to go on, I will post
> it.

Now this I'm not sure about.

You'll have the stdout and stderr descriptors in the select's readset, 
which for FD_SETSIZE=64 puts the limit at around 30 connections or so 
(assuming you're not port forwarding or something too).  What did you 
bump MAX_SESSIONS to?  It might be overrunning the fd_set.

To make this work, you would probably need to break the select into 
FD_SETSIZE chunks somehow.

> Is this because on Cygwin, the "fd_set" arrays, used with select(),
> can contain file descriptors (FD) from 0 to 63 (the fd_set array is
> 8-byte long). On Linux, this is 0 to 1023? From:
> http://www.ipflow.utc.fr/blog/?p=34 and
> http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=105321853321894&w=3
> I also found a post
> http://www.cygwin.com/ml/cygwin/2005-06/msg00511.html where Corinna
> said "Using a master/slave connection requires the ability to
> exchange file descriptors over AF_UNIX sockets.  That's not possible
> in Cygwin." I assume this has been addressed with USE_PIPES, since I
> AM able to use multiplexed connections most of the time?

That refers to the multiplexing (ControlMaster/ControlPath) 
functionality in the client, not the server side.

> Lastly, if security is not the biggest concern, should I even use
> ssh? I just need to be able to execute many remote shell commands in
> a short interval and return the output.

That's a local policy decision, but you're probably not going to get an 
unbiased opinion on this list :-)

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
     Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.