Hanging ssh

mouring at etoh.eviladmin.org mouring at etoh.eviladmin.org
Wed Dec 20 09:15:41 EST 2000 


At this point.. I don't know.. I'm taking a break from it (and a lot
of stuff to catch up with life).. I spent almost two week digging in the
depths of session.c and channel.c.. 

Don't know of Damien has any other words of wisdom.  I know I don't.

- Ben

On Tue, 19 Dec 2000, Rob Hagopian wrote:

> Your comments got me thinking; I tried my situation using Protocol 1
> (forced by sshd) and no problems so far... the repeatable problem using
> snaps later than 20001114 (I believe the last one to contain the
> patch) has gone away, and it survived the night without leaving a hung
> process around (which even 20001114 did).
> 
> Is this of any use? Is there anything else I can do to help track this
> down? I'd rather not have to convert to ssh1... <shudder>
> 								-Rob
> 
> On Mon, 18 Dec 2000 mouring at etoh.eviladmin.org wrote:
> 
> > 
> > What platform?  I know Linux has such an issue.  The patch to resolve the
> > issue caused scp to not complete it's transfers under meduim loads so
> > it was removed.  I have/had (somewhere) a hack to work around scp for
> > protocol 1. That did not SEEM to cause problems.. But I was not happy
> > with it. 
> > 
> > I've not been able to verify but does the old 1.2.16 source tree from
> > ssh.com produce the same results?  There are some major rewrites in the
> > lower level channel support that may be the problem.  
> > 
> > - Ben
> > 
> > On Mon, 18 Dec 2000, Rachit Siamwalla wrote:
> > 
> > > does this happen if you test this through rsh?
> > > 
> > > If so, i've had a similar problem (unfortunately no good solution
> > > though). The way i "solved" the problem is writing my own shellscript
> > > that had the same effect. apachectl is probably a shell script too. try
> > > cutting out only what is needed and narrow the problem down from there.
> > > 
> > > -rchit
> > > 
> > > 
> > > Rob Hagopian wrote:
> > > > 
> > > > OK, with openssh-SNAP-20001218 I still have the problem of a hanging ssh
> > > > when running (one specific) command on a server. On the server side,
> > > > there's a small suid C prog (/usr/dh/mgmt/HupServer) that runs system
> > > > "/usr/local/apache/bin/apachectl stop/start". Incidentally, apachectl is a
> > > > shell script, hence the need for a suid wrapper for non-root users... On
> > > > the client side, there's a shell script that removes the machine from load
> > > > balancing and then stops/starts the webserver by sshing to the server and
> > > > running the C prog there.
> > > > 
> > > > The problem doesn't arise if I run the HupServer program from the command
> > > > line, only from within the shell script that does the load balancing
> > > > enable/disable. In that script, the 'stop' command has no problems, only
> > > > 'start'.
> > > > 
> > > > - I assumed that a fd was being left open, so I added:
> > > >         close(2); close(1); close(0); exit(0);
> > > >   to the C prog, but with no affect...
> > > > - Last I checked the -t switch had no affect...
> > > > - Hitting Ctrl-C on either debug run finishes the process up cleanly
> > > > - During the hanging there is no HupServer executable running on the
> > > >   server
> > > > - The problem was diminished, but not eliminated, by the prior patches to
> > > >   end the connection before the fds were closed.
> > > > 
> > > > Any ideas on how I can work around this? Would forking HupServer be of any
> > > > use?
> > > >                                                                 -Rob
> > > > 
> > > > The ssh dump:
> > > > 
> > > > SSH Version OpenSSH_2.3.0p2, protocol versions 1.5/2.0.
> > > > Compiled with SSL (0x0090581f).
> > > > debug: Reading configuration data /etc/ssh/ssh_config
> > > > debug: Applying options for *
> > > > debug: ssh_connect: getuid 500 geteuid 0 anon 0
> > > > debug: Connecting to dhumb400 [10.40.10.10] port 22.
> > > > debug: Allocated local port 1022.
> > > > debug: Connection established.
> > > > debug: identity file /home/prod/.ssh/identity type 0
> > > > debug: identity file /home/prod/.ssh/id_dsa type 3
> > > > debug: Remote protocol version 1.99, remote software version
> > > > OpenSSH_2.3.0p2
> > > > debug: no match: OpenSSH_2.3.0p2
> > > > Enabling compatibility mode for protocol 2.0
> > > > debug: Local version string SSH-2.0-OpenSSH_2.3.0p2
> > > > debug: Seeding random number generator
> > > > debug: send KEXINIT
> > > > debug: done
> > > > debug: wait KEXINIT
> > > > debug: got
> > > > kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
> > > > debug: got kexinit: ssh-dss,ssh-rsa
> > > > debug: got
> > > > kexinit: 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes128-cbc,aes192-cbc,aes256-cbc,rijndael128-cbc,rijndael192-cbc,rijndael256-cbc,rijndael-cbc at lysator.liu.se
> > > > debug: got
> > > > kexinit: 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes128-cbc,aes192-cbc,aes256-cbc,rijndael128-cbc,rijndael192-cbc,rijndael256-cbc,rijndael-cbc at lysator.liu.se
> > > > debug: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> > > > debug: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> > > > debug: got kexinit: none,zlib
> > > > debug: got kexinit: none,zlib
> > > > debug: got kexinit:
> > > > debug: got kexinit:
> > > > debug: first kex follow: 0
> > > > debug: reserved: 0
> > > > debug: done
> > > > debug: kex: server->client arcfour hmac-sha1 none
> > > > debug: kex: client->server arcfour hmac-sha1 none
> > > > debug: Sending SSH2_MSG_KEX_DH_GEX_REQUEST.
> > > > debug: Wait SSH2_MSG_KEX_DH_GEX_GROUP.
> > > > debug: Got SSH2_MSG_KEX_DH_GEX_GROUP.
> > > > debug: bits set: 1002/2049
> > > > debug: Sending SSH2_MSG_KEX_DH_GEX_INIT.
> > > > debug: Wait SSH2_MSG_KEX_DH_GEX_REPLY.
> > > > debug: Got SSH2_MSG_KEXDH_REPLY.
> > > > debug: Host 'dhumb400' is known and matches the RSA host key.
> > > > debug: bits set: 1034/2049
> > > > debug: ssh_rsa_verify: signature correct
> > > > debug: Wait SSH2_MSG_NEWKEYS.
> > > > debug: GOT SSH2_MSG_NEWKEYS.
> > > > debug: send SSH2_MSG_NEWKEYS.
> > > > debug: done: send SSH2_MSG_NEWKEYS.
> > > > debug: done: KEX2.
> > > > debug: send SSH2_MSG_SERVICE_REQUEST
> > > > debug: service_accept: ssh-userauth
> > > > debug: got SSH2_MSG_SERVICE_ACCEPT
> > > > debug: authentications that can continue: publickey,password
> > > > debug: next auth method to try is publickey
> > > > debug: try pubkey: /home/prod/.ssh/id_dsa
> > > > debug: read SSH2 private key done: name dsa w/o comment success 1
> > > > debug: sig size 20 20
> > > > debug: ssh-userauth2 successfull: method publickey
> > > > debug: channel 0: new [client-session]
> > > > debug: send channel open 0
> > > > debug: Entering interactive session.
> > > > debug: client_init id 0 arg 0
> > > > debug: Sending command: /usr/dh/mgmt/scripts/HupServer start
> > > > debug: channel 0: open confirm rwindow 0 rmax 16384
> > > > Environment:
> > > >   USER=prod
> > > >   LOGNAME=prod
> > > >   HOME=/usr/dh/home/prod
> > > >   PATH=/usr/dh/home/prod/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
> > > >   MAIL=/var/mail/prod
> > > >   SHELL=/usr/local/bin/tcsh
> > > >   SSH_CLIENT=10.40.12.11 1022 22
> > > > /usr/local/apache/bin/apachectl start: httpd started
> > > > debug: client_input_channel_req: rtype exit-status reply 0
> > > > Connection to dhumb400 closed by remote host.
> > > > debug: Transferred: stdin 0, stdout 0, stderr 47 bytes in 138.8 seconds
> > > > debug: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.3
> > > > debug: Exit status 0
> > > > 
> > > > and the sshd dump:
> > > > debug1: sshd version OpenSSH_2.3.0p2
> > > > debug1: load_private_key_autodetect: type 0 RSA1
> > > > debug1: read SSH2 private key done: name dsa w/o comment success 1
> > > > debug1: load_private_key_autodetect: type 2 DSA
> > > > debug1: read SSH2 private key done: name rsa w/o comment success 1
> > > > debug1: load_private_key_autodetect: type 1 RSA
> > > > debug1: Bind to port 22 on 0.0.0.0.
> > > > Server listening on 0.0.0.0 port 22.
> > > > Generating 768 bit RSA key.
> > > > RSA key generation complete.
> > > > debug1: Server will not fork when running in debugging mode.
> > > > Connection from 10.40.12.11 port 1022
> > > > debug1: Client protocol version 2.0; client software version
> > > > OpenSSH_2.3.0p2
> > > > debug1: no match: OpenSSH_2.3.0p2
> > > > Enabling compatibility mode for protocol 2.0
> > > > debug1: Local version string SSH-1.99-OpenSSH_2.3.0p2
> > > > debug1: list_hostkey_types: ssh-dss,ssh-rsa
> > > > debug1: send KEXINIT
> > > > debug1: done
> > > > debug1: wait KEXINIT
> > > > debug1: got
> > > > kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
> > > > debug1: got kexinit: ssh-rsa,ssh-dss
> > > > debug1: got kexinit: arcfour,3des-cbc,blowfish-cbc,cast128-cbc
> > > > debug1: got kexinit: arcfour,3des-cbc,blowfish-cbc,cast128-cbc
> > > > debug1: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> > > > debug1: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> > > > debug1: got kexinit: none
> > > > debug1: got kexinit: none
> > > > debug1: got kexinit:
> > > > debug1: got kexinit:
> > > > debug1: first kex follow: 0
> > > > debug1: reserved: 0
> > > > debug1: done
> > > > debug1: kex: client->server arcfour hmac-sha1 none
> > > > debug1: kex: server->client arcfour hmac-sha1 none
> > > > debug1: Wait SSH2_MSG_KEX_DH_GEX_REQUEST.
> > > > debug1: Sending SSH2_MSG_KEX_DH_GEX_GROUP.
> > > > debug1: bits set: 1034/2049
> > > > debug1: Wait SSH2_MSG_KEX_DH_GEX_INIT.
> > > > debug1: bits set: 1002/2049
> > > > debug1: send SSH2_MSG_NEWKEYS.
> > > > debug1: done: send SSH2_MSG_NEWKEYS.
> > > > debug1: Wait SSH2_MSG_NEWKEYS.
> > > > debug1: GOT SSH2_MSG_NEWKEYS.
> > > > debug1: done: KEX2.
> > > > debug1: userauth-request for user prod service ssh-connection method none
> > > > debug1: attempt #1
> > > > debug1: Starting up PAM with username "prod"
> > > > Failed none for prod from 10.40.12.11 port 1022 ssh2
> > > > debug1: userauth-request for user prod service ssh-connection method
> > > > publickey
> > > > debug1: attempt #2
> > > > debug1: matching key found: file /usr/dh/home/prod/.ssh/authorized_keys2,
> > > > line 2
> > > > debug1: len 55 datafellows 0
> > > > debug1: ssh_dss_verify: signature correct
> > > > Could not reverse map address 10.40.12.11.
> > > > debug1: PAM setting rhost to "10.40.12.11"
> > > > Accepted publickey for prod from 10.40.12.11 port 1022 ssh2
> > > > debug1: Entering interactive session for SSH2.
> > > > debug1: server_init_dispatch_20
> > > > debug1: server_input_channel_open: ctype session rchan 0 win 65536 max
> > > > 32768
> > > > debug1: input_session_request
> > > > debug1: channel 0: new [server-session]
> > > > debug1: session_new: init
> > > > debug1: session_new: session 0
> > > > debug1: session_open: channel 0
> > > > debug1: session_open: session 0: link with channel 0
> > > > debug1: server_input_channel_open: confirm session
> > > > debug1: session_by_channel: session 0 channel 0
> > > > debug1: session_input_channel_req: session 0 channel 0 request exec reply
> > > > 0
> > > > debug1: PAM establishing creds
> > > > debug1: fd 7 setting O_NONBLOCK
> > > > debug1: fd 7 IS O_NONBLOCK
> > > > debug1: fd 9 setting O_NONBLOCK
> > > > debug1: Received SIGCHLD.
> > > > debug1: tvp!=NULL kid 1 mili 100
> > > > debug1: session_by_pid: pid 34260
> > > > debug1: session_exit_message: session 0 channel 0 pid 34260
> > > > debug1: session_exit_message: release channel 0
> > > > debug1: channel 0: write failed
> > > > debug1: channel 0: output open -> closed
> > > > debug1: channel 0: close_write
> > > > debug1: session_free: session 0 pid 34260
> > > 
> > 
> 
>

More information about the openssh-unix-dev mailing list