Hanging ssh

Tue Dec 19 07:15:36 EST 2000

does this happen if you test this through rsh?

If so, i've had a similar problem (unfortunately no good solution
though). The way i "solved" the problem is writing my own shellscript
that had the same effect. apachectl is probably a shell script too. try
cutting out only what is needed and narrow the problem down from there.

-rchit

Rob Hagopian wrote:
> 
> OK, with openssh-SNAP-20001218 I still have the problem of a hanging ssh
> when running (one specific) command on a server. On the server side,
> there's a small suid C prog (/usr/dh/mgmt/HupServer) that runs system
> "/usr/local/apache/bin/apachectl stop/start". Incidentally, apachectl is a
> shell script, hence the need for a suid wrapper for non-root users... On
> the client side, there's a shell script that removes the machine from load
> balancing and then stops/starts the webserver by sshing to the server and
> running the C prog there.
> 
> The problem doesn't arise if I run the HupServer program from the command
> line, only from within the shell script that does the load balancing
> enable/disable. In that script, the 'stop' command has no problems, only
> 'start'.
> 
> - I assumed that a fd was being left open, so I added:
>         close(2); close(1); close(0); exit(0);
>   to the C prog, but with no affect...
> - Last I checked the -t switch had no affect...
> - Hitting Ctrl-C on either debug run finishes the process up cleanly
> - During the hanging there is no HupServer executable running on the
>   server
> - The problem was diminished, but not eliminated, by the prior patches to
>   end the connection before the fds were closed.
> 
> Any ideas on how I can work around this? Would forking HupServer be of any
> use?
>                                                                 -Rob
> 
> The ssh dump:
> 
> SSH Version OpenSSH_2.3.0p2, protocol versions 1.5/2.0.
> Compiled with SSL (0x0090581f).
> debug: Reading configuration data /etc/ssh/ssh_config
> debug: Applying options for *
> debug: ssh_connect: getuid 500 geteuid 0 anon 0
> debug: Connecting to dhumb400 [10.40.10.10] port 22.
> debug: Allocated local port 1022.
> debug: Connection established.
> debug: identity file /home/prod/.ssh/identity type 0
> debug: identity file /home/prod/.ssh/id_dsa type 3
> debug: Remote protocol version 1.99, remote software version
> OpenSSH_2.3.0p2
> debug: no match: OpenSSH_2.3.0p2
> Enabling compatibility mode for protocol 2.0
> debug: Local version string SSH-2.0-OpenSSH_2.3.0p2
> debug: Seeding random number generator
> debug: send KEXINIT
> debug: done
> debug: wait KEXINIT
> debug: got
> kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
> debug: got kexinit: ssh-dss,ssh-rsa
> debug: got
> kexinit: 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes128-cbc,aes192-cbc,aes256-cbc,rijndael128-cbc,rijndael192-cbc,rijndael256-cbc,rijndael-cbc at lysator.liu.se
> debug: got
> kexinit: 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes128-cbc,aes192-cbc,aes256-cbc,rijndael128-cbc,rijndael192-cbc,rijndael256-cbc,rijndael-cbc at lysator.liu.se
> debug: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> debug: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> debug: got kexinit: none,zlib
> debug: got kexinit: none,zlib
> debug: got kexinit:
> debug: got kexinit:
> debug: first kex follow: 0
> debug: reserved: 0
> debug: done
> debug: kex: server->client arcfour hmac-sha1 none
> debug: kex: client->server arcfour hmac-sha1 none
> debug: Sending SSH2_MSG_KEX_DH_GEX_REQUEST.
> debug: Wait SSH2_MSG_KEX_DH_GEX_GROUP.
> debug: Got SSH2_MSG_KEX_DH_GEX_GROUP.
> debug: bits set: 1002/2049
> debug: Sending SSH2_MSG_KEX_DH_GEX_INIT.
> debug: Wait SSH2_MSG_KEX_DH_GEX_REPLY.
> debug: Got SSH2_MSG_KEXDH_REPLY.
> debug: Host 'dhumb400' is known and matches the RSA host key.
> debug: bits set: 1034/2049
> debug: ssh_rsa_verify: signature correct
> debug: Wait SSH2_MSG_NEWKEYS.
> debug: GOT SSH2_MSG_NEWKEYS.
> debug: send SSH2_MSG_NEWKEYS.
> debug: done: send SSH2_MSG_NEWKEYS.
> debug: done: KEX2.
> debug: send SSH2_MSG_SERVICE_REQUEST
> debug: service_accept: ssh-userauth
> debug: got SSH2_MSG_SERVICE_ACCEPT
> debug: authentications that can continue: publickey,password
> debug: next auth method to try is publickey
> debug: try pubkey: /home/prod/.ssh/id_dsa
> debug: read SSH2 private key done: name dsa w/o comment success 1
> debug: sig size 20 20
> debug: ssh-userauth2 successfull: method publickey
> debug: channel 0: new [client-session]
> debug: send channel open 0
> debug: Entering interactive session.
> debug: client_init id 0 arg 0
> debug: Sending command: /usr/dh/mgmt/scripts/HupServer start
> debug: channel 0: open confirm rwindow 0 rmax 16384
> Environment:
>   USER=prod
>   LOGNAME=prod
>   HOME=/usr/dh/home/prod
>   PATH=/usr/dh/home/prod/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
>   MAIL=/var/mail/prod
>   SHELL=/usr/local/bin/tcsh
>   SSH_CLIENT=10.40.12.11 1022 22
> /usr/local/apache/bin/apachectl start: httpd started
> debug: client_input_channel_req: rtype exit-status reply 0
> Connection to dhumb400 closed by remote host.
> debug: Transferred: stdin 0, stdout 0, stderr 47 bytes in 138.8 seconds
> debug: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.3
> debug: Exit status 0
> 
> and the sshd dump:
> debug1: sshd version OpenSSH_2.3.0p2
> debug1: load_private_key_autodetect: type 0 RSA1
> debug1: read SSH2 private key done: name dsa w/o comment success 1
> debug1: load_private_key_autodetect: type 2 DSA
> debug1: read SSH2 private key done: name rsa w/o comment success 1
> debug1: load_private_key_autodetect: type 1 RSA
> debug1: Bind to port 22 on 0.0.0.0.
> Server listening on 0.0.0.0 port 22.
> Generating 768 bit RSA key.
> RSA key generation complete.
> debug1: Server will not fork when running in debugging mode.
> Connection from 10.40.12.11 port 1022
> debug1: Client protocol version 2.0; client software version
> OpenSSH_2.3.0p2
> debug1: no match: OpenSSH_2.3.0p2
> Enabling compatibility mode for protocol 2.0
> debug1: Local version string SSH-1.99-OpenSSH_2.3.0p2
> debug1: list_hostkey_types: ssh-dss,ssh-rsa
> debug1: send KEXINIT
> debug1: done
> debug1: wait KEXINIT
> debug1: got
> kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
> debug1: got kexinit: ssh-rsa,ssh-dss
> debug1: got kexinit: arcfour,3des-cbc,blowfish-cbc,cast128-cbc
> debug1: got kexinit: arcfour,3des-cbc,blowfish-cbc,cast128-cbc
> debug1: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> debug1: got kexinit: hmac-sha1,hmac-md5,hmac-ripemd160 at openssh.com
> debug1: got kexinit: none
> debug1: got kexinit: none
> debug1: got kexinit:
> debug1: got kexinit:
> debug1: first kex follow: 0
> debug1: reserved: 0
> debug1: done
> debug1: kex: client->server arcfour hmac-sha1 none
> debug1: kex: server->client arcfour hmac-sha1 none
> debug1: Wait SSH2_MSG_KEX_DH_GEX_REQUEST.
> debug1: Sending SSH2_MSG_KEX_DH_GEX_GROUP.
> debug1: bits set: 1034/2049
> debug1: Wait SSH2_MSG_KEX_DH_GEX_INIT.
> debug1: bits set: 1002/2049
> debug1: send SSH2_MSG_NEWKEYS.
> debug1: done: send SSH2_MSG_NEWKEYS.
> debug1: Wait SSH2_MSG_NEWKEYS.
> debug1: GOT SSH2_MSG_NEWKEYS.
> debug1: done: KEX2.
> debug1: userauth-request for user prod service ssh-connection method none
> debug1: attempt #1
> debug1: Starting up PAM with username "prod"
> Failed none for prod from 10.40.12.11 port 1022 ssh2
> debug1: userauth-request for user prod service ssh-connection method
> publickey
> debug1: attempt #2
> debug1: matching key found: file /usr/dh/home/prod/.ssh/authorized_keys2,
> line 2
> debug1: len 55 datafellows 0
> debug1: ssh_dss_verify: signature correct
> Could not reverse map address 10.40.12.11.
> debug1: PAM setting rhost to "10.40.12.11"
> Accepted publickey for prod from 10.40.12.11 port 1022 ssh2
> debug1: Entering interactive session for SSH2.
> debug1: server_init_dispatch_20
> debug1: server_input_channel_open: ctype session rchan 0 win 65536 max
> 32768
> debug1: input_session_request
> debug1: channel 0: new [server-session]
> debug1: session_new: init
> debug1: session_new: session 0
> debug1: session_open: channel 0
> debug1: session_open: session 0: link with channel 0
> debug1: server_input_channel_open: confirm session
> debug1: session_by_channel: session 0 channel 0
> debug1: session_input_channel_req: session 0 channel 0 request exec reply
> 0
> debug1: PAM establishing creds
> debug1: fd 7 setting O_NONBLOCK
> debug1: fd 7 IS O_NONBLOCK
> debug1: fd 9 setting O_NONBLOCK
> debug1: Received SIGCHLD.
> debug1: tvp!=NULL kid 1 mili 100
> debug1: session_by_pid: pid 34260
> debug1: session_exit_message: session 0 channel 0 pid 34260
> debug1: session_exit_message: release channel 0
> debug1: channel 0: write failed
> debug1: channel 0: output open -> closed
> debug1: channel 0: close_write
> debug1: session_free: session 0 pid 34260