Hung ssh client connection

[Darren Tucker]

Damien Mascord wrote:
> Darren Tucker wrote:
>>When you "reset" your firewall, it flushes its state table, right?  If
>>so then would probably include the SSH TCP connection too.
> 
> *nods* ok, makes sense, so because the firewall isn't responding at all
> to the SSH connection (that it no longer knows about), the client is
> just waiting until it either times out or gets a connect reset ?

Yes.  There's nothing else it can do.

>>One place this might show up is in "netstat" on the client: if there's a
>>constant value in the "Send-Q" column for the ssh connection to your
>>firewall then that's almost certainly what's happening.
> 
> The Send-Q starts at 0, and starts increasing as I 'send' characters
> through the ssh client, until it grows until 5632:
[...]
> (Where does the value 5632 come from?)

It's just the number of bytes that have been sent by the client and not 
yet acknowledged by the server.  If it stops growing at 5632 then that's 
probably the limit of the send queue in the kernel (in which case strace 
would probably show writes to the network descriptor either blocking or 
returning EAGAIN).

[...]
> So, because the firewall silently ignores the ACKs sent to it, the
> client has nothing to do except wait for some indication?

Yes.

> Would the following sysctl value have an effect on the timeout ?
> 
> net.ipv4.tcp_retries2 = 15 # Defines how many times a TCP packet is
> retransmitted in established state before giving up.

Probably.  I've never tried it.

You could also set the ServerAliveInterval and ServerAliveCountMax in 
~/.ssh/config or ssh_config to make it detect the failure faster.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
     Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.