[openssh] Re: Frequent "Connection reset by peer"

[Darren Tucker]

Tuc at T-B-O-H.NET wrote:
>> Tuc at T-B-O-H wrote:
>>> 	Is there a way to tell why I'm getting frequent 
>>> "Connection reset by peer" all of a sudden? 
>>>
>>> 	I'm using a FreeBSD machine, plugged via eithernet to a Linksys 
>>> router running DD-WRT, to another Linksys 50 feet away running DD-WRT, 
>>> both of them WDS'd together, plugged via ethernet to another FreeBSD machine.
>>> With debug cranked, I see :
>> 1) check the SSH server's logs to see if there's any logged message 
>> corresponding to the disconnects.  If the problem is easy to reproduce, 
>> you could run sshd in debug mode in the foreground (eg "/path/to/sshd 
>> -ddde -p 2022" then connect to port 22).
>>
>> 2) if that doesn't have anything, or has the same "connection reset by 
>> peer message" then you probably will need to run tcpdump or similar to 
>> see which end is generating the TCP RST (in theory it should be only 
>> generated by one of the TCP endpoints but it could also be generated by, 
>> eg an intermediate NAT box).
>>
> Hi,
> 
> 	Thanks for the reply...
> 
> 	1) Nothing out of the ordinary in the logs. 
> 
> 	Last bit of logging from the server before/after it happens :
> 
> debug3: tty_parse_modes: 93 0
> debug1: server_input_channel_req: channel 0 request shell reply 0
> debug1: session_by_channel: session 0 channel 0
> debug1: session_input_channel_req: session 0 req shell
> debug1: PAM: setting PAM_TTY to "/dev/ttyp4"
> debug2: fd 5 setting TCP_NODELAY
> debug2: channel 0: rfd 10 isatty
> debug2: fd 10 setting O_NONBLOCK
> debug2: fd 9 is O_NONBLOCK
> debug1: Setting controlling tty using TIOCSCTTY.
> debug3: mm_answer_pty: tty /dev/ttyp4 ptyfd 3
> debug3: mm_request_receive entering
> Read error from remote host 10.0.0.1: Connection reset by peer
> debug1: do_cleanup
> debug1: PAM: cleanup
> debug3: PAM: sshpam_thread_cleanup entering
> debug1: do_cleanup
> debug1: PAM: cleanup
> debug3: PAM: sshpam_thread_cleanup entering
> debug1: session_pty_cleanup: session 0 release /dev/ttyp4
> 
> 	2) TCPDUMP available to anyone that wants it via email (21K)
> 
> The last "good" packet between the SSH server and client looked
> pretty normal, has Flags of PSH and ACK.
> 
> The next packet comes from the router closest to the SSH server.
> There is no sequence or acknowledgement number, but its claiming
> to originate from the same port that the client originally was
> using.  The window size is 1/2 of what the client/server window 
> was. Its sending to the SSH server an ACK. 
> 
> The next packet is from the SSH server to the router, there
> is no sequence number, window size is 0, Flags are RST.
> 
> Next packet is from the SSH server to the client, marked
> as a TCP retransmission, same sequence as the last "good"
> packet, PSH,ACK

Check the IP TOS bits on the packets as I think that's around the point 
that it sets IPTOS_LOWDELAY.  I have heard that some NAT/firewall 
devices can't handle a connection where the TOS changes mid-connection.

If you have netcat installed, you can try this:

ssh -o "ProxyCommand nc %h %p" yourserver

as a quick test.

> Last packet is from SSH client to server, with RST Flag
> set.
> 
> 	So why is the router jumping into the middle of
> the conversation and being a spoil sport?
> 
> 		Thanks, Tuc
> 


-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
     Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.