SuSE Linux Enterprise Server OpenSSH 5.1p1 nagle issue?

[Jeremy Guthrie]

I have another bit of detail, I think SSH keepalives are part of the 
issue.  This test didn't have me using the same values for keepalives 
that I use in production so I am going to re-conduct the test with 
matching SSH settings like in production.

When things look good:
debug2: channel 0: window 1966046 sent adjust 131106
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1998272 sent adjust 98880
debug2: channel 0: window 1982464 sent adjust 114688
debug2: channel 0: rcvd ext data 36
Wed Oct 17 15:26:19 2012 0.004193
debug2: channel 0: written 36 to e
d 6
debug2: channel 0: window 1966044 sent adjust 131108
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1998272 sent adjust 98880
debug2: channel 0: window 1982464 sent adjust 114688
debug2: channel 0: rcvd ext data 36
Wed Oct 17 15:26:34 2012 0.004004
debug2: channel 0: written 36 to e

When things are slow:
debug2: channel 0: window 1998812 sent adjust 98340
debug1: client_input_channel_req: channel 0 rtype keepalive at openssh.com 
reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive at openssh.com 
reply 1
debug2: channel 0: window 1966080 sent adjust 131072
debug1: client_input_channel_req: channel 0 rtype keepalive at openssh.com 
reply 1
debug2: channel 0: window 1966080 sent adjust 131072
debug1: client_input_channel_req: channel 0 rtype keepalive at openssh.com 
reply 1
debug2: channel 0: window 1966080 sent adjust 131072
debug1: client_input_channel_req: channel 0 rtype keepalive at openssh.com 
reply 1
debug2: channel 0: window 1966080 sent adjust 131072
debug1: client_input_channel_req: channel 0 rtype keepalive at openssh.com 
reply 1
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: window 1966080 sent adjust 131072
debug2: channel 0: rcvd ext data 37
Wed Oct 17 15:30:21 2012 196.41162



On 10/17/12 3:28 PM, Jeremy Guthrie wrote:
> One node: packet lengths > 1500 are TSO taking affect, if I turn off 
> TSO, I see under normal conditions the appropriate size/sequence of 
> packets.
>
> Example of the connection performing correctly:
> 450   1.746243 SSH 2642 Encrypted response packet len=2576 clientend 
> -> headend
> 451   1.746347 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074656266 
> Win=16666 Len=0 TSval=1456827748 TSecr=798139682 headend -> clientend
> 452   1.746369 SSH 3930 Encrypted response packet len=3864 clientend 
> -> headend
> 453   1.746386 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074657554 
> Win=16666 Len=0 TSval=1456827748 TSecr=798139682 headend -> clientend
> 454   1.746396 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074654978 
> Win=16666 Len=0 TSval=1456827748 TSecr=798139682 headend -> clientend
> 455   1.791951 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074660130 
> Win=16666 Len=0 TSval=1456827760 TSecr=798139694 headend -> clientend
> 456   1.791973 SSH 5218 Encrypted response packet len=5152 clientend 
> -> headend
> 457   1.792501 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074662706 
> Win=16666 Len=0 TSval=1456827760 TSecr=798139694 headend -> clientend
> 458   1.792692 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074665282 
> Win=16666 Len=0 TSval=1456827760 TSecr=798139694 headend -> clientend
> 459   1.792718 SSH 5218 Encrypted response packet len=5152 clientend 
> -> headend
> 460   1.792741 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074667858 
> Win=16666 Len=0 TSval=1456827760 TSecr=798139694 headend -> clientend
> 461   1.849577 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074670434 
> Win=16666 Len=0 TSval=1456827774 TSecr=798139706 headend -> clientend
> 462   1.849600 SSH 5218 Encrypted response packet len=5152 clientend 
> -> headend
> 463   1.849620 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074673010 
> Win=16666 Len=0 TSval=1456827774 TSecr=798139706 headend -> clientend
> 464   1.849746 TCP 66 43269 > 22 [ACK] Seq=555192630 Ack=3074675586 
> Win=16666 Len=0 TSval=1456827774 TSecr=798139706 headend -> clientend
> 465   1.849764 SSH 6506 Encrypted response packet len=6440 clientend 
> -> headend
>
> Example of the connection showing a ping-ponging effect using only 
> 1288 byte packets
> 1288  41.112341 TCP 66 43269 > 22 [ACK] Seq=555193638 Ack=3075980138 
> Win=16666 Len=0 TSval=1456837590 TSecr=798149524 headend -> clientend
> 1289  41.386131 SSH 1354 Encrypted response packet len=1288 clientend 
> -> headend
> 1290  41.432684 TCP 66 43269 > 22 [ACK] Seq=555193638 Ack=3075981426 
> Win=16666 Len=0 TSval=1456837670 TSecr=798149604 headend -> clientend
> 1291  41.702157 SSH 1354 Encrypted response packet len=1288 clientend 
> -> headend
> 1292  41.753245 TCP 66 43269 > 22 [ACK] Seq=555193638 Ack=3075982714 
> Win=16666 Len=0 TSval=1456837750 TSecr=798149683 headend -> clientend
> 1293  42.022128 SSH 1354 Encrypted response packet len=1288 clientend 
> -> headend
>
>
>


-- 
*Jeremy Guthrie*
Technical Architect - Orchestration | *CDW*
5520 Research Park | Madison, WI 53711
Phone: 608.298.1061 | Fax: 608.288.3007 | NOC: 608.298.1102 | Toll Free: 
866.202.1807