openssh PTY allocation

[Damien Miller]

Try this compat hack:


Index: ttymodes.c
===================================================================
RCS file: /cvs/src/usr.bin/ssh/ttymodes.c,v
retrieving revision 1.29
diff -u -p -r1.29 ttymodes.c
--- ttymodes.c	2 Nov 2008 00:16:16 -0000	1.29
+++ ttymodes.c	29 Jul 2011 07:58:29 -0000
@@ -295,8 +295,11 @@ tty_make_modes(int fd, struct termios *t
 	put_arg(&buf, tio.c_cc[NAME]);
 
 #define TTYMODE(NAME, FIELD, OP) \
-	buffer_put_char(&buf, OP); \
-	put_arg(&buf, ((tio.FIELD & NAME) != 0));
+	if (!compat20 || (datafellows & SSH_BUG_SCREENOS_PTY) == 0 || \
+	    buffer_len(&buf) < 256 - 5) { \
+		buffer_put_char(&buf, OP); \
+		put_arg(&buf, ((tio.FIELD & NAME) != 0)); \
+	}
 
 #include "ttymodes.h"
 
Index: compat.c
===================================================================
RCS file: /cvs/src/usr.bin/ssh/compat.c,v
retrieving revision 1.78
diff -u -p -r1.78 compat.c
--- compat.c	11 Sep 2008 14:22:37 -0000	1.78
+++ compat.c	29 Jul 2011 07:58:29 -0000
@@ -146,6 +146,8 @@ compat_datafellows(const char *version)
 					SSH_BUG_IGNOREMSG },
 		{ "*SSH Compatible Server*",			/* Netscreen */
 					SSH_BUG_PASSWORDPAD },
+		{ "NetScreen",
+					SSH_BUG_SCREENOS_PTY },
 		{ "*OSU_0*,"
 		  "OSU_1.0*,"
 		  "OSU_1.1*,"
Index: compat.h
===================================================================
RCS file: /cvs/src/usr.bin/ssh/compat.h,v
retrieving revision 1.42
diff -u -p -r1.42 compat.h
--- compat.h	11 Sep 2008 14:22:37 -0000	1.42
+++ compat.h	29 Jul 2011 07:58:29 -0000
@@ -58,6 +58,7 @@
 #define SSH_OLD_FORWARD_ADDR	0x01000000
 #define SSH_BUG_RFWD_ADDR	0x02000000
 #define SSH_NEW_OPENSSH		0x04000000
+#define SSH_BUG_SCREENOS_PTY	0x08000000
 
 void     enable_compat13(void);
 void     enable_compat20(void);

On Thu, 28 Jul 2011, Morty Abzug wrote:

> On Thu, Jul 28, 2011 at 06:00:38PM +0200, Gert Doering wrote:
> > Hi,
> > 
> > On Thu, Jul 28, 2011 at 11:52:47AM -0400, Morty Abzug wrote:
> > > On Wed, Jul 27, 2011 at 05:25:05PM +1000, Damien Miller wrote:
> > > 
> > > > The problem is a bug in ScreenOS, it refuses pty-req channel requests
> > > > when the tty modes blob exceeds 256 bytes in length. If you want a
> > > > workaround that preserves the usability of the tty, then comment out
> > > > a couple of less-important modes in ttymodes.h and recompile
> > > 
> > > Any suggestions on which modes are less important?
> > 
> > In that context, I think CS7, PARENB, PARODDB, IXON, IXOFF, IXANY, IUCLC,
> > PARMRK would be the ones I'd skip, given that use of 7-bit and parity
> > terminals is unlikely, and that the netscreens are not going to honour
> > xon/xoff flow control (IXON/IXOFF/IXANY) anyway.
> 
> Thanks.
> 
> I tested with #ifdef all of the above (CS7, PARENB, PARODDB, IXON,
> IXOFF, IXANY, IUCLC, and PARMRK.)  This worked to get to one of our
> firewalls (ScreenOS 6.3.0r7.0) but not another (ScreenOS 5.3.0r3.0).
> So the problem appears to depend to some extent on ScreenOS version or
> some other variable that is device-specific.
> 
> Meanwhile, I have that other workaround, i.e. to make the ssh client
> not consider PTY allocation failure a fatal exit.  It appears to work
> for all of our ScreenOS devices.
> 
> Questions/comments:
> 
> (1) From a patch perspective, which approach is preferable -- making
>     PTY allocation failure not a fatal error, or commenting out a
>     bunch of ttymodes?  [Assuming a set of ttymodes can be found that
>     makes this work, of course.]  I would lean towards the former
>     approach, since it seems inherently more robust/consistent.
> 
> (2) Commenting out stuff in ttymodes.h thing appears to be a
>     compile-time option.  Is there a way to make it a run-time option?
> 
> (3) What would be a good name for an option to workaround this?  I
>     lean towards ExitOnTTYFailure.
> 
> (4) What would be a good name for an option to workaround the scp "--"
>     problem?
> 
> - Morty
>