openssh PTY allocation

[Morty Abzug]

On Thu, Jul 28, 2011 at 06:00:38PM +0200, Gert Doering wrote:
> Hi,
> 
> On Thu, Jul 28, 2011 at 11:52:47AM -0400, Morty Abzug wrote:
> > On Wed, Jul 27, 2011 at 05:25:05PM +1000, Damien Miller wrote:
> > 
> > > The problem is a bug in ScreenOS, it refuses pty-req channel requests
> > > when the tty modes blob exceeds 256 bytes in length. If you want a
> > > workaround that preserves the usability of the tty, then comment out
> > > a couple of less-important modes in ttymodes.h and recompile
> > 
> > Any suggestions on which modes are less important?
> 
> In that context, I think CS7, PARENB, PARODDB, IXON, IXOFF, IXANY, IUCLC,
> PARMRK would be the ones I'd skip, given that use of 7-bit and parity
> terminals is unlikely, and that the netscreens are not going to honour
> xon/xoff flow control (IXON/IXOFF/IXANY) anyway.

Thanks.

I tested with #ifdef all of the above (CS7, PARENB, PARODDB, IXON,
IXOFF, IXANY, IUCLC, and PARMRK.)  This worked to get to one of our
firewalls (ScreenOS 6.3.0r7.0) but not another (ScreenOS 5.3.0r3.0).
So the problem appears to depend to some extent on ScreenOS version or
some other variable that is device-specific.

Meanwhile, I have that other workaround, i.e. to make the ssh client
not consider PTY allocation failure a fatal exit.  It appears to work
for all of our ScreenOS devices.

Questions/comments:

(1) From a patch perspective, which approach is preferable -- making
    PTY allocation failure not a fatal error, or commenting out a
    bunch of ttymodes?  [Assuming a set of ttymodes can be found that
    makes this work, of course.]  I would lean towards the former
    approach, since it seems inherently more robust/consistent.

(2) Commenting out stuff in ttymodes.h thing appears to be a
    compile-time option.  Is there a way to make it a run-time option?

(3) What would be a good name for an option to workaround this?  I
    lean towards ExitOnTTYFailure.

(4) What would be a good name for an option to workaround the scp "--"
    problem?

- Morty