[Bug 2945] New: sshd unstable on Illumos (and probably Solaris) when compiled in XPG4 mode

[bugzilla-daemon at bugzilla.mindrot.org]

https://bugzilla.mindrot.org/show_bug.cgi?id=2945

            Bug ID: 2945
           Summary: sshd unstable on Illumos (and probably Solaris) when
                    compiled in XPG4 mode
           Product: Portable OpenSSH
           Version: 7.9p1
          Hardware: amd64
                OS: Solaris
            Status: NEW
          Severity: major
          Priority: P5
         Component: sshd
          Assignee: unassigned-bugs at mindrot.org
          Reporter: openssh at sentfrom.com

Created attachment 3217
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3217&action=edit
Patch to check if the STREAMS modules are already pushed

My build of OpenSSH 7.9p1 on OpenIndiana oi151a9 and on SmartOS
joyent_20170302T081240Z was hanging after a few minutes of use.

After further investigation, the root cause was this issue in Illumos:
        https://www.illumos.org/issues/9042
(not sure if it is also present in Oracle’s version of Solaris,
probably so because the change dates to 2008)
Duplicate STREAMS terminal and line discipline modules were causing the
instability.

Dump of STREAMS modules in Solaris SSH:

ecthelion ~>m -p 22
mordac ~>strconf
ttcompat
ldterm
ptem
pts

In the affected OpenSSH 7.9p1

ecthelion ~>m -p 2223
Last login: Tue Dec 18 20:49:04 2018 from 38.99.40.226
mordac ~>strconf
ttcompat
ldterm
ptem
ttcompat
ldterm
ptem
ttcompat
ldterm
ptem
pts

The root cause is that if the binary is compiled in XPG4 compliant mode
(which is pretty much the default with reasonably modern versions of
GCC), libc pushes the module
http://src.illumos.org/source/xref/illumos-gate/usr/src/lib/libc/port/sys/open.c#163

but then again so does OpenSSH in bed_openpty.c, and we end up with
triplicate modules pushed, with unpredictable results that usually
manifest themselves as the connection hanging after a while:

Rather than checking if _XPG4 is defined to decide whether to push the
modules or not, the fix I adopted in the attached patch is to check if
the modules are already pushed before pushing them in
openbsd-compat/bsd-openpty.c. I am being conservative and made the
change Solaris-only as I have no idea if HP-UX supports the I_FIND
ioctl or not.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.