Ten second intermittent delay on login

[Nico Kadel-Garcia]

On Sat, Jul 22, 2023 at 3:20 PM Howard Chu <hyc at symas.com> wrote:
>
> Nico Kadel-Garcia wrote:
> > On Thu, Jul 20, 2023 at 1:49 PM Johnnie W Adams <jxadams at ualr.edu> wrote:
> >>
> >> Hi, folks,
> >>
> >>      We're experiencing an odd ten-second delay intermittently when logging
> >> into any of our Linux boxes which authenticate against LDAP. Here's where
> >> it happens:
> >>
> >> Jul 13 11:54:23 console2 sshd[1853]: debug1: temporarily_use_uid: <my
> >> uid\gid> (e=0/0)
> >>
> >> Jul 13 11:54:35 console2 sshd[1853]: debug1: trying public key file <my key
> >> file>
> >>
> >>      My assumption is there's something in sssd slowing it down, but I'm
> >> having a heck of a time figuring out what or why. Any guidance would be
> >> greatly appreciated.
> >>
> >> Thanks,
> >>
> >>      John A
> >
> > sssd is a pretty aggressively "optimized" tool. It's designed, not to
> > issue LDAP queries, but to pull from a locally stashed copy of the
> > *entire* upstream LDAP directory, or at least enough of the LDAP
> > directory to contain every dolder it may reference. The result can be
> > really nasty when the VPN connection between an internal AD and a
> > cloud environment, especially when it thinks it has to refresh that
> > cache. All of it. Without notice. And crash, if it doesn't succeed
> > within the hard-coded and un-tunable timeout periods.
> >
> > I'm not happy with some of sssd's behavior, especially the head games
> > it plays with systemd about "I'm started, I'm running, I'm allowing
> > logins via SSH, la-la-la-la-la, I failed to cache the full LDAP and
> > now I will crash hard with systemd not noticing and recovering the
> > service". It's an unpleasant problem.
>
> Sounds like you'd be better off using nslcd. And if you want caching
> of the LDAP info, use a local OpenLDAP slapd with slapo-pcache instead, which
> has all cache refresh/expiration/etc intervals fully configurable.

At that time, I probably would have been better off doing that for the
environment I was organizing. One of the problems of consulting work
is the need to leave something well documented and supportable by the
permanent staff. Another can be getting the people behind the
bureaucrati cwalls to admit what they're doing, and its consequences,
when you weren't hired as part of their department. I'm afraid that
they were fairly miffed at me for usin "ldapbrowser" to look around at
their AD structure. I was *not* supposed to have permission to look,
and the supervisor of security demanded that even his own personnel
not share insights with each other. It got weirder when I yanked out
the secret root user public SSH keys in the OS images I was supposed
to secure for deployment.

It took time to negotiate with the AD admins, and them to realize I
understood what I was asking for, to organize the relevant OU's in one
LDAP directory so sssd only had to stash that LDAP directory. It's a
good technique for improving sssd performance, especially in remotely
deployed environments without a high availability AD domain controller
at the end of a VPN for big cloud environments.

For our original poster here, the lesson is "take a look at your sssd
setup and see if you can reduce the relevant LDAP directory space".