Aw: Re: Howto log multiple sftpd instances with their chroot shared via NFS

[Hildegard Meier]

Thanks Peter,

I think both the LD_PRELOAD and overlay fs approach are too complicated and hacky for me. Additionally, the user's chroot NFS share is exported by a proprietary NFS storage appliance (Tegile), and I would like to keep that as a simple nfs server, as we experienced already some stramnge behaviour with that appliance in the past, and I am happy it is working now stable finally.

> Gesendet: Dienstag, 28. September 2021 um 20:27 Uhr
> Von: "Peter Stuge" <peter at stuge.se>
> An: openssh-unix-dev at mindrot.org
> Cc: "Hildegard Meier" <daku8938 at gmx.de>
> Betreff: Re: Howto log multiple sftpd instances with their chroot shared via NFS
>
> Hallo Hildegard,
>
> thanks for the summary.
>
> Hildegard Meier wrote:
> > ---------------------------------------------------------------
> > AllowGroups sftp-cust
> > Subsystem sftp internal-sftp -f LOCAL5 -l INFO
> > Match Group sftp-cust
> >     ChrootDirectory %h
> >     ForceCommand internal-sftp -u 0002 -f LOCAL5 -l INFO
> >     AllowTcpForwarding no
> > ---------------------------------------------------------------
>
> This is very sensible configuration but it unfortunately makes the
> LD_PRELOAD approach a bit less desirable. It will still work though.
>
>
> > So a solution for the log problem would be, to not use the chroot
> > logging, but to parse the global sftpd log
> ..
> > But I think this is not trivial to make this reliable and robust,
>
> Agreed. If going the log analysis path I would add -e to the
> internal-sftp command line and run sshd with stderr connected to that
> parser, to be able to keep track of events in a sensible way.
> Log rotation would happen only after (or in) the parser.
>
>
> > If we want to try to keep using the available session chroot
> > logging, man (8) sftp-server says:
> >
> > "For logging to work, sftp-server must be able to access /dev/log.  Use of sftp-server in a chroot configuration therefore requires that
> >      syslogd(8) establish a logging socket inside the chroot directory."
> >
> > As Peter has written here
> > https://lists.mindrot.org/pipermail/openssh-unix-dev/2021-September/039669.html
> >
> > as I understand it is hard coded in the system C library that the log
> > devices name is /dev/log, so this cannot be changed (e.g. to log to
> > chrooted /dev/log_hostname1 on hostname1 and chrooted /dev/log_hostname2
> > on hostname2).
>
> Using LD_PRELOAD e.g. with the code I attached to that email allows
> replacing the C library syslog functionality. My attachment does
> indeed log to /dev/log_hostname1 on hostname1 instead of /dev/log.
>
> However because of internal-sftp (which I still prefer over the alternative;
> maintaining an sftp-server binary and possibly libraries in every homedir)
> the entire sshd would have to be run with LD_PRELOAD.
>
> Note that LD_PRELOAD is only neccessary on n-1 servers; the first
> server can still use /dev/log and the full C library syslog() code.
>
> If the second server where LD_PRELOAD is used does not need to
> support anything besides SFTP, or if you can mandate that, then
> it could be okay to run the entire sshd with LD_PRELOAD. Your IT
> security colleagues may well object and they're not wrong; it's not
> much code but still quite a mighty hammer.
>
>
> > > Maybe someone with more Linux/NFS wit could suggest an OS-side solution
> > > for you?
>
> Before going down the LD_PRELOAD path I considered an overlayfs solution.
>
> It would be nice, but the writable (upper) directory can't be NFS
> with overlayfs so it can not work on the SFTP servers.
>
> But using overlayfs can work on the NFS server. There would then be
> one export for each SFTP server, all of them reaching the same
> homedir contents except for different (empty) <homedir>/dev/ subdirs.
>
> This requires maintaining a second homedir structure on the NFS server,
> but that structure only needs to contain a homedir with one empty dev
> subdir under and nothing else, so it may be doable. Homedirs need
> correct permissions and only one of the two directory inodes for the
> homedir itself will be changed by the respective SFTP server, but on
> the plus side it requires only a single overlay mount, nothing per-user
> and no LD_PRELOAD stuff on the SFTP servers.
>
>
> On the NFS server, assuming that /home is where real homedirs exist
> (this gets mounted directly by one SFTP server), that /var/home2 is
> the second homedir structure with only homedirs and an empty dev subdir
> in each, that /var/home2_work is an empty directory on the same filesystem
> as /var/home2 and that /nfs/home2 is the second export mounted by the
> second SFTP server, then set it up like so:
>
> mount -t overlay overlay -olowerdir=/home,upperdir=/var/home2,workdir=/var/home2_work /nfs/home2
>
>
> Then export /home and /nfs/home2 via NFS.
>
> The SFTP server mouting /home will access /home/<user>/dev/log
> (both syslog-ng and internal-sftp) and the SFTP server mouting
> /nfs/home2 will instead access /var/home2/<user>/dev/log but
> everything else within the homedir is to and from /home/<user>.
>
> If this change can be done on the NFS server I think it's what I'd prefer.
>
>
> Kind regards
>
> //Peter
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev at mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>