Aw: Re: Howto log multiple sftpd instances with their chroot shared via NFS

[Hildegard Meier]

Thanks David,

I think you describe the problem correctly, and your approach could work, but unfortunately it would be a way too complicated mess.

As I had written in my summary
https://lists.mindrot.org/pipermail/openssh-unix-dev/2021-September/039677.html

"Since we have 800 users, it would be impractical unrobust to use user-specifc e.g. bind) mounts (e.g. 800 bind-over-mounts). To keep it simple, clear and coherent, all user's homes must be on the same one singular NFS-Share."
 
 Best regards
 

Gesendet: Mittwoch, 29. September 2021 um 17:00 Uhr
Von: "David Newall" <openssh at davidnewall.com>
An: "Hildegard Meier" <daku8938 at gmx.de>
Cc: "Damien Miller" <djm at mindrot.org>, "OpenSSH List" <openssh-unix-dev at mindrot.org>
Betreff: Re: Howto log multiple sftpd instances with their chroot shared via NFS
Hi Hildegard,
This, I believe, explains your problem and how to achieve your goal:

On 29/9/21 7:55 pm, Hildegard Meier wrote:
What is /dev/log on each server (not the one in chroot).  I.e. output of ls -l /dev/log
On the next generation sftp server (Ubuntu 18.04.1 LTS with it's shipped OpenSSH 7.6p1-4ubuntu0.5):
ls -l /dev/log
lrwxrwxrwx 1 root root 28 Aug 27 11:54 /dev/log -> /run/systemd/journal/dev-log

On the old (to be replaced with the new) sftp server (Ubuntu 12.04.5 LTS with it's shipped OpenSSH 5.9p1-5ubuntu1.10):
ls -l /dev/log
srw-rw-rw- 1 root root 0 Sep 23 15:07 /dev/log
On Linux, a process opens /dev/log when it first uses syslog(3).  It writes log messages there.  On your new server, /dev/log is a link to /run/systemd/journal/dev-log, which is a UNIX domain socket.  On the old server, /dev/log is /dev/log, which is also a UNIX domain socket.
The syslog daemon conventionally also opens /dev/log to read incoming log messages.  That's how log messages get from arbitrary process to syslog daemon: they communicate via the same UNIX domain socket.
For external sftp, what is /dev/log in each NFS chroot?  I.e. output of ls -l /var/data/chroot/dev/log
Both above sftp servers (old and new generation) mount /var/data/chroot/ from a third NFS server appliance.
Every sftp user has it's (chroot) home dir /var/data/chroot/<username>/ under that single NFS mount.

E.g. for the user "sftp_nagios" this is 

ls  -l /var/data/chroot/sftp_nagios/dev/log
srw-rw-rw-+ 1 root root 0 Sep 29 11:26 /var/data/chroot/sftp_nagios/dev/log
When the syslog daemon starts, it needs to be able to open (in fact, create) the same UNIX domain socket that the external sftp server will write to.  When it starts!  That means, the socket that the external sftp server will write to must be visible in the chroot environment and also must be visible when syslog starts.
I'm imagining you have an NFS share to mount over /var/data/chroot on both of your SFTP servers so that you can pre-populate an empty directory for each sftp user.  I'm imagining that you NFS mount each user's home directory inside that when the user logs in (i.e. connects using sftp, in your case).
Finally, I infer that when a user connects with sftp, they get an external sftp process chrooted to /var/data/chroot/<username> with a /dev/log inside that.  This can't work.  The syslog daemon on neither old nor new server can open /var/data/chroot/<username>/dev/log at startup because there is no such file at startup.
What you should do is create /var/data/chroot/{dev,home} on both servers.  These are local directories, and not NFS mounts.  This allows you to configure syslog daemon to read log messages from /var/data/chroot/dev/log (wich syslog will create when it starts.)  You can't NFS mount over /var/data/chroot because you've got two different sftp servers, each running syslog daemon.  The server that starts second will remove the /var/data/chroot/dev/log socket that the first one created, and create it's own socket in that place.  The /var/data/chroot/dev directory must be truly local on each machine.
You'll also have /var/data/chroot/lib/sftp-server (or /var/data/chroot/lib/openssh/sftp-server) on each machine, so that sshd can exec that inside the chroot when a user connects.  The user will be in chroot at /var/data/chroot.
You mount the NFS share containing the empty directories for each user.  You mount it at /var/data/chroot/home.
You dynamically mount the user's home directory at /var/data/chroot/home/<username>.
When a user logs in using sftp, they're in a chroot at /var/data/chroot with their home directory at /home/<username> (as they see it).
What you might not like about this is that while two different users are logged on to the same server at the same time, their home directories are also mounted and accessible at the same time.  Mitigate this by chmod a=x on the NFS share that you'll mount over /var/data/chroot/home.  This is security through obscurity: Users can't see what's in home.  Also, something like chmod go= on each users NFS-shared home directory.  This prevents anybody from writing in any home directory other than their own.
Regards,
David