Howto log multiple sftpd instances with their chroot shared via NFS

[David Newall]

Hi Hildegard,

This, I believe, explains your problem and how to achieve your goal:

On 29/9/21 7:55 pm, Hildegard Meier wrote:
>> What is /dev/log on each server (not the one in chroot).  I.e. output of ls -l /dev/log
> On the next generation sftp server (Ubuntu 18.04.1 LTS with it's shipped OpenSSH 7.6p1-4ubuntu0.5):
> ls -l /dev/log
> lrwxrwxrwx 1 root root 28 Aug 27 11:54 /dev/log -> /run/systemd/journal/dev-log
>
> On the old (to be replaced with the new) sftp server (Ubuntu 12.04.5 LTS with it's shipped OpenSSH 5.9p1-5ubuntu1.10):
> ls -l /dev/log
> srw-rw-rw- 1 root root 0 Sep 23 15:07 /dev/log

On Linux, a process opens /dev/log when it first uses syslog(3). It 
writes log messages there.  On your new server, /dev/log is a link to 
/run/systemd/journal/dev-log, which is a UNIX domain socket.  On the old 
server, /dev/log is /dev/log, which is also a UNIX domain socket.

The syslog daemon conventionally also opens /dev/log to read incoming 
log messages.  That's how log messages get from arbitrary process to 
syslog daemon: they communicate via the same UNIX domain socket.

>> For external sftp, what is /dev/log in each NFS chroot?  I.e. output of ls -l /var/data/chroot/dev/log
> Both above sftp servers (old and new generation) mount/var/data/chroot/  from a third NFS server appliance.
> Every sftp user has it's (chroot) home dir/var/data/chroot/<username>/ under that single NFS mount.
>
> E.g. for the user "sftp_nagios" this is
>
> ls  -l /var/data/chroot/sftp_nagios/dev/log
> srw-rw-rw-+ 1 root root 0 Sep 29 11:26 /var/data/chroot/sftp_nagios/dev/log

When the syslog daemon starts, it needs to be able to open (in fact, 
create) the same UNIX domain socket that the external sftp server will 
write to.  When it starts!  That means, the socket that the external 
sftp server will write to must be visible in the chroot environment and 
also must be visible when syslog starts.

I'm imagining you have an NFS share to mount over /var/data/chroot on 
both of your SFTP servers so that you can pre-populate an empty 
directory for each sftp user.  I'm imagining that you NFS mount each 
user's home directory inside that when the user logs in (i.e. connects 
using sftp, in your case).

Finally, I infer that when a user connects with sftp, they get an 
external sftp process chrooted to /var/data/chroot/<username> with a 
/dev/log inside that. This can't work.  The syslog daemon on neither old 
nor new server can open /var/data/chroot/<username>/dev/log at startup 
because there is no such file at startup.

What you should do is create /var/data/chroot/{dev,home} on both 
servers.  These are local directories, and not NFS mounts.  This allows 
you to configure syslog daemon to read log messages from 
/var/data/chroot/dev/log (wich syslog will create when it starts.)  You 
can't NFS mount over /var/data/chroot because you've got two different 
sftp servers, each running syslog daemon.  The server that starts second 
will remove the /var/data/chroot/dev/log socket that the first one 
created, and create it's own socket in that place.  The 
/var/data/chroot/dev directory must be truly local on each machine.

You'll also have /var/data/chroot/lib/sftp-server (or 
/var/data/chroot/lib/openssh/sftp-server) on each machine, so that sshd 
can exec that inside the chroot when a user connects.  The user will be 
in chroot at /var/data/chroot.

You mount the NFS share containing the empty directories for each user.  
You mount it at /var/data/chroot/home.

You dynamically mount the user's home directory at 
/var/data/chroot/home/<username>.

When a user logs in using sftp, they're in a chroot at /var/data/chroot 
with their home directory at /home/<username> (as they see it).

What you might not like about this is that while two different users are 
logged on to the same server at the same time, their home directories 
are also mounted and accessible at the same time.  Mitigate this by 
chmod a=x on the NFS share that you'll mount over 
/var/data/chroot/home.  This is security through obscurity: Users can't 
see what's in home.  Also, something like chmod go= on each users 
NFS-shared home directory.  This prevents anybody from writing in any 
home directory other than their own.

Regards,

David