Howto log multiple sftpd instances with their chroot shared via NFS

[Peter Stuge]

Hi Hildegard,

Hildegard Meier wrote:
> > I think that works specifically because *no* new process is created
> > when using internal-sftp as opposed to executing the sftp-server binary.
> 
> For every sftp subsystem login (here with user "sftp_nagios"), I
> see a new sftpd process created:
..
> root      4192  0.0  0.1  72304  6512 ?        Ss   11:01   0:00 /usr/sbin/sftpd -D -f /etc/sftpd/sftpd_config
> root      4590  0.2  0.1  74736  6632 ?        Ss   11:05   0:00 sftpd: sftp_nagios [priv]
> sftp_na+  4592  0.0  0.0  74736  3432 ?        S    11:05   0:00 sftpd: sftp_nagios at notty
> sftp_na+  4593  0.0  0.0  74736  3108 ?        Ss   11:05   0:00 sftpd: sftp_nagios at internal-sftp

Thanks, yes, you're correct that there are new processes. I should
have written different, refering to sshd vs. sftp-server:

sshd creates one child process for each incoming TCP connection (4590)
and after authentication it creates another child process (4592) which
runs as the authenticated user to handle the session, and then the
sftp subsystem channel is handled by yet another child process (4593).

These processes all run (different parts of) sshd, they do not
execute any other program, when using internal-sftp not even 4593.

This is relevant because as mentioned there is then no point where
sshd executes the sftp-server binary, so also no opportunity to use
LD_PRELOAD only for SFTP. All of sftp-server is built into sshd,
specifically to enable internal-sftp, with the great benefit that no
sftp-server binary is needed inside the chroot.

My thought was still wrong though. It could have been relevant for
the syslog() socket because that is closed automatically on exec()
and not on fork(), but sshd re-opens the syslog() socket right after
fork() so the internal-sftp vs. sftp-server makes no difference for
logging, only for LD_PRELOAD convenience.


Hildegard Meier wrote:
> Jochen Bern wrote:
> > If a newly-started syslogd on server A does
> > indeed REMOVE AND RECREATE the /dev/log sockets,
> 
> If /dev dir under sftp user's chroot dir exists but there is no "log"
> file in it, it gets created by syslog-ng. It is never removed afterwards.

I had forgotten: AF_UNIX bind() to a filesystem address only succeeds
if the socket inode does not already exist in the filesystem.

So syslog-ng does what all AF_UNIX socket servers must: it unlinks the
socket from the filesystem first, so that a new socket inode can be
created by bind().

Jochen identified the problem.


Jochen Bern wrote:
> (The difference being that the socket itself would not have to store any 
> additional state about the open FDs pointing to it, which I doubt it has 
> any capacity for.)

The inode itself doesn't store anything, the kernel just maps the
filesystem inode to its socket object. Since the inode created by
daemon B means nothing within the kernel on system A, connecting to
that inode now also visible on system A doesn't reach daemon A.


Jochen Bern wrote:
> If that theory holds, then all we'd need to do is to force both 
> syslogd's to use *pre-existing* sockets instead ...

That's unfortunately not possible.


Hildegard Meier wrote:
> I think both the LD_PRELOAD and overlay fs approach are too
> complicated and hacky for me.

Fair enough, and overlayfs is only possible with a Linux NFS server.

There aren't very many other solutions:


* A patch to change SFTP logging

The attached patch adds an -E option to sftp-server (as exists for sshd)
to log to a specific file. With -E /sftp.log on the internal-sftp command
line in sshd_config on both servers they would then both log to a file in
the user's home directory. Beware of write contention on that log file
when the same user is connected to both servers.

Ensure that the file exists before using this configuration. Make file
ownership root:user and mode 620 if the user shouldn't be able to read it.

The patch is small, but until an equivalent is accepted in OpenSSH,
if ever, it requires maintaining a local package.

Note that as proposed this patch only works with a regular file, not
a syslog socket. Supporting a socket is of course possible but requires
a larger patch re-implementing much of syslog(). I can write that if
it would be accepted into OpenSSH.


* Add -e to the internal-sftp command line and create a custom parser

Because internal-sftp is still the sshd process this log message
parser must be connected already to the initial sshd stderr and must
handle anything that can be output on stderr by any sshd child process,
not just SFTP messages, making this alternative less attractive.


Kind regards

//Peter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-E-log_file-option-to-sftp-server.patch
Type: text/x-diff
Size: 2288 bytes
Desc: not available
URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20210930/9b40e2d4/attachment.bin>