Support for macOS feth devices

[Damien Miller]

On Wed, 15 Jul 2020, Charles Celerier wrote:

> Hi,
> 
> I am currently using the L2 tunnel feature of ssh between two Linux
> machines, and it works beautifully! As a result, I have come to prefer a
> workflow that uses an L2 tunnel, but I can't seem to find a long-term
> solution for this workflow on macOS. At the moment, tap devices on macOS
> can be generated using a kernel extension like tuntaposx
> <http://tuntaposx.sourceforge.net/>; however, all kernel extensions were
> deprecated recently and will likely be removed in a future macOS release
> this fall.
> 
> An alternative to tap devices on macOS is something called a feth
> interface. Luckily, the ZeroTierOne project released a program
> <https://github.com/zerotier/ZeroTierOne/blob/master/osdep/MacEthernetTapAgent.c>
> which
> can interact with a feth interface through stdin and stdout. Since ssh uses
> file descriptors for all of its tunnels, I think a similar program could be
> used in the ssh sys_tun_open logic to interact with a feth interface on
> macOS.

Apparently there's also "utun" mentioned on this bug
https://bugzilla.mindrot.org/show_bug.cgi?id=3139 - it's used AFAIK to
implement user-space PPP, so it seems like a good fit though I don't
know whether it does L2. Conversely, the feth interface seems to be L2-
only, so maybe we need both?

As others have observed, the ZeroTierOne code is incompatibly licensed
for inclusion in OpenSSH, so it would need to be reimplemented anyway.

Fortunately the procedure for using a feth interface pair is very easy:

Setting up an interface pair:

ifconfig feth0 create           # primary device
ifconfig feth5000 create        # peer device

ifconfig feth0 lladdr 00:00:de:ad:be:ef
ifconfig feth5000 peer feth0
ifconfig feth5000 mtu $MTU
ifconfig feth5000 mtu 16370     # Max peer MTU
ifconfig feth0 ...		# address and other config goes on primary

OpenSSH leaves that part to the user, I'm just including it for
anyone who wants to play with it manually.

The OpenSSH side would look like:

Create socket: domain AF_NDRV, type SOCK_RAW, protocol 0
bind socket sockaddr_ndrv { .snd_family = AF_NDRV, .snd_name = feth5000 }
connect socket same sockaddr_ndrv

find and open a free /dev/bpfN device (brute force open from unit 1 up)
ioctl bpf: BIOCSBLEN to set read buffer size, get back read packet size
ioctl bpf: BIOCIMMEDIATE/1 to disable bpf buffering in kernel
ioctl bpf: BIOCSSEESENT/0 to disable interception of sent packets
ioctl bpf: BIOCSHDRCMPLT/1 to disable lladdr completion
ioctl bpf: BIOCPROMISC/1 to enable promiscuous mode
ioctl bpf: BIOCSETIF to set peer interface (feth5000)

then read packets on the bpf fd as per usual, write packets on the
socket.

Someone want to hack this into openbsd-compat/port-net.c ? :)

The tricky part will be managing two file descriptors instead of the
usual one. This will require sys_tun_open() to return two fds instead
of one and to arrange for the calling code to assign the bdf fd to
channel_new()'s rfd and the socket to channel_new()'s wfd. The channels
code should transparently take care of the bookkeeping after that.

sys_tun_infilter would probably need to grow a new SSH_TUN_COMPAT_BPF
mode to deal with the trivia of the bpf(4) header, checking the header's
capture length against what was read, etc.

-d