ProxyJump: Connection reset by peer

[Eduardo Suarez]

Hi, I'm using ProxyJump via a local sshd (in a separate network namespace) to
connect to a remote host. It works fine the first time, but if I connect
several times in a row in a short period of time, I suddenly get a "Connection
reset by peer" error. This happens to be reproducible. If I wait a few seconds
(let's say 15 seconds), the connection is restored.

I have tried to debug the issue, but I'm not able to find the root cause.

These are the logs, with inline excerpts of the most relevant parts:

- local ssh client: https://dpaste.org/8ucke/raw

  debug1: Local version string SSH-2.0-OpenSSH_9.8
  kex_exchange_identification: Connection closed by remote host
  Connection closed by UNKNOWN port 65535

- local ssh proxy command ([60bc::2]): https://dpaste.org/jxzZv/raw

  debug3: receive packet: type 91
  debug2: channel_input_open_confirmation: channel 0: callback start
  debug2: channel_input_open_confirmation: channel 0: callback done
  debug2: channel 0: open confirm rwindow 2097152 rmax 32768
  debug3: receive packet: type 96
  debug2: channel 0: rcvd eof

- local intermediate sshd (localproxy, [60bc::1], 10.140.38.21):
  https://dpaste.org/v4hr2/raw

  debug3: send packet: type 91
  debug2: channel 0: read failed rfd 6 maxlen 32768: Connection reset by peer
  debug2: channel 0: read failed
  debug2: chan_shutdown_read: channel 0: (i0 o0 sock 6 wfd 6 efd -1 [closed])
  debug2: channel 0: input open -> drain
  debug2: channel 0: ibuf empty
  debug2: channel 0: send eof
  debug3: send packet: type 96

- remote sshd (target.gbdcs.net, 10.140.225.49): https://dpaste.org/fUhVN/raw

  debug3: send packet: type 20 [preauth]
  debug1: SSH2_MSG_KEXINIT sent [preauth]
  Connection reset by 10.140.38.21 port 51068 [preauth]
  debug1: do_cleanup [preauth]

So I figure out that the order of the events could be the following:

- The remote sshd sends a SSH2_MSG_KEXINIT to the local ssh client.
- For some reason the intermediate sshd cannot read from the input file
  descriptor.
- The intermediate sshd starts closing the connection.
- The local ssh proxy command receives the EOF from the intermediate sshd and
  closes the connection.
- A reset is sent to the remote sshd by the intermediate sshd, so the remote sshd
  closes the connection.
- The local ssh client receives the reset and closes the connection.
- The link from the local ssh client to the local ssh proxy command is closed.

What do you think about it?

Besides, I have been looking at the source code of OpenSSH, and the message:

  debug2: channel 0: read failed rfd 6 maxlen 32768: Connection reset by peer

may come from

https://github.com/openssh/openssh-portable/blob/05f2b141cfcc60c7cdedf9450d2b9d390c19eaad/channels.c#L2143

  debug2("channel %d: read failed rfd %d maxlen %zu: %s",
		c->self, c->rfd, maxlen, ssh_err(r));

However, I can't find then the 'Connection reset by peer' string in 'ssherr.c':

https://github.com/openssh/openssh-portable/blob/master/ssherr.c

Where does the string 'Connection reset by peer' come from?

I'm running locally OpenSSH 9.8p1, and remotely OpenSSH 8.7p1.

My ~/.ssh/config contains:

  Match host target.gbdcs.net
    ProxyJump localproxy
    User esuarez

Some details have been omited for brevity. If you need more information or some
more tests, please let me know.

Thanks in advance,
-Eduardo