portable sftp oddity: sftp, redirection of stderr and ControPersist

[Nick Holland]

hi,

Ran into a problem which I thought was an AIXism, but have since found 
that it can be reproduced on Linux and MacOS.  It can NOT be reproduced 
on OpenBSD.

Reproduced on:
AIXv7.1 OpenSSH v6.0p1
RedHat 6.8 OpenSSH 5.4p1
Redhat 7.2 OpenSSH 6.6.1p1
MacOS 10.11 (sorry, forgot to grab the OpenSSH version)

Could not reproduce on:
OpenBSD 5.9-current, march snapshot, OpenSSH_7.2
OpenBSD 5.3-current, April snapshot, OpenSSH_6.2

Demonstration: set up an account on a system that can ssh to itself and 
authenticate via keys.

Create and run this script:
=====
#!/usr/bin/ksh

mkdir -p ~/dest

for X in 1 2 3 4 5 6 7 8 9 10; do
     echo $X
     echo "cd dest
     put $0" | sftp -b - localhost 2>&1 | tee -a outfile
done
=====

Change first line to whatever shell your system uses (that's AIX's 
default), otherwise, should be pretty portable.

Run it, it should copy itself to a directory in your home directory ten 
times, should take just a few seconds.


Now, add the following to ~/.ssh/config :
=====
ControlMaster auto
ControlPath ~/.ssh/control/%h:%r:%p
ControlPersist 10s
=====

re-run script.
* What I think should happen is the persistent control channel should 
greatly reduce the SSH connection time, so it should run significantly 
faster.

* What DOES happen on platforms with a problem: the "ControlPersist" 
value becomes an SSH rate limiter -- instead of holding a connection 
OPEN for ten seconds, it PREVENTS another SSH session from starting for 
ten seconds!  So, instead of taking maybe four seconds before, now it 
takes 104 seconds (10 x 10 seconds + ssh connection overhead).  Change 
the ControlPersist to some other value, the overall speed goes up or 
down, but never as fast as without the ControlPersist options in place.


here's where it gets weird.
Remove the "2>&1" from the sftp line, and the problem goes away -- the 
script runs much faster with the .ssh/config file than without it. 
Unfortunately, I need the stderr output. :-/

It appears to be the "2>&1 | {cmd}" structure that is at fault -- 
doesn't matter if {cmd} is tee, wc, or a shell function (as it was in my 
"real" project).

I have found that 2>error.file |tee outfile works, and then I can append 
the error.file output into the end of the "outfile" and accomplish my 
goals, but that's kinda ugly.

Thanks for looking!

Nick.