portable sftp oddity: sftp, redirection of stderr and ControPersist

[Jakub Jelen]

On 07/07/2016 09:07 PM, Nick Holland wrote:
> hi,
>
> Ran into a problem which I thought was an AIXism, but have since found 
> that it can be reproduced on Linux and MacOS.  It can NOT be 
> reproduced on OpenBSD.
>
> Reproduced on:
> AIXv7.1 OpenSSH v6.0p1
> RedHat 6.8 OpenSSH 5.4p1
> Redhat 7.2 OpenSSH 6.6.1p1
> MacOS 10.11 (sorry, forgot to grab the OpenSSH version)
>
> Could not reproduce on:
> OpenBSD 5.9-current, march snapshot, OpenSSH_7.2
> OpenBSD 5.3-current, April snapshot, OpenSSH_6.2
>
> Demonstration: set up an account on a system that can ssh to itself 
> and authenticate via keys.
>
> Create and run this script:
> =====
> #!/usr/bin/ksh
>
> mkdir -p ~/dest
>
> for X in 1 2 3 4 5 6 7 8 9 10; do
>     echo $X
>     echo "cd dest
>     put $0" | sftp -b - localhost 2>&1 | tee -a outfile
> done
> =====
>
> Change first line to whatever shell your system uses (that's AIX's 
> default), otherwise, should be pretty portable.
>
> Run it, it should copy itself to a directory in your home directory 
> ten times, should take just a few seconds.
>
>
> Now, add the following to ~/.ssh/config :
> =====
> ControlMaster auto
> ControlPath ~/.ssh/control/%h:%r:%p
> ControlPersist 10s
> =====
>
> re-run script.
> * What I think should happen is the persistent control channel should 
> greatly reduce the SSH connection time, so it should run significantly 
> faster.
>
> * What DOES happen on platforms with a problem: the "ControlPersist" 
> value becomes an SSH rate limiter -- instead of holding a connection 
> OPEN for ten seconds, it PREVENTS another SSH session from starting 
> for ten seconds!  So, instead of taking maybe four seconds before, now 
> it takes 104 seconds (10 x 10 seconds + ssh connection overhead).  
> Change the ControlPersist to some other value, the overall speed goes 
> up or down, but never as fast as without the ControlPersist options in 
> place.
>
>
> here's where it gets weird.
> Remove the "2>&1" from the sftp line, and the problem goes away -- the 
> script runs much faster with the .ssh/config file than without it. 
> Unfortunately, I need the stderr output. :-/
>
> It appears to be the "2>&1 | {cmd}" structure that is at fault -- 
> doesn't matter if {cmd} is tee, wc, or a shell function (as it was in 
> my "real" project).
>
> I have found that 2>error.file |tee outfile works, and then I can 
> append the error.file output into the end of the "outfile" and 
> accomplish my goals, but that's kinda ugly.
This looks like related to the bug #1988 [1]. The mux process is holding 
the stderr. Could you test it with the patch from the referenced bug 
applied?
Also as I see you are using RHEL, you can contact your support to 
provide a test package (we plan to fix this in RHEL7.3).

[1] https://bugzilla.mindrot.org/show_bug.cgi?id=1988

-- 
Jakub Jelen
Associate Software Engineer
Security Technologies
Red Hat