SCP verbose output truncated

[CLOSE Dave]

Nico Kadel-Garcia wrote:

>> Fedora 27 x86_64. openssh-7.6p1-5.fc27.x86_64.
>> 
>> If I run a simple command like, "scp -v file target:", I get a new 
>> prompt before all the output appears. Running interactively, I see
>> all that output, though some of it is after the prompt. But if I
>> put this command in a script, the output is truncated at the point
>> where my prompt would have appeared interactively. I see a message
>> like, "Shared connection to target closed."
> 
> I think part of the issue is that some of the output is to stderr. 
> stderr and stdout can overwrite each other in fascinating ways.
> 
> I'd urge you to use "scp -v foo bar 2>&1" to keep all of stderr and 
> stdout, and avoid overwriting one with the other.

Basically, that is what I was doing. However, the problem remains that
the script continues when SCP returns and that is long before the output
is done. I don't know if this is due to buffering but the time involved
seems far too long for that (~5s on an otherwise idle system).

I've found that if I use "scp -v file target: 2>log >/dev/null", the log
file does eventually contain all the output. The problem then becomes
how the script can determine that the SCP command is finished and the
output file is complete. I'm currently using "lsof -f -- log 2>/dev/null
| grep -q " 2w ", but that seems a bit like a kludge.

> Have you considered using "rsync -e ssh" instead of scp, to get more 
> meaningful dialogue for most transfers?

I know there are other ways I can transfer a file. My interest is in
diagnosing a real-life transfer started by others.
-- 
Dave Close, Thales InFlyt Experience, Irvine California USA.
cell +1 949 394 2124, dave.close at us.thalesgroup.com

 "1800: Joseph Marie Jacquard teaches a loom to read punch cards,
  creating the first heavily multi-threaded processing unit."