cygwin performance problem

[Corinna Vinschen]

On Apr 18 12:44, Chris Rapier wrote:
> Could you tell me a bit more about your network setup including the RTT 
> between the hosts? I'm going to try this under OS X on 100bT and between 
> two linux hosts at 1Gb. The results will mean more if the differences in 
> the network environment are accounted for.

The hosts are on a 1Gb LAN.  The RTT is in the 0.2ms area.

However, while looking deeper into what's going on, it seems the bad
results are somehow based on a strange interaction between ssh and scp,
which triggers a performance problem in Cygwin.  Writing to the pipe to
scp and reading from the pipe in scp seem to take a lot of time.  I
still haven't understood what's going on, but the really weird thing is
this:

Not using scp for copying data results in far better performance.

For instance (118MB file, client_process_net_input buffer at 8K)

  cygwin> scp linux:file .

takes over 90 seconds.  Copying the file using tar over ssh like this:

  cygwin> ssh linux 'tar cf - file' | tar xf -

takes only about 8 seconds.  A Linux to Linux copy takes about 7 seconds,
with tar just slightly faster than using scp.

On Cygwin, changing the buffer size in client_process_net_input() only
affects the performance so drastically when using scp, but even in the
best case, the performance is rather bad.  I got the following average
values, using scp and tar with 8K and 20K buffer in
client_process_net_input():

secs / MB/s	 scp,8K     tar,8K     scp,20K    tar,20K

Linux->Cygwin   92.0/1.3   8.1/14.6   13.2/8.9   7.2/16.4

Linux->Linux    7.0/16.9   6.4/18.4   7.0/16.9   6.4/18.4

So, changing the buffer size on the input buffer doesn't really speed up
the network connection (which is apparently fast enough), it just
changes how ssh and scp interact.

Since in both cases (scp/tar) the underlying Cygwin is identical, and
since in both cases the file is locally copied over a pipe between the
two processes (ssh->scp resp. ssh->tar), I'm currently really at a loss
why scp is so slow.  I tried various buffer sizes in the "sink" function
in scp.c, values between 4K and 512K, but this had no influence at all.
I have to debug this a lot more, apparently, but...  if anybody has a
clue why copying over scp could be so much slower than copying over tar,
I'd really appreciate it.


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat