cygwin performance problem

[Corinna Vinschen]

On Apr 21 12:22, Corinna Vinschen wrote:
> On Apr 20 16:17, Chris Rapier wrote:
> > Corinna Vinschen wrote:
> > 
> > > Since in both cases (scp/tar) the underlying Cygwin is identical, and
> > > since in both cases the file is locally copied over a pipe between the
> > > two processes (ssh->scp resp. ssh->tar), I'm currently really at a loss
> > > why scp is so slow.  I tried various buffer sizes in the "sink" function
> > > in scp.c, values between 4K and 512K, but this had no influence at all.
> > > I have to debug this a lot more, apparently, but...  if anybody has a
> > > clue why copying over scp could be so much slower than copying over tar,
> > > I'd really appreciate it.
> > 
> > Do you have any tools under cygwin that could tel you what syscalls are 
> > being made during the transfer? If we can find out where its spending so 
> > much of its time that might help.
> 
> Yes, and I can see that the most time is spent in select as well as
> reading and writing on the pipe.  I still have to find out why this
> is, though.

I just applied a fix to Cygwin's CVS, so this should be better in the
next Cygwin release.

For the curious, the culprit was the select implementation for pipes.
There's no Windows equivalent for the select function which works on
pipes.  The only function available to test if data is available on a
pipe is a function called "PeekNamedPipe", which is non-blocking.

So a select implementation for pipes has to do something along the lines
of a busy loop, testing the pipe with PeekNamedPipe until data is
available.  However, such a loop has the disadvantage to take as much
CPU as it can get, so the Cygwin loop used a 10ms sleep per iteration,
to keep the CPU usage tolerable.

Unfortunately, when sending bulk data, this 10ms sleep is a problem.
It's not sure that always the first call to PeekNamedPipe returns with
success, so there's a high probability that the loop is at least run
twice, which introduces at least one extra sleep per select.

My fix changes that by starting the loop with a sleep time set to 0 ms,
and incrementing it every second iteration by 1 ms, up to the max value
of 10 ms.  This still saves CPU time, while giving a good performance
for transfering lots of data.

With this simple change, scp gets a throughput of ~13 MB/s instead of
the former 1.3 MB/s.  Changing the buffer size in client_process_net_input
is not necessary to get this performance.  I'm happy, and I hope Cygwin
scp users will be happy, too :-)


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat