cygwin performance problem

[Chris Rapier]

Corinna Vinschen wrote:

> Since in both cases (scp/tar) the underlying Cygwin is identical, and
> since in both cases the file is locally copied over a pipe between the
> two processes (ssh->scp resp. ssh->tar), I'm currently really at a loss
> why scp is so slow.  I tried various buffer sizes in the "sink" function
> in scp.c, values between 4K and 512K, but this had no influence at all.
> I have to debug this a lot more, apparently, but...  if anybody has a
> clue why copying over scp could be so much slower than copying over tar,
> I'd really appreciate it.

Do you have any tools under cygwin that could tel you what syscalls are 
being made during the transfer? If we can find out where its spending so 
much of its time that might help.

Also, regarding the buffer size: I'm not quite sure of which buffer you 
mean, however the value in

   if ((bp = allocbuf(&buffer, ofd, 4096)) == NULL) {

is, as far as I know, the value which dictates how much is read off the 
pipe at a time. The *possible* problem here is that the other side of 
the pipe is using 16k chunks so you have a 4:1 disparity in the 
read/writes calls on each side of the pipe.

If you are using the HPN version I've already change that line to

  if ((bp = allocbuf(&buffer, ofd, sizeof(buf))) == NULL) {

and changed buf to 16k. On the test systems it provided a small 
improvement in performance (3% or so, not much but on fast connections 
3% can be significant). So if you are using the HPN version to test this 
then try the canonical version. If you are, then I'd try changing that 
value and seeing what happens.

I was planning on running some tests between linux, cygwin, and os x 
using multiple versions of ssh at, when possible, 3 different RTTs. I 
was just going to cat a 200MB file via SSH but I think I'll be adding 
another variable and compare scp as well. Since I'm running 1000 
iterations its going to take a while (so 18,000 xfers). But maybe it 
will shed some light on this.