scp performance during local-to-remote file transfer

[Chris Rapier]

Hi,

Questions like this should be sent to hpn-ssh at psc.edu since the HPN 
functionality isn't part of the base OpenSSH distribution. You can also 
send the question directly to me because I wrote (in part) the patch.

Chris Rapier

ponraj wrote:
> Hi all,
> 
>>From this version of HPN, SO_RCVBUF can be set with the "TcpRcvBuf" option 
> on the client scp, but the scp launched by the sshd does not call the 
> setsockopt(). As a result the performance of scp is good when download a 
> file from remote-to-local but not when upload a file from local-to-remote. 
> Can you tell me what configuation settings makes the local-to-remote file 
> transfer faster and to get better throughput? Why don't we have an option 
> simillar to "TcpRcvBuf" for sshd ?
> 
> Regards,
> M.P
> 
> ----- Original Message ----- 
> From: "Chris Rapier" <rapier at psc.edu>
> To: <openssh-unix-dev at mindrot.org>
> Sent: Saturday, May 20, 2006 3:03 AM
> Subject: New HPN Patch Released
> 
> 
>> The HPN12 patch available from
>> http://www.psc.edu/networking/projects/hpn-ssh addresses performance
>> issues with bulk data transfer over high bandwidth delay paths. By
>> adjusting internal flow control buffers to better fit the outstanding
>> data capacity of the path significant improvements in bulk data
>> throughput performance are achieved.
>>
>> In other words, transfers over the internet are a lot faster with this
>> patch. Increase in performance of more than an order of magnitude are
>> pretty common. If you make use of the now included NONE cipher I've hit
>> data rates of more than 800Mbps between Pittsburgh and Chicago and
>> 650Mbps between Pittsburgh and San Diego. The NONE cipher is only used
>> during the bulk data transfer and *not* during authentication. This
>> version of the patch makes the NONE cipher a server side configuration
>> option so you don't have to enable it if you don't want to.
>>
>> Command line switches have been abandoned in favor of configuration
>> options which are outlined below and in the HPN12-README included in the
>> patch.
>>
>> Other versions of the patch showed some performance decrease in local
>> area network transfers. I've addressed this by changing the size of one
>> of the buffers (thanks Darren) and giving people the option of turning
>> off HPN functionality. I've done my best to verify this by running
>> several thousand tests and analyzing the results. These results are
>> available from http://www.psc.edu/networking/projects/hpn-ssh/results.html
>> However, there is always the chance that new problems have been
>> introduced that didn't show up in my environment. So while I am
>> reasonably confident there are no problems with this patch use al
>> appropriate caution especially in production environments.
>>
>>
>> Configuration options for HPN:
>> TcpRcvBuf=[int] (KB) client
>>       set the TCP socket receive buffer to n Kilobytes. It can be set
>> up to the maximum socket size allowed by the system. This is useful in
>> situations where the tcp receive window is set low but the maximum
>> buffer size is set higher (as is typical). This works on a per TCP
>> connection basis. You can also use this to artificially limit the
>> transfer rate of the connection. In these cases the throughput will be
>> no more than n/RTT. The minimum buffer size is 1KB. Default is the
>> current system wide tcp receive buffer size.
>>
>> TcpRcvBufPoll=[yes/no] client/server
>>       enable of disable the polling of the tcp receive buffer through
>> the life of the connection. You would want to make sure that this option
>> is enabled for systems making use of autotuning kernels (linux 2.4.24+,
>> 2.6, MS Vista) default is no.
>>
>> NoneEnabled=[yes/no] client/server
>>       enable or disable the use of the None cipher. Care must always be
>> used when enabling this as it will allow users to send data in the
>> clear. However, it is important to note that authentication information
>> remains encrypted even if this option is enabled. Set to no by default.
>>
>> NoneSwitch=[yes/no] client
>>       Switch the encryption cipher being used to the None cipher after
>> authentication takes place. NoneEnabled must be enabled on both the
>> client and server side of the connection. The connection attempt will
>> fail with an error if a client requests a NoneSwitch from the server
>> that does not explicitly have NoneEnabled set to yes. Set to no by 
>> default.
>>
>> HPNDisabled=[yes/no] client/server
>>      In some situations, such as transfers on a local area network, the
>> impact of the HPN code produces a net decrease in performance. In these
>> cases it is helpful to disable the HPN functionality. By default
>> HPNDisabled is set to no.
>>
>> HPNBufferSize=[int] (KB) client/server
>>      This is the default buffer size the HPN functionality uses when
>> interacting with nonHPN SSH installations. Conceptually this is similar
>> to the TcpRcvBuf option as applied to the internal SSH flow control.
>> This value can range from 1KB to 14MB (1-14336). Use of oversized or
>> undersized buffers can cause performance problems depending on the
>> length of the network path. The default size of this buffer is 2MB.
>> TcpRcvBufPoll, if set to yes, will override this value. This behaviour
>> may change in future versions.
>>
>>
>> Comments, questions, and especially critiques are always welcome.
>> Note: I did not attach the patch, as is customary, because its around
>> 1900 lines.
>>
>>
>> Chris Rapier
>> Network Research & Application Engineer
>> Pittsburgh Supercomputing Center
>>
>> _______________________________________________
>> openssh-unix-dev mailing list
>> openssh-unix-dev at mindrot.org
>> http://www.mindrot.org/mailman/listinfo/openssh-unix-dev 
> 
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev at mindrot.org
> http://www.mindrot.org/mailman/listinfo/openssh-unix-dev