Support for transferring sparse files via scp/sftp correctly?

[Ron Frederick]

Sorry - please ignore the e-mail below. I hit “send” accidentally. I’ll resend it shortly with the full details.

On Mar 28, 2025, at 9:45 PM, Ron Frederick <ronf at timeheart.net> wrote:
> I was inspired by this thread to add sparse file support to AsyncSSH, on OSes that support SEEK_DATA and SEEK_HOLE. It looks like I should also be able to get this to work on Windows with FSCTL_QUERY_ALLOCATED_RANGES and FSCTL_SET_SPARSE, but I haven’t gotten to that yet.
> 
> As Darren Tucker said, the put() operation here can be made to work with any SFTP server. However, an SFTP extension is required to support this for get() or copy(), or the  case where the copy-data extension is used to copy data between files on a remote server without reading and writing it back over the wire.
> 
> I’ve defined an extension called "ranges at asyncssh.com <mailto:ranges at asyncssh.com>” which is modeled somewhat after FXP_READDIR for getting valid data ranges in a remote file. Each call can return multiple ranges, but on files with a large number of ranges you may need call this new method multiple times to get the complete list. This allows for the copying to be interleaved with the range requests.
> 
> The extension looks like the following:
> 
>     uint32          id

-- 
Ron Frederick
ronf at timeheart.net