Support for transferring sparse files via scp/sftp correctly?

[Lionel Cons]

On Tue, 4 Mar 2025 at 11:34, Philipp Marek <philipp at marek.priv.at> wrote:
>
> > Does OpenSSH scp/sftp mode transfer sparse files correctly, i.e. are
> > holes skipped and not transferred as chunks of 0 bytes? [1]
> >
> > We're asking about sparse files in the >= 1PB range, which consists of
> > multi-TB holes with around 600-2000GB of valid data.
>
> Perhaps rsync would be a good fit here,
> it supports --sparse.

No, if we would use external tools then mounting NFSv4.2 filesystem
via https://github.com/kofemann/ms-nfs41-client/blob/master/cygwin/utils/sshnfs/sshnfs.ksh
would be the tool of choice.

I'm talking about NATIVE sparse file support in scp/sftp via
SEEK_DATA/SEEK_HOLE on POSIX
(https://pubs.opengroup.org/onlinepubs/9799919799/functions/lseek.html)
or Windows  FSCTL_QUERY_ALLOCATED_RANGES
(https://learn.microsoft.com/en-us/windows/win32/api/winioctl/ni-winioctl-fsctl_query_allocated_ranges).
We need that because people have tried to copy sparse file before, and
it either RUINED the files (by making them non-sparse, the holes were
filled-in with 0 byte data), or copies took forever because the holes
in sparse files are very large (e.g. multi TB size).

Lionel