Support for transferring sparse files via scp/sftp correctly?

[Cedric Blancher]

On Tue, 4 Mar 2025 at 11:05, Lionel Cons <lionelcons1972 at gmail.com> wrote:
>
> Does OpenSSH scp/sftp mode transfer sparse files correctly, i.e. are
> holes skipped and not transferred as chunks of 0 bytes? [1]
>
> We're asking about sparse files in the >= 1PB range, which consists of
> multi-TB holes with around 600-2000GB of valid data.
>
> [1] Note the difference: Sparse files can have data sections which
> consist of valid 0 bytes. Holes in sparse files represent regions
> where no data space is allocated, but they read as 0 bytes
>

I think proper sparse file support via SEEK_DATA+SEEK_HOLE is a very
good idea. Sparse files are abundantly in use (bioinformatiks in our
case), and it is very annoying to always have to wrap file transfers
in tar --sparse first. Error prone too.
Better sftp should support sparse files directly.

Ced
-- 
Cedric Blancher <cedric.blancher at gmail.com>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur