Support for transferring sparse files via scp/sftp correctly?

[Robinson, Herbie]

FWIW, a read operation which returns the offset the data came from will be better then using a separate control operation to find out the next offset.  In other words, you want a read that starts at the current offset, but skips missing data and returns the first offset where it found data.  It's always more efficient to have a single system call to do what you want.  Sometimes it doesn't matter, but I'm pretty sure it will in this case.

From: openssh-unix-dev <openssh-unix-dev-bounces+herbie.robinson=stratus.com at mindrot.org> On Behalf Of Lionel Cons
Sent: Wednesday, March 5, 2025 2:45 PM
To: openssh-unix-dev at mindrot.org
Subject: Re: Support for transferring sparse files via scp/sftp correctly?

You don't often get email from lionelcons1972 at gmail.com<mailto:lionelcons1972 at gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
SMART Modular Security Checkpoint: External email. Please make sure you trust this source before clicking links or opening attachments.
On Tue, 4 Mar 2025 at 11:01, Lionel Cons <lionelcons1972 at gmail.com<mailto:lionelcons1972 at gmail.com>> wrote:
>
> Does OpenSSH scp/sftp mode transfer sparse files correctly, i.e. are
> holes skipped and not transferred as chunks of 0 bytes? [1]
>
> We're asking about sparse files in the >= 1PB range, which consists of
> multi-TB holes with around 600-2000GB of valid data.
>
> [1] Note the difference: Sparse files can have data sections which
> consist of valid 0 bytes. Holes in sparse files represent regions
> where no data space is allocated, but they read as 0 bytes
>
> Lionel

I filed https://bugzilla.mindrot.org/show_bug.cgi?id=3798<https://url.us.m.mimecastprotect.com/s/lb17C4xGJxS9676nTBiWU4e09Y> for sparse
file copy support

Lionel
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev at mindrot.org<mailto:openssh-unix-dev at mindrot.org>
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev<https://url.us.m.mimecastprotect.com/s/Q2rVC5yKgyFpRWRzT2s0UkdF5d>