Debugging SFTP question

[Darren Tucker]

On Sun, May 05, 2013 at 01:39:06PM -0500, Graeme Wallace wrote:
> Corruption occurs early in the file. Filesystem is ext4 on Ubuntu 12.04.
> I've attached the transfer log at DEBUG3.

I notice the log does not show the file being closed.  Is that really
the case or is it just missing from the log?  In theory it could be an
error when the file is flushed to disk, and that wouldn't be detected
because of the missing close, but that seems pretty unlikely.

Looking at the server log I see huge chunks of the file missing.  I ran
the perl script below over the log and got:

chunks written:  0-2097152 41910272-44105728 95027200-97222656
   120356864-122552320 156434432-158629888 193921024-196116480
   237043712-239239168 269942784-272138240

the non-contiguous portions are suspiciously regular, and they
correspond to the jumps in the request numbers (the latter is not
necessarily a problem but the former probably is).  My guess is that the
client is multithreaded, doing multiple concurrent writes and is mixing
up the increments of both the request numbers and offsets.

How big is the file supposed to be?  Based on the number of packets I'd
guess about 17M, but the client seems to be creating a sparse file about
260M in size (compare the sizes reported by ls -l and du).

while (<>) {
	unless (/off (\d+) len (\d+)/) { next; }
	$off = $1; $len = $2;
	#print "off $off len $len\n";
	if (!defined($start)) {
		$start = $off;
		$end = $len;
	} elsif ($off == $end) {
		$end = $off + $len;
	} else {
		print "non-contig write at $off $len\n";
		$chunks = "$chunks $start-$end";
		$start = $off;
		$end = $off + $len;
	}
}
$chunks = "$chunks $start-$end";

print "chunks written: $chunks\n"

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.