HiddenStore option may be useful

[Jason]

William Ahern wrote:
> On Sat, Apr 14, 2007 at 10:54:43PM -0400, Jason wrote:
>> Thomas Blank wrote:
>>> I'm missing a HiddenStore option in OpenSSH, known from some ftp-server 
>>> implementations like ProFTPd.
>>>
>>> Consider the following scenario:
>>> - A process PROCA is frequently polling the directory for a file called 
>>> myfile.txt
>>> - Someone transfers this file via sftp or scp to the directory
>>> - While transfer is going on and the file is not completely written, 
>>> PROCA reads in the file and removes is
>>> -> Corrupt data is seen by PROCA
>>>
>>> Knowing this problem you have to solutions:
>>> 1. PROCA must check if myfile.txt is changing (filesize, mtime...) and 
>>> wait until it does not change any more
>>> 2. sftp and scp use a HiddenStore by writing the file with a unique 
>>> filename (eg. .myfile.txt) and renaming it at the end of the transfer 
>>> (mv .myfile.txt myfile.txt)
>>>
>>> What do you think about this?
>> Why not have PROCA use inotify?
>>
>> See /usr/src/linux/Documentation/filesystems/inotify.txt
>>
> 
> How does that address the race condition? inotify is just a better poll in
> this case. It doesn't even tell you how many processes have an open
> descriptor.

Given that PROCA is there before the file is created, inotify allows you
to detect when a new file is created (IN_CREATE), opened (IN_OPEN), and
how many times that occurred.  Once PROCA receives enough file close
events (IN_CLOSE) to match the create and all the opens (if they
occurred), then it can consider the file transfer is finished and pull
it out of the directory.

On a technical note, inotify is _not_ a better poll,  rather it is
interrupt driven (select() on file descriptor created by inotify_init()).

hth,

Jason.