advisory file locks in linux - do they work?

[Bob Proulx]

Jeff Gibson wrote:
> Hi - I'm trying to verify if OpenSSH/SFTP will in fact lock files
> with advisory file locking in Linux.

You seem to think that all programs do file locking?  Or perhaps you
are aware that they do not but you think they /should/ use file
locking and this is your way of lobbying to add it?  By the way ...
Most other programs such as 'cp', 'mv', 'cat', 'rsync', do not do file
locking either.  File locking is actually a rather trouble prone
technique and tends to have problems in practice on networked
filesystems such as NFS.  It is almost always better to avoid it and
use techniques that do not require it.  For one random example the
Maildir format often works better than mbox format since there is no
need for file locking with Maildirs.

> I can test locking with the linux "flock" command to verify that
> file locking does work - but when I upload or download a file with
> SFTP it will not detect a lock.

Correct.  But as you are aware the Unix philosophy is to use small
programs that each do one thing well and then to integrate them.  Your
use of the flock command from util-linux-ng is an example.  There is
no need to add file locking to sftp.  Instead use the flock program
(or other similar programs) to semaphore between processes.

> # get 100MB testfile from remote box via sftp
> (sftp to remote host, get testfile)
> # While that's running use the following command to check for the lock:
> flock -xn testfile -c "echo test > testfile"
> #flock command returns an exit status of 0.  Expected result is 1
> (failure).  Test "test" is in testfile

Since you already know about the 'flock -c cmd' then you already know
how to add this to sftp in your process.

  # Use 'sleep 20' here to simulate a large file transfer that takes a while.
  flock testfile -c "sleep 20 ; echo get testfile | sftp remote.example.com" &
  sleep 2  # give time for the above to get going and lock the file
  flock testfile -c "echo testfile is not locked now"

As such there is no need for file locking to be added to any
particular program.  You have the control to semaphore between any
process that you wish to coordinate.  This is much more flexible and
powerful than creating an infinite number of infinitely large
monolithic programs that contain all possible functionality.

Bob