Request add RwLock

[evan0greenup]

Similar to RwLock in Rust standard library https://doc.rust-lang.org/std/sync/struct.RwLock.html.

Single writer, or Multiple reader.

[gaborbernat]

PR welcome 👍

[leventov]

@gaborbernat @Yard1 @evan0greenup How about piggy-backing SQLite?

https://sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions

class RwLocker:
    def __init__(self, filename):
        self.procLock = threading.Lock()
        self.con = sqlite3.connect(filename)
        # Redundant unless there are "rogue" processes that open the db
        # and switch the the db to journal_mode=WAL
        self.con.execute('PRAGMA journal_mode=DELETE;')

    def writeLock(self):
        with self.procLock:
            if self.cur is not None:
                if self.lock_mode != "write":
                    # raise error because direct promotion of read locks to write is deadlock-prone
                    raise RwLockError
                return
            self.lock_mode = "write"
            with self.con.execute('BEGIN EXCLUSIVE TRANSACTION;') as cur:
                self.cur = cur
                cur.execute('CREATE TABLE IF NOT EXISTS lock (value INTEGER UNIQUE);')
                cur.execute('INSERT OR IGNORE INTO lock (value) VALUES (42);')

    def writeUnlock(self):
        with self.procLock:
            cur = self.cur
            cur.execute('END TRANSACTION;')
            self.cur = None
            self.lock_mode = None
            cur.close()

    def readLock(self):
        with self.procLock:
            if self.cur is not None:
                return
            self.lock_mode = "read"
            with self.con.execute('BEGIN TRANSACTION;') as cur:
                self.cur = cur
                cur.execute('CREATE TABLE IF NOT EXISTS lock (value INTEGER UNIQUE);')
                cur.execute('INSERT OR IGNORE INTO lock (value) VALUES (42);')
                if cur.rowcount == 0:
                    # I guess SQLite doesn't promote the lock to exclusive if the table and the row
                    # already exist, so we assume it's the beginning of the read transaction.
                    return
                # This readLock() accesses the lock for the very first time and created the table.
                # That table creation promoted the current transaction to EXCLUSIVE. We need to
                # need to exit it and start a new transaction that won't be promoted, but will stay SHARED.
                cur.execute('END TRANSACTION;')
                cur.execute('BEGIN TRANSACTION;')
                # BEGIN doesn't itself acquire a SHARED lock on the db, that is needed for
                # effective exclusion with writeLock(). A SELECT is needed.
                cur.execute('SELECT * from lock LIMIT 1;')

    def readUnlock(self):
        self.writeUnlock() # Read unlock is the same as write unlock

Should be cross-platform at least across Unix and Windows.

Using the legacy journal mode rather than more modern WAL mode because, apparently, in WAL mode it's impossible to enforce that read transactions (started with BEGIN TRANSACTION) are blocked if a concurrent write transaction, even EXCLUSIVE, is in progress, unless the read transactions actually read any pages modified by the write transaction. But in the legacy journal mode, it seems, it's possible to do this read-write locking without table data modification at each exclusive lock.

Disadvantage: lock files may not be "application" files, definitely should be separate files (obviously, because the file is a database).

WDYT?

[gaborbernat]

I think that's a good idea 🤔 as long as we only use https://docs.python.org/3/library/sqlite3.html I'm happy to accept it.

[Yard1]

That's a pretty interesting idea! I think it would work yeah.