Concurrent Connections (#145)

Author: stevensJourney

CHANGELOG.md

@@ -1,12 +1,23 @@
 # Changelog
 
-## 1.0.0-BETA29
-
-* Added queing protection and warnings when connecting multiple PowerSync clients to the same database file.
-
 ## 1.0.0-BETA28
 
 * Update PowerSync SQLite core extension to 0.3.12.
+* Added queing protection and warnings when connecting multiple PowerSync clients to the same database file.
+* Improved concurrent SQLite connection support accross various platforms. All platforms now use a single write connection and multiple read connections for concurrent read queries.
+* Added the ability to open a SQLite database given a custom `dbDirectory` path. This is currently not supported on iOS due to internal driver restrictions.
+* Internaly improved the linking of SQLite for iOS.
+* The Android SQLite driver now uses the [Xerial JDBC library](https://github.com/xerial/sqlite-jdbc). This removes the requirement for users to add the jitpack Maven repository to their projects.
+```diff
+// settings.gradle.kts example
+    repositories {
+        google()
+-        maven("https://jitpack.io") {
+-            content { includeGroup("com.github.requery") }
+-        }
+        mavenCentral()
+    }
+```
 
 ## 1.0.0-BETA27
 

README.md

@@ -46,14 +46,9 @@ The PowerSync Kotlin Multiplatform SDK is currently in a beta release. It can be
 Current limitations:
 
 - Integration with SQLDelight schema and API generation (ORM) is not yet supported.
-- Supports only a single database file.
 
 Future work/ideas:
-
-- Improved error handling.
 - Attachments helper package.
-- Management of DB connections on each platform natively.
-- Supporting additional targets (JVM, Wasm).
 
 ## Installation
 

build.gradle.kts

@@ -27,9 +27,6 @@ allprojects {
         maven("https://cache-redirector.jetbrains.com/intellij-dependencies")
         // Repo for the backported Android IntelliJ Plugin by Jetbrains used in Ultimate
         maven("https://maven.pkg.jetbrains.space/kotlin/p/kotlin/kotlin-ide-plugin-dependencies/")
-        maven("https://jitpack.io") {
-            content { includeGroup("com.github.requery") }
-        }
     }
 
     configurations.configureEach {

core-tests-android/src/androidTest/java/com/powersync/AndroidDatabaseTest.kt

@@ -93,4 +93,133 @@ class AndroidDatabaseTest {
                 query.cancel()
             }
         }
+
+    @Test
+    fun testConcurrentReads() =
+        runTest {
+            database.execute(
+                "INSERT INTO users (id, name, email) VALUES (uuid(), ?, ?)",
+                listOf(
+                    "steven",
+                    "[email protected]",
+                ),
+            )
+
+            val pausedTransaction = CompletableDeferred<Unit>()
+            val transactionItemCreated = CompletableDeferred<Unit>()
+            // Start a long running writeTransaction
+            val transactionJob =
+                async {
+                    database.writeTransaction { tx ->
+                        // Create another user
+                        // External readers should not see this user while the transaction is open
+                        tx.execute(
+                            "INSERT INTO users (id, name, email) VALUES (uuid(), ?, ?)",
+                            listOf(
+                                "steven",
+                                "[email protected]",
+                            ),
+                        )
+
+                        transactionItemCreated.complete(Unit)
+
+                        // Block this transaction until we free it
+                        runBlocking {
+                            pausedTransaction.await()
+                        }
+                    }
+                }
+
+            // Make sure to wait for the item to have been created in the transaction
+            transactionItemCreated.await()
+            // Try and read while the write transaction is busy
+            val result = database.getAll("SELECT * FROM users") { UserRow.from(it) }
+            // The transaction is not commited yet, we should only read 1 user
+            assertEquals(result.size, 1)
+
+            // Let the transaction complete
+            pausedTransaction.complete(Unit)
+            transactionJob.await()
+
+            val afterTx = database.getAll("SELECT * FROM users") { UserRow.from(it) }
+            assertEquals(afterTx.size, 2)
+        }
+
+    @Test
+    fun transactionReads() =
+        runTest {
+            database.execute(
+                "INSERT INTO users (id, name, email) VALUES (uuid(), ?, ?)",
+                listOf(
+                    "steven",
+                    "[email protected]",
+                ),
+            )
+
+            database.writeTransaction { tx ->
+                val userCount =
+                    tx.getAll("SELECT COUNT(*) as count FROM users") { cursor -> cursor.getLong(0)!! }
+                assertEquals(userCount[0], 1)
+
+                tx.execute(
+                    "INSERT INTO users (id, name, email) VALUES (uuid(), ?, ?)",
+                    listOf(
+                        "steven",
+                        "[email protected]",
+                    ),
+                )
+
+                // Getters inside the transaction should be able to see the latest update
+                val userCount2 =
+                    tx.getAll("SELECT COUNT(*) as count FROM users") { cursor -> cursor.getLong(0)!! }
+                assertEquals(userCount2[0], 2)
+            }
+        }
+
+    @Test
+    fun openDBWithDirectory() =
+        runTest {
+            val tempDir =
+                InstrumentationRegistry
+                    .getInstrumentation()
+                    .targetContext.cacheDir.canonicalPath
+            val dbFilename = "testdb"
+
+            val db =
+                PowerSyncDatabase(
+                    factory = DatabaseDriverFactory(InstrumentationRegistry.getInstrumentation().targetContext),
+                    schema = Schema(UserRow.table),
+                    dbDirectory = tempDir,
+                    dbFilename = dbFilename,
+                )
+
+            val path = db.get("SELECT file FROM pragma_database_list;") { it.getString(0)!! }
+
+            assertEquals(path.contains(tempDir), true)
+
+            db.close()
+        }
+
+    @Test
+    fun readConnectionsReadOnly() =
+        runTest {
+            val exception =
+                assertThrows(PowerSyncException::class.java) {
+                    // This version of assertThrows does not support suspending functions
+                    runBlocking {
+                        database.getOptional(
+                            """
+                            INSERT INTO 
+                                 users (id, name, email)
+                             VALUES
+                                 (uuid(), ?, ?) 
+                             RETURNING *
+                            """.trimIndent(),
+                            parameters = listOf("steven", "[email protected]"),
+                        ) {}
+                    }
+                }
+            // The exception messages differ slightly between drivers
+            assertEquals(exception.message!!.contains("write a readonly database"), true)
+        }
 }

core/.gitignore

@@ -1 +1,5 @@
 binaries/
+# Required for the JDBC SQLite driver, but should not be commiteed
+src/androidMain/jni
+
+testdb-*

core/README.md

@@ -1,19 +1,31 @@
 # PowerSync core module
 
-The PowerSync core module provides the core functionality for the PowerSync Kotlin Multiplatform SDK.
+The PowerSync core module provides the core functionality for the PowerSync Kotlin Multiplatform
+SDK.
 
 ## Structure
 
-This is a Kotlin Multiplatform project targeting Android, iOS platforms, with the following structure:
+This is a Kotlin Multiplatform project targeting Android, iOS platforms, with the following
+structure:
 
-- `commonMain` - Shared code for all targets, which includes the `PowerSyncBackendConnector` interface and `PowerSyncBuilder` for building a `PowerSync` instance. It also defines
+- `commonMain` - Shared code for all targets, which includes the `PowerSyncBackendConnector`
+  interface and `PowerSyncBuilder` for building a `PowerSync` instance. It also defines
   the `DatabaseDriverFactory` class to be implemented in each platform.
-- `androidMain` - Android specific code, which includes a implementation of `DatabaseDriverFactory` class that creates an instance of `app.cash.sqldelight.driver.android.AndroidSqliteDriver` using
-  a `io.requery.android.database.sqlite.RequerySQLiteOpenHelperFactory`. It also includes native SQLite bindings for Android.
-- `iosMain` - iOS specific code, which includes a implementation of `DatabaseDriverFactory` class that creates an instance of `app.cash.sqldelight.driver.native.NativeSqliteDriver` and also sets up native SQLite bindings for iOS.
+- `commonJava` - Common Java code including a Java SQLite driver using
+  the [Xerial JDBC Driver](https://github.com/xerial/sqlite-jdbc). This is used by both the Android
+  and JVM drivers.
+- `androidMain` - Android specific code, which includes an implementation of
+  `DatabaseDriverFactory`.
+- `jvmMain` - JVM specific code which includes an implementation of `DatabaseDriverFactory`.
+- `iosMain` - iOS specific code, which includes am implementation of `DatabaseDriverFactory` class
+  that creates an instance of `app.cash.sqldelight.driver.native.NativeSqliteDriver` and also sets
+  up native SQLite bindings for iOS.
 
 ## Note on SQLDelight
 
-The PowerSync core module, internally makes use of [SQLDelight](https://cashapp.github.io/sqldelight) for it database API and typesafe database query generation.
+The PowerSync core module, internally makes use
+of [SQLDelight](https://sqldelight.github.io/sqldelight/latest/) for it database API and typesafe database
+query generation.
 
-The PowerSync core module does not currently support integrating with SQLDelight from client applications.
+The PowerSync core module does not currently support integrating with SQLDelight from client
+applications.