Run indxdb in a WebWorker

[dskvr]

SurrealDB solves a lot of frontend issues with me, primarily utilizing IndexedDB as a persistence layer while having a memory first approach. However, the one issue that does not seem to be addressed is moving the IndexedDB processing off of the main thread, while keeping the Surreal interface in the main thread (in my case while using @surrealdb/wasm)

Is this presently possible, and if not, are there any plans for implementation flexiblity with regards to threads in Surreal 2.0?

[nickchomey]

I don't think this is something that surreal needs to handle. Rather, it would happen in your application code - instantiate a web worker and put surrealdb.wasm there.

I noticed yesterday that surrealist.app uses the partytown js library, which helps with moving things off the main thread. So perhaps that's a good place to start looking?

https://partytown.builder.io/

And an article about wasm in web workers. https://www.sitepen.com/blog/using-webassembly-with-web-workers

[orimay]

I don't think this is something that surreal needs to handle. Rather, it would happen in your application code - instantiate a web worker and put surrealdb.wasm there.

I noticed yesterday that surrealist.app uses the partytown js library, which helps with moving things off the main thread. So perhaps that's a good place to start looking?

https://partytown.builder.io/

And an article about wasm in web workers. https://www.sitepen.com/blog/using-webassembly-with-web-workers

But we don't instantiate wasm, it's handled by the library: https://github.com/surrealdb/surrealdb.wasm?tab=readme-ov-file#example-usage
Also, partytown doesn't seem to be mentioned anywhere in the surrealist repo

[nickchomey]

So use the surrealdbjs library in a web worker...

I am currently using surrealdb.wasm and surrealdb.js from a web worker. Main thread creates a new Worker, and you pass messages to it via one of various messaging APIs (im using broadcast channel) and then it does what it needs to do, and passes the result back via message. The Worker can contain the exact same surrealdbjs code that you would use in main thread. You could also use Comlink or various other similar libraries to abstract away the messaging and just "call" the worker's functions from the main thread.

I'm actually doing it in an even more complicated way, because reasons, and the service worker is first intercepting http requests from main thread and then messaging the dedicated worker as described above.

The ideal would be to have a single Shared Worker that could handle all db needs of all tabs/main threads, but shared workers still aren't supported on Android chromium (50%) of web. The "polyfill" for this is to do fancy stuff with leader election of dedicated workers via web locks and broadcast channel