Add: WASM Emscripten example

[raphaelmenges]

Hello 👋,

I noticed that support for the WASM (WASI?) target in this crate has been officially dropped. However, I recently encountered a use case that requires me running onnxruntime on the Web. This led me to investigate whether ort could work within an Emscripten environment.

I discovered that this crate can be used as-is with the wasm32-unknown-emscripten Rust compilation target! While the setup can get a bit complex—especially when enabling multi-threading in onnxruntime—it works well. I would love to see this example merged into the official repository, but I also understand if it is considered too experimental to be officially endorsed.

Here’s a .gif for motivation, showcasing my example using the YoloV8 model to classify objects in pictures on Chromium:

---

The Less Crazy Part

Microsoft does not provide precompiled static libraries for WASM, so I created a GitHub Actions workflow to handle this. The generated libonnxruntime.a can be linked with ort as usual—even when targeting wasm32-unknown-emscripten.

To expose Rust functions to JavaScript, the Rust main file must provide a C interface, which Emscripten can export. I use rust-embed to bundle the .onnx model into the .wasm. An index.html file then incorporates the .js and .wasm outputs, which include the compiled Rust code, onnxruntime, and the model.

Additionally, the Emscripten SDK version used by the Rust compiler must match the exact version used by onnxruntime for successful linking.

The Crazy Part

Things get trickier when enabling multi-threading in onnxruntime. Since v1.19.0, Microsoft recommends enabling multi-threading using the --enable_wasm_threads build flag. This links libonnxruntime.a to pthread, meaning all linked objects—including Rust’s standard library—must also be compiled with pthread support.

However, Rust’s standard library is not compiled this way by default, so you must switch to Rust nightly and compile the Rust standard library with +atomics,+bulk-memory,+mutable-globals.

Additionally, the server must set specific CORS flags, as multi-threaded Emscripten uses SharedArrayBuffer in the Web browser, which requires these settings.

Verdict

I am already opening the pull request as a draft to get early feedback. However, at least following ToDos are pending before a pull request could happen:

• Enable debug builds of the example.
• Add CI to automatically test the target.
• Use/enhance ort's precompiled static libonnxruntime mechanism.

In the future, I would love to see Web-exclusive execution providers to be available in ort:

• Add XNNPACK execution provider. Cannot be built at the moment.
• Add WebGL execution provider. Stable but deprecated. Might be the best option for now?
• Add WebGPU execution provider. Still requires Firefox Nightly, but is the most promising future option. However, might not yet be ready.
• Add WebNN execution provider. WebNN is not yet a standard.

[raphaelmenges]

• Add CI to automatically test the target.

I imagine an even more minimal example / test setup than YoloV8 that is first compiled and then run in the CI within a headless Chromium. We could check the developer console output to determine whether the model was correctly inferred. What do you think? Which model would be suited? Any good example how to use headless Chromium in GitHub workflows? Would this become its own workflow or rather a job in an existing workflow?

PS: I guess we could use puppeteer to execute JavaScript code without any .html?

[decahedron1]

YOLOv8 is already a pretty good example model - not too simple like the MNIST model, but not too large like GPT-2. I don't know off the top of my head if either supports WASM threads but we should give Node.js or ideally Deno a shot before jumping straight to headless Chromium for CI.

Btw, great work here! Until now I didn't think it was even possible to get wasm32-unknown-emscripten working 😅

[raphaelmenges]

I don't know off the top of my head if either supports WASM threads but we should give Node.js or ideally Deno a shot before jumping straight to headless Chromium for CI.

Sure, I can use Deno to test the .wasm build. I am wondering what would be the best approach to integrate such a test into this repository:

• We require Rust code that is compiled into the .wasm. Where should it live? I guess inside the tests directory, only Rust tests should live? Should I reuse the code from the example for the test? Feels also strange to use an example project for testing. Perhaps I can create a new top-level directory tests-wasm and put all relevant files there?
• In the workflow, the Rust code should be compiled into the .wasm. Should that happen in an existing workflow like the test.yml and there be a job on its own?

[raphaelmenges]

• Use/enhance ort's precompiled static libonnxruntime mechanism.

I have observed that you have added mechanisms to provide a precompiled static libonnxruntime for the Emscripten environment. However, when I try to use the download-binaries feature, cargo build panics with "downloaded binaries not available for target wasm32-unknown-emscripten". Am I doing something wrong, or do require any contribution in that regard from me or should I leave the current mechanism to get static libonnxruntime in the example code as-is?

[decahedron1]

Like I said in #349 I couldn't get binaries for 1.20.2 but they should be available for 1.21. Keep the current binary mechanism, and don't worry about CI, I'll handle those when 1.21 rolls around 👍