Skip to content

Add opt-in materialized weight loading override#1993

Open
k1x0r wants to merge 1 commit into
osaurus-ai:mainfrom
k1x0r:pr-materialized-weights
Open

Add opt-in materialized weight loading override#1993
k1x0r wants to merge 1 commit into
osaurus-ai:mainfrom
k1x0r:pr-materialized-weights

Conversation

@k1x0r

@k1x0r k1x0r commented Jul 12, 2026

Copy link
Copy Markdown

Summary

Add an opt-in OSAURUS_MATERIALIZE_WEIGHTS environment variable that forces safetensor weights to be materialized instead of memory-mapped.

This is useful on systems where memory-mapped model weights can be evicted or cause unstable performance under memory pressure. Materializing the weights keeps them resident in allocated memory and provides a workaround without changing the default loading behavior for other users.

When OSAURUS_MATERIALIZE_WEIGHTS=1 is set, Osaurus uses:

useMmapSafetensors = false

When the variable is not set, the existing memory-mapped loading behavior remains unchanged.

Test Plan

Model used: minimax-m2.7-jangtq

  1. Build and launch Osaurus without the environment variable.

  2. Load the model and verify that the existing memory-mapped safetensor path is used.

  3. Quit Osaurus.

  4. Launch it with the environment variable enabled:

    OSAURUS_MATERIALIZE_WEIGHTS=1 \
      /Applications/Osaurus.app/Contents/MacOS/Osaurus

    Alternatively, add OSAURUS_MATERIALIZE_WEIGHTS=1 to the environment variables in the Xcode scheme.

  5. Load the same model.

  6. Verify that useMmapSafetensors is set to false and the safetensor weights are materialized.

  7. Run an inference request and confirm that model loading and generation complete successfully.

  8. Remove the environment variable and confirm that the default memory-mapped behavior is restored.

Screenshots

Not applicable. No UI changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant