Skip to content

Conversation

@shiyi9801
Copy link
Owner

This PR is a very initial implementation of the model cache proposal:

partial interface MLContext {
  Promise<MLGraph> loadGraph(DOMString key);
  Promise<undefined> saveGraph(DOMString key, MLGraph graph);
};

since ORT does not yet provide a function for saving the EPContext model directly, I added an experimental ORT API SaveEpContextModel() to expose this functionality:
microsoft/onnxruntime@main...shiyi9801:onnxruntime:save_ep_context_model

@shiyi9801
Copy link
Owner Author

/cc @huningxin @Honry

@shiyi9801
Copy link
Owner Author

shiyi9801 commented Apr 24, 2025

Add some error messages for saveGraph() and loadGraph() APIs when they failed.

Simple usage of model cache API by webnn-samples: webmachinelearning/webnn-samples@master...shiyi9801:webnn-samples:model_cache

  async build(outputOperand) {
    try {
      console.log("try to load graph...");
      this.graph_ = await this.context_.loadGraph(this.modelCacheKey_);
      console.log("load graph succeed!");
    } catch (e) {
      console.log("failed to load graph: ", e.message, " try to build graph...");
      this.graph_ = await this.builder_.build({'output': outputOperand});
      await this.context_.saveGraph(this.modelCacheKey_, this.graph_);
    }
  }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants