[7/N] Add sugar syntax for module.update #11534

cccclai · 2025-06-10T18:39:16Z

Stack from ghstack (oldest at bottom):

The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following:

Module module(stub_model_path_);
int new_num_threads = 4;

const auto update_result = module.update("forward",
      {
        {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}}
      },
  );

Differential Revision: D76242292

The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following: ``` Module module(stub_model_path_); int new_num_threads = 4; const auto update_result = module.update("forward", { {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}} }, ); ``` Differential Revision: [D76242292](https://our.internmc.facebook.com/intern/diff/D76242292/) [ghstack-poisoned]

pytorch-bot · 2025-06-10T18:39:20Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11534

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 3 Unrelated Failures

As of commit 8b3d944 with merge base f7cc72f ():

NEW FAILURES - The following jobs have failed:

Lint / lintrunner / linux-job (gh)
>>> Lint for runtime/executor/test/stub_backend.h:
pull / test-llava-runner-linux / linux-job (gh)
RuntimeError: Command docker exec -t 2e8259098bf6c4b4375febe9d03c32417d7b3e9defc529feac39e5c69225fa71 /exec failed with exit code 139
pull / unittest / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8
pull / unittest-editable / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / unittest-buck / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / linux / linux-job (gh) (trunk failure)
.ci/scripts/tests/test_gather_benchmark_configs.py::TestGatehrBenchmarkConfigs::test_generate_compatible_configs_quantized_llama_model
pull / unittest-editable / linux / linux-job (gh) (trunk failure)
.ci/scripts/tests/test_gather_benchmark_configs.py::TestGatehrBenchmarkConfigs::test_generate_compatible_configs_quantized_llama_model

This comment was automatically generated by Dr. CI and updates every 15 minutes.

The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following: ``` Module module(stub_model_path_); int new_num_threads = 4; const auto update_result = module.update("forward", { {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}} }, ); ``` Differential Revision: [D76242292](https://our.internmc.facebook.com/intern/diff/D76242292/) ghstack-source-id: 289276526 Pull Request resolved: #11534

github-actions · 2025-06-10T18:40:05Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

facebook-github-bot · 2025-06-10T18:41:02Z

This pull request was exported from Phabricator. Differential Revision: D76242292

shoumikhin · 2025-06-10T19:11:14Z

extension/module/dynamic_backend_options_map.h

+#include <vector>
+#include <string>
+#include <initializer_list>
+#include <executorch/runtime/backend/backend_options.h>


Suggested change

#include <vector>

#include <string>

#include <initializer_list>

#include <executorch/runtime/backend/backend_options.h>

#include <executorch/runtime/backend/backend_options.h>

#include <vector>

#include <string>

#include <initializer_list>

shoumikhin · 2025-06-10T19:11:27Z

extension/module/module.h

 #include <executorch/runtime/executor/program.h>
+#include <executorch/extension/module/dynamic_backend_options_map.h>


Suggested change

#include <executorch/runtime/executor/program.h>

#include <executorch/extension/module/dynamic_backend_options_map.h>

#include <executorch/extension/module/dynamic_backend_options_map.h>

#include <executorch/runtime/executor/program.h>

shoumikhin · 2025-06-10T19:11:59Z

extension/module/module.h

@@ -483,6 +484,12 @@ class Module {
    return update("forward", backend_options);
  }

+  ET_EXPERIMENTAL ET_NODISCARD inline runtime::Error update(


Any comments on what this API does?

Update the comments

shoumikhin · 2025-06-10T19:14:10Z

extension/module/dynamic_backend_options_map.h

+namespace executorch {
+namespace runtime {
+
+class DynamicBackendOptionsMap {


Can we somehow use only standard containers for this map in order to skip this extra class? E.g. std::vector<std::pair<const char*, ArrayRef<BackendOption>>> or something?

That's an option, not sure how simple it is compared with update API exposed in #11533 If that one is simple enough for users, we may not need to add this PR. What do you think?

If I understand this PR correctly, it just makes it easier for devs to use the other update() API so that they don't need to keep the backend options preallocated and pass refs to them, but employ standard containers to keep the memory alive?

What I propose here is to skip any custom classes to keep the backend options and just use the standard containers directly. Then inside this new convenient update() override do the due conversion and call the original update().

Ah I see, update the PR based on the suggestion

The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following: ``` Module module(stub_model_path_); int new_num_threads = 4; const auto update_result = module.update("forward", { {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}} }, ); ``` Differential Revision: [D76242292](https://our.internmc.facebook.com/intern/diff/D76242292/) [ghstack-poisoned]

Pull Request resolved: #11534 The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following: ``` Module module(stub_model_path_); int new_num_threads = 4; const auto update_result = module.update("forward", { {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}} }, ); ``` ghstack-source-id: 290073340 @exported-using-ghexport Differential Revision: [D76242292](https://our.internmc.facebook.com/intern/diff/D76242292/)

facebook-github-bot · 2025-06-12T21:37:10Z

This pull request was exported from Phabricator. Differential Revision: D76242292

The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following: ``` Module module(stub_model_path_); int new_num_threads = 4; const auto update_result = module.update("forward", { {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}} }, ); ``` Differential Revision: [D76242292](https://our.internmc.facebook.com/intern/diff/D76242292/) [ghstack-poisoned]

Pull Request resolved: #11534 The update API in method is supposed to be portable, but we can make it more user friendly for the update API in module. Add a bit sugar syntax in module to improve UX. Such that user can update backend option in module like following: ``` Module module(stub_model_path_); int new_num_threads = 4; const auto update_result = module.update("forward", { {"StubBackend", {{IntKey("NumberOfThreads"), new_num_threads}} }, ); ``` ghstack-source-id: 290372285 @exported-using-ghexport Differential Revision: [D76242292](https://our.internmc.facebook.com/intern/diff/D76242292/)

facebook-github-bot · 2025-06-13T22:33:45Z

This pull request was exported from Phabricator. Differential Revision: D76242292

cccclai requested a review from shoumikhin as a code owner June 10, 2025 18:39

This was referenced Jun 10, 2025

[1/N] Add BackendOptions class #11389

Open

[2/N] Add option context #11390

Open

This was referenced Jun 7, 2025

[3/N] Add get_option/set_option function in backend interface #11391

Open

[4/N] Add backend options map #11462

Open

[5/N] Add update in method #11463

Open

[6/N] Add update in module #11533

Open

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 10, 2025

facebook-github-bot added the fb-exported label Jun 10, 2025

shoumikhin reviewed Jun 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[7/N] Add sugar syntax for module.update #11534

[7/N] Add sugar syntax for module.update #11534

Uh oh!

cccclai commented Jun 10, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Jun 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jun 10, 2025

Uh oh!

facebook-github-bot commented Jun 10, 2025

Uh oh!

shoumikhin Jun 10, 2025

Uh oh!

shoumikhin Jun 10, 2025

Uh oh!

shoumikhin Jun 10, 2025 •

edited

Loading

Uh oh!

cccclai Jun 12, 2025

Uh oh!

shoumikhin Jun 10, 2025

Uh oh!

cccclai Jun 10, 2025

Uh oh!

shoumikhin Jun 11, 2025

Uh oh!

cccclai Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 13, 2025

Uh oh!

Uh oh!

		#include <executorch/runtime/executor/program.h>
		#include <executorch/extension/module/dynamic_backend_options_map.h>

[7/N] Add sugar syntax for module.update #11534

Are you sure you want to change the base?

[7/N] Add sugar syntax for module.update #11534

Uh oh!

Conversation

cccclai commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11534

❌ 4 New Failures, 3 Unrelated Failures

Uh oh!

github-actions bot commented Jun 10, 2025

This PR needs a release notes: label

Uh oh!

facebook-github-bot commented Jun 10, 2025

Uh oh!

shoumikhin Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

shoumikhin Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

shoumikhin Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cccclai Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

shoumikhin Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

cccclai Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

shoumikhin Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

cccclai Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 13, 2025

Uh oh!

Uh oh!

cccclai commented Jun 10, 2025 •

edited

Loading

pytorch-bot bot commented Jun 10, 2025 •

edited

Loading

This PR needs a `release notes:` label

shoumikhin Jun 10, 2025 •

edited

Loading