Skip to content

[Example] Yolo12 Detection sample with OpenVINO/XNNPACK backend #10156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

daniil-lyakhov
Copy link
Contributor

@daniil-lyakhov daniil-lyakhov commented Apr 14, 2025

Summary

Ultralytics Yolo12 Detection sample to expand example models with the detection task:

  1. Export script to get executorch yolo model lowered to OpenVINO/XNNPACK
  2. Demo app which gets a model and a video as an input and outputs an annotated video

Changes in the OpenVINO backend:

  • OpenVINO repo branch is updated to support yolo12 models (Commit to Openvino master WIP)
  • Minor fixes in quantizer.py

Test plan

Warning:

OpenCV should be installed in the environment.

OpenVINO:

./.ci/scripts/test_yolo12.sh  -model yolo12n -mode openvino -pt2e_quantize OFF -upload tmp_ov_run -video_path  path/to/mp4/file

XNNPACK:

./.ci/scripts/test_yolo12.sh  -model yolo12n -mode xnnpack -pt2e_quantize OFF -upload tmp_xnnpack_run -video_path  path/to/mp4/file

Issues:

  • Quantization does not work in both backends, issues WIP. Marked by NonImplemented error by now
  • OpenVINO is being build twice because it is not present in the executorch-config.cmake file. Perhaps we could add the OpenVINO backend to the config file?

CC: @ynimmaga @suryasidd @alexsu52

Copy link

pytorch-bot bot commented Apr 14, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10156

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit b3e69c5 with merge base 85485b8 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 14, 2025
@GregoryComer
Copy link
Member

FYI On the mlock error, you can try switching the load mode in the Module constructor (or data loader) to Mmap (from Mlock). IMO that should be the default.

@ynimmaga ynimmaga added partner: intel For backend delegation, kernels, demo, etc. from the 3rd-party partner, Intel release notes: openvino OpenVino backend related issues, features, bugs etc. labels Apr 14, 2025
Copy link
Contributor

@alexsu52 alexsu52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some sanity checking needs to be added to this example.

@@ -88,7 +88,7 @@ def set_ignored_scope(
names: Optional[List[str]] = None,
patterns: Optional[List[str]] = None,
types: Optional[List[str]] = None,
subgraphs: Optional[List[Tuple[List[str], List[str]]]] = None,
subgraphs: Optional[List[nncf.Subgraph]] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you manage subgraphs without changing the function signature?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@daniil-lyakhov
Copy link
Contributor Author

FYI On the mlock error, you can try switching the load mode in the Module constructor (or data loader) to Mmap (from Mlock). IMO that should be the default.

This helped, thanks!

@daniil-lyakhov daniil-lyakhov requested a review from alexsu52 April 23, 2025 09:15
@daniil-lyakhov
Copy link
Contributor Author

@lucylq, @jackzhxng, could you please take a look?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. partner: intel For backend delegation, kernels, demo, etc. from the 3rd-party partner, Intel release notes: openvino OpenVino backend related issues, features, bugs etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants