Skip to content

Replace qwen3-asr custom helper with optimum-intel OVModelForSpeechSeq2Seq - #3461

Draft
openvino-dev-samples wants to merge 1 commit into
openvinotoolkit:latestfrom
openvino-dev-samples:qwen3-asr
Draft

Replace qwen3-asr custom helper with optimum-intel OVModelForSpeechSeq2Seq#3461
openvino-dev-samples wants to merge 1 commit into
openvinotoolkit:latestfrom
openvino-dev-samples:qwen3-asr

Conversation

@openvino-dev-samples

Copy link
Copy Markdown
Collaborator

What does this PR do?

Replaces the custom qwen_3_asr_helper.py (1442 lines of manual model conversion and inference code) with the standard optimum-intel OVModelForSpeechSeq2Seq API, following the pattern established in huggingface/optimum-intel#1677.

Changes

  • Delete qwen_3_asr_helper.py — removes custom OpenVINO conversion code, manual GenerationMixin wrapper, and custom inference pipeline
  • Rewrite qwen3-asr.ipynb — use \OVModelForSpeechSeq2Seq.from_pretrained(export=True)\ for export and \OVModelForSpeechSeq2Seq\ for inference
  • Simplify gradio_helper.py — use standard processor + model.generate() pattern instead of custom transcribe API
  • Update README.md — reflect new optimum-intel based workflow

Key benefits

  • ~1600 lines deleted of fragile custom code (manual model tracing, stateful model patching, custom KV-cache management, etc.)
  • Standard API — uses the same \OVModelForSpeechSeq2Seq\ pattern as distil-whisper-asr and other speech notebooks
  • Easier maintenance — model export and inference handled by optimum-intel library
  • No Qwen3-ASR repo clone needed — only \pip install qwen-asr\ required to register the model

Dependencies

Testing

Tested end-to-end in a fresh virtual environment:

  1. Export Qwen3-ASR-0.6B to OpenVINO format ✅
  2. Load model and processor ✅
  3. Run inference on English sample audio ✅

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

- Use optimum-cli export openvino for model export (--trust-remote-code, --weight-format fp16)
- Use OVModelForSpeechSeq2Seq from optimum-intel for inference
- Delete qwen_3_asr_helper.py (~1442 lines of custom code)
- Update gradio_helper.py to use optimum-intel API
- Pin transformers>=4.53,<5
- Requires optimum-intel with Qwen3-ASR support (huggingface/optimum-intel#1677)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant