Relax Transformers modeling backend MoE experts check #28952
+11
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The experts model could now also be a 3D tensor Note that there are other issues on Transformers main which currently break Transformers modeling backend MoE support. These are being worked on separately. Signed-off-by: Harry Mellor <[email protected]>
|
Documentation preview: https://vllm--28952.org.readthedocs.build/en/28952/ |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request relaxes the check for Mixture-of-Experts (MoE) layers in the Transformers modeling backend. It now supports identifying packed expert modules where parameters are 3D tensors, in addition to the existing check for nn.ModuleList. The documentation has been updated accordingly. My review focuses on the implementation of this new check. I've found a potential edge case where a module with no parameters could be incorrectly identified as a packed expert module and have provided a suggestion to make the check more robust.
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Isotr0py
approved these changes
Nov 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The experts module could now also be a 3D tensor.
Note that there are other issues on Transformers main which currently break Transformers modeling backend MoE support. These are being worked on separately.