feat(kernel): add SM120 FlashInfer MXFP4 MoE#645
Conversation
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b63f409a4b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2b90b110f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d34fb6e23d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
2 participants
Summary
Motivation
DeepGEMM
mega_moeserves the existing SM100 path, but it is not available on SM120. FlashInfer 0.6.13, already pinned by TokenSpeed, provides the corresponding CUTLASS MXFP8 x MXFP4 fused MoE kernel on SM120/SM121.Impact
flashinfer_cutlasscan now satisfy MXFP4, SwiGLU, precomputed-top-k MoE plans on SM120/SM121, including expert-parallel plans. Existing architectures and dependency pins are unchanged.Validation
inferlab run --environment tokenspeed -- pytest tokenspeed/tokenspeed-kernel/test/test_kernel_api_selection.py -q -k mxfp4_cutlass(2 passed)inferlab run --environment tokenspeed -- pytest tokenspeed/tokenspeed-kernel/test/test_runtime_callsite_selection.py -q(7 passed, 14 skipped)prek run --all-files