[WebGPU] Replace subgroup matrix tiling table with decision tree - #31833
[WebGPU] Replace subgroup matrix tiling table with decision tree#31833Jie Chen (jchen10) wants to merge 2 commits into
Conversation
Replace static lookup table with ML-based decision tree for Intel Xe3-LPG subgroup matrix tiling selection. The decision tree accounts for batch size in its predictions, eliminating the need for post-hoc split-K clamping. This provides more accurate performance predictions and better occupancy management across varying problem sizes and batch dimensions.
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
| @@ -0,0 +1,4116 @@ | |||
| // Auto-generated by tools/python/gen_sgmm_tree.py. Do not edit. | |||
There was a problem hiding this comment.
Where is tools/python/gen_sgmm_tree.py? Please include the generator and document the rationale and data used to derive these configurations. For maintainability, I would prefer shared, explainable tuning rules across Intel devices unless Xe3-LPG requires architecture-specific tuning. If so, please provide benchmark evidence demonstrating the need and benefit.
There was a problem hiding this comment.
It can be found here: jchen10@8827c95
The static pre-tuned table only covers [B, M, N, K] to [1, 4k, 4k, 4k]. We may run into the table size exploration problem if we need to use larger sizes. In fact, B can be as large as 1k, and N, K can be 40k~50k regarding the models we are benchmarking. With this decision tree, we can now expand to [1k, 4k, 64k, 64k].
There was a problem hiding this comment.
Updated the comment as we discussed.
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| @@ -0,0 +1,4123 @@ | |||
| // Auto-generated. Do not edit. | |||
| // Distilled decision tree (depth 12, 1024 leaves, 43 tilings) | |||
There was a problem hiding this comment.
Cleanup: the header says "43 tilings" although the tree emits 40 unique configurations; its closing namespace comment is incorrect and fails clang-format; and the disabled large-batch test still references the removed ClampSplitKForBatch function.
Replace static lookup table with ML-based decision tree for Intel Xe3-LPG subgroup matrix tiling selection. The decision tree accounts for batch size in its predictions, eliminating the need for post-hoc split-K clamping. This provides more accurate performance predictions and better occupancy management across varying problem sizes and batch dimensions.