Skip to content

[XPU] opt FD_XPU_MOE_FFN_QUANT_TYPE_MAP#8008

Merged
zhupengyang merged 1 commit into
PaddlePaddle:developfrom
zhupengyang:w8a8
Jun 5, 2026
Merged

[XPU] opt FD_XPU_MOE_FFN_QUANT_TYPE_MAP#8008
zhupengyang merged 1 commit into
PaddlePaddle:developfrom
zhupengyang:w8a8

Conversation

@zhupengyang

Copy link
Copy Markdown
Collaborator

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick,PR标题需遵循格式,在最开始加上[Cherry-Pick]标签,以及最后面加上原PR ID,例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

Modifications

Usage or Command

Accuracy Tests

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

PaddlePaddle-bot

This comment was marked as outdated.

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Paddle-CI-Agent | pr_review | 2026-06-05 11:19:11

📋 Review 摘要

PR 概述:修复 _set_xpu_moe_quant_type 中对 FD_XPU_MOE_FFN_QUANT_TYPE_MAP 的解析,新增对无冒号全局默认量化类型格式的支持,并修正注释中的环境变量名称。
变更范围fastdeploy/model_executor/layers/backends/xpu/moe/fused_moe.py
影响面 Tag[XPU] [OP]

问题

级别 文件 概述
❓ 疑问 fused_moe.py:102-103 全局默认条目放在非首位时会静默覆盖前面所有范围条目

❓ 疑问 详情_set_xpu_moe_quant_type,新增 len == 1 分支)

FD_XPU_MOE_FFN_QUANT_TYPE_MAP 中全局默认条目不在首位(如 "w_channelwise_int8_a_tokenwise_float16:3->5;w_bfloat16_a_bfloat16")时,循环末轮会无条件执行 self.xpu_moe_quant_type = "w_bfloat16_a_bfloat16",把之前对层 3-5 的范围设置全部静默覆盖。注释已说明「should be placed first」,但代码未产生任何警告,用户误操作时难以排查。

建议在检测到非首位全局默认条目时输出 warning 日志,示例:

if len(quant_type_info) == 1:
    if quant_type_map != xpu_moe_ffn_quant_type_map.split(";")[0]:
        logger.warning(
            f"FD_XPU_MOE_FFN_QUANT_TYPE_MAP: global default entry "
            f"'{quant_type_info[0]}' is not placed first and will "
            f"override previous range-specific entries."
        )
    self.xpu_moe_quant_type = quant_type_info[0]

历史 Findings 修复情况

Finding 问题 状态
F1(上次 Review) PR 描述模板各段为空,Checklist 未勾选 ⚠️ 仍存在

📝 PR 规范检查

PR 标题包含官方 Tag [XPU] ✓,但描述模板各段(Motivation / Modifications / Usage or Command / Accuracy Tests)均为空,Checklist 未勾选。

标题建议(可直接复制):

  • [XPU][BugFix] Fix empty-string edge case in FD_XPU_MOE_FFN_QUANT_TYPE_MAP parsing
PR 描述建议(点击展开,可直接复制)
## Motivation
`_set_xpu_moe_quant_type` 在解析 `FD_XPU_MOE_FFN_QUANT_TYPE_MAP` 时,旧逻辑(`if len != 2: continue`)无法支持不含层索引范围的全局量化类型格式(如 `FD_XPU_MOE_FFN_QUANT_TYPE_MAP="w_bfloat16_a_bfloat16"`)。本 PR 新增对此格式的支持,并同步将注释中错误的变量名 `XPU_MOE_FFN_QUANT_TYPE_MAP` 修正为 `FD_XPU_MOE_FFN_QUANT_TYPE_MAP`## Modifications
- `fastdeploy/model_executor/layers/backends/xpu/moe/fused_moe.py`- 修正 docstring 中环境变量名称(`XPU_MOE_FFN_QUANT_TYPE_MAP``FD_XPU_MOE_FFN_QUANT_TYPE_MAP`- `_set_xpu_moe_quant_type`:新增 `len(quant_type_info) == 1` 分支,支持不含 `:` 分隔符的全局量化类型格式

## Usage or Command
```bash
# 新格式:对所有层统一设置量化类型(无需指定层索引范围)
export FD_XPU_MOE_FFN_QUANT_TYPE_MAP="w_bfloat16_a_bfloat16"

# 原格式(层范围)仍然兼容
export FD_XPU_MOE_FFN_QUANT_TYPE_MAP="w_channelwise_int8_a_tokenwise_float16:3->5,7->9"
```

## Accuracy Tests
N/A

## Checklist

- [x] Add at least a tag in the PR title.
  - Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
  - You can add new tags based on the PR content, but the semantics must be clear.
- [x] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.

总体评价

变更逻辑正确,成功修复了旧代码无法支持全局默认量化类型的问题,同时新增的空字符串守卫(if not quant_type_map: continue)也处理了 "" 环境变量边界情况。建议对全局默认条目非首位的场景补充 warning 日志,以提升可调试性;PR 描述仍需完善。

@codecov-commenter

codecov-commenter commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@fab344e). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...odel_executor/layers/backends/xpu/moe/fused_moe.py 0.00% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #8008   +/-   ##
==========================================
  Coverage           ?   66.38%           
==========================================
  Files              ?      468           
  Lines              ?    65927           
  Branches           ?    10168           
==========================================
  Hits               ?    43764           
  Misses             ?    19385           
  Partials           ?     2778           
Flag Coverage Δ
GPU 76.31% <ø> (?)
XPU 7.02% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hong19860320 hong19860320 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhupengyang
zhupengyang merged commit f2f7120 into PaddlePaddle:develop Jun 5, 2026
52 of 56 checks passed
@zhupengyang
zhupengyang deleted the w8a8 branch June 5, 2026 06:36
@paddle-bot

paddle-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

Thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants