[Others]add logporbs chunked process

[bukejiyu]

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick，PR标题需遵循格式，在最开始加上[Cherry-Pick]标签，以及最后面加上原PR ID，例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

Modifications

在不开chunkedprefill的情况下，为了防止OOM 在计算logprobs的时候分chunked计算，chunked默认8K

Usage or Command

Accuracy Tests

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | pr_review | 2026-07-13 15:22:05

📋 Review 摘要

PR 概述：为 GPU prompt logprobs 计算增加 8192 token 分块，降低长 prompt 下 logits/logprobs 峰值。
变更范围：fastdeploy/worker/gpu_model_runner.py
影响面 Tag：[Optimization] [XPU] [Metax]

问题

级别	文件	概述
🟡 建议	fastdeploy/worker/gpu_model_runner.py:3657	prompt logprobs 分块只同步到 GPU runner，XPU/Metax 同名路径仍保留整段 logits 计算

📝 PR 规范检查

标题 Tag 不是 FastDeploy 官方 Tag（当前为 [logporbs]），且 PR 描述各必填 section 仍为空/占位。建议替换为下面内容。

标题建议（可直接复制）：

• [Optimization] Chunk prompt logprobs computation to reduce memory peak

PR 描述建议（点击展开，可直接复制）

## Motivation
Reduce the memory peak when generating prompt logprobs for long chunked prefill requests by avoiding one large `[num_logits, vocab]` logits/logprobs tensor.

## Modifications
- In `fastdeploy/worker/gpu_model_runner.py`, split prompt logprobs logits computation into chunks of 8192 prompt tokens.
- Copy each chunk's token ids, logprobs, and selected ranks into the accumulated CPU `LogprobsTensors`.

## Usage or Command
N/A

## Accuracy Tests
N/A

## Checklist

- [x] Add at least a tag in the PR title.
  - Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
  - You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.

总体评价

CUDA/GPU 路径的分块实现与现有 compute_logits / gather_logprobs 形状契约一致，但还需要补齐多硬件同步或明确 CUDA-only 范围。PR 元信息也需要按 FastDeploy 模板补全。

[PaddlePaddle-bot]

🟡 建议 这里仅给 GPU runner 的 prompt logprobs 加了分块，但同一逻辑在 XPU/Metax runner 仍保持整段计算 logits。

fastdeploy/worker/xpu_model_runner.py:276-293 和 fastdeploy/worker/metax_model_runner.py:2817-2832 仍会一次性对 num_logits 个 prompt token 计算 [num_logits, vocab]，长 prompt + prompt_logprobs 场景下会保留本 PR 要解决的显存/内存峰值问题。FastDeploy checklist A6 要求通用 GPU runner 变更检查多硬件同步。

建议把同样的 max_prompt_logprobs_chunk_size 分块处理同步到这些同名实现；如果本次只支持 CUDA，请在代码/PR 描述中明确范围，并补充对应硬件不受支持或待同步的说明。

[codecov-commenter]

Codecov Report

❌ Patch coverage is 75.00000% with 5 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@9a48fab). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/worker/gpu_model_runner.py	75.00%	3 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #8098   +/-   ##
==========================================
  Coverage           ?   67.49%           
==========================================
  Files              ?      475           
  Lines              ?    66965           
  Branches           ?    10330           
==========================================
  Hits               ?    45199           
  Misses             ?    18887           
  Partials           ?     2879           

Flag	Coverage Δ
GPU	77.52% <75.00%> (?)
XPU	6.94% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[paddle-bot]

Thanks for your contribution!