[Feature] add the magicmtp speculative decoding acceleration algorithm

[chenaoxuan]

What this PR does / why we need it?

1. MagicMTP (paper: "Block Verification Accelerates Speculative Decoding") was introduced to consider the influence among multiple draft tokens, improving the acceptance rate without compromising accuracy.
2. Added Triton and PyTorch implementations, and added E2E test cases.

Does this PR introduce any user-facing change?

MagicMTP will automatically take effect when the parameter "num_speculative_tokens" >= 3.

• vLLM version: v0.13.0
• vLLM main: vllm-project/vllm@7157596

[gemini-code-assist]

Code Review

This pull request introduces the MagicMTP speculative decoding acceleration algorithm, adding both Triton and PyTorch implementations, along with end-to-end tests. The new algorithm is enabled when num_speculative_tokens is 3 or more. My review identified a critical logical error in the Triton kernel's implementation of the cumulative acceptance probability, which deviates from the reference PyTorch implementation and the MagicMTP paper, potentially leading to incorrect sampling results. Additionally, a critical typo was found in the new test file that would prevent it from running. I have provided code suggestions to address both of these critical issues.

[gemini-code-assist]

There is a typo in the keyword argument for torch.full. The argument detype should be dtype. This typo will raise a TypeError and cause the test to fail.

Suggested change

	detype=torch.int64,
	dtype=torch.int64,

[gemini-code-assist]

The logic for calculating the cumulative acceptance probability pi is incorrect and does not match the block verification algorithm from the MagicMTP paper, nor the provided PyTorch reference implementation. The current formula pi = min(pi * target_prob / draft_prob, 1.0) incorrectly inflates the probability when target_prob / draft_prob > 1. The correct formula should be pi = pi * min(target_prob / draft_prob, 1.0).

Furthermore, the current implementation has unsafe handling of division by zero when draft_prob is 0. This relies on floating-point inf behavior and breaks the cumulative product logic in subsequent iterations.

The suggested change corrects the formula and handles the division-by-zero case safely, ensuring the logic is correct and robust.

Suggested change

	pi = min(pi * target_prob / draft_prob, 1.0)
	if draft_prob > 0 and pi >= uniform_prob:
	last_accepted_token_pos = pos
	rejected = False
	else:
	rejected = True
	ratio = 0.0
	if draft_prob > 0.0:
	ratio = target_prob / draft_prob
	pi = pi * tl.min(ratio, 1.0)
	if pi >= uniform_prob:
	last_accepted_token_pos = pos
	rejected = False
	else:
	rejected = True

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[jianzs]

What performance improvement does this feature offer?

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[chenaoxuan]

Performance Improvement:

Environment:
CANNVERSION=CANN 8.5.0.B100
PTAVERSION=FrameworkPTAdapter 7.2.RC1.B130
VLLM_ASCEND_VERSION=e07938047e4b117258f8cb564a41c472d2b6e4ab
VLLM_VERSION=v0.13.0
MOONCAKE_VERSION=v0.3.7.post2
TORCH_VERSION=2.8.0

Parameter:
--max-num-seqs 32
--max-model-len 8192
--max-num-batched-tokens 4096
--data-parallel-size 2
--tensor-parallel-size 8

Data:
select 256 requests from gsm8k
max_out_len = 1500
temperature = 0.6
top_p = 0.95
ignore_eos=True

Model:
DeepSeek-R1_w8a8 + MTP = 3

Improvement:

• Triton:
  TOPT: 76.44ms -> 73.12ms
  Acceptance Rate: 46.94% -> 51.37%
• Torch(Only turn off Triton in rejection_sampler.py):
  TOPT: 76.87ms -> 75.19ms
  Acceptance Rate: 46.99% -> 51.17%

[chenaoxuan]

What performance improvement does this feature offer?

Increase the draft token acceptance rate by accepting more potential draft tokens.
With Deepseek-R1 model and gsm8k dataset:
TOPT: 76.44ms -> 73.12ms
Acceptance Rate: 46.94% -> 51.37%