[Cherry-Pick][CI][BugFix] Fix rotary_embs initialization(#8012)#8017
Merged
EmmonsCurse merged 1 commit intoJun 8, 2026
Merged
Conversation
5 tasks
Collaborator
Author
|
/skip-ci ci_iluvatar |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## release/2.6 #8017 +/- ##
==============================================
Coverage ? 72.12%
==============================================
Files ? 386
Lines ? 55715
Branches ? 8748
==============================================
Hits ? 40186
Misses ? 12691
Partials ? 2838
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ZhangYulongg
approved these changes
Jun 8, 2026
|
Thanks for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #8012 (authored by @EmmonsCurse) to
release/2.6.devPR:#8012
Motivation
A CUDA kernel out-of-bounds issue was observed when running in environments with the 580 driver stack. The problem may lead to test failures, runtime instability, or unexpected kernel execution behavior under specific workloads.
This change addresses the compatibility issue to ensure stable execution in 580 driver environments.
tests/layers/test_plas_attention.pyfails with CUDA illegal memory access errors during execution:Investigation using
compute-sanitizer --tool memcheckrevealed that the failures originate from out-of-bounds memory accesses insidefused_block_mean_and_rope_kernel.The test initializes
rotary_embsusing:However, the kernel computes the sin buffer offset using
plas_max_seq_lengthrather than the actual testseq_len. As a result, the kernel accesses memory beyond the allocated buffer, causing CUDA illegal memory access and corrupting the CUDA context. Once the illegal access occurs, subsequent GPU operations also fail, which explains whytest_serverreports the same error during initialization.In addition, the test validation expects RoPE to behave as an identity transformation, which requires:
The previous initialization incorrectly set both cosine and sine values to 1.
Modifications
rotary_embsallocation to useplas_max_seq_lengthinstead ofseq_len.10test_plas_attentiontest_serverUsage or Command
N/A
Accuracy Tests
N/A
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.