-
Notifications
You must be signed in to change notification settings - Fork 789
[CI] change ds32 cudagraph_sizes #6399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] change ds32 cudagraph_sizes #6399
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request updates the E2E test configuration for the deepseek_v3_2_w8a8 model. The changes include increasing max-num-seqs, adjusting gpu-memory-utilization, updating cudagraph_capture_sizes to align with the new sequence count, and enabling layer sharding for memory optimization. These adjustments appear to be for performance tuning of the test. My review includes one suggestion to remove a commented-out line to improve code clarity.
| "8192", "--max-num-seqs", "8", "--trust-remote-code", "--quantization", | ||
| "ascend", "--gpu-memory-utilization", "0.98", "--compilation-config", | ||
| '{"cudagraph_capture_sizes":[8, 16, 24, 32], "cudagraph_mode":"FULL_DECODE_ONLY"}', | ||
| # '{"cudagraph_capture_sizes":[24, 48], "cudagraph_mode":"FULL_DECODE_ONLY"}', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This commented-out line appears to be a remnant of testing or debugging. Leaving commented-out code can lead to confusion for future maintainers, who might not know its purpose or whether it should be restored. To improve code clarity and maintainability, it's best to remove this line. If this configuration is important, it should be documented elsewhere or managed through a more explicit configuration mechanism.
a161bb1 to
2890b5d
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
2890b5d to
86e59f3
Compare
86e59f3 to
cb1cf22
Compare
|
please fix the lint error |
cb1cf22 to
53b89a7
Compare
fixed |
Signed-off-by: guozr <[email protected]>
53b89a7 to
2a8b3c1
Compare
What this PR does / why we need it?
This PR fixes the single-node nightly test for DeepSeek V3.2 (W8A8) model to ensure CI stability. The changes include:
Does this PR introduce any user-facing change?
No. This PR only affects CI/CD test configurations and does not introduce any user-facing changes.
How was this patch tested?
The nightly test workflow will run the DeepSeek V3.2 W8A8 test case with the updated configuration to verify the changes work correctly.