Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge upstream commits #7

Merged
merged 54 commits into from
Feb 6, 2025
Merged

Merge upstream commits #7

merged 54 commits into from
Feb 6, 2025

Conversation

daniel-geon-park
Copy link
Collaborator

Motivation

Merge upstream commits to keep our dev branch up-to-date.

Modifications

Merge upstream commits

Checklist

ByronHsu and others added 30 commits January 30, 2025 18:01
@ByronHsu
[test] Lower number of top logprobs to get rid of -inf (sgl-project…
20453ce 
…#3212)
@zhyncs
update 3rdparty and rms norm for sgl-kernel (sgl-project#3213)
c38b5fb
@zhyncs
update setup for sgl-kernel (sgl-project#3214)
468d23c
@zhyncs @BBuf
add tensorrt_llm common and cutlass_extensions as 3rdparty (sgl-proje…
222ce6f 
…ct#3216)

Co-authored-by: BBuf <[email protected]>
@zhyncs
add tensorrt_llm moe_gemm as 3rdparty (sgl-project#3217)
e81d7f1
@zhyncs
keep the parts needed for moe_kernels (sgl-project#3218)
9602c2a
@Ying1123
docs: add Novita for adoption and sponsorship (sgl-project#3227)
cde4bbd
@ravi03071991
Docs: Update supported models with Mistral 3 (sgl-project#3229)
9829e77 
Co-authored-by: Ravi Theja Desetty <[email protected]>
@zhyncs
revert the MoE dependence (sgl-project#3230)
3ee6223
@ByronHsu
[fix] Clamp logprob with dtype min to prevent -inf (sgl-project#3224)
734daed
@ispobock
Fix block wise fp8 torch compile (sgl-project#3232)
c02e313
@zhyncs
support 12.5 CUDA runtime (sgl-project#3231)
b49d6d0
@zhyncs
chore: bump v0.4.2.post1 (sgl-project#3233)
cf0f7ea
@jhinpan @zhaochenyang20
Docs: Quick fix for Speculative_decoding doc (sgl-project#3228)
656f7fc 
Co-authored-by: Chayenne <[email protected]>
Co-authored-by: Chayenne <[email protected]>
@zhyncs
compatible with flashinfer v0.2 (sgl-project#3235)
7811bfd
@ispobock
Optimize MoE topk with torch compile (sgl-project#3236)
1ebe1d6
@zhyncs
update sgl-kernel version for sglang (sgl-project#3238)
34e405e
@zhyncs
update cutlass dependency (sgl-project#3240)
7876279
@zhyncs @HandH1998
add tuning block wise fp8 (sgl-project#3242)
7b020cc 
Co-authored-by: HandH1998 <[email protected]>
@Edenzzzz
[Docs] Add more details to profiling docs (sgl-project#3221)
d7c0b32
@ispobock
Add test for fp8 torch compile (sgl-project#3246)
5317902
@HaiShaw
update ENV to ROCm dockers (sgl-project#3248)
17dbf97
@zhyncs
update and simplify CustomOp (sgl-project#3249)
4eb4b40
@zhyncs
support QuickGELU (sgl-project#3250)
8db776f
@zhyncs
add contact us in README (sgl-project#3251)
ad67409
@zhyncs
Update README
f2b3a31
@zhyncs
use srt VocabParallelEmbedding (sgl-project#3252)
959dca4
@whchung
Tune paged attention parameters for AMD GPU. (sgl-project#3255)
d9eb935
@simveit @shuaills @zhaochenyang20
docs/accuracy evaluation (sgl-project#3114)
c27c378 
Co-authored-by: Shi Shuai <[email protected]>
Co-authored-by: zhaochenyang20 <[email protected]>
@zhaochenyang20
Docs: Update accuracy evaluation (sgl-project#3261)
55f5fc6
BBuf and others added 20 commits February 3, 2025 18:56
@BBuf
optimize test_fused_moe style (sgl-project#3268)
3c8ac78
@zhyncs @Ying1123 @merrymercy
refactor EAGLE 2 (sgl-project#3269)
013021b 
Co-authored-by: Ying Sheng <[email protected]>
Co-authored-by: merrymercy <[email protected]>
Co-authored-by: Ying1123 <[email protected]>
@zhyncs
add copyright for sgl-kernel (sgl-project#3270)
00fa7d0
@kushanam
adding Triton configs for DeepSeekV3 on Blackwell (sgl-project#3272)
d54cee1
@zhyncs
add Nebius for Adoption and Sponsorship (sgl-project#3274)
897e2e2
@zhyncs
add Atlas Cloud for Adoption and Sponsorship (sgl-project#3276)
4b6f62e
@simveit @shuaills
Update server args doc (sgl-project#3273)
7b5a374 
Co-authored-by: Shi Shuai <[email protected]>
@Fridge003 @Ying1123
[Feature] Define backends and add Triton backend for Lora (sgl-projec…
70817a7 
…t#3161)

Co-authored-by: Ying Sheng <[email protected]>
@zhyncs @pankajroark
upgrade flashinfer v0.2.0.post2 (sgl-project#3288)
d39899e 
Co-authored-by: pankajroark <[email protected]>
@HaiShaw
ROCm: sgl-kernel enablement starting with sgl_moe_align_block (sgl-pr…
2c1a695 
…oject#3287)
@ispobock
Update Triton decode backend interface (sgl-project#3292)
a07364c
@zhyncs
update flashinfer install index url (sgl-project#3293)
6186a8f
@whchung
[ROCm] Add tuning configs for AMD Radeon Graphics. (sgl-project#3294)
c7256ca
@whchung
[ROCm] Manually unroll _w8a8_block_fp8_matmul kernel on AMD GPU. (sgl…
c2723a4 
…-project#3299)
@kkHuang-amd
Use forward_cuda to execute custom op for hip platform (sgl-project#3305
4885b90 
)

Co-authored-by: wunhuang <[email protected]>
@whchung
[ROCm] Logic to decide whether to used manually unrolled kernel. (sgl…
7ab8494 
…-project#3306)
@Fridge003
Fix lora flashinfer import bug on ROCM (sgl-project#3312)
76fa2d1
@zhyncs
chore: bump v0.4.2.post2 (sgl-project#3313)
7aad8d1
@ispobock
Update Triton extend backend interface (sgl-project#3309)
de55333
@daniel-geon-park
Merge branch 'main' into deepauto/feat/update-upstream
883a547
kbumsik
kbumsik approved these changes Feb 6, 2025
View reviewed changes
Copy link

@kbumsik kbumsik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go

gmlwns2000
gmlwns2000 approved these changes Feb 6, 2025
View reviewed changes
Copy link
Collaborator

@gmlwns2000 gmlwns2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kbumsik kbumsik merged commit 022c502 into deepauto/dev Feb 6, 2025
2 checks passed
@kbumsik kbumsik deleted the deepauto/feat/update-upstream branch February 6, 2025 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gmlwns2000 gmlwns2000 gmlwns2000 approved these changes

@kbumsik kbumsik kbumsik approved these changes

Assignees

@gmlwns2000 gmlwns2000

@daniel-geon-park daniel-geon-park

@kbumsik kbumsik

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.