Cherry-pick of: "Delayed sampling tp fix #834" #1126

Job	Run time
CodeQL Scan	3m 11s
Dependency Review	3s
Calculate Tests To Trigger	6s
Test / gsm8k_small_g3_tp1	3s
Test / gsm8k_small_g3_tp2	3s
Test / gsm8k_small_g2_tp1	2s
Test / gsm8k_small_g2_tp2	2s
Test / gsm8k_large_g3_tp2	2s
Test / gsm8k_large_g2_tp4	2s
Test / gsm8k_small_g3_tp1_fp8	2s
Test / gsm8k_small_g3_tp2_fp8	2s
Test / gsm8k_small_g3_tp1_mss	3s
Test / gsm8k_small_g2_tp1_mss	2s
Test / gsm8k_small_g3_tp2_mss	3s
Test / gsm8k_small_g2_tp2_mss	2s
Test / gsm8k_small_g2_tp1_spec_decode	4s
Test / gsm8k_small_g2_tp1_mlp_spec_decode	3s
Test / gsm8k_small_g2_tp1_medusa_spec_decode	2s
Test / gsm8k_small_g2_tp1_eagle_spec_decode	3s
Test / test_llama_lora	4s
Test / test_multilora	2s
Test / test_long_context	3s
Test / multimodal_small_g3_tp1	4s
Test / multimodal_small_g3_tp2	3s
Test / multimodal_small_g3_tp1_mss	3s
Test / multimodal_small_g3_tp2_mss	4s
Test / test_awq	4s
Test / test_gptq	2s
Test / test_lazy_outlines	4s
Test / test_guided_generate	3s
	4m 36s

Provide feedback