perf(multimodal): optimize SHM input transport

[yechank-nvidia]

Summary

• Report the loaded vision encoder dtype so SMG can serialize multimodal tensors in the destination dtype.
• Interleave large SHM-to-pinned copies with asynchronous H2D transfers.
• Overlap SHM unlink cleanup with H2D for single large tensors.
• For large tensor-parallel inputs, copy SHM on one vision TP rank and broadcast the GPU tensor instead of repeating the host copy on every rank.
• Copy single TP source tensors directly from SHM into the destination GPU buffer.
• Preserve the existing staged path for small, mixed-dtype, inline, and non-CUDA inputs.

Test Plan

• Verified vision encoder dtype discovery for direct and nested vision towers.
• Verified SHM copy, pinned-memory staging, direct destination copy, cleanup, and dtype validation.
• Verified TP broadcast selection thresholds and fallback behavior for mixed dtype and inline tensors.
• Verified CUDA value preservation for both small staged tensors and large interleaved tensors.
• Benchmarked SHM/H2D paths on B200 with single-GPU and TP configurations.
• pytest -q test/runtime/test_multimodal_encoder_dtype.py test/runtime/test_multimodal_shm_transport.py test/runtime/test_multimodal_tp_broadcast.py
• pre-commit run --all-files

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d749e519b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Order TP broadcast with the model stream

When this large-SHM TP path runs under overlap scheduling, _move_pixel_features_to_device has switched to the H2D stream and that stream never waits for the current model stream before issuing this NCCL broadcast. Unlike the other work on this stream, this collective uses the same TP process group as model collectives, so if the current stream still has queued TP all-reduces from the overlapped forward, this broadcast can overtake them on some ranks and create a different collective order, hanging the communicator. Please run the broadcast on the model stream or add an explicit dependency on the current stream before entering the collective path.

Useful? React with 👍 / 👎.