You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context Parallel has 2 ways to dispatch SDPA to the corresponding KV all-gather variant: 1. through monkey-patch; 2. using TorchFunctionMode. The first approach works well with SAC but the second is failing with error "RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead."