What would you like to be added: llmd includes GPU rebalancing based on Kubernetes quota this guide will use Kueue quotas and enable borrowing of resources between the models requesting for GPUs.
Why is this needed: This is needed to support sharing amoung different models requesting for GPUs.
What would you like to be added: llmd includes GPU rebalancing based on Kubernetes quota this guide will use Kueue quotas and enable borrowing of resources between the models requesting for GPUs.
Why is this needed: This is needed to support sharing amoung different models requesting for GPUs.