TGI 2.3.1

[philschmid]

The TGI team worked really hard on adding M-Llama i.e. Vision Llama support so kudos to them! :robot_clap:
Some of the features of the new TGI release besides M-Llama are:

• Renamed HUGGINGFACE_HUB_CACHE to use HF_HOME (to harmonize environment variables across HF ecosystem)
• Add prefix caching by default. To help with long running queries TGI will use prefix caching a reuse pre-existing queries in the KV-cache in order to speed up Time-To-First-Token (TTFT).
• Changed default kernels from paged_attention to flashinfer (and flashdecoding as a fallback for some specific models that aren't supported by flashinfer).
• Fixed support for Gemma2 and variants such as DataGemma.
• Included lots of performance improvements with Marlin and quantization.
• Uses Python 3.11 instead of Python 3.10; and uses CUDA 12.4 instead of CUDA 12.1
• Adds support for M-Llama i.e. latest Vision Llama 3.2
• FP8 and MoE improvements