TGI

This release introduces support for limiting image fetching size and fixes an issue related to automatic device count computation. The system is also entering Maintenance mode.

This patch release (v3.3.6) focuses primarily on bug fixes, including correcting an issue with flashinfer masking and removing Azure references, alongside minor documentation and code cleanup.

This release introduces significant hardware acceleration updates, including V2 Pydantic migration, XPU LoRA support, and various Gaudi optimizations for models like Gemma3 and Deepseek v2. It also bumps core dependencies like transformers and huggingface_hub.

This release introduces initial support for Gemma 3 models on Gaudi and fixes a bug related to Neuron models exported with batch_size 1.

This release focuses on updating the Neuron backend, including bumping the SDK version and adding support for the Qwen3_moe model on Gaudi. Several Gaudi-specific fixes and performance optimizations were also implemented.

This release focuses on Gaudi improvements, including OOM fixes and new hardware support, alongside an upgrade to vllm extension operations and the addition of the Qwen3 model.

This release updates TGI to Torch 2.7 and CUDA 12.8, incorporating HPU warmup logic refinements, kernel updates, and bug fixes.

This release introduces prefill chunking for VLMs and includes numerous stability fixes across various hardware backends like Gaudi and L4. Key updates involve dependency bumps and specific model support enhancements.

This release introduces patching for Llama 4 and updates underlying dependencies like ROCM and transformers. It also includes a fix for a compute type typo.

This release introduces support for the llama4 model, adds a configurable termination timeout, and includes several fixes, notably for Gaudi hardware.

This release introduces support for the Gemma 3 text model type and the official release of the Gaudi Backend. It also includes necessary updates for Triton kernel compilation and various bug fixes.

This release introduces support for the Gemma 3 model and brings significant updates to tool calling behavior, aligning it more closely with OpenAI's specification, alongside various backend and model-specific bug fixes.

This release focuses on backend expansion, adding support for Llamacpp, Neuron, and Gaudi backends, alongside significant improvements to Qwen VL handling and template features. It also includes various stability fixes and dependency updates.

This release introduces full hardware support for Deepseek R1 on AMD and Nvidia, adds fp8 support for MoE models, and includes several stability fixes and dependency updates.

This release introduces a major new transformers backend supporting flashattention for unsupported models and adds support for several new models including Cohere2 and OLMo variants. Numerous bug fixes target specific model issues, VLM handling, and hardware acceleration improvements across CUDA, ROCm, and XPU platforms.