Hugging Face updated its transformers modeling backend for vLLM on July 8, 2026, enabling model authors to achieve native inference speeds without writing custom vLLM code. The updated backend meets or beats the throughput of hand-written native vLLM implementations across three tested Qwen3 model configurations.
Benchmarking against vLLM's native code evaluated a single-GPU Qwen3-4B dense model, a Qwen3-32B dense model running tensor parallelism across two GPUs, and a 235-billion-parameter FP8 Mixture-of-Experts model using data and expert parallelism across eight Nvidia H100 GPUs. Operators activate the backend by passing the --model-impl transformers flag during deployment, composing directly with standard vLLM parallelization flags.
The updated backend relies on torch.fx to run static analysis on a model's computational graph and identify candidate optimization patterns. Following graph analysis, abstract syntax tree manipulation rewrites source code operations in place to apply inference-specific layer fusions dynamically. These fusions map operations to optimized vLLM kernels, including MergedColumnParallelLinear and QKVParallelLinear blocks for tensor parallelism, as well as Expert Parallelization routines for Mixture-of-Experts architectures.
Models modified by this backend remain fully compatible with torch.compile and CUDA Graphs. Unlike dedicated vLLM model code, transformers model definitions can also serve during training, evaluation, and reinforcement learning rollouts. Hugging Face stated that models relying on linear attention are currently unsupported, while custom Hub repository models may fail if they lack compliant code implementations.
