vllm-project/vllm
High-throughput LLM inference and serving engine
- Chart
- Not on a chart right now
- Stars
- 97.1k
What it is
vLLM is an open-source library for fast, memory-efficient LLM inference and serving using PagedAttention and continuous batching. It supports over 200 Hugging Face model architectures and provides an OpenAI-compatible API server.
Who it is for
AI engineers and researchers deploying LLMs in production or running high-throughput inference workloads.
Why it is trending
It gained 3,311 stars today to reach 97,099 total stars, alongside the recent v0.34.0 release.
Main features
- High serving throughput with PagedAttention, continuous batching, and prefix caching
- Quantization (FP8, INT8, INT4, GPTQ/AWQ, GGUF) and speculative decoding
- Distributed inference across NVIDIA, AMD, and Intel GPUs, CPUs, and TPUs
- OpenAI-compatible API, Anthropic Messages API, gRPC, and multi-LoRA support
Summarized on November 18, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary