Back to the chart

vllm-project/vllm

High-throughput LLM inference and serving engine

Chart
Not on a chart right now
Stars
97.1k

Open on GitHubHomepage

What it is

vLLM is an open-source library for fast, memory-efficient LLM inference and serving using PagedAttention and continuous batching. It supports over 200 Hugging Face model architectures and provides an OpenAI-compatible API server.

Who it is for

AI engineers and researchers deploying LLMs in production or running high-throughput inference workloads.

Why it is trending

It gained 3,311 stars today to reach 97,099 total stars, alongside the recent v0.34.0 release.

Main features

  • High serving throughput with PagedAttention, continuous batching, and prefix caching
  • Quantization (FP8, INT8, INT4, GPTQ/AWQ, GGUF) and speculative decoding
  • Distributed inference across NVIDIA, AMD, and Intel GPUs, CPUs, and TPUs
  • OpenAI-compatible API, Anthropic Messages API, gRPC, and multi-LoRA support

Summarized on November 18, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary