Back to the chart

kvcache-ai/Mooncake

KVCache-centric disaggregated LLM serving platform

Chart
Not on a chart right now
Stars
7.1k

Open on GitHubHomepage

What it is

Mooncake is a KVCache-centric disaggregated serving platform built for Moonshot AI's Kimi LLM service. It supports large-scale LLM inference through distributed KV cache pooling and RDMA-based data transfer.

Who it is for

AI infrastructure engineers operating large-scale LLM inference and disaggregated serving with engines like vLLM, SGLang, or TensorRT-LLM would use it.

Why it is trending

It recently released v0.3.14 and gained 10 stars today, reaching 7,081 total stars. It stands out for enabling Kimi to handle 75% more requests under SLOs and its broad integration across major inference engines.

Main features

  • Distributed KV cache pooling and prefix reuse via Mooncake Store
  • Zero-copy RDMA data transfer with Mooncake Transfer Engine
  • Native integrations with vLLM, SGLang, TensorRT-LLM, and LMDeploy
  • Support for Prefill-Decode (PD) and Encode-Prefill-Decode (EPD) disaggregation

Summarized on November 19, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary