kvcache-ai/Mooncake
KVCache-centric disaggregated LLM serving platform
- Chart
- Not on a chart right now
- Stars
- 7.1k
What it is
Mooncake is a KVCache-centric disaggregated serving platform built for Moonshot AI's Kimi LLM service. It supports large-scale LLM inference through distributed KV cache pooling and RDMA-based data transfer.
Who it is for
AI infrastructure engineers operating large-scale LLM inference and disaggregated serving with engines like vLLM, SGLang, or TensorRT-LLM would use it.
Why it is trending
It recently released v0.3.14 and gained 10 stars today, reaching 7,081 total stars. It stands out for enabling Kimi to handle 75% more requests under SLOs and its broad integration across major inference engines.
Main features
- Distributed KV cache pooling and prefix reuse via Mooncake Store
- Zero-copy RDMA data transfer with Mooncake Transfer Engine
- Native integrations with vLLM, SGLang, TensorRT-LLM, and LMDeploy
- Support for Prefill-Decode (PD) and Encode-Prefill-Decode (EPD) disaggregation
Summarized on November 19, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary