JustVugg/colibri
Hierarchical inference engine for large MoE models
- Chart
- Not on a chart right now
- Stars
- 52k
What it is
colibri is a C-based inference engine that runs large mixture-of-experts models on consumer hardware. It keeps dense layers and hot experts in RAM or on a GPU while streaming cold experts from disk.
Who it is for
Developers and researchers who want to run large open-source AI models locally without high-end GPU clusters.
Why it is trending
It gained 9,415 stars today, reaching 51,669 total stars, following the v2.1.0 release. It stands out for running 13 engines—including GLM-5.2, DeepSeek V4 Flash, and Kimi K3—across CPU, Vulkan, and CUDA.
Main features
- Supports CPU, Vulkan, and CUDA backends plus GGUF files
- One-step setup script with hardware-based model recommendations
- Built-in web dashboard and OpenAI/Anthropic-compatible API endpoints
- Includes coli mcp server for AI assistant setup
Summarized on November 17, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary