Back to the chart

JustVugg/colibri

Hierarchical inference engine for large MoE models

Chart
Not on a chart right now
Stars
52k

Open on GitHubHomepage

What it is

colibri is a C-based inference engine that runs large mixture-of-experts models on consumer hardware. It keeps dense layers and hot experts in RAM or on a GPU while streaming cold experts from disk.

Who it is for

Developers and researchers who want to run large open-source AI models locally without high-end GPU clusters.

Why it is trending

It gained 9,415 stars today, reaching 51,669 total stars, following the v2.1.0 release. It stands out for running 13 engines—including GLM-5.2, DeepSeek V4 Flash, and Kimi K3—across CPU, Vulkan, and CUDA.

Main features

  • Supports CPU, Vulkan, and CUDA backends plus GGUF files
  • One-step setup script with hardware-based model recommendations
  • Built-in web dashboard and OpenAI/Anthropic-compatible API endpoints
  • Includes coli mcp server for AI assistant setup

Summarized on November 17, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary