Back to the chart

FareedKhan-dev/kimi-k3-in-c

Portable C99 inference engine for Kimi K3 on a single CPU

Chart
Not on a chart right now
Stars
12.4k

Open on GitHubHomepage

What it is

A portable C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU without BLAS, frameworks, or GPUs. It streams the 1.56 TB checkpoint from disk so it can run in 8.24 GB of RAM while producing byte-identical output across memory budgets.

Who it is for

Systems and AI developers who want to run or study a massive MoE model on standard CPUs without external frameworks.

Why it is trending

It gained 3,259 stars today following the v2.0.0 release, which added macOS, Windows, and NEON ports along with chat mode. It stands out for producing byte-identical outputs from 8 GB to 128 GB+ of RAM using a 176 KB engine.

Main features

  • 176 KB C99 engine with zero BLAS or ML framework dependencies
  • Streams trunk layers and computes directly on packed 4-bit experts in 8.24 GB RAM
  • Supports Linux x86-64, macOS arm64, and Windows x86-64 with AVX2 and NEON
  • Includes a weightless test suite verified against a PyTorch reference

Summarized on November 18, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary