Back to the chart

ashhart/TensorFold

Zig-based LLM inference engine for Metal, CUDA, and Vulkan

Chart
Not on a chart right now
Stars
2.3k

Open on GitHubHomepage

What it is

TensorFold serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. It reads checkpoints, tokenizes requests, and runs Metal or CUDA kernels directly without requiring Python or MLX.

Who it is for

Developers and teams who want to serve LLMs locally or on GPU servers without a Python runtime.

Why it is trending

It gained 954 stars today to reach 2,298 total stars, following the TensorFold 1.2.9 release on November 15, 2026. Its native Zig binary serves OpenAI- and Anthropic-compatible APIs without Python dependencies.

Main features

  • Serves OpenAI-compatible chat, completions, Responses, and Anthropic Messages APIs
  • Manages concurrent streams and request queues with --parallel
  • Includes a local web UI (tensorfold web) and a macOS desktop app
  • Experimental Living Weights (--slide) mode for teaching new facts into model weights

Summarized on November 17, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary