ashhart/TensorFold
Zig-based LLM inference engine for Metal, CUDA, and Vulkan
- Chart
- Not on a chart right now
- Stars
- 2.3k
What it is
TensorFold serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. It reads checkpoints, tokenizes requests, and runs Metal or CUDA kernels directly without requiring Python or MLX.
Who it is for
Developers and teams who want to serve LLMs locally or on GPU servers without a Python runtime.
Why it is trending
It gained 954 stars today to reach 2,298 total stars, following the TensorFold 1.2.9 release on November 15, 2026. Its native Zig binary serves OpenAI- and Anthropic-compatible APIs without Python dependencies.
Main features
- Serves OpenAI-compatible chat, completions, Responses, and Anthropic Messages APIs
- Manages concurrent streams and request queues with --parallel
- Includes a local web UI (tensorfold web) and a macOS desktop app
- Experimental Living Weights (--slide) mode for teaching new facts into model weights
Summarized on November 17, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary