Back to the chart

Mesh-LLM/mesh-llm

Distributed GPU and memory pooling for LLM inference

Chart
Not on a chart right now
Stars
4.2k

Open on GitHubHomepage

What it is

Mesh LLM pools GPUs and memory across multiple machines and exposes them as a single OpenAI-compatible API on localhost port 9337. It automatically decides whether to run a model locally, route it to a peer, or split it across nodes using Skippy stages.

Who it is for

Developers and AI agent users who want to run large models across multiple machines or share compute in private or public meshes.

Why it is trending

It gained 646 stars today following its v0.80.0 release on November 16, 2026. Its ability to split models that exceed a single machine's capacity into layer stages across peers stands out.

Main features

  • End-to-end QUIC encrypted peer transport and routing
  • Skippy stage splits for models too large for one machine
  • Public mesh discovery via Nostr and invite-token private meshes
  • Automatic compatible model selection via model: "mesh"

Summarized on November 17, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary