Neroued/ninfer
Native C++/CUDA inference engine for RTX 5090 and DGX Spark
- Chart
- Not on a chart right now
- Stars
- 3.4k
What it is
NInfer is a native C++/CUDA inference engine for selected models such as Qwen3.6, Qwen3.8, and GLM-5.3-Flash on RTX 5090 and DGX Spark. It runs text and image/video generation through a local CLI or OpenAI- and Anthropic-compatible HTTP APIs.
Who it is for
Developers and engineers deploying high-performance local or dual-node inference servers on RTX 5090 or DGX Spark hardware would use this.
Why it is trending
With 3,376 total stars and 9 gained today, it stands out for its native C++/CUDA implementation featuring CUDA Graph decode, speculative decoding, and shared-prefix caching tailored to RTX 5090 and DGX Spark.
Main features
- Local CLI and OpenAI/Anthropic-compatible HTTP APIs
- CUDA Graph decode, speculative decoding (MTP/DFlash), and shared-prefix caching
- Single-device execution and dual DGX Spark model-parallel execution via RDMA or NCCL
- Support for text, vision, tool calling, constrained output, and CPU expert offload
Summarized on November 18, 2026 from the README and repository data. It can be wrong; check the source before relying on it. Report a wrong summary