RustingStudio / RustingStrata
Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 80B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.
Why this job: says: llama.cpp, inference engine; topics: llama-cpp, local-llm, gguf, inference, gguf
What it does
A pure-Rust inference engine for Qwen GGUF models: Qwen3.5/3.6 (qwen35moe, qwen35), Qwen3-Next (qwen3next), Qwen3.8-Flash-Next (qwen4exp), Qwen3 (qwen3, qwen3moe), DeepSeek-V4.1-Flash (deepseek41) and GLM-5.3-Flash (glm5-next). CUDA kernels are compiled at run time with NVRTC through cudarc, and the same kernels also run on Vulkan (--backend vulkan): no C++ build step, no ggml. Experts that do not fit in VRAM run on the CPU at the same time as the GPU work, and the expert placement adapts to the routing it sees.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 171
- +159 this week
- Licence
- Custom
- Read before business use
- Downloads for
- 2 systems
- Windows, Linux
- Latest version
- v1.2.1
- Released 2 days ago
- Built with
- Rust
- 75% of the code
Is it looked after?
- Contributors
- 3
- Last 90 days
- Issues
- 50% closed
- 7 still open
- Pull requests
- 8 merged
- 0 waiting
- Releases
- 2
- 2 in the last year
- Contributing guide (opens in new tab)
- No security policy
- Code of conduct (opens in new tab)
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| simplexai-labs / PolyStrata | 956 | 3 days ago | MIT | Windows |
| mw00 / project-maya | 2,741 | yesterday | MIT | No downloads |
| raketenkater / ggrun | 284 | 3 days ago | MIT | Windows, Linux |
More: download files, languages, topics, project record and preview image
Download files, v1.2.1
- Windows1 file, 220 downloads
- Linux1 file, 93 downloads
Languages
- Rust 75%
- Cuda 11%
- Python 5%
- GLSL 4%
Topics
- cuda
- gguf
- inference
- llama-cpp
- llm
- local-llm
- mixture-of-experts
- moe
- openai-api
- qwen
- rust
- anthropic-api
Project record
- JobLocal LLMs
- StatusNew to our list
- Commits811
- Forks20
- Watching1
- CreatedOct 10, 2026
- First seen by usNov 7, 2026
- CheckedNov 10, 2026