giannisanni / pulsar
SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.
Why this job: says: inference engine; topics: local-llm, gguf, inference-engine, quantization
What it does
- Expert streaming needs link bandwidth → the fastest measured card.
- Attention residency (MLA models. the whole ~14GB attn stack + KV
- Expert tiers. leftover cards are filled with the hottest expert
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 2 months ago
- Stars
- 236
- +2 this week
- Licence
- Custom
- Read before business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Rust
- 59% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 50% closed
- 6 still open
- Pull requests
- 24 merged
- 0 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| mw00 / project-maya | 2,741 | yesterday | MIT | No downloads |
| simplexai-labs / PolyStrata | 956 | 3 days ago | MIT | Windows |
| prabanta-dev / janas | 685 | today | GPL-3.0 | Windows, Linux |
More: languages, topics and project record
Languages
- Rust 59%
- C++ 20%
- Cuda 14%
- HTML 5%
Topics
- cuda
- gguf
- glm
- inference-engine
- llm
- local-llm
- mixture-of-experts
- moe
- multi-gpu
- quantization
- rust
- ssd-streaming
Project record
- JobLocal LLMs
- StatusListed
- Commits562
- Forks28
- Watching2
- Stars in 28 days+8
- CreatedJul 13, 2026
- First seen by usNov 7, 2026
- CheckedNov 8, 2026