wici-ai / inferssd
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Why this job: says: llama.cpp; topics: apple-silicon, llama-cpp, gguf, inference
What it does
These are the tested GGUF files, community conversions of the named model releases. DeepSeek's Q8_0 keeps the official bit widths: routed experts stay native 4-bit MXFP4, and FP8 tensors become 8-bit Q8_0. See weight-format details.
Get started
curl -fsSL https://github.com/wici-ai/inferssd/releases/latest/download/install.sh | sh - Maintenance
- Active
- Last commit today
- Stars
- 364
- +101 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- 2 systems
- Mac, Windows
- Latest version
- v0.1.4
- Released today
- Built with
- C++
- 56% of the code
Is it looked after?
- Contributors
- 5
- Last 90 days
- Issues
- 44% closed
- 9 still open
- Pull requests
- 122 merged
- 7 waiting
- Releases
- 5
- 4 in the last year
- Contributing guide (opens in new tab)
- No security policy
- Code of conduct (opens in new tab)
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| Helldez / BigMoeOnEdge | 734 | yesterday | Apache-2.0 | Mac, Windows, Linux |
| incoai / splash | 2,159 | 3 days ago | Apache-2.0 | Mac |
| trymirai / uzu | 3,244 | 2 days ago | MIT | Mac |
More: download files, languages, topics and project record
Download files, v0.1.4
- Mac2 files, 0 downloads
- Windows2 files, 0 downloads
Languages
- C++ 56%
- Python 16%
- C 10%
- Cuda 5%
Topics
- apple-silicon
- cuda
- gguf
- inference
- llama-cpp
- llm
- metal
- mixture-of-experts
- ssd
Project record
- JobLocal LLMs
- StatusNew to our list
- Commits216
- Forks60
- Watching10
- CreatedOct 22, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026