vitoplantamura / OnnxStream
Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available.
Why this job: says: stable diffusion; topics: stable-diffusion
What it does
- Inference engine decoupled from the WeightsProvider
- WeightsProvider can be DiskNoCache, DiskPrefetch, Ram or custom
- Dynamic quantization (8 bit unsigned, asymmetric, percentile)
- Static quantization (W8A8 unsigned, asymmetric, percentile)
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 4 months ago
- Stars
- 2,088
- +1 this week
- Licence
- Custom
- Read before business use
- Downloads for
- Windows
- From the latest release
- Latest version
- v0.2
- Released 2 years ago
- Built with
- C++
- 91% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 35% closed
- 67 still open
- Pull requests
- 30 merged
- 0 waiting
- Releases
- 2
- 0 in the last year
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| xlite-dev / lite.ai.toolkit | 4,448 | 2 months ago | GPL-3.0 | No downloads |
More: download files, languages, topics and project record
Languages
- C++ 91%
- Python 2%
- C# 2%
- Jupyter Notebook 2%
Topics
- machine-learning
- onnx
- raspberry-pi
- stable-diffusion
- llama
- mistral
- tinyml
- wasm
- webassembly
- yolov8
- whisper
Project record
- JobImage generation
- StatusListed
- Commits115
- Forks99
- Watching28
- Stars in 28 days+4
- CreatedJul 14, 2023
- First seen by usNov 7, 2026
- CheckedNov 8, 2026