DominikBucko / qwen38-flash-next-2x3090
Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).
Why this job: topics: local-llm, llm-inference, vllm, quantization
What it does
Qwen3.8-Flash-Next, with its high sparsity and low active param count is a great candidate for CPU offloading under right setup. This build keeps the full expert set and an FP8 Ngram table in system memory, caches active (LRU) experts on the GPUs, and uses a small MTP3 drafter to recover decode speed.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 25 days ago
- Stars
- 136
- +4 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.5.0
- Released 40 days ago
- Built with
- Python
- 96% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 75% closed
- 4 still open
- Pull requests
- 21 merged
- 5 waiting
- Releases
- 5
- 5 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| youssofal / MTPLX | 2,685 | 3 days ago | Apache-2.0 | Mac |
| syv-ai / HyperQwen | 2,626 | 6 days ago | Apache-2.0 | No downloads |
| jpezzulli / sglang-rtxpro6000 | 195 | 3 days ago | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- Python 96%
- C++ 2%
- Shell 1%
- Cuda <1%
Topics
- 256k-context
- cpu-offloading
- dual-gpu
- fp8
- int4
- llm-inference
- local-llm
- moe
- mtp
- multi-gpu
- quantization
- qwen
Project record
- JobLocal LLMs
- StatusListed
- Commits29
- Forks16
- Watching4
- Stars in 28 days+22
- CreatedAug 30, 2026
- First seen by usNov 7, 2026
- CheckedNov 7, 2026