OpenBitSys / BitDecoding-FP4
Accurate, fast FP4 KV-cache decoding through algorithm-system co-design.
Why this job: topics: llm-inference, vllm
What it does
BitDecoding-NVFP4 achieves near-lossless accuracy: across seven benchmarks, the average score stays within 0.5 percentage points of BF16 KV on Qwen3-4B-Thinking-2507 and Qwen3-30B-A3B-Thinking-2507.
Get started
git clone --recurse-submodules https://github.com/OpenBitSys/BitDecoding-FP4.git
cd BitDecoding-FP4
bash tools/bootstrap_env.sh You need: 1 × NVIDIA B200 or B300 GPU (SM100 / SM103); CUDA 13.0 or newer (NVIDIA driver 580+); Python 3.12 and uv (pip install uv)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 12
- +11 this week
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Python
- 100% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- None
- No issues filed
- Pull requests
- 0 merged
- 0 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| LMCache / LMCache | 12,321 | today | Apache-2.0 | Linux |
| syv-ai / HyperQwen | 2,639 | today | Apache-2.0 | No downloads |
| jaylfc / taOS | 605 | 16 days ago | AGPL-3.0 | No downloads |
More: languages, topics and project record
Languages
- Python 100%
- Shell <1%
Topics
- cuda
- kv-cache-quantization
- llm-inference
- low-bit-quantization
- vllm
Project record
- JobLocal LLMs
- StatusNew to our list
- Commits8
- Forks0
- Watching0
- CreatedNov 3, 2026
- First seen by usNov 9, 2026
- CheckedNov 10, 2026