quantumaikr / quant.cpp
LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.
Why this job: says: llm inference; topics: llm-inference, gguf, quantization
What it does
Chunking was a workaround for small context windows. We just made it unnecessary. 6.4× KV compression brings full-document understanding to consumer hardware.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Inactive
- Last commit 6 months ago
- Stars
- 407
- +1 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- 3 systems
- Mac, Windows, Linux
- Latest version
- v0.13.0
- Released 7 months ago
- Built with
- C
- 54% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 78% closed
- 11 still open
- Pull requests
- 44 merged
- 0 waiting
- Releases
- 16
- 16 in the last year
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| Dynamis-Labs / spectralquant | 204 | 6 months ago | MIT | No downloads |
| syv-ai / HyperQwen | 2,687 | 2 days ago | Apache-2.0 | No downloads |
| huawei-csl / KVarN | 519 | 4 months ago | Apache-2.0 | No downloads |
More: download files, languages, topics and project record
Download files, v0.13.0
- Mac1 file, 4 downloads
- Windows1 file, 29 downloads
- Linux1 file, 15 downloads
Languages
- C 54%
- Makefile 14%
- C++ 10%
- Python 8%
Topics
- delta-compression
- embeddable
- gguf
- kv-cache
- llm
- llm-inference
- pure-c
- quantization
- transformer
- turboquant
Project record
- JobLocal LLMs
- StatusListed
- Commits788
- Forks44
- Watching6
- Stars in 28 days+3
- CreatedMar 28, 2026
- First seen by usNov 7, 2026
- CheckedNov 8, 2026