ModelEngine-Group / unified-cache-management
Persist and reuse KV Cache to speedup your LLM.
Why this job: works in the areas we cover, but does not say which job it does
What it does
Agent workflows, multi-turn conversations, and multimodal applications often revisit the same context. UCM makes reusable KV Cache available beyond an individual inference process and extends cache capacity through external storage. This helps reduce repeated computation and the pressure on accelerator memory.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit today
- Stars
- 354
- +5 this week
- Licence
- MIT
- Allows business use
- Downloads for
- Linux
- From the latest release
- Latest version
- v0.9.1
- Released today
- Built with
- C++
- 54% of the code
Is it looked after?
- Contributors
- 33
- Last 90 days
- Issues
- 78% closed
- 44 still open
- Pull requests
- 1,032 merged
- 87 waiting
- Releases
- 27
- 27 in the last year
- No contributing guide
- No security policy
- Code of conduct (opens in new tab)
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| Zefan-Cai / R-KV | 1,223 | 3 days ago | Apache-2.0 | No downloads |
| NVIDIA / cuml | 5,321 | 6 days ago | Apache-2.0 | No downloads |
| datawhalechina / torch-rechub | 1,241 | 6 days ago | MIT | No downloads |
More: download files, languages, topics and project record
Languages
- C++ 54%
- Python 40%
- CMake 2%
- HTML 2%
Topics
- ascend
- cuda
- gpu
- kvcache
- llm
- npu
- dram
- hbm
- nfs
- ssd
- torch
- ucm
Project record
- JobOther AI tools
- StatusListed
- Commits821
- Forks132
- Watching0
- Stars in 28 days+14
- CreatedJul 10, 2025
- First seen by usNov 7, 2026
- CheckedNov 12, 2026