EfficientContext / ContextPilot
Accelerating Long Context LLM Inference with Accuracy-Preserving Context Optimization in SGLang, vLLM, llama.cpp, OpenClaw, RAG, and Agentic AI.
Why this job: says: llama.cpp, llm inference
What it does
Long-context workloads (RAG, memory chat, tool-augmented agents) prepend many context blocks. Across requests, these blocks often overlap but get reordered or duplicated, changing token prefixes and triggering cache misses and redundant KV recomputation. Common examples include (1) Trending Topic QA, (2) Closed-Domain Long-Context QA, (3) Batched Long-Context Inference, (4) multi-turn conversations with long-term memory and many more.
Get started
pip install contextpilot
xcode-select --install
# one-time: provides clang++ to compile the native hook- Maintenance
- Active
- Last commit 38 days ago
- Stars
- 146
- +1 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.5.0
- Released 45 days ago
- Built with
- Python
- 86% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 71% closed
- 4 still open
- Pull requests
- 41 merged
- 10 waiting
- Releases
- 13
- 13 in the last year
- No contributing guide
- Security policy (opens in new tab)
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| amitshekhariitbhu / llm-inference-engineering | 405 | 8 days ago | Apache-2.0 | No downloads |
| syv-ai / HyperQwen | 2,626 | 6 days ago | Apache-2.0 | No downloads |
| htobty / ReadyLLM | 101 | 3 days ago | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 86%
- TypeScript 13%
- Shell 1%
- Dockerfile <1%
Topics
- ai-agents
- context-api
- context-engineering
- inference-optimization
- prompt-engineering
- openclaw
- hermes-agent
Project record
- JobLocal LLMs
- StatusListed
- Commits454
- Forks9
- Watching2
- Stars in 28 days+6
- CreatedJan 9, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026