pegainfer-project / pegainfer
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
Why this job: says: inference engine, llm inference; topics: llm-inference, vllm, inference, inference-engine, llm-serving, model-serving
What it does
PegaInfer serves LLMs through an OpenAI-compatible API. Each model owns its scheduler, state, and kernels; serving and KV infrastructure are shared. No PyTorch or Python runtime.
Get started
curl -fsSL https://raw.githubusercontent.com/pegainfer-project/pegainfer/main/install.sh | bash - Maintenance
- Active
- Last commit yesterday
- Stars
- 738
- +6 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- Linux
- From the latest release
- Latest version
- v0.1.1
- Released 2 months ago
- Built with
- Rust
- 76% of the code
Is it looked after?
- Contributors
- 15
- Last 90 days
- Issues
- 90% closed
- 40 still open
- Pull requests
- 754 merged
- 20 waiting
- Releases
- 2
- 2 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| vllm-project / vllm | 96,634 | today | Apache-2.0 | Mac, Linux |
| SemiAnalysisAI / InferenceX | 2,016 | today | Apache-2.0 | No downloads |
| thu-pacman / chitu | 2,999 | 3 days ago | Apache-2.0 | No downloads |
More: download files, languages, topics, project record and preview image
Languages
- Rust 76%
- Python 13%
- Cuda 8%
- C++ 1%
Topics
- inference
- kimi
- llm-serving
- model-serving
- cuda
- cuda-kernels
- deepseek
- gpu
- inference-engine
- kimi-k2
- kv-cache
- llm
Project record
- JobLocal LLMs
- StatusListed
- Commits943
- Forks120
- Watching2
- Stars in 28 days+21
- CreatedFeb 17, 2026
- First seen by usNov 7, 2026
- CheckedNov 10, 2026