slwang-ustc / nano-vllm-v1
Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill
Why this job: topics: llm-inference, vllm, inference
What it does
- Fast inference. Comparable online inference speeds and offline throughput performance to vLLM v1.
- Readable codebase. Clean implementation in ~ 1,200 lines of Python code
- Optimization Suite. Paged Attention, Prefix Caching, Chunked Prefill, Tensor Parallelism, Torch Compilation, and full reproduction of the vLLM v1 scheduling strategy, etc.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Inactive
- Last commit 9 months ago
- Stars
- 104
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Python
- 100% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 0% closed
- 2 still open
- Pull requests
- 0 merged
- 3 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| vllm-project / vllm-ascend | 3,073 | today | Apache-2.0 | No downloads |
| microsoft / vidur | 723 | 2 months ago | MIT | No downloads |
| LMCache / LMCache | 12,344 | today | Apache-2.0 | Linux |
More: languages, topics and project record
Languages
- Python 100%
Topics
- deep-learning
- inference
- llm
- llm-inference
- nano-vllm
- nlp
- pytorch
- transformer
- vllm
Project record
- JobLocal LLMs
- StatusListed
- Commits19
- Forks27
- Watching0
- Stars in 28 days+4
- CreatedJan 22, 2026
- First seen by usNov 7, 2026
- CheckedNov 8, 2026