vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Why this job: topics: inference, llm-serving, model-serving, llama
What it does
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has grown into one of the most active open-source AI projects built and maintained by a diverse community of many dozens of academic institutions and companies from over 2000 contributors.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit today
- Stars
- 96,634
- +386 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- 2 systems
- Mac, Linux
- Latest version
- v0.33.0rc3
- Released 8 days ago
- Built with
- Python
- 86% of the code
Is it looked after?
- Contributors
- 57
- Last 90 days
- Issues
- 86% closed
- 2,825 still open
- Pull requests
- 24,887 merged
- 7,246 waiting
- Releases
- 109
- 30+ in the last year
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| alibaba / rtp-llm | 1,394 | yesterday | Apache-2.0 | No downloads |
| GradientHQ / parallax | 1,427 | 4 months ago | Apache-2.0 | No downloads |
| pegainfer-project / pegainfer | 738 | yesterday | Apache-2.0 | Linux |
More: download files, languages, topics and project record
Download files, v0.33.0rc3
- Mac1 file, 1,918 downloads
- Linux5 files, 5,491 downloads
Languages
- Python 86%
- Rust 7%
- Cuda 4%
- C++ 3%
Topics
- gpt
- llm
- pytorch
- model-serving
- transformer
- llm-serving
- inference
- llama
- amd
- cuda
- tpu
- deepseek
Project record
- JobLocal LLMs
- StatusListed
- Commits24,925
- Forks24,582
- Watching603
- Stars in 28 days+1,838
- CreatedFeb 9, 2023
- First seen by usNov 7, 2026
- CheckedNov 10, 2026