bentoml / OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Why this job: topics: llm-inference, llm-serving, llama
What it does
OpenLLM allows developers to run any open-source LLMs (Llama 3.3, Qwen2.5, Phi3 and more) or custom models as OpenAI-compatible APIs with a single command. It features a built-in chat UI, state-of-the-art inference backends, and a simplified workflow for creating enterprise-grade cloud deployment with Docker and Kubernetes.
Get started
pip install openllm
# or pip3 install openllm
openllm hello- Maintenance
- Active
- Last commit 18 days ago
- Stars
- 12,587
- +8 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.6.30
- Released 18 months ago
- Built with
- Python
- 96% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 98% closed
- 6 still open
- Pull requests
- 758 merged
- 16 waiting
- Releases
- 147
- 0 in the last year
- No contributing guide
- Security policy (opens in new tab)
- Code of conduct (opens in new tab)
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| liguodongiot / llm-action | 25,266 | 26 days ago | Apache-2.0 | No downloads |
| vllm-project / vllm-ascend | 3,063 | today | Apache-2.0 | No downloads |
| alibaba / rtp-llm | 1,394 | today | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- Python 96%
- Shell 4%
Topics
- llm
- llmops
- model-inference
- fine-tuning
- llm-serving
- llama
- vicuna
- bentoml
- llama2
- llm-inference
- llm-ops
- open-source-llm
Project record
- JobLocal LLMs
- StatusFeatured
- Commits1,926
- Forks847
- Watching81
- Stars in 28 days+44
- CreatedApr 19, 2023
- First seen by usNov 7, 2026
- CheckedNov 9, 2026