ModelTC / LightLLM
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Why this job: works in the areas we cover, but does not say which job it does
What it does
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance. LightLLM harnesses the strengths of numerous well-regarded open-source implementations, including but not limited to FasterTransformer, TGI, vLLM, and FlashAttention.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit today
- Stars
- 4,358
- +9 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v1.2.0
- Released 3 months ago
- Built with
- Python
- 99% of the code
Is it looked after?
- Contributors
- 14
- Last 90 days
- Issues
- 60% closed
- 101 still open
- Pull requests
- 1,081 merged
- 84 waiting
- Releases
- 4
- 1 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| botpress / botpress | 14,986 | 3 days ago | MIT | No downloads |
| openlake-project / openlake | 2,860 | 5 days ago | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- Python 99%
- Shell 1%
- Jinja <1%
- Dockerfile <1%
Topics
- deep-learning
- gpt
- llama
- llm
- model-serving
- nlp
- openai-triton
Project record
- JobOther AI tools
- StatusListed
- Commits1,043
- Forks384
- Watching34
- Stars in 28 days+42
- CreatedJul 22, 2023
- First seen by usNov 7, 2026
- CheckedNov 9, 2026