ling0322 / libllm
A lightweight vLLM-style inference engine in Rust
Why this job: says: inference engine; topics: llama
What it does
It will automatically download the model from Huggingface, and start the chat CLI in llm.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 2 months ago
- Stars
- 154
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- C++
- 59% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- 0% closed
- 1 still open
- Pull requests
- 138 merged
- 1 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| mlc-ai / web-llm | 19,365 | 6 days ago | Apache-2.0 | No downloads |
| Tiiny-AI / PowerInfer | 9,875 | 6 months ago | MIT | No downloads |
| alibaba / rtp-llm | 1,394 | today | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- C++ 59%
- Rust 25%
- Cuda 10%
- Python 4%
Topics
- ai
- cpp
- language-model
- llama
- llm
- python
- rust
Project record
- JobLocal LLMs
- StatusListed
- Commits140
- Forks7
- Watching2
- Stars in 28 days0
- CreatedOct 23, 2023
- First seen by usNov 7, 2026
- CheckedNov 7, 2026