lordmathis / llamactl
Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.
Why this job: topics: llama-cpp, llamacpp, llm-inference, vllm, mlx, llama.cpp
What it does
- Built-in Model Downloader. Download GGUF and Safetensors models directly from HuggingFace for llama.cpp, vLLM, and MLX backends
- Dynamic Multi-Model Instances. llama.cpp router mode - serve multiple models from a single instance with on-demand loading
- Smart Resource Management. Automatic idle timeout, LRU eviction, instance groups with per-group limits, and configurable instance limits
- Web Dashboard. Modern React UI for managing instances, monitoring health, and viewing logs
- API Compatible. OpenAI chat completions and resources endpoints, Anthropic messages endpoint (depending on backend) - route requests to different models by instance name
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 2 days ago
- Stars
- 161
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- 3 systems
- Mac, Windows, Linux
- Latest version
- v0.23.0
- Released 2 days ago
- Built with
- Go
- 63% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 96% closed
- 1 still open
- Pull requests
- 230 merged
- 1 waiting
- Releases
- 42
- 23 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| xorbitsai / inference | 9,647 | yesterday | Apache-2.0 | No downloads |
| thomas9120 / LLama-GUI | 139 | 3 days ago | GPL-3.0 | No downloads |
| XHToken / Spark-X2.6 | 901 | 7 days ago | Apache-2.0 | No downloads |
More: download files, languages, topics and project record
Download files, v0.23.0
- Mac1 file, 2 downloads
- Windows1 file, 1 downloads
- Linux2 files, 5 downloads
Languages
- Go 63%
- TypeScript 36%
- CSS <1%
- Python <1%
Topics
- llama-cpp
- llama-server
- llamacpp
- llm
- llm-inference
- openai-api
- localllama
- localllm
- self-hosted
- mlx
- mlx-lm
- llm-router
Project record
- JobLocal LLMs
- StatusListed
- Commits1,050
- Forks33
- Watching2
- Stars in 28 days+4
- CreatedJul 16, 2025
- First seen by usNov 7, 2026
- CheckedNov 9, 2026