htobty / ReadyLLM

What it does

• Unified target management. add the local machine or remote LAN machines (SSH, key or password), auto-detect the OS, and configure the inference engine and model directory.
• Hardware detection & model recommendation. probe the target's GPU / VRAM / CPU / RAM and recommend models that actually fit.
• Model store. a curated catalog of text, image and video models, plus dynamic fetching of popular models from HuggingFace. Filter by category (text / image / video, with live per-category counts), by "fits this machine", or by keyword. Every
• One-click deployment. start / stop inference services (llama.cpp, vLLM, SGLang) with manually tweakable runtime parameters; when the engine is missing on the target, install it in one click (auto download / compile / pip and back-fill the p
• Real-time monitoring. push GPU utilization, VRAM, throughput, and latency curves live over WebSocket.