ArkaneFans / servllama

What it does

• Local and cloud models. support OpenAI-compatible, Anthropic, and Gemini APIs alongside on-device GGUF and MNN models.
• Your own assistants. give each assistant a name, instructions, model, generation settings, and enabled tools.
• Conversation management. streaming replies, collapsible reasoning, Markdown and code blocks, supported image input, conversation search, drafts, message editing, and reply versions.
• Two inference engines. llama.cpp for GGUF files, and MNN for complete MNN model directories.
• Model discovery and downloads. browse featured models or search Hugging Face and ModelScope; pause, resume, retry, and switch download sources.