loomax-labs / InferrLM

What it does

• Core local inference through llama.cpp with support for GGUF models on both Android and iOS.
• Local MLX inference for Apple Silicon devices.
• Seamless integration with cloud-based models from OpenAI, Gemini, and Anthropic. You need your own API keys and an InferrLM registered account for remote models. Using remote models is optional.
• Customizable base URLs for OpenAI-compatible providers like OpenRouter, Groq, Ollama, LM Studio, Together AI. This allows you to access alternative API endpoints within the app.
• Apple Foundation model support for Apple Intelligence supported devices.