jegly / OfflineLLM

What it does

• 100% Offline. no INTERNET permission in the manifest, cannot phone home
• On-Device Inference. GGUF models via llama.cpp; runtime CPU dispatch picks the best kernel set (dotprod / fp16 / i8mm / SVE) for your exact SoC
• GPU Acceleration (Vulkan). opt-in toggle in Settings with per-layer offload control and automatic CPU fallback; detects and names your GPU
• Fast Multi-Turn Chat. incremental prompt processing: each turn feeds only the new message into the KV cache instead of re-processing the whole conversation
• Performance Controls. CPU thread count, prompt-phase threading across all cores, memory-mapped loading, RAM lock, experimental quantized KV cache