Local LLMs, page 5
510 repos on our list for this job.
Ranked by a score of how recently it was updated, contributors, growth, stars gained and total stars, and releases. How we rank
More repos in this job
| Rank | Name |
|---|---|
| 428. | akx /ollama-dl Download models from the Ollama library, without Ollama 154 stars0 this week |
| 429. | gokayfem /ComfyUI_VLM_nodes ComfyUI nodes for vision-language models: Qwen3-VL, Moondream 3, Florence-2, SmolVLM2, InternVL, Gemma 3, MiniCPM-V. Plus open-vocabulary detection, SAM2/SAM3 segmentation, video temporal reasoning, GGUF via llama.cpp, and hosted LLM/VLM APIs. 593 stars+1 this week |
| 430. | ymcui /Chinese-Mixtral 中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs） 613 stars0 this week |
| 431. | lmg-anon /mikupad LLM Frontend in a single html file 759 stars+2 this week |
| 432. | varunvasudeva1 /llm-server-docs End-to-end documentation to set up your own local & fully private LLM server on Debian. Equipped with chat, web search, RAG, model management, MCP servers, image generation, and TTS. 849 stars0 this week |
| 433. | ThunderAgent-org /ThunderAgent A simple, fast and robust program-aware agentic inference system. 479 stars+6 this week |
| 434. | huawei-csl /KVarN KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag. 519 stars+3 this week |
| 435. | dswah /pyGAM [CONTRIBUTORS WELCOME] Generalized Additive Models in Python 1,023 stars0 this week |
| 436. | matt-k-wong /mlx-flash Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon. 142 stars+1 this week |
| 437. | fikrikarim /volocal Fully local voice AI for iOS 103 stars+1 this week |
| 438. | AmyangXYZ /PoPo Pose and animate MMD model with LLM and MPL code 284 stars+1 this week |
| 439. | Leon-Sander /Local-Multimodal-AI-Chat Self-hostable multimodal chat with local LLMs (Ollama/OpenAI): PDF RAG, image chat, and Whisper voice, Streamlit + Docker. 209 stars+1 this week |
| 440. | 10Nates /ollama-autocoder A simple to use Ollama autocompletion engine with options exposed and streaming functionality 157 stars0 this week |
| 441. | zhihu /ZhiLight A highly optimized LLM inference acceleration engine for Llama and its variants. 906 stars0 this week |
| 442. | phronmophobic /llama.clj Run LLMs locally. A clojure wrapper for llama.cpp. 178 stars0 this week |
| 443. | little51 /llm-dev 《大模型项目实战：多领域智能应用开发》配套资源 238 stars0 this week |
| 444. | docusealco /rllama Ruby FFI bindings for llama.cpp to run open-source LLMs such as GPT-OSS, Qwen 3.5, Gemma 4, and Llama 3 locally with Ruby. 104 stars0 this week |
| 445. | ivanfioravanti /wine_variety_classification Examples on how to use various LLM providers with a Wine Classification problem 129 stars0 this week |
| 446. | Sandermage /sndr_core_engine SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI. 133 stars0 this week |
| 447. | cgbur /llama2.zig Inference Llama 2 in one file of pure Zig 225 stars+1 this week |
| 448. | pythops /tenere 🤖 TUI for LLMs 687 stars+1 this week |
| 449. | jmerelnyc /Talos GPU worker client for the Talos network. Pairs with your Talos account, serves open-model inference jobs over a WebSocket, and reports uptime for payouts. 629 stars-5 this week |
| 450. | antirez /gguf-tools GGUF implementation in C as a library and a tools CLI program 373 stars0 this week |
| 451. | airesearch-official /Z-Image-Turbo-Windows One-click Windows installer for Z-Image Turbo AI image generation. Optimized for low-VRAM GPUs (4GB+). Features Gradio web UI, automatic setup, and GGUF model support. 125 stars+1 this week |
| 452. | axon-rl /gem A Gym for Agentic LLMs 518 stars+2 this week |
| 453. | synsense /sinabs A deep learning library for spiking neural networks which is based on PyTorch, focuses on fast training and supports inference on neuromorphic hardware. 127 stars0 this week |
| 454. | clearml /clearml-serving ClearML - Model-Serving Orchestration and Repository Solution 170 stars0 this week |
| 455. | ma2za /telegram-llm-bot Local-first Telegram AI bot with Ollama tool calling, voice transcription, MinIO storage, SearchApi web search, and readiness checks 114 stars0 this week |
| 456. | RayFernando1337 /LLM-Calc Instantly calculate the maximum size of quantized language models that can fit in your available RAM, helping you optimize your models for inference. 258 stars0 this week |
| 457. | Chen-zexi /vllm-cli A command-line interface tool for serving LLM using vLLM. 506 stars+1 this week |
| 458. | slwang-ustc /nano-vllm-v1 Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill 104 stars0 this week |
| 459. | Dynamis-Labs /spectralquant SpectralQuant: Calibrated Eigenbasis Rotation and Water-Filled Bit Allocation for KV-Cache Compression 204 stars0 this week |
| 460. | microsoft /sarathi-serve A low-latency & high-throughput serving engine for LLMs 536 stars+1 this week |
| 461. | AI-Hypercomputer /JetStream JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome). 473 stars0 this week |
| 462. | datawhalechina /code-your-own-llm 一份全栈式大语言模型参考指南，用最简洁的代码帮助你端到端定义模型从零训练到工程落地的每一个细节 227 stars0 this week |
| 463. | psmarter /CUDA-Practice CUDA编程练习项目-Hands-on CUDA kernels and performance optimization, covering GEMM, FlashAttention, Tensor Cores, CUTLASS, quantization, KV cache, NCCL, and profiling. 114 stars0 this week |
| 464. | alipay /PainlessInferenceAcceleration Accelerate inference without tears 368 stars0 this week |
| 465. | axhiao /QuickNote Capture what you want with LLM 163 stars0 this week |
| 466. | nekomeowww /ollama-operator 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫 242 stars0 this week |
| 467. | KolosalAI /kolosal-cli Super lightweight Ollama + Qwen Code alternative to run Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other large language models. 465 stars+2 this week |
| 468. | cxumol /promptmask Never give AI companies your secrets! A local LLM-based privacy filter for LLM users. Seamless integration with your existing AI tools as a Python library / OpenAI SDK replacement / API Gatetway / Web Server. 157 stars0 this week |
| 469. | domschl /HuggingFaceGuidedTourForMac A guided tour on how to use HuggingFace large language models on Macs with Apple Silicon 205 stars0 this week |
| 470. | hyperai /vllm-cn vLLM Documentation in Chinese Simplified / vLLM 中文文档 199 stars0 this week |
| 471. | LeanModels /DFloat11 DFloat11 [NeurIPS '25]: Lossless Compression of LLMs and DiTs for Efficient GPU Inference 665 stars+1 this week |
| 472. | SeldonIO /MLServer An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more 903 stars0 this week |
| 473. | lumia431 /photon_infer A High-Performance LLM Inference Engine with vLLM-Style Continuous Batching 121 stars0 this week |
| 474. | withcatai /catai Run AI ✨ assistant locally! with simple API for Node.js 🚀 498 stars0 this week |
| 475. | jy-yuan /KIVI [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache 439 stars+1 this week |
| 476. | promplate /partial-json-parser Parse partial JSON generated by LLM 141 stars0 this week |