jasonacox / TinyLLM
Setup and run a local LLM and Chatbot using consumer grade hardware.
Why this job: says: local llm; topics: vllm
What it does
- Supports multiple LLMs (see list below)
- Builds a local OpenAI API web service via Ollama, llama.cpp or vLLM.
- Serves up a Chatbot web interface with customizable prompts, accessing external websites (URLs), vector databases and other sources (e.g. news, stocks, weather).
Get started
# Clone the project
git clone https://github.com/jasonacox/TinyLLM.git
cd TinyLLM You need: CPU: Intel, AMD or Apple Silicon; Memory: 8GB+ DDR4; Disk: 128G+ SSD
- Maintenance
- Active
- Last commit 15 days ago
- Stars
- 354
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.16.5
- Released 11 months ago
- Built with
- JavaScript
- 47% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 12% closed
- 7 still open
- Pull requests
- 25 merged
- 9 waiting
- Releases
- 35
- 1 in the last year
- No contributing guide
- No security policy
- No code of conduct
More: languages, topics and project record
Languages
- JavaScript 47%
- Python 37%
- HTML 11%
- Shell 2%
Topics
- artificial-intelligence
- chatbot
- large-language-models
- llama-cpp-python
- llm
- openai
- rag
- retrieval-augmented-generation
- vllm
Project record
- JobLocal LLMs
- StatusListed
- Commits443
- Forks43
- Watching10
- Stars in 28 days+2
- CreatedSep 12, 2023
- First seen by usNov 7, 2026
- CheckedNov 7, 2026