FareedKhan-dev / train-llm-from-scratch
A straightforward method for training your LLM, from downloading data to generating text.
Why this job: says: from scratch
What it does
I implemented a transformer model from scratch using PyTorch, based on the paper Attention is All You Need. You can use my scripts to train your own billion or million parameter LLM using a single GPU, or a small one on a laptop CPU with no GPU at all.
Get started
git clone https://github.com/FareedKhan-dev/train-llm-from-scratch.git
cd train-llm-from-scratch
pip install -e . - Maintenance
- Active
- Last commit 38 days ago
- Stars
- 13,172
- +83 this week
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Python
- 74% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 93% closed
- 1 still open
- Pull requests
- 21 merged
- 1 waiting
- Releases
- None
- Never published
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| huggingface / alignment-handbook | 5,711 | 47 days ago | Apache-2.0 | No downloads |
| Danielskry / Awesome-RAG | 1,423 | 34 days ago | CC0-1.0 | No downloads |
| analyticalrohit / llms-from-scratch | 386 | 5 months ago | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 74%
- Jupyter Notebook 22%
- Mermaid 3%
- Shell 1%
Topics
- gemini
- large-language-models
- llm
- openai
- training
- transformers
Project record
- JobLists and guides
- StatusFeatured
- Commits175
- Forks1,829
- Watching93
- Stars in 28 days+1,096
- CreatedJan 12, 2025
- First seen by usNov 7, 2026
- CheckedNov 8, 2026