ashhart / TensorFold
LLM Inference Engine for Metal, CUDA and Vulkan.
Why this job: says: inference engine, llm inference; topics: llm-inference
What it does
TensorFold serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly. Serving needs no Python or MLX installation.
Get started
brew install ashhart/tensorfold/tensorfold
tensorfold pull mlx-community/Qwen3.5-2B-MLX-4bit@93760be4f1f69842a46bc13dbdc0f19e291392a3
tensorfold serve mlx-community/Qwen3.5-2B-MLX-4bit --name local-model --context 8192 --no-thinking - Maintenance
- Active
- Last commit today
- Stars
- 1,712
- +106 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- 2 systems
- Mac, Linux
- Latest version
- v1.2.5
- Released yesterday
- Built with
- Zig
- 64% of the code
Is it looked after?
- Contributors
- 10
- Last 90 days
- Issues
- 81% closed
- 132 still open
- Pull requests
- 951 merged
- 182 waiting
- Releases
- 59
- 30+ in the last year
- Contributing guide (opens in new tab)
- Security policy (opens in new tab)
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| liguodongiot / llm-action | 25,266 | 26 days ago | Apache-2.0 | No downloads |
| incoai / splash | 2,159 | 3 days ago | Apache-2.0 | Mac |
| abhibambhaniya / GenZ-LLM-Analyzer | 129 | 3 months ago | MIT | No downloads |
More: download files, languages, topics and project record
Download files, v1.2.5
- Mac4 files, 292 downloads
- Linux6 files, 236 downloads
Languages
- Zig 64%
- TypeScript 22%
- Python 5%
- Metal 4%
Topics
- ai
- ai-tools
- llm
- llm-inference
- llm-tools
Project record
- JobLocal LLMs
- StatusFeatured
- Commits7,459
- Forks372
- Watching10
- Stars in 28 days+439
- CreatedJun 19, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026