b4rtaz / distributed-llama
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
Why this job: says: llm inference; topics: llm-inference
What it does
Connect home devices into a powerful cluster to accelerate LLM inference. More devices mean faster performance, leveraging tensor parallelism and high-speed synchronization over Ethernet.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 3,132
- +7 this week
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.17.0
- Released 2 days ago
- Built with
- C++
- 97% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- 65% closed
- 46 still open
- Pull requests
- 100 merged
- 2 waiting
- Releases
- 65
- 3 in the last year
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| SciSharp / LLamaSharp | 3,825 | yesterday | MIT | No downloads |
| Picovoice / picollm | 319 | 3 days ago | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- C++ 97%
- Python 3%
- Makefile <1%
- C <1%
Topics
- distributed-computing
- llama2
- llm
- llm-inference
- neural-network
- llms
- open-llm
- distributed-llm
- llama3
Project record
- JobLocal LLMs
- StatusListed
- Commits425
- Forks260
- Watching52
- Stars in 28 days+30
- CreatedDec 4, 2023
- First seen by usNov 7, 2026
- CheckedNov 9, 2026