UbiquitousLearning / mllm
Fast Multimodal LLM on Mobile Devices
Why this job: matched by what the repo talks about: llm, device, fast, multimodal
What it does
For input_len=2048 and output_len=128, pymllm shows strong prefill gains over llama.cpp on Jetson Orin. Qwen3-VL-2B W8A8 reaches up to 3.12x prefill speedup on AGX Orin 32GB and about 12243 tok/s prefill throughput. Decode throughput is generally close to llama.cpp, with small wins or losses depending on model, device, and quantization.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 43 days ago
- Stars
- 1,636
- +1 this week
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- 2.0.0
- Released 8 months ago
- Built with
- C++
- 79% of the code
Is it looked after?
- Contributors
- 2
- Last 90 days
- Issues
- 65% closed
- 66 still open
- Pull requests
- 466 merged
- 22 waiting
- Releases
- 2
- 1 in the last year
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| Tiiny-AI / PowerInfer | 9,879 | 6 months ago | MIT | No downloads |
| richardgill / llm-ui | 1,751 | 44 days ago | MIT | No downloads |
| microsoft / sarathi-serve | 536 | 10 months ago | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- C++ 79%
- Python 17%
- Cuda 2%
- CMake 1%
Topics
- llama
- multimodal
- ai
- llm
- mobile
Project record
- JobLocal LLMs
- StatusListed
- Commits3,230
- Forks221
- Watching30
- Stars in 28 days+9
- CreatedAug 30, 2023
- First seen by usNov 7, 2026
- CheckedNov 12, 2026