alibaba / MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
Why this job: says: inference engine
What it does
- Optimized for devices, no dependencies, can be easily deployed to mobile devices and a variety of embedded devices.
- iOS platform. static library size will full option for armv7+arm64 platforms is about 12MB, size increase of linked executables is about 2M.
- Android platform. core so size is about 800KB (armv7a - c++_shared).
- Using MNN_BUILD_MINI can reduce package size by about 25%, with a limit of fixed model input size
- Support FP16 / Int8 quantize, can reduce model size 50%-70%
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 16,462
- +120 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- 3 systems
- Mac, Windows, Linux
- Latest version
- 3.6.2
- Released 12 days ago
- Built with
- C++
- 89% of the code
Is it looked after?
- Contributors
- 22
- Last 90 days
- Issues
- 99% closed
- 41 still open
- Pull requests
- 1,002 merged
- 25 waiting
- Releases
- 64
- 6 in the last year
- Contributing guide (opens in new tab)
- Security policy (opens in new tab)
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| antirez / gguf-tools | 373 | 5 months ago | MIT | No downloads |
| ArkaneFans / servllama | 110 | yesterday | AGPL-3.0 | No downloads |
More: download files, languages, topics and project record
Download files, 3.6.2
- Mac1 file, 96 downloads
- Windows1 file, 254 downloads
- Linux1 file, 115 downloads
Languages
- C++ 89%
- Python 2%
- C 2%
- Kotlin 2%
Topics
- mnn
- convolution
- vulkan
- embedded-devices
- winograd-algorithm
- machine-learning
- deep-learning
- ml
- arm
- llm
- transformer
Project record
- JobLocal LLMs
- StatusListed
- Commits3,272
- Forks2,511
- Watching237
- Stars in 28 days+264
- CreatedApr 15, 2019
- First seen by usNov 7, 2026
- CheckedNov 10, 2026