mosecorg / mosec
A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine
Why this job: says: model serving; topics: llm-serving, model-serving
What it does
Mosec is a high-performance and flexible model serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API.
Get started
make package- Maintenance
- Active
- Last commit 7 days ago
- Stars
- 903
- +1 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- 0.9.8
- Released 28 days ago
- Built with
- Python
- 75% of the code
Is it looked after?
- Contributors
- 4
- Last 90 days
- Issues
- 90% closed
- 11 still open
- Pull requests
- 484 merged
- 2 waiting
- Releases
- 54
- 3 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| thu-pacman / chitu | 3,000 | 2 days ago | Apache-2.0 | No downloads |
| vllm-project / vllm-ascend | 3,063 | today | Apache-2.0 | No downloads |
| bentoml / BentoML | 8,935 | 11 days ago | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- Python 75%
- Rust 23%
- Dockerfile 1%
- Makefile 1%
Topics
- cv
- deep-learning
- gpu
- hacktoberfest
- jax
- llm
- llm-serving
- machine-learning
- machine-learning-platform
- mlops
- model-serving
- mxnet
Project record
- JobLocal LLMs
- StatusListed
- Commits498
- Forks75
- Watching9
- Stars in 28 days+2
- CreatedMar 13, 2021
- First seen by usNov 7, 2026
- CheckedNov 7, 2026