xLLM-AI / xllm
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
Why this job: says: inference engine; topics: llm-inference, inference, inference-engine
What it does
- Top-tier Performance. Delivers high-throughput, low-latency inference through many advanced features.
- Mainstream Hardware Support. Purpose-built and deeply optimized for Chinese AI accelerators.
- Service-Engine Decoupled Architecture. Service layer handles scheduling and availability; engine layer handles computation.
- Enterprise-grade Deployment. Battle-tested at scale across JD.com's core retail business.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit today
- Stars
- 1,618
- +3 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.12.0
- Released 9 days ago
- Built with
- C++
- 78% of the code
Is it looked after?
- Contributors
- 7
- Last 90 days
- Issues
- 53% closed
- 85 still open
- Pull requests
- 2,003 merged
- 158 waiting
- Releases
- 12
- 10 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| EfficientMoE / MoE-Infinity | 381 | 2 days ago | Apache-2.0 | Linux |
| zhongkaifu / TensorSharp | 667 | today | BSD-3-Clause | Mac, Windows, Linux |
| pegainfer-project / pegainfer | 738 | yesterday | Apache-2.0 | Linux |
More: languages, topics and project record
Languages
- C++ 78%
- Python 18%
- Cuda 2%
- CMake 1%
Topics
- deepseek
- inference
- inference-engine
- llm-inference
- qwen
- large-language-models
- glm
Project record
- JobLocal LLMs
- StatusListed
- Commits2,238
- Forks331
- Watching20
- Stars in 28 days+22
- CreatedAug 12, 2025
- First seen by usNov 7, 2026
- CheckedNov 9, 2026