waybarrios / vllm-mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Why this job: says: inference server, llm inference; topics: apple-silicon, local-llm, vllm, mlx
What it does
- OpenAI-compatible. /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/rerank, /v1/responses
- Anthropic-compatible. /v1/messages (streaming, tool use, system prompts)
- MCP Tool Calling. 19 parsers (OpenAI, Anthropic, Gemini, Qwen, DeepSeek, Gemma, and more)
- Structured output. JSON Schema via response_format (lm-format-enforcer)
- Continuous batching. high throughput for concurrent requests
Get started
python -m pip install 'vllm-mlx==0.4.1'
vllm-mlx serve mlx-community/Llama-3.2-3B-Instruct-4bit --host 127.0.0.1 --port 8000 - Maintenance
- Active
- Last commit 2 days ago
- Stars
- 1,651
- +7 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.6.0
- Released 20 days ago
- Built with
- Python
- 100% of the code
Is it looked after?
- Contributors
- 11
- Last 90 days
- Issues
- 73% closed
- 69 still open
- Pull requests
- 394 merged
- 38 waiting
- Releases
- 16
- 16 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| jundot / omlx | 23,475 | 4 days ago | Apache-2.0 | Mac |
| vllm-project / vllm-metal | 1,974 | today | Apache-2.0 | Mac |
| carloslfu / slotstream | 659 | today | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 100%
Topics
- anthropic
- anthropic-api
- apple-silicon
- claude-code
- continuous-batching
- inference-server
- llm
- local-llm
- macos
- mcp
- mlx
- multimodal-ai
Project record
- JobLocal LLMs
- StatusListed
- Commits735
- Forks237
- Watching11
- Stars in 28 days+34
- CreatedDec 6, 2025
- First seen by usNov 7, 2026
- CheckedNov 7, 2026