shawnshekari / strixite

LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context

Why this job: says: inference engine, llm inference; topics: local-llm, llm-inference

What it does

OpenAI-compatible server. chat completions with streaming, tool calls, reasoning, structured output and
Images. PNG, JPEG, WebP, GIF, BMP and TGA in chat requests, from a separate ~0.9 GB file in the weights repo
Multi-token prediction. the model's own draft head, up to 7 tokens checked per step
Thinking that ends. a request can cap its reasoning with its own budget, and a nudge steps in when the
Prompt cache in RAM and on disk. long agent conversations resume instead of being re-read every turn, and

git clone https://github.com/shawnshekari/strixite && cd strixite
cmake --preset strix && cmake --build --preset strix

Repos similar to this one
Repo	Stars	Last commit	Licence	Downloads for
Yamz-Labs / kyojin	234	yesterday	MIT	No downloads
LMCache / LMCache	12,321	today	Apache-2.0	Linux
SemiAnalysisAI / InferenceX	2,016	today	Apache-2.0	No downloads

More: languages, topics and project record