FonaTech / Project_Chronos
⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.
Why this job: matched by what the repo talks about: inference, moe, optimiz, window
What it does
Mainstream MoE models such as Mixtral, DeepSeek-MoE, and Qwen-MoE make routing decisions at decode time, token by token. The model discovers which experts it needs only when the next token is already on the critical path. If those experts are not resident in VRAM, generation stalls while weights are moved from SSD to RAM to VRAM.
Get started
pip install Project_Chronos- Maintenance
- Active
- Last commit 2 days ago
- Stars
- 207
- +1 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.3.0
- Released 4 days ago
- Built with
- Python
- 96% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- None
- No issues filed
- Pull requests
- 10 merged
- 0 waiting
- Releases
- 12
- 12 in the last year
- Contributing guide (opens in new tab)
- Security policy (opens in new tab)
- No code of conduct
More: languages, topics and project record
Languages
- Python 96%
- JavaScript 3%
- CSS <1%
- HTML <1%
Topics
- high-throughput
- mixture-of-experts
- predictive-inference
- sliding-window-attention
- streaming-llm
- async-dma
- dual-layer-moe
- io-latency-hiding
- lookahead-routing
- mla-attention
- ssd-offloading
- generative-ai
Project record
- JobLocal LLMs
- StatusListed
- Commits60
- Forks23
- Watching21
- Stars in 28 days+1
- CreatedApr 19, 2026
- First seen by usNov 7, 2026
- CheckedNov 10, 2026