mit-han-lab / EvolvingInfra
A inference framework that adapt to diverse and dynamic agent workload
Why this job: works in the areas we cover, but does not say which job it does
What it does
Evolving Infra is an LLM serving system that adapts itself to the agent workload purely online, with no offline profiling. Serving coding, math and computer-use agents with GLM-5.2 and Kimi K2.6 on up to 256 GB300 GPUs, it reduces mean end-to-end latency by up to 4.1× and P95 time to first token by up to 14× compared with NVIDIA Dynamo, with up to 2.7× its throughput.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 12
- On GitHub
- Licence
- Custom
- Read before business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Python
- 80% of the code
Is it looked after?
- Contributors
- 3
- Last 90 days
- Issues
- 100% closed
- 0 still open
- Pull requests
- 13 merged
- 1 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| dmlc / xgboost | 28,967 | 2 days ago | Apache-2.0 | No downloads |
| professorpalmer / Puppetmaster | 472 | yesterday | MIT | No downloads |
| yzhao062 / SUOD | 395 | 2 months ago | BSD-2-Clause | No downloads |
More: languages, topics and project record
Languages
- Python 80%
- Rust 8%
- Cuda 8%
- C++ 4%
Topics
- distributed-systems
- llm
- llm-agents
- system
Project record
- JobOther AI tools
- StatusNew to our list
- Commits142
- Forks0
- Watching1
- CreatedNov 10, 2026
- First seen by usNov 12, 2026
- CheckedNov 13, 2026