microsoft / SWE-bench-Live
[NeurIPS 2025 D&B] 🚀 SWE-bench Goes Live!
Why this job: works in the areas we cover, but does not say which job it does
What it does
SWE-bench-Live is the first automatically-updating, multi-language and multi-os SWE task set designed for agentic benchmarking and training. This repository provides: 1. The evaluation script to evaluate the prediction patches of your agent on our public huggingface datasets: _SWE-bench-Live/SWE-bench-Live (Python)_, _SWE-bench-Live/MultiLang_ and _SWE-bench-Live/Windows_. 2. The task-creation source code for you to create your customized SWE tasks for large-scale agentic RFT/RL, each paired with an executable docker sandbox.
Get started
git clone https://github.com/microsoft/SWE-bench-Live --recursive
cd SWE-bench-Live
pip install -e . ./launch/ - Maintenance
- Active
- Last commit 4 days ago
- Stars
- 262
- +1 this week
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest tag
- v1.0-multi-language-multi-os-benchmarking
- Released 8 months ago
- Built with
- Python
- 93% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 87% closed
- 4 still open
- Pull requests
- 23 merged
- 56 waiting
- Releases
- 3
- 2 in the last year
- No contributing guide
- Security policy (opens in new tab)
- Code of conduct (opens in new tab)
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| mlcommons / inference | 1,660 | 2 months ago | Apache-2.0 | No downloads |
| GoodStartLabs / AI_Diplomacy | 713 | 5 months ago | Custom | No downloads |
| Tencent-Hunyuan / ArtifactsBenchmark | 282 | 10 months ago | Custom | No downloads |
More: languages, topics and project record
Languages
- Python 93%
- Shell 7%
Topics
- benchmark
- llm
- swe
Project record
- JobOther AI tools
- StatusListed
- Commits130
- Forks36
- Watching6
- Stars in 28 days+5
- CreatedMar 28, 2025
- First seen by usNov 7, 2026
- CheckedNov 9, 2026