UiPath / coder_eval
Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.
Why this job: topics: agent-skills, claude-skills, claude-code-skills, claude-code, codex, opencode
What it does
SWE-bench or SkillsBench, which rank models on a shared task set, you bring the tasks and you bring the scoring — weighted 0.0–1.0 criteria, a skill_triggered activation check, an A/B experiment layer, and per-tool cost telemetry, over whatever work *you* care about. See How it compares. 📚 Full docs: coder-eval.com/docs.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 3 days ago
- Stars
- 157
- +1 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.12.17
- Released 3 days ago
- Built with
- Python
- 84% of the code
Is it looked after?
- Contributors
- 18
- Last 90 days
- Issues
- 57% closed
- 15 still open
- Pull requests
- 149 merged
- 17 waiting
- Releases
- 34
- 30+ in the last year
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| wshobson / agents | 42,338 | yesterday | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 84%
- TypeScript 15%
- JavaScript <1%
- Shell <1%
Topics
- agent-evaluation
- claude-code
- codex
- coding-agents
- evaluation-framework
- claude
- claude-skills
- agent-testing
- claude-code-skills
- gemini
- swe-bench
- agent-skills
Project record
- JobAgent skills
- StatusListed
- Commits212
- Forks8
- Watching2
- Stars in 28 days+6
- CreatedJul 9, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026