Evals and observability, page 3
252 repos on our list for this job.
Ranked by a score of how recently it was updated, contributors, growth, stars gained and total stars, and releases. How we rank
More repos in this job
| Rank | Name |
|---|---|
| 227. | MantisAI /nervaluate Full named-entity (i.e., not tag/token) evaluation metrics based on SemEval’13 225 stars0 this week |
| 228. | CMU-CREATE-Lab /deep-smoke-machine Deep learning models and dataset for recognizing industrial smoke emissions 125 stars0 this week |
| 229. | snap-stanford /stark (NeurIPS D&B 2024) STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases 338 stars0 this week |
| 230. | Minqi824 /ADBench Official Implement of "ADBench: Anomaly Detection Benchmark", NeurIPS 2022. 1,033 stars+1 this week |
| 231. | primitive-bench /primitive-bench The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer. 122 stars0 this week |
| 232. | VibeBench /VibeSearchBench 🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1. 107 stars+1 this week |
| 233. | microsoft /eureka-ml-insights A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. 187 stars0 this week |
| 234. | uber /petastorm Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code. 1,895 stars0 this week |
| 235. | InternScience /SGI-Bench Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows 172 stars0 this week |
| 236. | principia-ai /PhysGym A benchmark suite for evaluating LLM-based interactive scientific reasoning. 103 stars+1 this week |
| 237. | X-PLUG /WritingBench WritingBench: A Comprehensive Benchmark for Generative Writing 207 stars0 this week |
| 238. | AIRI-Institute /nablaDFT nablaDFT: Large-Scale Conformational Energy and Hamiltonian Prediction benchmark and dataset 241 stars0 this week |
| 239. | benedekrozemberczki /datasets A repository of pretty cool datasets that I collected for network science and machine learning research. 657 stars0 this week |
| 240. | heilcheng /openevals Benchmarking suite for open-weight language models 139 stars0 this week |
| 241. | sisinflab /elliot Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluation 298 stars0 this week |
| 242. | TheAgentCompany /TheAgentCompany An agent benchmark with tasks in a simulated software company. 814 stars+3 this week |
| 243. | trekhleb /machine-learning-experiments 🤖 Interactive Machine Learning experiments: 🏋️models training + 🎨models demo 1,837 stars0 this week |
| 244. | Scale3-Labs /langtrace Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊 1,235 stars+1 this week |
| 245. | IAAR-Shanghai /xFinder [ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation 179 stars0 this week |