holistic-ai / holisticai

What it does

An open-source library for assessing and improving the safety aspects of AI systems. It answers five questions a validation accuracy cannot: does the model treat groups differently, can its decisions be accounted for, does it hold up under shift and attack, what does it leak about its training data, and is it actually effective. Each answer is a metric you can compute in a few lines against a scikit-learn-shaped model — and for bias, a mitigator you can drop into the pipeline that produced it.