Machine learning models can largely outperform classical algorithms to make predictions about complexe problems, e.g. recognizing trees (which can vary a lot depending on the season, the species...). To do so, they learn from data (either from examples or experience) instead of following a well-defined sequence of instructions (like a cooking recipe). We humans do the same to teach our kids to recognize trees: we do not provide instructions but examples.
It leads to the kids not always being able to clearly explain their decisions. Similarly, machine learning models can look like black-box decision-making programs. The question is: can we trust them? Understanding why a prediction was made is crucial to answer this question. As a bank customer, I want to know why the machine doesn't want to lend me money. The GDPR even makes it mandatory in the EU. This is known as model interpretability.
Model interpretability opens the door to model fairness. Was the decision made based on an unfair view of the world? We don't want our models to reproduce human discrimination that probably reflects in data. Model fairness consists in making sure a decision was not based on protected attributes (e.g. gender, race... for a bank loan).
ethik is a Python package to explain trained decision rules and
to make sure they are fair.
At it's core, the approach of ethik is to build
counterfactual distributions that permit answering "what if?"
scenarios. The key principle is that we stress one or more variables of a test
set and we then observe how the trained machine learning model reacts to the
stress. The stress is based on a statistical re-weighting scheme called
entropic variable projection. The main benefit of our method
is that it will only consider realistic scenarios, and will not build fake
examples. It additionally scales well to large datasets. You may find more
information by reading this paper, the
Get started
tutorial and the How it works
page.
Currently, ethik can be used for:
- Detecting model influence with respect to one or more (protected) attributes.
- Identifying causes for why a model performs poorly on certain inputs.
- Visualizing regions of an image that influence a model's predictions.
We have more plans for the future.
Gallery
About
This work is led by members of the Toulouse Institute of Mathematics, namely:
You can contact us at jm.loubes@gmail.com lrisser@math.univ-toulouse.fr and/or maxhalford25@gmail.com.
This work is done at the Toulouse Mathematics Institute. It is supported by the Centre National de la Recherche Scientifique (CNRS) and in collaboration with the Artificial and Natural Intelligence Toulouse Institute (ANITI) project.
This software is released under the GPL license.