Ethik AI
Ethik AI

Module ethik

Installation

warning Python 3.6 or above is required

Via PyPI

>>> pip install ethik

Via GitHub for the latest development version

>>> pip install git+<https://github.com/XAI-ANITI/ethik.git>
>>> # Or through SSH:
>>> pip install git+ssh://git@github.com/XAI-ANITI/ethik.git

Development installation

>>> git clone <https://github.com/MaxHalford/ethik>
>>> cd ethik
>>> make install_dev
Expand source code 
"""
## Installation

<div style="display: flex; align-items: center; margin-bottom: 20px;">
    <i class="material-icons" style="margin-right: 10px; color: red;">warning</i> Python 3.6 or above is required
</div>

**Via [PyPI](https://pypi.org/project/ethik/)**

```shell
>>> pip install ethik
```

**Via GitHub for the latest development version**

```shell
>>> pip install git+https://github.com/XAI-ANITI/ethik.git
>>> # Or through SSH:
>>> pip install git+ssh://git@github.com/XAI-ANITI/ethik.git
```

**Development installation**

```shell
>>> git clone https://github.com/MaxHalford/ethik
>>> cd ethik
>>> make install_dev
```
"""
from . import datasets
from .__version__ import __version__
from .classification_explainer import ClassificationExplainer
from .regression_explainer import RegressionExplainer
from .image_classification_explainer import ImageClassificationExplainer
from .utils import extract_category


__all__ = [
    "__version__",
    "datasets",
    "RegressionExplainer",
    "ClassificationExplainer",
    "ImageClassificationExplainer",
    "extract_category",
]

Sub-modules

ethik.base_explainer
ethik.cache_explainer
ethik.classification_explainer
ethik.datasets
ethik.image_classification_explainer
ethik.query
ethik.regression_explainer
ethik.utils
ethik.warnings

Functions

def extract_category(X, cat)
Expand source code 
def extract_category(X, cat):
    return pd.get_dummies(X)[cat].rename(f"{X.name}={cat}")

Classes

class ClassificationExplainer (alpha=0.05, n_taus=41, n_samples=1, sample_frac=0.8, conf_level=0.05, max_iterations=15, tol=0.0001, n_jobs=1, memoize=False, verbose=True)

Explains the influence of features on model predictions and performance.

Parameters

alpha : float
A float between 0 and 0.5 which indicates by how close the CacheExplainer should look at extreme values of a distribution. The closer to zero, the more so extreme values will be accounted for. The default is 0.05 which means that all values beyond the 5th and 95th quantiles are ignored.
n_taus : int
The number of τ values to consider. The results will be more fine-grained the higher this value is. However the computation time increases linearly with n_taus. The default is 41 and corresponds to each τ being separated by it's neighbors by 0.05.
n_samples : int
The number of samples to use for the confidence interval. If 1, the default, no confidence interval is computed.
sample_frac : float
The proportion of lines in the dataset sampled to generate the samples for the confidence interval. If n_samples is 1, no confidence interval is computed and the whole dataset is used. Default is 0.8.
conf_level : float
A float between 0 and 0.5 which indicates the quantile used for the confidence interval. Default is 0.05, which means that the confidence interval contains the data between the 5th and 95th quantiles.
max_iterations : int
The maximum number of iterations used when applying the Newton step of the optimization procedure. Default is 5.
tol : float
The bottom threshold for the gradient of the optimization procedure. When reached, the procedure stops. Otherwise, a warning is raised about the fact that the optimization did not converge. Default is 1e-4.
n_jobs : int
The number of jobs to use for parallel computations. See joblib.Parallel(). Default is -1.
memoize : bool
Indicates whether or not memoization should be used or not. If True, then intermediate results will be stored in order to avoid recomputing results that can be reused by successively called methods. For example, if you call plot_influence followed by plot_influence_ranking and memoize is True, then the intermediate results required by plot_influence will be reused for plot_influence_ranking. Memoization is turned off by default because it can lead to unexpected behavior depending on your usage.
verbose : bool
Whether or not to show progress bars during computations. Default is True.
Expand source code 
class ClassificationExplainer(CacheExplainer):
    def plot_influence(
        self, X_test, y_pred, colors=None, yrange=None, size=None, constraints=None
    ):
        """Plot the influence for the features in `X_test`.

        See `ethik.cache_explainer.CacheExplainer.plot_influence()`.
        """
        if yrange is None:
            yrange = [0, 1]

        X_test = pd.DataFrame(to_pandas(X_test))
        y_pred = pd.DataFrame(to_pandas(y_pred))

        if len(y_pred.columns) == 1:
            return super().plot_influence(
                X_test=X_test,
                y_pred=y_pred.iloc[:, 0],
                colors=colors,
                yrange=yrange,
                size=size,
                constraints=constraints,
            )

        if colors is None:
            features = X_test.columns
            #  Skip the lightest color as it is too light
            scale = cl.interp(cl.scales["10"]["qual"]["Paired"], len(features) + 1)[1:]
            colors = {feat: scale[i] for i, feat in enumerate(features)}

        labels = y_pred.columns
        plots = []
        for label in labels:
            plots.append(
                super().plot_influence(
                    X_test=X_test,
                    y_pred=y_pred[label],
                    colors=colors,
                    yrange=yrange,
                    constraints=constraints,
                )
            )

        fig = make_subplots(rows=len(labels), cols=1, shared_xaxes=True)
        for ilabel, (label, plot) in enumerate(zip(labels, plots)):
            fig.update_layout({f"yaxis{ilabel+1}": dict(title=f"Average {label}")})
            for trace in plot["data"]:
                trace["showlegend"] = ilabel == 0 and trace["showlegend"]
                trace["legendgroup"] = trace["name"]
                fig.add_trace(trace, row=ilabel + 1, col=1)

        fig.update_xaxes(
            nticks=5,
            showline=True,
            showgrid=True,
            zeroline=False,
            linecolor="black",
            gridcolor="#eee",
        )
        fig.update_yaxes(
            range=yrange,
            showline=True,
            showgrid=True,
            linecolor="black",
            gridcolor="#eee",
        )
        fig.update_layout(
            {
                f"xaxis{len(labels)}": dict(
                    title="tau"
                    if len(X_test.columns) > 1
                    else f"Average {X_test.columns[0]}"
                )
            }
        )
        set_fig_size(fig, size)
        return fig

    def plot_influence_2d(
        self, X_test, y_pred, z_range=None, colorscale=None, size=None, constraints=None
    ):
        """Plot the combined influence for the features in `X_test`.

        See `ethik.cache_explainer.CacheExplainer.plot_influence_2d()`.
        """
        if z_range is None:
            z_range = [0, 1]

        X_test = pd.DataFrame(to_pandas(X_test))
        y_pred = pd.DataFrame(to_pandas(y_pred))

        if len(y_pred.columns) == 1:
            return super().plot_influence_2d(
                X_test=X_test,
                y_pred=y_pred,
                z_range=z_range,
                colorscale=colorscale,
                size=size,
                constraints=constraints,
            )

        labels = y_pred.columns
        plots = []
        for label in labels:
            plots.append(
                super().plot_influence_2d(
                    X_test=X_test,
                    y_pred=y_pred[label],
                    z_range=z_range,
                    colorscale=colorscale,
                    size=size,
                    constraints=constraints,
                )
            )

        fig = make_subplots(
            rows=len(labels),
            cols=1,
            shared_xaxes=True,
            subplot_titles=[p["data"][0]["colorbar"]["title"].text for p in plots],
        )
        for ilabel, (label, plot) in enumerate(zip(labels, plots)):
            fig.update_layout({f"yaxis{ilabel+1}": dict(title=plot.layout.yaxis.title)})
            heatmap, ds_mean = plot["data"]
            heatmap["showscale"] = ilabel == 0
            heatmap["colorbar"]["title"] = ""
            heatmap["hoverinfo"] = "x+y+z"
            fig.add_trace(heatmap, row=ilabel + 1, col=1)
            fig.add_trace(ds_mean, row=ilabel + 1, col=1)

        fig.update_xaxes(
            showline=True,
            showgrid=True,
            zeroline=False,
            mirror=True,
            linecolor="black",
            gridcolor="#eee",
        )
        fig.update_yaxes(
            showline=True,
            showgrid=True,
            mirror=True,
            linecolor="black",
            gridcolor="#eee",
        )
        fig.update_layout(
            {
                f"xaxis{len(labels)}": dict(title=plots[0].layout.xaxis.title),
                "title": plots[0].layout.title,
            }
        )
        set_fig_size(fig, size)
        return fig

    def plot_distributions(
        self,
        feature_values,
        y_pred=None,
        bins=None,
        show_hist=False,
        show_curve=True,
        targets=None,
        colors=None,
        dataset_color="black",
        size=None,
    ):
        """Plot the stressed distribution of `feature_values` or `y_pred` if specified
        for each mean of `feature_values` in `targets`.

        See `ethik.base_explainer.BaseExplainer.plot_distributions()`.
        """
        if y_pred is None:
            return super().plot_distributions(
                feature_values=feature_values,
                bins=bins,
                show_hist=show_hist,
                show_curve=show_curve,
                targets=targets,
                colors=colors,
                dataset_color=dataset_color,
                size=size,
            )

        y_pred = pd.DataFrame(to_pandas(y_pred))
        if len(y_pred.columns) == 1:
            return super().plot_distributions(
                feature_values=feature_values,
                y_pred=y_pred.iloc[:, 0],
                bins=bins,
                show_hist=show_hist,
                show_curve=show_curve,
                targets=targets,
                colors=colors,
                dataset_color=dataset_color,
                size=size,
            )

        labels = y_pred.columns
        plots = []
        for label in labels:
            plots.append(
                super().plot_distributions(
                    feature_values=feature_values,
                    y_pred=y_pred[label],
                    bins=bins,
                    show_hist=show_hist,
                    show_curve=show_curve,
                    targets=targets,
                    colors=colors,
                    dataset_color=dataset_color,
                )
            )

        fig = make_subplots(rows=len(labels), cols=1, shared_xaxes=True)
        for ilabel, (label, plot) in enumerate(zip(labels, plots)):
            fig.update_layout(
                {
                    f"xaxis{ilabel+1}": dict(title=label),
                    f"yaxis{ilabel+1}": dict(title=f"Probability density"),
                }
            )
            for itrace, trace in enumerate(plot["data"]):
                trace["legendgroup"] = itrace
                fig.add_trace(trace, row=ilabel + 1, col=1)

        fig.update_xaxes(
            showline=True,
            showgrid=True,
            zeroline=False,
            linecolor="black",
            gridcolor="#eee",
        )
        fig.update_yaxes(
            showline=True, showgrid=True, linecolor="black", gridcolor="#eee"
        )
        set_fig_size(fig, size)
        return fig

    def plot_influence_comparison(
        self, X_test, y_pred, reference, compared, colors=None, yrange=None, size=None
    ):
        """Plot the influence of features in `X_test` on `y_pred` for the
        individual `compared` compared to `reference`. Basically, we look at how
        the model would behave if the average individual were `compared` and take
        the difference with what the output would be if the average were `reference`.

        See `ethik.base_explainer.BaseExplainer.plot_influence_comparison()`.
        """
        if yrange is None:
            yrange = [-1, 1]

        X_test = pd.DataFrame(to_pandas(X_test))
        y_pred = pd.DataFrame(to_pandas(y_pred))
        features = X_test.columns
        labels = y_pred.columns

        if len(labels) == 1:
            return super().plot_influence_comparison(
                X_test,
                y_pred.iloc[:, 0],
                reference,
                compared,
                colors=colors,
                yrange=yrange,
                size=size,
            )

        if colors is None:
            #  Skip the lightest color as it is too light
            scale = cl.interp(cl.scales["10"]["qual"]["Paired"], len(features) + 1)[1:]
            colors = {feat: scale[i] for i, feat in enumerate(features)}

        plots = []
        for label in labels:
            plots.append(
                super().plot_influence_comparison(
                    X_test, y_pred[label], reference, compared, colors=colors
                )
            )

        fig = make_subplots(rows=len(labels), cols=1, shared_xaxes=True)
        title = None
        shapes = []
        for ilabel, (label, plot) in enumerate(zip(labels, plots)):
            fig.update_layout({f"yaxis{ilabel+1}": dict(title=label)})
            for trace in plot["data"]:
                shapes.append(
                    go.layout.Shape(
                        type="line",
                        x0=0,
                        y0=-0.5,
                        x1=0,
                        y1=len(features) - 0.5,
                        yref=f"y{ilabel+1}",
                        line=dict(color="black", width=1),
                    )
                )
                fig.add_trace(trace, row=ilabel + 1, col=1)

        fig.update_xaxes(
            range=yrange,
            showline=True,
            linecolor="black",
            linewidth=1,
            zeroline=False,
            showgrid=True,
            gridcolor="#eee",
            side="top",
            fixedrange=True,
            showticklabels=True,
        )
        fig.update_layout(
            showlegend=False,
            xaxis1=dict(title=plots[0].layout.xaxis.title),
            shapes=shapes,
        )
        set_fig_size(
            fig, size, width=500, height=100 + 60 * len(features) + 30 * len(labels)
        )
        return fig

Ancestors

Methods

def plot_distributions(self, feature_values, y_pred=None, bins=None, show_hist=False, show_curve=True, targets=None, colors=None, dataset_color='black', size=None)

Plot the stressed distribution of feature_values or y_pred if specified for each mean of feature_values in targets.

See BaseExplainer.plot_distributions().

Expand source code 
def plot_distributions(
    self,
    feature_values,
    y_pred=None,
    bins=None,
    show_hist=False,
    show_curve=True,
    targets=None,
    colors=None,
    dataset_color="black",
    size=None,
):
    """Plot the stressed distribution of `feature_values` or `y_pred` if specified
    for each mean of `feature_values` in `targets`.

    See `ethik.base_explainer.BaseExplainer.plot_distributions()`.
    """
    if y_pred is None:
        return super().plot_distributions(
            feature_values=feature_values,
            bins=bins,
            show_hist=show_hist,
            show_curve=show_curve,
            targets=targets,
            colors=colors,
            dataset_color=dataset_color,
            size=size,
        )

    y_pred = pd.DataFrame(to_pandas(y_pred))
    if len(y_pred.columns) == 1:
        return super().plot_distributions(
            feature_values=feature_values,
            y_pred=y_pred.iloc[:, 0],
            bins=bins,
            show_hist=show_hist,
            show_curve=show_curve,
            targets=targets,
            colors=colors,
            dataset_color=dataset_color,
            size=size,
        )

    labels = y_pred.columns
    plots = []
    for label in labels:
        plots.append(
            super().plot_distributions(
                feature_values=feature_values,
                y_pred=y_pred[label],
                bins=bins,
                show_hist=show_hist,
                show_curve=show_curve,
                targets=targets,
                colors=colors,
                dataset_color=dataset_color,
            )
        )

    fig = make_subplots(rows=len(labels), cols=1, shared_xaxes=True)
    for ilabel, (label, plot) in enumerate(zip(labels, plots)):
        fig.update_layout(
            {
                f"xaxis{ilabel+1}": dict(title=label),
                f"yaxis{ilabel+1}": dict(title=f"Probability density"),
            }
        )
        for itrace, trace in enumerate(plot["data"]):
            trace["legendgroup"] = itrace
            fig.add_trace(trace, row=ilabel + 1, col=1)

    fig.update_xaxes(
        showline=True,
        showgrid=True,
        zeroline=False,
        linecolor="black",
        gridcolor="#eee",
    )
    fig.update_yaxes(
        showline=True, showgrid=True, linecolor="black", gridcolor="#eee"
    )
    set_fig_size(fig, size)
    return fig
def plot_influence(self, X_test, y_pred, colors=None, yrange=None, size=None, constraints=None)

Plot the influence for the features in X_test.

See CacheExplainer.plot_influence().

Expand source code 
def plot_influence(
    self, X_test, y_pred, colors=None, yrange=None, size=None, constraints=None
):
    """Plot the influence for the features in `X_test`.

    See `ethik.cache_explainer.CacheExplainer.plot_influence()`.
    """
    if yrange is None:
        yrange = [0, 1]

    X_test = pd.DataFrame(to_pandas(X_test))
    y_pred = pd.DataFrame(to_pandas(y_pred))

    if len(y_pred.columns) == 1:
        return super().plot_influence(
            X_test=X_test,
            y_pred=y_pred.iloc[:, 0],
            colors=colors,
            yrange=yrange,
            size=size,
            constraints=constraints,
        )

    if colors is None:
        features = X_test.columns
        #  Skip the lightest color as it is too light
        scale = cl.interp(cl.scales["10"]["qual"]["Paired"], len(features) + 1)[1:]
        colors = {feat: scale[i] for i, feat in enumerate(features)}

    labels = y_pred.columns
    plots = []
    for label in labels:
        plots.append(
            super().plot_influence(
                X_test=X_test,
                y_pred=y_pred[label],
                colors=colors,
                yrange=yrange,
                constraints=constraints,
            )
        )

    fig = make_subplots(rows=len(labels), cols=1, shared_xaxes=True)
    for ilabel, (label, plot) in enumerate(zip(labels, plots)):
        fig.update_layout({f"yaxis{ilabel+1}": dict(title=f"Average {label}")})
        for trace in plot["data"]:
            trace["showlegend"] = ilabel == 0 and trace["showlegend"]
            trace["legendgroup"] = trace["name"]
            fig.add_trace(trace, row=ilabel + 1, col=1)

    fig.update_xaxes(
        nticks=5,
        showline=True,
        showgrid=True,
        zeroline=False,
        linecolor="black",
        gridcolor="#eee",
    )
    fig.update_yaxes(
        range=yrange,
        showline=True,
        showgrid=True,
        linecolor="black",
        gridcolor="#eee",
    )
    fig.update_layout(
        {
            f"xaxis{len(labels)}": dict(
                title="tau"
                if len(X_test.columns) > 1
                else f"Average {X_test.columns[0]}"
            )
        }
    )
    set_fig_size(fig, size)
    return fig
def plot_influence_2d(self, X_test, y_pred, z_range=None, colorscale=None, size=None, constraints=None)

Plot the combined influence for the features in X_test.

See CacheExplainer.plot_influence_2d().

Expand source code 
def plot_influence_2d(
    self, X_test, y_pred, z_range=None, colorscale=None, size=None, constraints=None
):
    """Plot the combined influence for the features in `X_test`.

    See `ethik.cache_explainer.CacheExplainer.plot_influence_2d()`.
    """
    if z_range is None:
        z_range = [0, 1]

    X_test = pd.DataFrame(to_pandas(X_test))
    y_pred = pd.DataFrame(to_pandas(y_pred))

    if len(y_pred.columns) == 1:
        return super().plot_influence_2d(
            X_test=X_test,
            y_pred=y_pred,
            z_range=z_range,
            colorscale=colorscale,
            size=size,
            constraints=constraints,
        )

    labels = y_pred.columns
    plots = []
    for label in labels:
        plots.append(
            super().plot_influence_2d(
                X_test=X_test,
                y_pred=y_pred[label],
                z_range=z_range,
                colorscale=colorscale,
                size=size,
                constraints=constraints,
            )
        )

    fig = make_subplots(
        rows=len(labels),
        cols=1,
        shared_xaxes=True,
        subplot_titles=[p["data"][0]["colorbar"]["title"].text for p in plots],
    )
    for ilabel, (label, plot) in enumerate(zip(labels, plots)):
        fig.update_layout({f"yaxis{ilabel+1}": dict(title=plot.layout.yaxis.title)})
        heatmap, ds_mean = plot["data"]
        heatmap["showscale"] = ilabel == 0
        heatmap["colorbar"]["title"] = ""
        heatmap["hoverinfo"] = "x+y+z"
        fig.add_trace(heatmap, row=ilabel + 1, col=1)
        fig.add_trace(ds_mean, row=ilabel + 1, col=1)

    fig.update_xaxes(
        showline=True,
        showgrid=True,
        zeroline=False,
        mirror=True,
        linecolor="black",
        gridcolor="#eee",
    )
    fig.update_yaxes(
        showline=True,
        showgrid=True,
        mirror=True,
        linecolor="black",
        gridcolor="#eee",
    )
    fig.update_layout(
        {
            f"xaxis{len(labels)}": dict(title=plots[0].layout.xaxis.title),
            "title": plots[0].layout.title,
        }
    )
    set_fig_size(fig, size)
    return fig
def plot_influence_comparison(self, X_test, y_pred, reference, compared, colors=None, yrange=None, size=None)

Plot the influence of features in X_test on y_pred for the individual compared compared to reference. Basically, we look at how the model would behave if the average individual were compared and take the difference with what the output would be if the average were reference.

See BaseExplainer.plot_influence_comparison().

Expand source code 
def plot_influence_comparison(
    self, X_test, y_pred, reference, compared, colors=None, yrange=None, size=None
):
    """Plot the influence of features in `X_test` on `y_pred` for the
    individual `compared` compared to `reference`. Basically, we look at how
    the model would behave if the average individual were `compared` and take
    the difference with what the output would be if the average were `reference`.

    See `ethik.base_explainer.BaseExplainer.plot_influence_comparison()`.
    """
    if yrange is None:
        yrange = [-1, 1]

    X_test = pd.DataFrame(to_pandas(X_test))
    y_pred = pd.DataFrame(to_pandas(y_pred))
    features = X_test.columns
    labels = y_pred.columns

    if len(labels) == 1:
        return super().plot_influence_comparison(
            X_test,
            y_pred.iloc[:, 0],
            reference,
            compared,
            colors=colors,
            yrange=yrange,
            size=size,
        )

    if colors is None:
        #  Skip the lightest color as it is too light
        scale = cl.interp(cl.scales["10"]["qual"]["Paired"], len(features) + 1)[1:]
        colors = {feat: scale[i] for i, feat in enumerate(features)}

    plots = []
    for label in labels:
        plots.append(
            super().plot_influence_comparison(
                X_test, y_pred[label], reference, compared, colors=colors
            )
        )

    fig = make_subplots(rows=len(labels), cols=1, shared_xaxes=True)
    title = None
    shapes = []
    for ilabel, (label, plot) in enumerate(zip(labels, plots)):
        fig.update_layout({f"yaxis{ilabel+1}": dict(title=label)})
        for trace in plot["data"]:
            shapes.append(
                go.layout.Shape(
                    type="line",
                    x0=0,
                    y0=-0.5,
                    x1=0,
                    y1=len(features) - 0.5,
                    yref=f"y{ilabel+1}",
                    line=dict(color="black", width=1),
                )
            )
            fig.add_trace(trace, row=ilabel + 1, col=1)

    fig.update_xaxes(
        range=yrange,
        showline=True,
        linecolor="black",
        linewidth=1,
        zeroline=False,
        showgrid=True,
        gridcolor="#eee",
        side="top",
        fixedrange=True,
        showticklabels=True,
    )
    fig.update_layout(
        showlegend=False,
        xaxis1=dict(title=plots[0].layout.xaxis.title),
        shapes=shapes,
    )
    set_fig_size(
        fig, size, width=500, height=100 + 60 * len(features) + 30 * len(labels)
    )
    return fig

Inherited members

class ImageClassificationExplainer (alpha=0.05, max_iterations=10, tol=0.0001, n_jobs=-1, memoize=True, verbose=True)

An explainer specially suited for image classification.

This has exactly the same API as Explainer, but expects to be provided with an array of images instead of a tabular dataset.

TODO: add a note about n_taus being 2

Parameters

alpha : float
A float between 0 and 0.5 which indicates by how close the Explainer should look at extreme values of a distribution. The closer to zero, the more so extreme values will be accounted for. The default is 0.05 which means that all values beyond the 5th and 95th quantiles are ignored.
max_iterations : int
The number of iterations used when applying the Newton step of the optimization procedure. Default is 5.
tol : float
The bottom threshold for the gradient of the optimization procedure. When reached, the procedure stops. Otherwise, a warning is raised about the fact that the optimization did not converge. Default is 1e-4.
n_jobs : int
The number of jobs to use for parallel computations. See joblib.Parallel(). Default is -1.
memoize : bool
Indicates whether or not memoization should be used or not. If True, then intermediate results will be stored in order to avoid recomputing results that can be reused by successively called methods. For example, if you call plot_influence followed by plot_influence_ranking and memoize is True, then the intermediate results required by plot_influence will be reused for plot_influence_ranking. Memoization is turned on by default because computations are time-consuming for images.
verbose : bool
Whether or not to show progress bars during computations. Default is True.
Expand source code 
class ImageClassificationExplainer(CacheExplainer):
    """An explainer specially suited for image classification.

    This has exactly the same API as `Explainer`, but expects to be provided with an array of
    images instead of a tabular dataset.

    TODO: add a note about n_taus being 2

    Parameters:
        alpha (float): A `float` between `0` and `0.5` which indicates by how close the `Explainer`
            should look at extreme values of a distribution. The closer to zero, the more so
            extreme values will be accounted for. The default is `0.05` which means that all values
            beyond the 5th and 95th quantiles are ignored.
        max_iterations (int): The number of iterations used when applying the Newton step
            of the optimization procedure. Default is `5`.
        tol (float): The bottom threshold for the gradient of the optimization
            procedure. When reached, the procedure stops. Otherwise, a warning
            is raised about the fact that the optimization did not converge.
            Default is `1e-4`.
        n_jobs (int): The number of jobs to use for parallel computations. See
            `joblib.Parallel()`. Default is `-1`.
        memoize (bool): Indicates whether or not memoization should be used or not. If `True`, then
            intermediate results will be stored in order to avoid recomputing results that can be
            reused by successively called methods. For example, if you call `plot_influence` followed by
            `plot_influence_ranking` and `memoize` is `True`, then the intermediate results required by
            `plot_influence` will be reused for `plot_influence_ranking`. Memoization is turned on by
            default because computations are time-consuming for images.
        verbose (bool): Whether or not to show progress bars during
            computations. Default is `True`.
    """

    def __init__(
        self,
        alpha=0.05,
        max_iterations=10,
        tol=1e-4,
        n_jobs=-1,
        memoize=True,
        verbose=True,
    ):
        super().__init__(
            alpha=alpha,
            n_taus=2,
            max_iterations=max_iterations,
            tol=tol,
            n_jobs=n_jobs,
            memoize=memoize,
            verbose=verbose,
        )

    def _set_image_shape(self, images):
        self.img_shape = images[0].shape
        if self.img_shape[-1] == 1:
            self.img_shape = self.img_shape[:-1]

    def explain_influence(self, X_test, y_pred):
        """Compute the influence of the model for the features in `X_test`.

        Args:
            X_test (np.array): An array of images, i.e. a 3d numpy array of
                dimension `(n_images, n_rows, n_cols)`.
            y_pred (pd.DataFrame or pd.Series): The model predictions
                for the samples in `X_test`. For binary classification and regression,
                `pd.Series` is expected. For multi-label classification, a
                pandas dataframe with one column per label is
                expected. The values can either be probabilities or `0/1`
                (for a one-hot-encoded output).

        Returns:
            pd.DataFrame:
                See `ethik.explainer.Explainer.explain_influence()`.
        """
        self._set_image_shape(images=X_test)

        with warnings.catch_warnings():
            warnings.simplefilter("ignore", category=ConstantWarning)
            return super().explain_influence(
                X_test=images_to_dataframe(X_test), y_pred=y_pred
            )

    def explain_performance(self, X_test, y_test, y_pred, metric):
        """Compute the change in model's performance for the features in `X_test`.

        Args:
            X_test (np.array): An array of images, i.e. a 3d numpy array of
                dimension `(n_images, n_rows, n_cols)`.
            y_test (pd.DataFrame or pd.Series): The true values
                for the samples in `X_test`. For binary classification and regression,
                a `pd.Series` is expected. For multi-label classification,
                a pandas dataframe with one column per label is
                expected. The values can either be probabilities or `0/1`
                (for a one-hot-encoded output).
            y_pred (pd.DataFrame or pd.Series): The model predictions
                for the samples in `X_test`. The format is the same as `y_test`.
            metric (callable): A scikit-learn-like metric
                `f(y_true, y_pred, sample_weight=None)`. The metric must be able
                to handle the `y` data. For instance, for `sklearn.metrics.accuracy_score()`,
                "the set of labels predicted for a sample must exactly match the
                corresponding set of labels in `y_true`".

        Returns:
            pd.DataFrame:
                See `ethik.explainer.Explainer.explain_performance()`.
        """
        self._set_image_shape(images=X_test)

        with warnings.catch_warnings():
            warnings.simplefilter("ignore", category=ConstantWarning)
            return super().explain_performance(
                X_test=images_to_dataframe(X_test),
                y_test=y_test,
                y_pred=y_pred,
                metric=metric,
            )

    def _get_fig_size(self, cell_width, n_rows, n_cols):
        if cell_width is None:
            cell_width = 800 / n_cols
        im_height, im_width = self.img_shape
        ratio = im_height / im_width
        cell_height = ratio * cell_width
        return n_cols * cell_width, n_rows * cell_height

    def plot_influence(self, X_test, y_pred, n_cols=3, cell_width=None):
        """Plot the influence of the model for the features in `X_test`.

        Args:
            X_test (pd.DataFrame or np.array): See `ImageClassificationExplainer.explain_influence()`.
            y_pred (pd.DataFrame or pd.Series): See `ImageClassificationExplainer.explain_influence()`.
            n_cols (int): The number of classes to render per row. Default is `3`.
            cell_width (int, optional): The width of each cell in pixels.

        Returns:
            plotly.graph_objs.Figure:
                A Plotly figure. It shows automatically in notebook cells but you
                can also call the `.show()` method to plot multiple charts in the
                same cell.
        """
        influences = self.explain_influence(X_test=X_test, y_pred=y_pred)
        z_values = {}

        for label, group in influences.groupby("label"):
            diffs = (
                group.query("tau == 1")["influence"]
                - group.query("tau == -1")["influence"].values
            )
            diffs = diffs.to_numpy().reshape(self.img_shape)
            z_values[label] = diffs

        n_plots = len(z_values)
        labels = sorted(z_values)
        n_rows = n_plots // n_cols + 1
        fig = make_subplots(
            rows=n_rows,
            cols=n_cols,
            subplot_titles=list(map(str, labels)),
            shared_xaxes="all",
            shared_yaxes="all",
            horizontal_spacing=0.2 / n_cols,
            vertical_spacing=0.2 / n_rows,
        )

        # We want all the heatmaps to share the same scale
        zmin = min(np.min(z) for z in z_values.values())
        zmax = max(np.max(z) for z in z_values.values())

        # We want to make sure that 0 is at the center of the scale
        zmin, zmax = min(zmin, -zmax), max(zmax, -zmin)

        im_height, im_width = self.img_shape
        colorbar_width = 30
        colorbar_ticks_width = 27
        colorbar_x = 1.02

        for i, label in enumerate(labels):
            fig.add_trace(
                go.Heatmap(
                    z=z_values[label][::-1],
                    x=list(range(im_width)),
                    y=list(range(im_height)),
                    zmin=zmin,
                    zmax=zmax,
                    colorscale="RdBu",
                    zsmooth="best",
                    showscale=(i == 0),
                    name=label,
                    hoverinfo="x+y+z",
                    reversescale=True,
                    colorbar=dict(
                        thicknessmode="pixels",
                        thickness=colorbar_width,
                        xpad=0,
                        x=colorbar_x,
                    ),
                ),
                row=i // n_cols + 1,
                col=i % n_cols + 1,
            )

        for i in range(n_plots):
            fig.update_layout(
                {
                    f"xaxis{i+1}": dict(visible=False),
                    f"yaxis{i+1}": dict(scaleanchor=f"x{i+1}", visible=False),
                }
            )

        width, height = self._get_fig_size(cell_width, n_rows, n_cols)
        width += (colorbar_x - 1) * width + colorbar_width + colorbar_ticks_width

        fig.update_layout(
            margin=dict(t=20, l=20, b=20, r=20),
            width=width,
            height=height,
            autosize=False,
        )
        return fig

    def plot_performance(self, X_test, y_test, y_pred, metric):
        """Plot the performance of the model for the features in `X_test`.

        Args:
            X_test (pd.DataFrame or np.array): See `ImageClassificationExplainer.explain_performance()`.
            y_test (pd.DataFrame or pd.Series): See `ImageClassificationExplainer.explain_performance()`.
            y_pred (pd.DataFrame or pd.Series): See `ImageClassificationExplainer.explain_performance()`.
            metric (callable): See `ImageClassificationExplainer.explain_performance()`.

        Returns:
            plotly.graph_objs.Figure:
                A Plotly figure. It shows automatically in notebook cells but you
                can also call the `.show()` method to plot multiple charts in the
                same cell.

                TODO: explain what is represented on the image.
        """
        perf = self.explain_performance(
            X_test=X_test, y_test=y_test, y_pred=y_pred, metric=metric
        )
        metric_name = self.get_metric_name(metric)

        mask = perf["label"] == perf["label"].iloc[0]
        diffs = (
            perf[mask].query(f"tau == 1")[metric_name].to_numpy()
            - perf[mask].query(f"tau == -1")[metric_name].to_numpy()
        )
        diffs = diffs.reshape(self.img_shape)

        # We want to make sure that 0 is at the center of the scale
        zmin, zmax = diffs.min(), diffs.max()
        zmin, zmax = min(zmin, -zmax), max(zmax, -zmin)

        height, width = self.img_shape

        fig = go.Figure()
        fig.add_trace(
            go.Heatmap(
                z=diffs[::-1],
                x=list(range(width)),
                y=list(range(height)),
                zmin=zmin,
                zmax=zmax,
                colorscale="RdBu",
                zsmooth="best",
                showscale=True,
                hoverinfo="x+y+z",
                reversescale=True,
            )
        )

        fig_width = 500
        fig.update_layout(
            margin=dict(t=20, l=20, b=20),
            width=fig_width,
            height=fig_width * height / width,
            xaxis=dict(visible=False),
            yaxis=dict(visible=False, scaleanchor="x", scaleratio=height / width),
        )

        return fig

Ancestors

Methods

def explain_influence(self, X_test, y_pred)

Compute the influence of the model for the features in X_test.

Args

X_test : np.array
An array of images, i.e. a 3d numpy array of dimension (n_images, n_rows, n_cols).
y_pred : pd.DataFrame or pd.Series
The model predictions for the samples in X_test. For binary classification and regression, pd.Series is expected. For multi-label classification, a pandas dataframe with one column per label is expected. The values can either be probabilities or 0/1 (for a one-hot-encoded output).

Returns

pd.DataFrame:
See ethik.explainer.Explainer.explain_influence().
Expand source code 
def explain_influence(self, X_test, y_pred):
    """Compute the influence of the model for the features in `X_test`.

    Args:
        X_test (np.array): An array of images, i.e. a 3d numpy array of
            dimension `(n_images, n_rows, n_cols)`.
        y_pred (pd.DataFrame or pd.Series): The model predictions
            for the samples in `X_test`. For binary classification and regression,
            `pd.Series` is expected. For multi-label classification, a
            pandas dataframe with one column per label is
            expected. The values can either be probabilities or `0/1`
            (for a one-hot-encoded output).

    Returns:
        pd.DataFrame:
            See `ethik.explainer.Explainer.explain_influence()`.
    """
    self._set_image_shape(images=X_test)

    with warnings.catch_warnings():
        warnings.simplefilter("ignore", category=ConstantWarning)
        return super().explain_influence(
            X_test=images_to_dataframe(X_test), y_pred=y_pred
        )
def explain_performance(self, X_test, y_test, y_pred, metric)

Compute the change in model's performance for the features in X_test.

Args

X_test : np.array
An array of images, i.e. a 3d numpy array of dimension (n_images, n_rows, n_cols).
y_test : pd.DataFrame or pd.Series
The true values for the samples in X_test. For binary classification and regression, a pd.Series is expected. For multi-label classification, a pandas dataframe with one column per label is expected. The values can either be probabilities or 0/1 (for a one-hot-encoded output).
y_pred : pd.DataFrame or pd.Series
The model predictions for the samples in X_test. The format is the same as y_test.
metric : callable
A scikit-learn-like metric f(y_true, y_pred, sample_weight=None). The metric must be able to handle the y data. For instance, for sklearn.metrics.accuracy_score(), "the set of labels predicted for a sample must exactly match the corresponding set of labels in y_true".

Returns

pd.DataFrame:
See ethik.explainer.Explainer.explain_performance().
Expand source code 
def explain_performance(self, X_test, y_test, y_pred, metric):
    """Compute the change in model's performance for the features in `X_test`.

    Args:
        X_test (np.array): An array of images, i.e. a 3d numpy array of
            dimension `(n_images, n_rows, n_cols)`.
        y_test (pd.DataFrame or pd.Series): The true values
            for the samples in `X_test`. For binary classification and regression,
            a `pd.Series` is expected. For multi-label classification,
            a pandas dataframe with one column per label is
            expected. The values can either be probabilities or `0/1`
            (for a one-hot-encoded output).
        y_pred (pd.DataFrame or pd.Series): The model predictions
            for the samples in `X_test`. The format is the same as `y_test`.
        metric (callable): A scikit-learn-like metric
            `f(y_true, y_pred, sample_weight=None)`. The metric must be able
            to handle the `y` data. For instance, for `sklearn.metrics.accuracy_score()`,
            "the set of labels predicted for a sample must exactly match the
            corresponding set of labels in `y_true`".

    Returns:
        pd.DataFrame:
            See `ethik.explainer.Explainer.explain_performance()`.
    """
    self._set_image_shape(images=X_test)

    with warnings.catch_warnings():
        warnings.simplefilter("ignore", category=ConstantWarning)
        return super().explain_performance(
            X_test=images_to_dataframe(X_test),
            y_test=y_test,
            y_pred=y_pred,
            metric=metric,
        )
def plot_influence(self, X_test, y_pred, n_cols=3, cell_width=None)

Plot the influence of the model for the features in X_test.

Args

X_test : pd.DataFrame or np.array
See ImageClassificationExplainer.explain_influence().
y_pred : pd.DataFrame or pd.Series
See ImageClassificationExplainer.explain_influence().
n_cols : int
The number of classes to render per row. Default is 3.
cell_width : int, optional
The width of each cell in pixels.

Returns

plotly.graph_objs.Figure:
A Plotly figure. It shows automatically in notebook cells but you can also call the .show() method to plot multiple charts in the same cell.
Expand source code 
def plot_influence(self, X_test, y_pred, n_cols=3, cell_width=None):
    """Plot the influence of the model for the features in `X_test`.

    Args:
        X_test (pd.DataFrame or np.array): See `ImageClassificationExplainer.explain_influence()`.
        y_pred (pd.DataFrame or pd.Series): See `ImageClassificationExplainer.explain_influence()`.
        n_cols (int): The number of classes to render per row. Default is `3`.
        cell_width (int, optional): The width of each cell in pixels.

    Returns:
        plotly.graph_objs.Figure:
            A Plotly figure. It shows automatically in notebook cells but you
            can also call the `.show()` method to plot multiple charts in the
            same cell.
    """
    influences = self.explain_influence(X_test=X_test, y_pred=y_pred)
    z_values = {}

    for label, group in influences.groupby("label"):
        diffs = (
            group.query("tau == 1")["influence"]
            - group.query("tau == -1")["influence"].values
        )
        diffs = diffs.to_numpy().reshape(self.img_shape)
        z_values[label] = diffs

    n_plots = len(z_values)
    labels = sorted(z_values)
    n_rows = n_plots // n_cols + 1
    fig = make_subplots(
        rows=n_rows,
        cols=n_cols,
        subplot_titles=list(map(str, labels)),
        shared_xaxes="all",
        shared_yaxes="all",
        horizontal_spacing=0.2 / n_cols,
        vertical_spacing=0.2 / n_rows,
    )

    # We want all the heatmaps to share the same scale
    zmin = min(np.min(z) for z in z_values.values())
    zmax = max(np.max(z) for z in z_values.values())

    # We want to make sure that 0 is at the center of the scale
    zmin, zmax = min(zmin, -zmax), max(zmax, -zmin)

    im_height, im_width = self.img_shape
    colorbar_width = 30
    colorbar_ticks_width = 27
    colorbar_x = 1.02

    for i, label in enumerate(labels):
        fig.add_trace(
            go.Heatmap(
                z=z_values[label][::-1],
                x=list(range(im_width)),
                y=list(range(im_height)),
                zmin=zmin,
                zmax=zmax,
                colorscale="RdBu",
                zsmooth="best",
                showscale=(i == 0),
                name=label,
                hoverinfo="x+y+z",
                reversescale=True,
                colorbar=dict(
                    thicknessmode="pixels",
                    thickness=colorbar_width,
                    xpad=0,
                    x=colorbar_x,
                ),
            ),
            row=i // n_cols + 1,
            col=i % n_cols + 1,
        )

    for i in range(n_plots):
        fig.update_layout(
            {
                f"xaxis{i+1}": dict(visible=False),
                f"yaxis{i+1}": dict(scaleanchor=f"x{i+1}", visible=False),
            }
        )

    width, height = self._get_fig_size(cell_width, n_rows, n_cols)
    width += (colorbar_x - 1) * width + colorbar_width + colorbar_ticks_width

    fig.update_layout(
        margin=dict(t=20, l=20, b=20, r=20),
        width=width,
        height=height,
        autosize=False,
    )
    return fig
def plot_performance(self, X_test, y_test, y_pred, metric)

Plot the performance of the model for the features in X_test.

Args

X_test : pd.DataFrame or np.array
See ImageClassificationExplainer.explain_performance().
y_test : pd.DataFrame or pd.Series
See ImageClassificationExplainer.explain_performance().
y_pred : pd.DataFrame or pd.Series
See ImageClassificationExplainer.explain_performance().
metric : callable
See ImageClassificationExplainer.explain_performance().

Returns

plotly.graph_objs.Figure:

A Plotly figure. It shows automatically in notebook cells but you can also call the .show() method to plot multiple charts in the same cell.

TODO: explain what is represented on the image.

Expand source code 
def plot_performance(self, X_test, y_test, y_pred, metric):
    """Plot the performance of the model for the features in `X_test`.

    Args:
        X_test (pd.DataFrame or np.array): See `ImageClassificationExplainer.explain_performance()`.
        y_test (pd.DataFrame or pd.Series): See `ImageClassificationExplainer.explain_performance()`.
        y_pred (pd.DataFrame or pd.Series): See `ImageClassificationExplainer.explain_performance()`.
        metric (callable): See `ImageClassificationExplainer.explain_performance()`.

    Returns:
        plotly.graph_objs.Figure:
            A Plotly figure. It shows automatically in notebook cells but you
            can also call the `.show()` method to plot multiple charts in the
            same cell.

            TODO: explain what is represented on the image.
    """
    perf = self.explain_performance(
        X_test=X_test, y_test=y_test, y_pred=y_pred, metric=metric
    )
    metric_name = self.get_metric_name(metric)

    mask = perf["label"] == perf["label"].iloc[0]
    diffs = (
        perf[mask].query(f"tau == 1")[metric_name].to_numpy()
        - perf[mask].query(f"tau == -1")[metric_name].to_numpy()
    )
    diffs = diffs.reshape(self.img_shape)

    # We want to make sure that 0 is at the center of the scale
    zmin, zmax = diffs.min(), diffs.max()
    zmin, zmax = min(zmin, -zmax), max(zmax, -zmin)

    height, width = self.img_shape

    fig = go.Figure()
    fig.add_trace(
        go.Heatmap(
            z=diffs[::-1],
            x=list(range(width)),
            y=list(range(height)),
            zmin=zmin,
            zmax=zmax,
            colorscale="RdBu",
            zsmooth="best",
            showscale=True,
            hoverinfo="x+y+z",
            reversescale=True,
        )
    )

    fig_width = 500
    fig.update_layout(
        margin=dict(t=20, l=20, b=20),
        width=fig_width,
        height=fig_width * height / width,
        xaxis=dict(visible=False),
        yaxis=dict(visible=False, scaleanchor="x", scaleratio=height / width),
    )

    return fig

Inherited members

class RegressionExplainer (alpha=0.05, n_taus=41, n_samples=1, sample_frac=0.8, conf_level=0.05, max_iterations=15, tol=0.0001, n_jobs=1, memoize=False, verbose=True)

Explains the influence of features on model predictions and performance.

Parameters

alpha : float
A float between 0 and 0.5 which indicates by how close the CacheExplainer should look at extreme values of a distribution. The closer to zero, the more so extreme values will be accounted for. The default is 0.05 which means that all values beyond the 5th and 95th quantiles are ignored.
n_taus : int
The number of τ values to consider. The results will be more fine-grained the higher this value is. However the computation time increases linearly with n_taus. The default is 41 and corresponds to each τ being separated by it's neighbors by 0.05.
n_samples : int
The number of samples to use for the confidence interval. If 1, the default, no confidence interval is computed.
sample_frac : float
The proportion of lines in the dataset sampled to generate the samples for the confidence interval. If n_samples is 1, no confidence interval is computed and the whole dataset is used. Default is 0.8.
conf_level : float
A float between 0 and 0.5 which indicates the quantile used for the confidence interval. Default is 0.05, which means that the confidence interval contains the data between the 5th and 95th quantiles.
max_iterations : int
The maximum number of iterations used when applying the Newton step of the optimization procedure. Default is 5.
tol : float
The bottom threshold for the gradient of the optimization procedure. When reached, the procedure stops. Otherwise, a warning is raised about the fact that the optimization did not converge. Default is 1e-4.
n_jobs : int
The number of jobs to use for parallel computations. See joblib.Parallel(). Default is -1.
memoize : bool
Indicates whether or not memoization should be used or not. If True, then intermediate results will be stored in order to avoid recomputing results that can be reused by successively called methods. For example, if you call plot_influence followed by plot_influence_ranking and memoize is True, then the intermediate results required by plot_influence will be reused for plot_influence_ranking. Memoization is turned off by default because it can lead to unexpected behavior depending on your usage.
verbose : bool
Whether or not to show progress bars during computations. Default is True.
Expand source code 
class RegressionExplainer(CacheExplainer):
    pass

Ancestors

Inherited members