Counterfactual Explanations.pdf

Counterfactual explanations are obtained by identifying the smallest change to an input to change a prediction made by a fixed model. However, recent work has shown that there often exists multiple different classifiers that give almost equal solutions, known as predictive multiplicity. This challenges the assumption that there is one superior solution to a prediction problem.

The authors derive a general upper bound for the costs of counterfactual explanations under predictive multiplicity, which depends on the discrepancy between two classifiers. They compare sparse and data support approaches empirically on real-world data and find that data support methods are more robust to multiplicity of different models.

However, the authors also show that data support methods have provably higher costs of generating counterfactual explanations under one fixed model. The results challenge the commonly held view that counterfactual recommendations should be sparse in general.

The authors investigate the effect of assumption A1 (the underlying model is stable over time) for counterfactual explanations in theory and in practice. They consider two scenarios: (a) the decision maker changes the deployed model, and (b) the decision maker is unsure about the correct classifier.

The authors conclude that their theoretical and empirical results challenge the commonly held assumptions in the field of counterfactual explanations. They highlight the importance of considering the costs and robustness of counterfactual explanations under predictive multiplicity.

Key contributions include:

Deriving a general upper bound for the costs of counterfactual explanations under predictive multiplicity
Comparing sparse and data support approaches empirically on real-world data
Showing that data support methods are more robust to multiplicity of different models
Highlighting the importance of considering the costs and robustness of counterfactual explanations under predictive multiplicity.