An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning

The Rashomon Effect describes the following phenomenon: for a given dataset there may exist many models with equally good performance but with different solution strategies. The Rashomon Effect has implications for Explainable Machine Learning, especially for the comparability of explanations. We provide a unified view on three different comparison scenarios and conduct a quantitative evaluation across different datasets, models, attribution methods, and metrics. We find that hyperparameter-tuning plays a role and that metric selection matters. Our results provide empirical support for previously anecdotal evidence and exhibit challenges for both scientists and practitioners.

  • Published in:
    European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases
  • Type:
    Inproceedings
  • Authors:
    Müller, Sebastian; Toborek, Vanessa; Beckh, Katharina; Jakobs, Matthias; Bauckhage, Christian; Welke, Pascal
  • Year:
    2023

Citation information

Müller, Sebastian; Toborek, Vanessa; Beckh, Katharina; Jakobs, Matthias; Bauckhage, Christian; Welke, Pascal: An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023, https://link.springer.com/chapter/10.1007/978-3-031-43418-1_28, Mueller.etal.2023a,

Associated Lamarr Researchers

lamarr institute person Mueller Sebastian e1663925309673 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Sebastian Müller

Scientist to the profile
- Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Vanessa Toborek

Author to the profile
Portrait of Katharina Beckh.

Katharina Beckh

Author to the profile
Portrait of Matthias Jakobs.

Matthias Jakobs

Scientist to the profile
Kopie von LAMARR Person 500x500 1 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Christian Bauckhage

Director to the profile
Portrait of Pascal Welke.

Pascal Welke

Autor to the profile