An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning

The Rashomon Effect describes the following phenomenon: for a given dataset there may exist many models with equally good performance but with different solution strategies. The Rashomon Effect has implications for Explainable Machine Learning, especially for the comparability of explanations. We provide a unified view on three different comparison scenarios and conduct a quantitative evaluation across different datasets, models, attribution methods, and metrics. We find that hyperparameter-tuning plays a role and that metric selection matters. Our results provide empirical support for previously anecdotal evidence and exhibit challenges for both scientists and practitioners.

  • Published in:
    European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases
  • Type:
    Inproceedings
  • Authors:
    Müller, Sebastian; Toborek, Vanessa; Beckh, Katharina; Jakobs, Matthias; Bauckhage, Christian; Welke, Pascal
  • Year:
    2023

Citation information

Müller, Sebastian; Toborek, Vanessa; Beckh, Katharina; Jakobs, Matthias; Bauckhage, Christian; Welke, Pascal: An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023, https://link.springer.com/chapter/10.1007/978-3-031-43418-1_28, Mueller.etal.2023a,

Associated Lamarr Researchers

lamarr institute person Mueller Sebastian e1663925309673 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Sebastian Müller

Scientist to the profile
- Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Vanessa Toborek

Author to the profile
lamarr institute person Beckh Katharina - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Katharina Beckh

Author to the profile
lamarr institute person Jakobs Matthias - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Matthias Jakobs

Scientist to the profile
lamarr institute person Bauckhage Christian - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Christian Bauckhage

Director to the profile
lamarr institute person Welke Pascal - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Pascal Welke

Autor to the profile