A Quantitative Human-Grounded Evaluation Process for Explainable ML
Methods from explainable machine learning are increasingly applied. However, evaluation of these methods is often anecdotal and not systematic. Prior work has identified properties of explanation quality and we argue that evaluation should be based on them. In this work, we provide an evaluation process that follows the idea of property testing. The process acknowledges the central role of the human, yet argues for a quantitative approach for the evaluation. We find that properties can be divided into two groups, one to ensure trustworthiness, the other to assess comprehensibility. Options for quantitative property tests are discussed. Future research should focus on the standardization of testing procedures.
- Published in:
HCXAI Workshop at CHI Workshop on Human-Centered Explainable AI (HCXAI) at the Conference on Human Factors in Computing Systems (CHI) - Type:
Inproceedings - Authors:
K. Beckh, S. Müller, S. Rüping - Year:
2022
Citation information
K. Beckh, S. Müller, S. Rüping: A Quantitative Human-Grounded Evaluation Process for Explainable ML, Workshop on Human-Centered Explainable AI (HCXAI) at the Conference on Human Factors in Computing Systems (CHI), HCXAI Workshop at CHI, 2022, Beckh.etal.2022,
@Inproceedings{Beckh.etal.2022,
author={K. Beckh, S. Müller, S. Rüping},
title={A Quantitative Human-Grounded Evaluation Process for Explainable ML},
booktitle={Workshop on Human-Centered Explainable AI (HCXAI) at the Conference on Human Factors in Computing Systems (CHI)},
journal={HCXAI Workshop at CHI},
year={2022},
abstract={Methods from explainable machine learning are increasingly applied. However, evaluation of these methods is often anecdotal and not systematic. Prior work has identified properties of explanation quality and we argue that evaluation should be based on them. In this work, we provide an evaluation process that follows the idea of property testing. The process acknowledges the central role of the...}}