{XALIGN}: A Human-centric Visual Analytics Approach to Understanding {AI} Explanations

Explainable {AI} ({XAI}) methods are usually evaluated for technical properties such as faithfulness and robustness. In high-stakes application domains, experts also need to check model predictions against established knowledge. Different explanation methods often disagree, and this disagreement is hard to see when each method is inspected separately. We propose {XALIGN}, a visual analytics workflow and interface that shows rule-based explanations and several feature attribution methods ({TreeSHAP}, {KernelSHAP}, and {LIME}) side by side for the same Random Forest model. Agreements and contradictions between the methods become visible signals that users can explore and compare with domain knowledge. We apply {XALIGN} in two case studies: the German Credit dataset and a meteorological dataset, in which events derived from {ICON} model output and Cloudnet observations are classified. In both, the views reveal discrepancies between the rule-based explanation and the attribution methods that a single method would not show. In an expert evaluation with three specialists, participants found {XALIGN} useful for identifying influential features and for avoiding over-trust in predictions. Linking rules to attributions was harder for the participant without an {XAI} background.