Explainable Hallucination through Natural Language Inference Mapping
Large language models ({LLMs}) often generate hallucinated content, making it crucial to identify and quantify inconsistencies in their outputs. We introduce {HaluMap}, a post-hoc framework that detects hallucinations by mapping entailment and contradiction relations between source inputs and generated outputs using a natural language inference ({NLI}) model. To improve reliability, we propose a calibration step leveraging intra-text relations to refine predictions. {HaluMap} outperforms state-of-the-art {NLI}-based methods by five percentage points compared to other training-free approaches, while providing clear, interpretable explanations. As a training-free and model-agnostic approach, {HaluMap} offers a practical solution for verifying {LLM} outputs across diverse {NLP} tasks. The resources of this paper are available at https://github.com/caisa-lab/acl25-halumap.
- Published in:
Findings of the Association for Computational Linguistics: {ACL} 2025 - Type:
Inproceedings - Authors:
- Year:
2025 - Source:
https://aclanthology.org/2025.findings-acl.96/
Citation information
: Explainable Hallucination through Natural Language Inference Mapping, Findings of the Association for Computational Linguistics: {ACL} 2025, 2025, 1888--1896, July, Association for Computational Linguistics, https://aclanthology.org/2025.findings-acl.96/, Chen.etal.2025a,
@Inproceedings{Chen.etal.2025a,
author={Chen, Wei-Fan; Zhao, Zhixue; Karimi, Akbar; Flek, Lucie},
title={Explainable Hallucination through Natural Language Inference Mapping},
booktitle={Findings of the Association for Computational Linguistics: {ACL} 2025},
pages={1888--1896},
month={July},
publisher={Association for Computational Linguistics},
url={https://aclanthology.org/2025.findings-acl.96/},
year={2025},
abstract={Large language models ({LLMs}) often generate hallucinated content, making it crucial to identify and quantify inconsistencies in their outputs. We introduce {HaluMap}, a post-hoc framework that detects hallucinations by mapping entailment and contradiction relations between source inputs and generated outputs using a natural language inference ({NLI}) model. To improve reliability, we propose a...}}