Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generation

Financial institutions operate under dense, frequently amended rulebooks, and answering a compliance question correctly requires not only fluency but verifiable grounding in the authoritative text. Large language models are attractive for this task, yet the models that firms can realistically deploy on-premise are compact ones, and compact models hallucinate obligations. We study whether a carefully domain-adapted retrieval-augmented generation pipeline closes that gap. Our retriever is built in three stages on top of {LegalBERT}: entailment tuning that recasts question–passage matching as premise–hypothesis reconstruction, contrastive tuning with in-batch negatives, and score-level fusion with {BM}25. Our generator is a compact model (2B–12B parameters) served under 4-bit quantization, either prompted or adapted with retrieval-aware fine-tuning ({RAFT}) through {LoRA}. On {ObliQA}, a question-answering benchmark built from the Abu Dhabi Global Market rulebooks, the staged retriever raises Recall at 10 from 0.256 to 0.774 and outperforms {BM}25 (0.678) and E5-large-v2 (0.758), the strongest general-purpose dense encoder we tested. {RAFT}-{LoRA} then improves the composite {RePASs} answer-quality score for every model we could adapt, with the largest gain on the weakest one. However, the adapted models do not transfer to Australian case-law questions, and a closed-book model that receives no passages at all scores within 0.011 {RePASs} of the full pipeline while producing answers that cite nothing and misstate obligations. The retrieval gain is therefore measured directly, the generation gain is a gain in {RePASs} rather than demonstrated grounding, and grounding itself requires an evaluation protocol that {RePASs} does not provide.