Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models

Auditing financial documents is a very tedious and time-consuming process. As of today, it can already be simplified by employing AI-based solutions to recommend relevant text passages from a report for each legal requirement of rigorous accounting standards. However, these methods need to be fine-tuned regularly, and they require abundant annotated data, which is often lacking in industrial environments. Hence, we present ZeroShotALI, a novel recommender system that leverages a state-of-the-art large language model (LLM) in conjunction with a domain-specifically optimized transformer-based text-matching solution. We find that a two-step approach of first retrieving a number of best matching document sections per legal requirement with a custom BERT-based model and second filtering these selections using an LLM yields significant performance improvements over existing approaches.

  • Published in:
    ACM Symposium on Document Engineering
  • Type:
    Inproceedings
  • Authors:
    Hillebrand, Lars; Berger, Armin; Deußer, Tobias; Dilmaghani, Tim; Khaled, Mohamed; Kliem, Bernd; Loitz, Rüdiger; Pielka, Maren; Leonhard, David; Bauckhage, Christian; Sifa, Rafet
  • Year:
    2023

Citation information

Hillebrand, Lars; Berger, Armin; Deußer, Tobias; Dilmaghani, Tim; Khaled, Mohamed; Kliem, Bernd; Loitz, Rüdiger; Pielka, Maren; Leonhard, David; Bauckhage, Christian; Sifa, Rafet: Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models, ACM Symposium on Document Engineering, 2023, https://arxiv.org/abs/2308.06111, Hillebrand.etal.2023b,

Associated Lamarr Researchers

lamarr institute person Pielka Maren - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Maren Pielka

Autorin to the profile
lamarr institute person Bauckhage Christian - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Christian Bauckhage

Director to the profile
Prof. Dr. Rafet Sifa

Prof. Dr. Rafet Sifa

Principal Investigator Hybrid ML to the profile