Advancing Risk and Quality Assurance: A {RAG} Chatbot for Improved Regulatory Compliance

Risk and Quality (R&Q) assurance in highly regulated industries requires constant navigation of complex regulatory frameworks, with employees handling numerous daily queries demanding accurate policy interpretation. Traditional methods relying on specialized experts create operational bottlenecks and limit scalability. We present a novel Retrieval Augmented Generation ({RAG}) system leveraging Large Language Models ({LLMs}), hybrid search and relevance boosting to enhance R&Q query processing. Evaluated on 124 expert-annotated real-world queries, our actively deployed system demonstrates substantial improvements over traditional {RAG} approaches. Additionally, we perform an extensive hyperparameter analysis to compare and evaluate multiple configuration setups, delivering valuable insights to practitioners.

  • Published in:
    2024 {IEEE} International Conference on Big Data ({BigData})
  • Type:
    Inproceedings
  • Authors:
    Hillebrand, Lars; Berger, Armin; Uedelhoven, Daniel; Berghaus, David; Warning, Ulrich; Dilmaghani, Tim; Kliem, Bernd; Schmid, Thomas; Loitz, Rüdiger; Sifa, Rafet
  • Year:
    2024
  • Source:
    https://ieeexplore.ieee.org/abstract/document/10825431

Citation information

Hillebrand, Lars; Berger, Armin; Uedelhoven, Daniel; Berghaus, David; Warning, Ulrich; Dilmaghani, Tim; Kliem, Bernd; Schmid, Thomas; Loitz, Rüdiger; Sifa, Rafet: Advancing Risk and Quality Assurance: A {RAG} Chatbot for Improved Regulatory Compliance, 2024 {IEEE} International Conference on Big Data ({BigData}), 2024, 8668--8670, December, https://ieeexplore.ieee.org/abstract/document/10825431, Hillebrand.etal.2024b,

Associated Lamarr Researchers

Prof. Dr. Rafet Sifa

Prof. Dr. Rafet Sifa

Principal Investigator Hybrid ML to the profile