AI Colloquium with Dr. Jonas Landsgesell, Pascall Knoll and Dr. Tizian Wenzel on “ScoringBench: A Benchmark for Evaluating Tabular Foundation Models with Proper Scoring Rules”

On Thursday, August 4, 2026, Dr Jonas Landsgesell and Pascal Knoll from the University of Stuttgart as well as Dr. Tizian Wenzel from the Ludwig Maximilian University of Munich and the Munich Center for Machine Learning will give a presentation on “ScoringBench: A Benchmark for Evaluating Tabular Foundation Models with Proper Scoring Rules” at TU Dortmund University.

About the AI Colloquium

The AI Colloquium, organized by the Lamarr Institute, the Research Center Trustworthy Data Science and Security (RC Trust) and the Center for Data Science & Simulation at TU Dortmund University (DoDas), provides a platform for leading researchers to present groundbreaking work in the field of Machine Learning and Artificial Intelligence. These 90-minute sessions, unlike other colloquia, focus on interactive dialog and international collaboration and include one-hour lectures and 30-minute Q&A sessions. The colloquium will be held mainly in English. The hybrid format of the colloquium ensures that all interested parties can participate either in person or online via Zoom.

About the Talk

While select tabular foundation models (TFMs) can produce full predictive distributions, current benchmarks predominantly rely on point-estimate metrics like RMSE, often overlooking valuable probabilistic information. To bridge this gap, the speakers present ScoringBench, an open benchmark evaluating tabular regression models via proper scoring rules alongside standard metrics. Their research reveals that model rankings shift significantly under probabilistic evaluation, distinct loss functions impose unique inductive biases, and fine-tuning with unseen scoring rules enhances targeted metric performance. Ultimately, the talk emphasizes that metric selection is a core modeling decision and advocates for prioritizing distributional evaluation aligned with downstream tasks.

About the Organizer

Matthias Feurer is an Assistant Professor (Juniorprofessor) for Automated  Machine Learning and Optimization at TU Dortmund University since 2025 and faculty member of the Lamarr Institute for Machine Learning and Artificial Intelligence. Before joining TU Dortmund University, he was a Thomas Bayes Fellow of the Munich Center for Machine Learning and an interim professor at LMU Munich. Previously, he was a PostDoc at the machine learning lab at the Albert-Ludwigs-Universität Freiburg, from which he also obtained his PhD.

Matthias aims to simplify the usage of Machine Learning by researching methods and developing tools that allow the usage of Machine Learning by domain scientists and also make it more efficient for expert users. His focus is on Automated Machine Learning (AutoML), which encompasses methods for hyperparameter optimization, meta-learning, and model selection. In addition, he embraces thorough benchmarking and researches tools and methods to support these. As part of these efforts, he is  also a member of the OpenML project, where researchers can share machine learning artifacts – mostly datasets – for collaborative research.

Jun. Prof. Dr. Matthias Feurer

Principal Investigator to the profile