How does Knowledge Injection Help in Informed Machine Learning?

Informed machine learning describes the injection of prior knowledge into learning systems. It can help to improve generalization, especially when training data is scarce. However, the field is so application-driven that general analyses about the effect of knowledge injection are rare. This makes it difficult to transfer existing approaches to new applications, or to estimate potential improvements. Therefore, in this paper, we present a framework for quantifying the value of prior knowledge in informed machine learning. Our main contributions are threefold. Firstly, we propose a set of relevant metrics for quantifying the benefits of knowledge injection, comprising in-distribution accuracy, out-of-distribution robustness, and knowledge conformity. We also introduce a metric that combines performance improvement and data reduction. Secondly, we present a theoretical framework that represents prior knowledge in a function space and relates it to data representations and a trained model. This suggests that the distances between knowledge and data influence potential model improvements. Thirdly, we perform a systematic experimental study with controllable toy problems. All in all, this helps to find general answers to the question how knowledge injection helps in informed machine learning.

  • Published in:
    International Joint Conference on Neural Networks
  • Type:
    Inproceedings
  • Authors:
    von Rueden, Laura; Garcke, Jochen; Bauckhage, Christian
  • Year:
    2023

Citation information

von Rueden, Laura; Garcke, Jochen; Bauckhage, Christian: How does Knowledge Injection Help in Informed Machine Learning?, International Joint Conference on Neural Networks, 2023, https://ieeexplore.ieee.org/document/10191994, Rueden.etal.2023b,

Associated Lamarr Researchers

lamarr institute person von Rueden Laura Platzhalter - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Laura von Rueden

Autorin to the profile
Kopie von LAMARR Person 500x500 1 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Christian Bauckhage

Director to the profile