Hybrid Machine Learning
Hybrid Machine Learning aims at an integrated treatment of empirical data, world knowledge, and application contexts. The overall goal is to develop ML solutions that are efficient, robust, explainable, and trustworthy. Hybrid ML therefore constitutes an integral part of our research and development activities at the Lamarr institute.
Structured Representations for Interpretable Machine Learning
Hybrid Machine Learning models and algorithms learn structured representations from empirical data. These representations are transformed views of the data which make it more interpretable or more usable for downstream modelling and prediction. The key word here is “structured”. On the one hand, Hybrid Machine Learning incorporates structure into data representations by leveraging inductive biases which come from mathematics, the natural sciences, the social sciences, or economics. On the other hand, Hybrid Machine Learning uses such structures to better solve problems in these fields.
Indeed, our research on Hybrid Machine Learning considers three broad directions:
- Mathematical Concepts and Representation Learning
The study and application of concepts from discrete mathematics, geometry, group theory, and probability theory to build and analyze representation learning algorithms. Here, we focus on algorithms that process and leverage the underlying structure of relational-, of image- and of video data. - Knowledge for Transcending Black-Box Deep Learning
The development of neural models of empirical data which transcend black-box deep learning as they are consistent with mathematical knowledge or theories from either the natural or the social sciences. Examples include our models for reasoning in natural language which infer representations that rely on classical theories of semantics, or our neural stochastic process models which constrain representations of time series data to satisfy physics-inspired difference- or differential equations. - Automatic Scientific Discovery
The development of methods for automatic scientific discovery through inference from neural representations of (carefully curated) algebraic equations. These representations are designed to be manipulated by neural reasoners (artificial scientists) which are trained via reinforcement learning to automatically construct novel scientific theories from data.
Our research therefore fits into the triangular AI paradigm of the Lamarr institute as it combines data (empirical observations), structured knowledge (scientific theories or mathematical models), and context (reward signals of reinforcement algorithms) to better understand and predict physical and social processes in practical applications.
