Resource-aware Machine Learning
Resource-aware Machine Learning aims to adapt Machine Learning and the underlying hardware technologies to save energy, memory, and computational resources.
Researchers of the Lamarr Institute are dedicated to developing sustainable and environmental-friendly Machine Learning solutions that save energy and computational resources. For this purpose, we study the connection between hardware and Machine Learning. It is our goal to make Machine Learning available even on devices with restricted computing power and limited energy and memory resources.
ML Approaches for High Performance and Low Resource Consumption
To advance resource-aware ML, we are investigating resource-friendly variations of high-performance ML approaches. Topics of our current and future research include resource-aware transformer models for Natural Language Processing (NLP), the choice of optimal hyperparameter to reduce training times and improve performance, research on model quantization as well as efficient search algorithms.
Contact persons
Adjusting Hardware for More Efficiency
At the Lamarr Institute we also focus on adjustments to hardware which has the potential to further reduce energy consumption and improve the computation efficiency. For instance, in-memory computing, that mitigates costly data transfers between memory and the processing unit, provides potentials for efficient execution as well as energy reduction on modern memory technologies. Moreover, we see a need to develop more flexible hardware accelerators that improve the execution and training of models on resource-constrained devices.
Anytime Models
To further improve responsiveness and adaptability in constrained environments, we investigate anytime Machine Learning models. These models are designed to deliver predictions at any point in time during inference even when interrupted before completion, making them ideal for time-critical applications such as real-time monitoring or edge-based decision-making.
Our research includes the integration of early-exit strategies into deep learning architectures, learning with abstention and rejection, as well as the formalization of metrics that capture the trade-off between computational cost and prediction quality. This enables graceful degradation in performance under tight resource constraints while maintaining reliability.
Quantum Machine Learning
Finally, part of our research is dedicated to computational methods at the intersection of Machine Learning and quantum computing, aiming both to exploit quantum hardware for learning tasks and to apply Machine Learning techniques to characterize and enhance quantum devices.
Naive formulations of quantum algorithms cannot be implemented on current and future generations of quantum computers due to resource constraints such as the limited number of qubits, limited qubit connectivity, and limited circuit depth. Therefore, we are researching resource-efficient alternatives, particularly decomposition techniques, to enable the execution of quantum algorithms for practical applications on today’s quantum processors.
Furthermore, we investigate the relationship between classical Machine Learning methods and their corresponding quantum algorithms, including support vector machines, Markov random fields, restricted Boltzmann machines, and neural networks.
Publications
News from the Area Resource-aware ML
- All
- News
- Events
- Projects
