{MetaQuRe}: Meta-Learning from Model Quality and Resource Consumption
Automated machine learning (AutoML) allows for selecting, parametrizing, and composing learning algorithms for a given data set. While resources play a pivotal role in neural architecture search, it is less pronounced by classical AutoML approaches. In fact, they generally focus on only maximizing predictive quality and disregard the importance of finding resource-efficient solutions. To push resource-awareness further, our work explicitly explores how measures such as running time or energy consumption can be better considered in AutoML. Firstly, we propose a novel method for algorithm selection that balances multiple performance aspects (including resource demand) as prioritized by the user with the help of compositional meta-learning. Secondly, to foster research on green meta-learning and AutoML, we release the MetaQuRe data set, which contains information on predictive (Qu)ality and (Re)source consumption of models evaluated across hundreds of data sets and four execution environments. We use this data to put our methodology into practice and conduct an in-depth analysis of how our approach and data set can help in making AutoML more resource-aware, which represents our third contribution. Lastly, we publish MetaQuRe alongside an extensive code base, allowing for reproducing all results, expanding our data with results from custom environments, and exploring MetaQuRe interactively. In short, our work demonstrates both the importance as well as benefits of rethinking AutoML and meta-learning in a resource-aware way, thus paving the path for making future ML solutions more sustainable.
- Published in:
European Conference on Machine Learning and Knowledge Discovery in Databases - Type:
Inproceedings - Authors:
Fischer, Raphael; Wever, Marcel; Buschjäger, Sebastian - Year:
2024
Citation information
Fischer, Raphael; Wever, Marcel; Buschjäger, Sebastian: {MetaQuRe}: Meta-Learning from Model Quality and Resource Consumption, European Conference on Machine Learning and Knowledge Discovery in Databases, 2024, Fischer.etal.2024b,
@Inproceedings{Fischer.etal.2024b,
author={Fischer, Raphael; Wever, Marcel; Buschjäger, Sebastian},
title={{MetaQuRe}: Meta-Learning from Model Quality and Resource Consumption},
booktitle={European Conference on Machine Learning and Knowledge Discovery in Databases},
year={2024},
abstract={Automated machine learning (AutoML) allows for selecting, parametrizing, and composing learning algorithms for a given data set. While resources play a pivotal role in neural architecture search, it is less pronounced by classical AutoML approaches. In fact, they generally focus on only maximizing predictive quality and disregard the importance of finding resource-efficient solutions. To push...}}