Progress Report: Towards European LLMs
We present two multilingual LLMs designed to embrace Europe’s linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60\% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the models‘ development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate competitive performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, MMLU, and TruthfulQA.
- Veröffentlicht in:
arXiv - Typ:
Article - Autoren:
- Jahr:
2024 - Source:
https://arxiv.org/abs/2410.03730v1
Informationen zur Zitierung
: Progress Report: Towards European LLMs, arXiv, 2024, https://arxiv.org/abs/2410.03730v1, Ali.etal.2024b,
@Article{Ali.etal.2024b,
author={Ali, Mehdi; Fromm, Michael; Thellmann, Klaudia; Ebert, Jan; Weber, Alexander Arno; Rutmann, Richard; Jain, Charvi; Lübbering, Max; Steinigen, Daniel; Leveling, Johannes; others},
title={Progress Report: Towards European LLMs},
journal={arXiv},
url={https://arxiv.org/abs/2410.03730v1},
year={2024},
abstract={We present two multilingual LLMs designed to embrace Europe’s linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60\% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the...}}
