Open language models have improved at a remarkable pace with a broad ecosystem of capable models to choose from. Yet for anyone deciding what to actually deploy, three gaps remain conspicuous. The first is openness: most releases still ship weights only, while omitting the data, recipes, and design decisions needed to reproduce or audit them. The second is language: general-purpose multilingual models are either English-centric or spread their capacity thinly across dozens of languages, leaving German underrepresented relative to its economic and scientific weight. The third gap is the one that most directly governs deployment cost, the inference efficiency that determines whether a capable model is actually affordable to run at scale.
With Soofi S 30B-A3B, we set out to close these three gaps through a joint, cross-institutional effort. Soofi S is a sovereign foundation model designed to be open source for German and English, developed by a consortium of German research institutions including the Lamarr Institute, Fraunhofer IAIS, DFKI, TU Darmstadt, hessian.AI, ellamind, L3S Research Center and the University of Würzburg while being coordinated by KI Bundesverband and funded by the Federal Ministry for Economic Affairs and Energy (BMWE). In this blog article we take a closer look at the underlying architecture, the training recipe, the sovereign European infrastructure, the evaluation results, and explain why we believe transparency is as much a part of the contribution as the model itself.
An architecture built for efficient inference
Active parameter count is only a proxy for what a model costs to run. In production, performance is actually bottlenecked by memory speed: every generated token must re-read the active model weights and the attention cache of every sequence in the batch. As contexts grow into the tens or hundreds of thousands of tokens, this key–value (KV) cache comes to dominate, and full-attention dense models slow down accordingly.
Soofi S is designed for exactly this regime. It adopts the openly published Nemotron 3 Nano reference architecture: a 52-layer network that interleaves 23 Mamba-2 sequence-mixing layers, 23 granular MoE layers with shared experts, and just 6 Grouped-Query Attention layers distributed sparsely throughout the network. The model totals roughly 31.6 billion parameters but activates only about 3.2 billion per token, the capacity of a 30B network at roughly the inference cost of a 3B one. Because only 6 of 52 layers maintain a KV cache while the Mamba-2 layers carry a fixed-size recurrent state, the per-sequence memory footprint stays small and near-constant as context grows.
Reusing an established, openly specified architecture rather than designing a bespoke one was a deliberate decision, providing a proven, highly efficient foundation for long-context inference in a short amount of time. Moreover, it makes the model deployable in mature open serving stacks from day one, it delivers the long-context efficiency we were targeting, and it turns the architecturally identical Nemotron 3 Nano into a clean baseline, so the effect of our German–English data recipe can be measured in isolation. Parallel to this, we are also actively researching other model architectures, including custom architectures of our own.
Trained on sovereign European infrastructure
Soofi S was trained on the Industrial AI Cloud operated by Deutsche Telekom in Munich, built together with NVIDIA and brought into operation in February 2026. We used up to 512 NVIDIA B200 GPUs — 64 DGX B200 nodes of eight GPUs each — over roughly 253,000 GPU-hours between 24 March and 13 May 2026, making Soofi S one of the first flagship workloads on this new sovereign HPC-scale infrastructure.
Training on German soil, under European operational and data-protection requirements rather than on extra-European hyperscale compute, is itself part of what we mean by a sovereign model. The Munich facility is powered entirely by renewable energy, cooled with water drawn from the nearby Eisbach canal, and integrated with a waste-heat-reuse concept that feeds the surrounding Tucherpark district.
A German-first data curriculum, fully documented
Soofi S was pretrained on approximately 27 trillion tokens under a three-phase Warmup–Stable–Decay curriculum: around 20 trillion tokens of diverse, quality-tiered pretraining, roughly 6.6 trillion tokens of high-quality annealing, and a final long-context phase that extends the usable context window to one million tokens. German was deliberately up-weighted to 7.2% of the stable phase and 15.3% of the annealing mixture, more than triple the multilingual share of the reference recipe. The goal was a genuine German–English champion rather than a thinly spread multilingual model.
Just as important as what went into the model is our commitment to documenting it. The Soofi S tech report is released with the complete pretraining corpus statistics for all three phases, including sources we evaluated and deliberately excluded due to poor performance, with per-source and per-language token accounting. We publish the full learning-rate schedule, the optimizer, every hyperparameter, and the per-phase token budgets.
How does Soofi S perform?
We evaluated Soofi S against 15 open base models using a common lm-evaluation-harness pipeline with identical prompts, few-shot configurations, and task settings, covering code, mathematics, knowledge, reasoning, science, reading comprehension, and German-language proficiency. We separate the comparison into two groups: fully open-source models that, like Soofi S, release data and code (Alia 40B, EuroLLM 22B, Apertus 70B, and Olmo 3 32B), and larger open-weight models that release weights only (Qwen3.5 35B-A3B, Ministral 3 14B, and Gemma 3 27B).
Among the fully open-source models, Soofi S is the strongest overall in our comparison (as of 11th of July 2026). It obtains the highest English aggregate (77.3, ahead of Olmo 3 32B by 1.5 points) and the highest German aggregate (85.3, ahead of Apertus 70B by 5.9 points), despite activating only 3B parameters per token against baselines up to 70B. The largest margins appear on German and technical benchmarks. Against the much larger open-weight models, Soofi S is not the top scorer on every aggregate, but it is competitive with dense 14–27B models — and it improves on its architectural twin, Nemotron 3 Nano, by 0.6 points in English and 4.6 points in German, isolating the effect of our data recipe. Across all 16 models in the study, Soofi S achieves the best English and German code aggregates.
Strong German capabilities
The German results are where Soofi S most clearly stands apart. Against every European sovereign baseline in our comparison — including models an order of magnitude larger in active parameters — Soofi S matches or outperforms them on every German benchmark in the suite, often by 10 to 30 points.
Concretely, Soofi S leads the open-source comparison on the German aggregate (+5.9), German-language proficiency (+7.6), German reasoning on ARC-Challenge-DE (+7.0), the knowledge benchmark INCLUDE-DE (+10.1), German grade-school math on GSM8K-Platinum-DE (+9.7), and German code generation on MBPP-DE (+13.4). To our knowledge, this makes Soofi S the strongest open German base model in its inference-cost class among the evaluated models.
Efficient long-context serving
The architectural bet pays off most visibly in serving efficiency. Because only 6 of 52 layers maintain a KV cache, Soofi S carries an incremental attention-cache footprint of only about 6 KB per token per sequence, 11 to 53 times smaller than the dense models in our comparison. At a 40K-token context and batch size 32, Soofi S sustains a measured aggregate decode rate of 4.82k tokens per second per GPU, roughly 9 times higher than a dense 14B model, while fitting all weights and 32 sequence states on a single GPU.
Crucially, this advantage widens with context length. Dense-model throughput falls off as KV-cache reads dominate decoding, whereas Soofi S remains nearly flat from 4K to 256K tokens. The same holds for time-to-first-token: at a 256K context, Soofi S completes the prompt-processing sweep faster than every baseline we measured. This is the structural advantage that compounds in exactly the regime that matters most in production (long context & high concurrency).
Contamination Disclosure
Like other open projects (e.g. OLMo), we train with rephrased practice questions from well-known benchmarks: mock exams, not the real test, so that the model can practice the formats it will later be tested on. These come from the training splits, which are normally published separately from the test splits used for evaluation. For GPQA and GPQA Diamond, no such separation exists: the complete evaluation set is published as a single split, labelled “Train” on Hugging Face. Exam questions therefore ended up in the practice set, and from there in our data. Community members reading the published QA-base dataset found this shortly after release, and we confirmed it against our construction pipeline.
A full audit of the dataset revealed four affected benchmarks in total: GPQA, TruthfulQA, BLiMP and the Inverse Scaling tasks. Only GPQA is in our evaluation suite, but we name all four so that scores obtained on them are not mistaken for capability. We removed GPQA Diamond and its German variant from every table and figure and recomputed the English and German aggregates without them for all models alike, so the comparisons stay fair and the rankings above are unchanged. A corrected dataset release accompanies this revision.
Open by design
We believe the way a model is released is part of its scientific contribution. Soofi S will be released under highly permissive, open-access terms: the model weights, selected intermediate training checkpoints, the full per-source data accounting, all hyperparameters, and the complete training and evaluation code. Our aim is to move the open ecosystem a step further, from open-weight toward genuinely open-source and to offer a reproducible template for other language communities seeking capable, efficient, and sovereign foundation models.
Looking ahead, we plan to extend Soofi S along three axes: open post-training toward instruction-following and reasoning variants, broader and deeper German evaluation suites, and continued scaling of the high-quality German data pipeline that this release identified as the principal bottleneck for further gains. We invite the research community to build on, scrutinize, and extend this work.
Beta Testing & Community Access
Soofi S is currently undergoing closed beta testing and optimization in collaboration with the IPCEI-CIS/8ra initiative, a European project involving over 100 partners from industry, SMEs and the research sector across 12 EU Member States. This phase allows us to gather early insights from practical business use cases and refine the model directly against enterprise needs.
While our team is focusing resources on this closed testing phase, public access is temporarily restricted. However, Soofi S will be released as soon as possible. We deeply value the open-source community and look forward to opening up broad testing, gathering your feedback, and continuing to improve the model together with researchers, developers, and partner organizations alike.
