Soofi Consortium Releases Soofi S 30B-A3B: Hybrid Mamba-Transformer MoE Base Model in German and English

A German research organization has published a report on training Soofi S 30B-A3B. It is an open basic model in German and English. The training was completed in Deutsche Telekom’s Industrial AI Cloud in Munich. Preview weights are on Hugging Face. It is worth noting that among the other fully tested basic models, the Soofi S records the highest scores for English and German.

What is Soofi S 30B-A3B?

The Soofi S is the base model of the Mixture-of-Experts (MoE) hybrid Mamba Transformer. It includes ~31.6B parameters and unlocks ~3.2B per token. Like the base model, it has no instruction tuning, alignment, or safety tuning. The KI Bundesverband directs the consortium, which is funded by the German Ministry of Economic Affairs and Energy. Partners include Fraunhofer IAIS, DFKI, TU Darmstadt, ellamind, and Meantix Momentum.

How architecture works?

The claim of efficiency starts with the layer stack. The network hosts 52 layers. Those layers include 23 Mamba-2 sequences, 23 granular MoE layers, and 6 Grouped-Query Attention (GQA) layers. Only those 6 GQA layers store the KV cache. Each MoE layer manages 128 routing experts, activating 6 per token, and adding 2 shared experts. Other details: model size 2688, squared ReLU, RMSNorm, and no position embeddings.

Soofi S adopts the Nemotron 3 Nano reference design without modification. The research team gives three reasons for that choice. That’s deployment in stacks like vLLM, efficiency, and scientific management. Because the core is fixed, the Nemotron 3 Nano becomes a base like structure. The data recipe is the only part that moves.