# NVIDIA's Nemotron TwoTower is a MoE experiment

> An experimental 30B mixture-of-experts base model blends diffusion and Mamba ideas under a two-tower design.

Published by The Open Weights on Apr 11, 2026. Canonical: https://theopenweights.com/news/nemotron-twotower-30b-a3b-1pls

## Key facts

- Company: NVIDIA
- Model: Nemotron TwoTower 30B-A3B Base
- Version: 1.0
- Category: Text / LLM
- Modalities: Text / LLM
- License: Other (Open weights, commercial use allowed)
- Parameters: 30B (mixture of experts)
- Active parameters: 3B
- Precision: BF16
- Significance: notable
- Published: 2026-04-11
- Last verified: 2026-07-06
- Hugging Face: https://huggingface.co/nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16
- Canonical URL: https://theopenweights.com/news/nemotron-twotower-30b-a3b-1pls

NVIDIA has quietly published Nemotron TwoTower 30B-A3B Base, an experimental language model that departs from the standard transformer recipe. Released on [Hugging Face](https://huggingface.co/nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16) in BF16, it is a base model — meaning it ships without instruction tuning or alignment, intended as a foundation for further research and fine-tuning.

The architecture is where things get interesting. The model is described as a two-tower design that draws on diffusion and Mamba-style approaches, wrapped in a mixture-of-experts (MoE) layout. With 30 billion total parameters but only about 3 billion active per token, it aims for the efficiency profile that has made sparse MoE models attractive: large capacity at a fraction of the inference cost.

## Why it matters

Most frontier open-weight models still lean on dense or MoE transformers. NVIDIA's willingness to publish a hybrid that mixes diffusion and state-space (Mamba) components signals continued exploration of alternatives to attention-heavy designs, particularly for long-context and throughput-sensitive workloads.

- 30B total parameters, roughly 3B active per token
- Mixture-of-experts with a two-tower structure
- Base checkpoint in BF16, no instruction tuning
- Released under a custom NVIDIA license

As an experimental, notable-tier release, this is aimed squarely at researchers rather than production users. The unusual architecture makes it a useful testbed for anyone probing how diffusion and Mamba ideas hold up at scale — but expect rough edges, and check the license terms before building on it.

## Get the model

- [Hugging Face](https://huggingface.co/nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16)

## Sources

- [nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16](https://huggingface.co/nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16) — Hugging Face, Apr 11, 2026

---
Source: The Open Weights (https://theopenweights.com/). Aggregated and written by Claude, curated by humans. Cite as: "NVIDIA's Nemotron TwoTower is a MoE experiment", The Open Weights, Apr 11, 2026, https://theopenweights.com/news/nemotron-twotower-30b-a3b-1pls