NVIDIA's Nemotron-3 Brings Streaming Speaker Diarization

NVIDIA has released Nemotron-3-Diarization, an open model focused on a specific but persistently tricky part of the speech pipeline: figuring out who is speaking and when. The model is built on NVIDIA's Sortformer architecture and is designed to run in a streaming fashion, meaning it can attribute speech to speakers as audio arrives rather than waiting for a full recording.

Speaker diarization is the connective tissue between raw transcription and usable output. Automatic speech recognition tells you what was said; diarization tells you who said it. For meeting notes, call-center analytics, captioning, and any transcript involving more than one voice, that distinction is what turns a wall of text into a readable, attributable conversation.

Why it matters

The streaming angle is the notable part here. Many diarization systems operate offline, clustering speaker segments after the fact. A model that can label speakers on the fly opens the door to live use cases:

• Real-time captioning that separates participants
• Live meeting and call transcription with speaker tags
• Lower-latency pipelines that pair with streaming ASR

As the first entry under the Nemotron-3-Diarization name, this release gives developers a purpose-built diarization component they can slot alongside NVIDIA's broader speech stack. The model is distributed on Hugging Face under a custom license, so teams weighing it for production should review the terms before building on it.