NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt
NVIDIA Nemotron 3.5 Transcribe is an open‑weight, promptable multilingual speech model that jointly performs transcription, speaker diarization, translation, and audio Q&A across 40 languages (31 for translation) and supports up to eight speakers per hour‑long recording. It leads the Hugging Face Open ASR leaderboard for English and achieves the lowest diarization error on David AI’s 21‑language leaderboard, with quantized variants runnable on RTX Spark, DGX Spark, Jetson AGX Thor, or Blackwell data‑center GPUs. The accompanying Granary v2 data pipeline and NeMo Speech fine‑tuning recipes enable domain adaptation, cutting word‑error‑rate by 45.8 % relative to prompt‑only baselines in a medical case study. Engineers can therefore deploy a single checkpoint for diverse transcription‑centric workloads while retaining the ability to fine‑tune for specialized vocabularies and accents.
⚡ Key Takeaways
- Leaderboard performance: first on Hugging Face Open ASR (English) and lowest diarization error on David AI’s 21‑language leaderboard.
- Architecture: single checkpoint delivering transcription, speaker attribution (13 languages), translation (31 languages), and audio Q&A (English) with word‑level timestamps.
- Trade‑off: quantized models enable edge deployment on RTX Spark/DGX Spark/Jetson AGX Thor but may require hardware‑specific tuning.
- Integration: model and Granary v2 data are available on Hugging Face and ModelScope; fine‑tuning uses NeMo Speech tools.
- Limitation: audio‑question answering is trained only in English; language support varies by task.
By consolidating multiple speech‑processing capabilities into one open model and providing a ready‑to‑use fine‑tuning pipeline, engineers can reduce system complexity, lower latency, and achieve higher accuracy for domain‑specific transcription use‑cases without maintaining separate models.
✅ Practical Steps
- Pull the Nemotron 3.5 Transcribe checkpoint from Hugging Face or ModelScope and run inference with the provided prompt guide to obtain transcription, diarization, or translation.
- Use the Granary v2 dataset from Hugging Face/ModelScope together with NeMo Speech fine‑tuning scripts to adapt the model to a target domain (e.g., medical terminology).
- Deploy the quantized version on edge hardware such as RTX Spark, DGX Spark, Jetson AGX Thor, or scale to Blackwell GPUs for production workloads.
Want the full story? Read the original article.
Read on Hugging Face Blog ↗