Rumble-2-Flash: an open 0.5B speech model for 40+ languages
Rumble‑2‑Flash is an open‑source 0.5 B‑parameter speech‑recognition model that supports automatic language detection for 40+ languages and matches the accuracy of much larger models (e.g., NVIDIA’s 30 B Nemotron 3.5 Transcribe) while delivering extreme speed—over 7,000 RTFx, processing an hour of audio in roughly half a second. It runs natively in 🤗 Transformers and vLLM without custom code, and uses on‑policy distillation with the GOLD token‑merging technique to train a student model that does not share a tokenizer with its teacher. Engineers can integrate the model via the Hugging Face Hub or the provider’s API for low‑latency, multilingual ASR workloads.
⚡ Key Takeaways
- 0.5 B‑parameter model achieves 6.96 % average error rate on the Open ASR Leaderboard multilingual track (better than Meta’s 15× larger omniASR‑LLM 7B and Whisper large‑v3).
- Architecture: built on Qwen3‑ASR with on‑policy distillation; GOLD merges differing tokenizers to enable teacher‑student training without a shared vocab.
- Performance trade‑off: delivers >7,000 RTFx (≈0.5 s per hour of audio) while maintaining parity with a 30 B model on a 302‑hour call‑centre benchmark (7.86 % WER vs. 7.78 %).
- Integration: supported out‑of‑the‑box in 🤗 Transformers and vLLM; accepts local paths, URLs, NumPy arrays, or lists thereof as audio inputs.
- Limitation: requires the Qwen3‑ASR vocabulary; token merging (GOLD) is needed only because the student and teacher tokenizers differ.
🔧 Tools & Libraries
Engineers can now deploy a tiny, multilingual ASR model that offers near‑state‑of‑the‑art accuracy and ultra‑low latency without bespoke runtime changes, dramatically reducing compute costs for real‑time speech applications.
✅ Practical Steps
- Pull the model from the Hugging Face Hub (Reson8/Rumble-2-Flash) and load it with `from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor` (or the equivalent vLLM API).
- Provide audio input as a local file path, URL, NumPy array, or list of these to the model’s `forward` method for inference.
- Use the built‑in automatic language detection and output control flags (e.g., digits vs. words, filler handling) to customize transcription behavior.
Want the full story? Read the original article.
Read on Hugging Face Blog ↗