← Back
Hugging Face Blog

nemotron-halloween

•12 min read•
#agents#llm#nvidia
Level:Advanced
For:AI Engineers
✦TL;DR

MonsterHouse is an interactive Halloween projection built on NVIDIA’s Nemotron Voice Agent stack, using Nemotron 3 ASR (nemotron‑speech‑streaming‑en‑0.6b.q8_0.gguf) for speech‑to‑text, Nemotron 3.5 Bolt E4B 4GB with a LoRA adapter for dialogue generation, Magpie TTS Zeroshot for voice synthesis, and Audio2Face‑3D for real‑time mouth animation in Unreal Engine 5. The entire pipeline—listening, thinking, speaking, and animating—runs on a single workstation equipped with an RTX PRO 6000 Blackwell Max‑Q and an RTX PRO 4000 Blackwell. Engineers can replicate this end‑to‑end agent by wiring the same NVIDIA models together via the Nemotron Voice Agent coordinator.

⚡ Key Takeaways

  • Nemotron 3 ASR model used: **nemotron‑speech‑streaming‑en‑0.6b.q8_0.gguf**.
  • Conversational core: **Nemotron 3.5 Bolt E4B 4GB** with a LoRA adapter for personality.
  • Voice synthesis: **NVIDIA Magpie TTS Zeroshot** driven by a short reference audio sample.
  • Real‑time facial animation: **NVIDIA Audio2Face‑3D** integrated in **Unreal Engine 5**.
  • Vision components: **NVIDIA Nemotron 3 Nano Omni** for costume description plus **YOLO** and **ByteTrack** for visitor counting and tracking.

🔧 Tools & Libraries

Nemotron Voice AgentNemotron ASRNemotron 3.5 Bolt E4B 4GBMagpie TTS ZeroshotAudio2Face‑3DYOLOByteTrackUnreal Engine 5RTX PRO 6000RTX PRO 4000
💡 Why It Matters

This demo shows that a full multimodal voice agent—speech recognition, LLM reasoning, TTS, and 3D animation—can be deployed on a single workstation, lowering the barrier for interactive AI installations in physical spaces.

✅ Practical Steps

  1. Install the Nemotron Voice Agent stack and load **nemotron‑speech‑streaming‑en‑0.6b.q8_0.gguf** for ASR.
  2. Deploy **Nemotron 3.5 Bolt E4B 4GB** with the provided LoRA adapter to generate dialogue responses.
  3. Connect **Magpie TTS Zeroshot** to the LLM output and pipe the audio into **Audio2Face‑3D** within Unreal Engine 5 for mouth animation.

Want the full story? Read the original article.

Read on Hugging Face Blog ↗

More like this

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

Mathematical proof, powered by AI. And no hallucinations!

Amazon Science•#agents

AI’s GPU focus meets a CPU reality check at Dell and AMD

SiliconANGLE AI•#agents

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING