← Back
AWS ML Blog

Speaker-labeled transcription with WhisperX on SageMaker AI

•13 min read•
#deployment
Level:Intermediate
For:AI Engineers
✦TL;DR

WhisperX extends OpenAI’s Whisper by adding per‑word timestamps, speaker diarization, and faster batched inference, packaged in an AWS Deep Learning Container (DLC) for GPU‑ready deployment on SageMaker. The container follows the SageMaker AI serving contract, exposing POST /invocations on port 8080 and supporting multipart/form‑data requests with optional language, diarize, and response_format fields. It can output json, verbose_json, srt, or vtt, enabling both real‑time and asynchronous endpoints for use cases ranging from contact‑center analytics to caption generation. Engineers can now obtain accurate, speaker‑labeled transcripts at scale without building custom images.

⚡ Key Takeaways

  • WhisperX provides per‑word timestamps and speaker labels in addition to Whisper’s phrase‑level timestamps.
  • The DLC includes Whisper, wav2vec2 forced‑alignment models, and diarization weights, requiring no Hugging Face token.
  • Real‑time vs. asynchronous endpoints let you trade latency for throughput depending on workload size.
  • The container serves on port 8080 with POST /invocations for inference and GET /ping for health checks.
  • Requests must be multipart/form‑data; supported optional fields are language, diarize, and response_format.
💡 Why It Matters

Engineers can integrate high‑precision, speaker‑aware transcription into production pipelines with a single SageMaker deployment, reducing custom infrastructure effort and improving compliance‑critical analytics.

✅ Practical Steps

  1. Deploy the AWS WhisperX Deep Learning Container to a SageMaker real‑time or asynchronous endpoint following the standard SageMaker AI serving contract.
  2. Send audio via a multipart/form‑data POST to /invocations on port 8080, including optional language, diarize, and response_format fields as needed.
  3. Choose the desired output format (json, verbose_json, srt, or vtt) via the response_format field to feed downstream analytics or video editing pipelines.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Hugging Face Blog•#llm

NVIDIA and Microsoft Expand Accelerated Computing From Azure to Windows PCs

NVIDIA Blog•#compute

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING