Speaker-labeled transcription with WhisperX on SageMaker AI
WhisperX extends OpenAI’s Whisper by adding per‑word timestamps, speaker diarization, and faster batched inference, packaged in an AWS Deep Learning Container (DLC) for GPU‑ready deployment on SageMaker. The container follows the SageMaker AI serving contract, exposing POST /invocations on port 8080 and supporting multipart/form‑data requests with optional language, diarize, and response_format fields. It can output json, verbose_json, srt, or vtt, enabling both real‑time and asynchronous endpoints for use cases ranging from contact‑center analytics to caption generation. Engineers can now obtain accurate, speaker‑labeled transcripts at scale without building custom images.
⚡ Key Takeaways
- WhisperX provides per‑word timestamps and speaker labels in addition to Whisper’s phrase‑level timestamps.
- The DLC includes Whisper, wav2vec2 forced‑alignment models, and diarization weights, requiring no Hugging Face token.
- Real‑time vs. asynchronous endpoints let you trade latency for throughput depending on workload size.
- The container serves on port 8080 with POST /invocations for inference and GET /ping for health checks.
- Requests must be multipart/form‑data; supported optional fields are language, diarize, and response_format.
Engineers can integrate high‑precision, speaker‑aware transcription into production pipelines with a single SageMaker deployment, reducing custom infrastructure effort and improving compliance‑critical analytics.
✅ Practical Steps
- Deploy the AWS WhisperX Deep Learning Container to a SageMaker real‑time or asynchronous endpoint following the standard SageMaker AI serving contract.
- Send audio via a multipart/form‑data POST to /invocations on port 8080, including optional language, diarize, and response_format fields as needed.
- Choose the desired output format (json, verbose_json, srt, or vtt) via the response_format field to feed downstream analytics or video editing pipelines.
Want the full story? Read the original article.
Read on AWS ML Blog ↗