← Back
AWS ML Blog

Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot

•7 min read•
#deployment#amazon
Level:Advanced
For:AI Engineers
✦TL;DR

AssemblyAI built a cost‑optimized speech‑to‑text inference platform on AWS by mixing Amazon EC2 Spot and On‑Demand GPU instances within Amazon ECS clusters and using an Amazon SQS‑based retry loop to survive Spot interruptions. The fleet runs thousands of GPUs across multiple instance families (g4dn.xlarge, g4dn.2xlarge, g5.xlarge, g5.4xlarge, g6.2xlarge) in both US West (us‑west‑2) and Europe (eu‑west‑2) to meet latency SLAs while keeping data residency. This architecture delivers significant cost savings without compromising availability, enabling engineers to run large‑scale GPU inference workloads affordably.

⚡ Key Takeaways

  • Instance families used: g4dn.xlarge, g4dn.2xlarge, g5.xlarge, g5.4xlarge, g6.2xlarge.
  • ECS capacity providers blend Spot and On‑Demand capacity for resilient scheduling.
  • Spot interruptions are handled via an EventBridge‑triggered SQS retry mechanism.
  • Deployment spans US West (us‑west‑2) and Europe (eu‑west‑2) to satisfy data‑residency requirements.

🔧 Tools & Libraries

Amazon ECSAmazon EC2 SpotAmazon SQSAmazon EventBridgeAmazon S3
💡 Why It Matters

Engineers can achieve dramatic GPU cost reductions for latency‑sensitive inference by combining Spot diversification, ECS capacity providers, and queue‑driven retries, while still honoring regional compliance constraints.

✅ Practical Steps

  1. Configure an Amazon ECS cluster with capacity providers that specify both Spot and On‑Demand GPU instance types (g4dn.xlarge, g4dn.2xlarge, g5.xlarge, g5.4xlarge, g6.2xlarge).
  2. Set up an Amazon SQS queue and an EventBridge rule that captures Spot interruption notices and re‑queues in‑flight jobs for retry.
  3. Deploy the ECS tasks in the required AWS Regions (us‑west‑2 and eu‑west‑2) to enforce data residency.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING