← Back
AWS ML Blog

Introducing usage-based pricing for Amazon SageMaker AI bidirectional streaming

•12 min read•
#deployment#inference#amazon
Level:Advanced
For:AI Engineers
✦TL;DR

Amazon SageMaker AI now supports usage‑based metering for bidirectional streaming inference, enabling model providers to bill AWS Marketplace customers based on actual streamed usage rather than static endpoint time. The feature adds a companion WebSocket metadata stream (opened at /​invocations‑bidirectional‑stream‑metadata) that carries JSON metering records, and model servers signal support with the X‑Amzn‑SageMaker‑Metadata‑Stream‑Supported: true header. Deepgram is the launch partner, bringing real‑time Speech‑to‑Text and Text‑to‑Speech billing to the marketplace. Engineers can integrate this metering without changing existing data streams, aligning costs with real‑time AI consumption.

⚡ Key Takeaways

  • Deepgram is the launch partner for usage‑based billing of real‑time STT and TTS on SageMaker AI.
  • Model servers must return the header X‑Amzn‑SageMaker‑Metadata‑Stream‑Supported: true to opt‑in to the metadata stream.
  • A second WebSocket connection to /​invocations‑bidirectional‑stream‑metadata carries JSON metering frames alongside the primary inference stream.
  • SageMaker AI endpoint infrastructure charges continue to apply separately from the usage‑based software charge.
  • Billing aligns with actual streamed usage, avoiding charges during idle periods of long‑lived streams.

🔧 Tools & Libraries

DeepgramAWS MarketplaceAWS IAMAmazon SageMaker AI
💡 Why It Matters

Aligning software charges with real‑time streaming consumption lets enterprises pay only for the AI work performed, reducing waste and simplifying cost forecasting for production‑grade, low‑latency AI services.

✅ Practical Steps

  1. Ensure your model server returns X‑Amzn‑SageMaker‑Metadata‑Stream‑Supported: true in the WebSocket upgrade response to enable the companion metadata stream.
  2. Implement a WebSocket client that connects to /​invocations‑bidirectional‑stream‑metadata and sends metering data as JSON text frames, using the same X‑Amzn‑SageMaker‑Request‑Id as the inference stream for correlation.
  3. Deploy the model on SageMaker AI as usual; the platform will automatically open the metadata WebSocket alongside the inference stream for usage‑based metering.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING