← Back
AWS ML Blog

Four Amazon Bedrock inference choices that affect cost

•11 min read•
#bedrock#inference
Level:Advanced
For:AI Engineers
✦TL;DR

Amazon Bedrock inference cost can be tuned via request‑level settings such as prompt cache TTL, service tier, reasoning effort, and maxTokens without changing the model or prompt. For supported Anthropic models, explicit cache TTLs of 5 minutes or 1 hour are available, while supported GPT‑5.6 models have a 30‑minute minimum TTL, and the pricing differs: a 5‑minute write costs 1.25 × the input rate, a 1‑hour write costs 2 ×, and a cache read costs 0.1 ×. Engineers can verify the impact of these knobs by inspecting usage metadata, logs, and billing data from the Bedrock response. Applying the right TTL and cache strategy can substantially lower token‑based charges for repeated prompt prefixes.

⚡ Key Takeaways

  • 5‑minute write cost = 1.25 × input rate; 1‑hour write cost = 2 × input rate; cache read cost = 0.1 × input rate (Anthropic Claude Sonnet 5 & Opus 5).
  • Explicit prompt caching uses the JSON field `{ "cachePoint": { "type": "default", "ttl": "1h" } }` (or equivalent `cache_control` in InvokeModel).
  • Longer TTLs are not automatically cheaper; 5‑minute cache pays back on first reuse within 5 min, 1‑hour cache becomes cheaper from the second request within the hour.
  • Prompt caching does not work with batch inference; use on‑demand inference and keep stable content before dynamic content.
  • Verify cost impact via response usage metadata, CloudWatch logs, and Bedrock billing reports.

✅ Practical Steps

  1. Run the “Mixed TTL Caching” and “Tenant Isolation with Prompt Caching” notebooks from the Amazon Bedrock Samples repository.
  2. Modify the sample’s model ID and inference parameters to match your target model, then set `"cachePoint": {"type":"default","ttl":"1h"}` (or `"5m"` as needed) in your request payload.
  3. After deployment, inspect the `usage` fields in Bedrock responses and CloudWatch logs to confirm cache read/write token counts and compare against billing data.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Helping AI text detectors keep up with changing conditions

MIT News AI•#llm

Hugging Face and OpenRouter join forces to make open-weight models easy to run

Hugging Face Blog•#inference

Activation and Floating-Point Quantization

Machine Learning Mastery•#inference

How Airbnb uses cost per task to operate coding agents on Amazon Bedrock

AWS ML Blog•#amazon

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING