← Back
AWS ML Blog

Optimizing cost and latency with Amazon Bedrock prompt caching

•54 min read•
#bedrock#deployment
Level:Intermediate
For:AI Engineers
✦TL;DR

Prompt caching in Amazon Bedrock can cut input‑token costs by up to 90 percent for repeated context, while also lowering time‑to‑first‑token. The feature works by inserting a **cachePoint** marker in a Converse API request; if the preceding tokens match a cached entry (minimum 1,024 tokens for Claude Sonnet 4.5/4.6, 4,096 tokens for Opus), the model skips reprocessing them. Engineers can apply six ready‑to‑use patterns—including message, system‑prompt, and tool‑definition caching—and control cache lifetimes (default 5 minutes, up to 1 hour) per tenant.

⚡ Key Takeaways

  • Cost reduction of up to 90 percent on cached input tokens (e.g., a 10,000‑token contract with 50 questions would otherwise bill 500,000 tokens).
  • CachePoint syntax in the Converse API defines the checkpoint; it is model‑agnostic across Anthropic Claude and Amazon Nova families.
  • Trade‑off: shorter prompts lower token count but may degrade context quality; caching preserves prompt quality while saving compute.
  • Integration: add a **cachePoint** marker and optional **TTL** parameter to your Converse API payload; LangChain users can enable caching via the provided LangChain wrapper.
  • Limitation: cache entries are scoped to a single AWS account and Region, and each checkpoint must meet the model‑specific token threshold.
💡 Why It Matters

By reusing processed context at the infrastructure layer, engineers can dramatically reduce Bedrock inference spend and latency without altering model prompts, enabling more cost‑effective, responsive AI services.

✅ Practical Steps

  1. Insert a **cachePoint** marker in your Converse API request payload and set the desired **TTL** (default 5 minutes, up to 1 hour) to enable prompt caching.
  2. For LangChain workflows, use the LangChain integration example to wrap Bedrock calls with caching enabled.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Hugging Face Blog•#llm

NVIDIA and Microsoft Expand Accelerated Computing From Azure to Windows PCs

NVIDIA Blog•#compute

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING