← Back
AWS ML Blog

Use prompt caching with open-weight models on Amazon Bedrock

•16 min read•
#bedrock#llm
Level:Intermediate
For:AI Engineers
✦TL;DR

Amazon Bedrock now lets you reuse long prompt prefixes with open‑weight models via implicit and explicit prompt‑caching modes, cutting input‑token costs and latency. Implicit caching automatically reuses any stable prefix, while explicit caching requires you to mark reusable sections with `prompt_cache_options` set to `explicit` and one‑or‑more `prompt_cache_breakpoint` markers. Cache reads are billed at a lower rate than normal input, and cache writes may be billed higher, so maximizing reuse of static content yields the biggest savings for production agents and coding assistants.

⚡ Key Takeaways

  • Implicit caching reuses eligible prefixes automatically on a best‑effort basis; explicit caching uses `prompt_cache_options` = `explicit` plus `prompt_cache_breakpoint` markers.
  • You can add several breakpoints (up to the model‑specific maximum) to create layered caches that change at different rates.
  • Cache reads are billed at a lower rate than standard input, while cache writes can be billed at a higher rate (see Amazon Bedrock pricing).
  • To get the most benefit, place static content (system prompt, reference docs, tool definitions) at the start of the prompt and dynamic content (user query) at the end.
  • The cache entry lifetime, minimum prefix length, and maximum breakpoints vary by model; consult the “Models at a glance” table for each model’s limits.
💡 Why It Matters

Engineers can halve prompt‑processing latency and reduce token‑based costs for chat‑style or coding agents by structuring prompts to exploit Bedrock’s built‑in caching, enabling cheaper, faster production AI services.

✅ Practical Steps

  1. In a Chat Completions API call, set `prompt_cache_options` to `explicit` and insert one or more `prompt_cache_breakpoint` markers at the end of each reusable prefix.
  2. For workloads that already have a stable prefix, simply order the prompt with static content first and rely on implicit caching; no extra parameters are needed.
  3. Monitor the `cache_usage` fields in each response to verify read/write billing and adjust prefix lengths or breakpoint placement accordingly.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

NVIDIA RTX Spark Compact Desktops Out Now

NVIDIA Blog•#compute

Helping AI text detectors keep up with changing conditions

MIT News AI•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING