Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock
This article explains how Amazon Bedrock’s prompt caching can dramatically reduce token costs for multi‑turn AI agents by storing and reusing the KV state of repeated prompt prefixes such as tool definitions and system instructions. Prompt caching works either implicitly (Bedrock auto‑selects the reusable prefix) or explicitly via the prompt_cache_breakpoint API, allowing agents to pay only for new user inputs on each turn. Engineers can combine both methods to keep agents responsive and cost‑effective while preserving full conversational context.
⚡ Key Takeaways
- Prompt caching saves the model’s key‑value (KV) state for repeated input tokens, eliminating the need to recompute the prefix on every request.
- Explicit caching is enabled by calling prompt_cache_breakpoint to mark the reusable portion of the prompt.
- Implicit caching lets Bedrock automatically determine the reusable prefix as the conversation grows.
- Using both implicit and explicit caching together is recommended for agentic workloads.
- Without caching, agents must resend the entire history (tool list, instructions, prior turns) on each turn, incurring full token cost each time.
Reducing repeated prompt token processing cuts operational spend and latency for long‑running agents, making large‑scale, multi‑step AI applications viable in production.
✅ Practical Steps
- Insert prompt_cache_breakpoint in your agent code at the boundary between static content (e.g., tool definitions, system prompt) and dynamic content (new user query).
- Enable Bedrock’s implicit caching (default behavior) so the service can automatically reuse unchanged prefixes as the session progresses.
- Combine both approaches by marking known static sections with prompt_cache_breakpoint while allowing Bedrock to handle any additional reusable text.
Want the full story? Read the original article.
Read on AWS ML Blog ↗