Claude Prompt Caching: Why Agent Loops Miss the 20-Block Lookback

The Failure Mode

Your agent begins a run with healthy prompt-cache reads. cache_read_input_tokens starts around 40,000 and increases as the conversation grows. After several tool calls, the metric suddenly falls to zero while cache_creation_input_tokens jumps to approximately the full conversation length.

The obvious explanations are usually wrong. You did not necessarily change the system prompt, reorder a tool, add a timestamp, or switch models. The serialized prefix can remain byte-identical and still miss the cache.

The likely cause is the cache breakpoint's lookback limit.

A Claude Messages API cache breakpoint is both a cache write point and a lookup point. It tells the service to cache the prefix through that content block. On a later request, Claude searches backward from the breakpoint for a prior cache entry. That search reaches back through at most 20 content blocks.

This is a block limit, not a token limit. One block may contain a small text fragment, a tool_use object, a tool_result, or another structured content item. A single agentic turn can therefore consume the lookback window surprisingly quickly.

For example, an agent may emit:

• An assistant message containing several tool_use blocks.
• A user message containing the corresponding tool_result blocks.
• Additional reasoning or status blocks produced by the harness.
• Structured tool payloads split into multiple content blocks.

With 11 parallel tool calls, the turn can add 22 or more tool-related blocks before the next request is constructed. If the next breakpoint is placed only on the newest block, the previous cache write may be more than 20 blocks away. The lookup does not find it, even though the earlier prefix has not changed.

The result is a silent miss followed by a new cache write for the entire prefix. Cache creation input is billed at a premium relative to ordinary input. The supplied scenario describes this as a 1.25x write price, so repeated misses can cost more than expected and can make caching look ineffective or even counterproductive.

What Is Actually Stable?

Caching depends on exact prefix identity. A cache hit requires the relevant serialized content to match. A one-character change can produce a different prefix hash.

Good cache candidates include:

• Long system instructions that remain unchanged for a session.
• Deterministically serialized tool definitions.
• Stable policy, schema, and domain-reference content.
• Earlier conversation history that is included identically on each request.

Poor cache candidates include:

• The current user question.
• Newly retrieved RAG passages that change on every turn.
• Current timestamps, random identifiers, or request IDs.
• Tool output that is reformatted nondeterministically.
• A cache breakpoint placed after content that changes every request.

The breakpoint does not make unstable content stable. It only tells Claude where to write and where to search for a previous matching prefix.

A Practical Breakpoint Strategy

The simplest robust design has two parts: a stable prefix anchor and rolling breakpoints in the message history.

Reserve one of the four available breakpoints for the stable tools-plus-system prefix. Then rotate the other three through the conversation or message list. In a long agent loop, place rolling breakpoints roughly every 15 content blocks. This leaves margin below the 20-block search limit for the next turn's new tool activity.

The exact placement depends on how your harness serializes messages. Count content blocks after serialization, not application-level messages. One application message may contain many blocks.

A useful layout is:

[stable tool definitions and system instructions]
[static cache breakpoint]

[conversation history]
[rolling breakpoint A]
[tool-use and tool-result blocks]
[rolling breakpoint B]
[tool-use and tool-result blocks]
[rolling breakpoint C]
[current turn content]

The rolling markers should advance as the conversation grows. They are not three permanent markers attached to the same old messages. Their job is to ensure that at least one recent cache write remains within the next breakpoint's 20-block backward search.

Template 1: Stable Prefix

The following is a wire-format pattern for a Messages API request. The precise SDK representation may differ, but the important properties are stable ordering, deterministic tool serialization, and a breakpoint after the stable prefix.

{
  "model": "YOUR_MODEL",
  "system": [
    {
      "type": "text",
      "text": "Stable agent policy, operating rules, output schema, and domain instructions.",
      "cache_control": {
        "type": "ephemeral"
      }
    }
  ],
  "tools": [
    {
      "name": "search_records",
      "description": "Search the approved record store.",
      "input_schema": {
        "type": "object",
        "properties": {
          "query": { "type": "string" }
        },
        "required": ["query"]
      }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "Current request goes here. Keep changing request data after the stable prefix."
    }
  ]
}

This template is appropriate when the system instructions and tools are stable across requests. Do not add a timestamp, per-request trace ID, or changing retrieval result inside the cached prefix. Keep tool definitions serialized in the same order and with the same field representation on every request.

Whether your SDK places the breakpoint on a system content block, at the end of the tools collection, or through a helper abstraction depends on the API surface you are using. Verify the generated request rather than assuming that a high-level framework placed the marker where you intended.

Template 2: Rolling Message Breakpoints

Use rolling markers when the transcript grows through many tool calls. This example is intentionally schematic: your harness must attach cache_control using the content-block structure supported by its Claude client.

{
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Earlier user request and compacted context.",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "tool_use",
          "id": "tool_001",
          "name": "search_records",
          "input": { "query": "open incidents" }
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "tool_result",
          "tool_use_id": "tool_001",
          "content": "Compact, deterministic result summary."
        },
        {
          "type": "text",
          "text": "Continue the workflow.",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    }
  ]
}

The marker should be moved forward as the agent produces more blocks. The example does not mean that every block needs a marker. Four breakpoints per request is the stated maximum for this strategy, so use them deliberately.

A production harness should calculate the content-block position of each candidate marker and choose recent positions that are approximately 15 blocks apart. It should also avoid placing a marker after volatile content unless that volatile content is intentionally part of the cacheable prefix.

Inputs and Outputs

Inputs to Track

At request construction time, record:

• The model identifier.
• The ordered tool names and a stable serialization hash of tool definitions.
• The system prompt hash.
• The number of content blocks in each message.
• The absolute positions of all cache breakpoints.
• Which content blocks changed since the previous request.
• The request interval, because a five-minute cache entry can expire during a quiet period.
• Whether the request uses the same model as the preceding request.

The most important input is the serialized block layout. Counting only user and assistant turns will hide the actual problem.

Outputs to Inspect

For every response, log the cache-related usage fields when available:

{
  "usage": {
    "input_tokens": 5200,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 43800
  }
}

A healthy repeated-prefix request generally shows a meaningful cache_read_input_tokens value and little or no new cache creation for the already-written prefix. A suspicious request shows cache_read_input_tokens at zero while cache_creation_input_tokens is close to the complete serialized input size.

Do not judge caching from one request. The first request must create an entry, and a concurrent request may arrive before the first response begins streaming. In that case, identical parallel requests can all pay the creation cost because the entry is not yet readable. Measure a sequence of requests with the same static prefix and record the block count at the first miss.

A useful diagnostic table is:

The third row illustrates the failure: the newest breakpoint is more than 20 blocks beyond the previous cached block.

Implementation Tips and Trade-Offs

Compact Tool Results Before Caching Them

Tool output is often the fastest way to exhaust the lookback window and context budget. Store large raw results outside the transcript when possible, then insert a deterministic summary, selected fields, or a reference that the agent can resolve later.

This changes behavior, so validate that the agent still has enough detail to make correct decisions. Aggressive trimming saves cache and context costs but can remove evidence needed for later steps.

Keep Serialization Deterministic

Stable application data can still miss if your serializer changes key ordering, whitespace, numeric formatting, or field presence. Define one canonical serialization path for tools and structured tool results. Do not let unrelated middleware inject changing fields into the prefix.

Separate Static and Volatile Instructions

Put durable policy and tool definitions before the rolling conversation. Put the current question, current time, request metadata, and fresh retrieval data after the stable prefix. A breakpoint after changing data writes a new prefix every time, which provides little reuse.

Account for TTL

A cache entry can expire during a quiet period. The supplied guidance identifies a five-minute entry lifetime and recommends a one-hour TTL for bursty or batch workloads where that option is available. Treat TTL as a workload decision: short-lived interactive bursts may be fine with the shorter lifetime, while scheduled batches may need a longer lifetime or a deliberate pre-warm request.

Avoid Unnecessary Model Switching

Caches are model-scoped. Switching models in the main loop does not preserve the same cache entry. Keep the primary loop on one model when cache reuse matters, and use separate subagents for cheaper parallel work when that architecture fits the task.

Do Not Assume Automatic Caching Solves Layout

Automatic caching can place a breakpoint conveniently, but it cannot understand your agent's future block growth. Inspect the emitted request and explicitly manage rolling breakpoints when a turn can produce many tool calls.

Test the Harness With a Block-Count Fixture

Create a deterministic test transcript containing 5, 10, 15, 20, and 21 new blocks after a cache write. Assert that the harness inserts or rotates a breakpoint before the next lookup must cross more than 20 blocks. Then test parallel tool results, because that is where application-level turn counts most often diverge from content-block counts.

The test should also cover changed tool ordering, changed system text, a model switch, a timestamp in the prefix, and a cache-expiry gap. These are separate causes of misses and should not be misdiagnosed as lookback failures.

A Debugging Checklist

1. Capture the actual serialized request for a cache hit and the first following miss.
2. Compare model, tools, system content, and message content byte-for-byte where practical.
3. Count content blocks from the previous cache write to the new breakpoint.
4. Check whether parallel tool_use and tool_result blocks added more than 20 positions.
5. Confirm that the stable breakpoint is before volatile request data.
6. Add rolling breakpoints approximately every 15 blocks.
7. Keep within the four-breakpoint-per-request budget.
8. Check TTL gaps and concurrent-request timing.
9. Check for model switches and tool-definition changes.
10. Compare cache_read_input_tokens and cache_creation_input_tokens over multiple turns.

The key operational lesson is that a cache miss is a layout event as often as it is a content event. Prompt review alone will not reveal it. You need block-level request telemetry.

FAQ

Does a byte-identical prompt guarantee a cache hit?

No. Exact prefix identity is necessary, but the breakpoint must also be able to find the previous cache entry within its backward search window. If the prior write is more than 20 content blocks away, an identical prefix can still miss from that breakpoint.

Is the limit 20 tokens or 20 messages?

Neither. It is a lookback through at most 20 content blocks. A message can contain multiple blocks, and tool calls commonly create several structured blocks. Count the blocks in the serialized request.

Should I put a breakpoint on every tool result?

No. The strategy described here uses up to four breakpoints per request. Reserve one for the stable tools-and-system prefix and rotate the remaining three through the message list. Place rolling markers at roughly 15-block intervals rather than marking every block.

Why did cache creation equal the full conversation length?

When the lookup misses, Claude may write a new cache entry for the complete prefix through the breakpoint. That makes cache_creation_input_tokens appear close to the full input size, even if most of the content was unchanged from the previous request.

Can I cache changing RAG results or the current user question?

You can mark them, but doing so usually creates a fresh prefix on each request. Put changing retrieval data and the current question after stable cached content unless those values are intentionally reused across requests.

Does changing the model preserve the cache?

No. The supplied design notes state that caches are model-scoped. A model switch should be treated as a new cache path.

Why do parallel identical requests sometimes all miss?

A cache entry becomes readable only after the first response begins streaming. Concurrent requests that arrive before that point may all pay the creation cost. Coordinate or stagger requests when pre-warming a shared prefix matters.

What should I fix first in production?

First instrument block positions and cache usage. Then make tool and system serialization deterministic, add a stable prefix breakpoint, and rotate message breakpoints before the 20-block boundary. Only after that should you tune TTL, result compaction, model routing, or prompt size.

Related Reading

• Reverse Engineering Claude's System Prompts
• Cutting API Costs with LLM Prompt Caching
• Agentic AI Executive Briefing Prompts: 4 Ready-to-Copy Templates for Building Real Enterprise Agents in 2026