A team enabled prompt caching on an application whose requests share a 40,000-token knowledge base. They placed the per-user dynamic data at the…

Anthropic's prompt caching mechanism relies on exact prefix matching. The cache is evaluated sequentially from the very beginning of the prompt. If dynamic, user-specific data is placed at the start of the prompt, the prefix changes with every single request, causing a cache miss for the entire prompt, including the knowledge base. To utilize caching effectively, stable content (like the 40,000-token knowledge base and system instructions) must be placed at the beginning of the prompt, followed by the dynamic data at the end.

A. Prompt caching is designed specifically for large contexts and supports prompts well beyond 10,000 tokens, up to the model's maximum context window limit.

B. Temperature settings control output randomness and do not affect the prompt caching mechanism, which operates strictly on the input tokens.

C. The scenario states the knowledge base is shared across requests, implying it is stable; the issue is its structural placement, not its mutation rate.

1. Anthropic Official Documentation, "Prompt Caching", Section: How prompt caching works (Prefix matching), 2024. (https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching)

2. Anthropic Official Documentation, "Prompt Caching", Section: Best practices for caching, 2024.