[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"blog-en-llm-prompt-caching-measure-costs":3},{"id":4,"title":5,"author":6,"body":7,"cover":325,"description":326,"draft":327,"extension":328,"locale":329,"meta":330,"navigation":331,"path":332,"publishedAt":333,"seo":334,"seoTitle":335,"slug":336,"stem":337,"tags":338,"__hash__":340},"blog\u002Fen\u002Fblog\u002Fllm-prompt-caching-measure-costs.md","Prompt caching: structuring prompts and measuring savings","Manuel Tardivo",{"type":8,"value":9,"toc":312},"minimark",[10,31,36,47,54,71,75,104,108,145,152,156,164,238,250,257,272,276,287,291,294,298],[11,12,13,14,18,19,22,23,30],"p",{},"Several vendors (OpenAI, Anthropic, Google, …) expose ",[15,16,17],"strong",{},"discounted accounting"," or separate counters for reused prompt prefixes—often branded “prompt caching”, “context caching”, or cheaper repeated blocks. ",[15,20,21],{},"Policies shift frequently",": this article covers measurement methodology, not live dollar math. Always open the vendor ",[24,25,29],"a",{"href":26,"rel":27},"https:\u002F\u002Fopenai.com\u002Fapi\u002Fpricing\u002F",[28],"nofollow","pricing page"," plus their caching guide for the model you call.",[32,33,35],"h2",{"id":34},"what-cache-hit-means-upstream","What “cache hit” means upstream",[11,37,38,39,42,43,46],{},"Roughly: a ",[15,40,41],{},"deterministic prefix"," (long system instructions, tool manifests, static RAG bundle) that the vendor already ingested recently is billed differently from “new” prompt text in the current turn. This is ",[15,44,45],{},"not"," your HTTP cache—it lives inside their infra.",[11,48,49,50,53],{},"Common ",[15,51,52],{},"false negatives"," (no hit):",[55,56,57,65,68],"ul",{},[58,59,60,61,64],"li",{},"You change ",[15,62,63],{},"one byte"," in the supposedly static system block (timestamps, random UUIDs).",[58,66,67],{},"Identical chunks appear in a different order.",[58,69,70],{},"You call a model or route where caching is unsupported—verify the matrix.",[32,72,74],{"id":73},"stable-prompt-structure","Stable prompt structure",[76,77,78,89,96],"ol",{},[58,79,80,81,84,85,88],{},"Push the ",[15,82,83],{},"largest reusable corpus"," into ",[15,86,87],{},"system"," (or a fixed prefix) from immutable templates.",[58,90,91,92,95],{},"Place user-specific variables (question text, uploaded doc digest) ",[15,93,94],{},"after"," the stable block—many systems compare prefixes.",[58,97,98,99,103],{},"Avoid dynamic strings in system (",[100,101,102],"code",{},"new Date()"," unless required).",[32,105,107],{"id":106},"measurement-protocol-ab","Measurement protocol (A\u002FB)",[76,109,110,116,126],{},[58,111,112,115],{},[15,113,114],{},"Baseline",": N real requests (e.g. 200) with minimal reuse (or a tiny system prompt).",[58,117,118,121,122,125],{},[15,119,120],{},"Cached",": same N requests after ",[15,123,124],{},"warming"," with at least one identical call if the vendor requires priming.",[58,127,128,129,132,133,136,137,140,141,144],{},"Read ",[100,130,131],{},"usage"," from responses (or billing exports): total ",[15,134,135],{},"input tokens",", optional split like ",[100,138,139],{},"cached_prompt_tokens",", and ",[15,142,143],{},"estimated cost"," using current price sheets.",[11,146,147,148,151],{},"Compare ",[15,149,150],{},"mean and p95"," cost per request—retries in the middle can skew averages.",[32,153,155],{"id":154},"openai-operational-numbers-official-cookbook-api-guides","OpenAI: operational numbers (official cookbook + API guides)",[11,157,158,159,163],{},"OpenAI’s ",[160,161,162],"em",{},"Prompt Caching 201"," cookbook captures mechanics you can rely on in design reviews:",[55,165,166,181,192,201,221],{},[58,167,168,169,172,173,176,177,180],{},"Hits require ",[15,170,171],{},"byte-identical prefixes"," recently seen upstream—system text, ",[15,174,175],{},"tool definitions",", even ",[15,178,179],{},"structured-output schemas"," participate, not just prose.",[58,182,183,184,187,188,191],{},"Practical threshold called out in the cookbook: prompts around ",[15,185,186],{},"≥ 1024 tokens",", with cache granularity of ",[15,189,190],{},"128-token chunks"," on the prefix (reconfirm when docs change).",[58,193,194,195,200],{},"Responses expose ",[15,196,197],{},[100,198,199],{},"usage.prompt_tokens_details.cached_tokens"," for Chat\u002FResponses—never guess from string length.",[58,202,203,204,209,210,215,216,220],{},"Optional ",[15,205,206],{},[100,207,208],{},"prompt_cache_key"," (when enabled for your workload) improves colocation for traffic sharing huge static prefixes—read the live ",[24,211,214],{"href":212,"rel":213},"https:\u002F\u002Fdevelopers.openai.com\u002Fdocs\u002Fguides\u002Fprompt-caching",[28],"Prompt caching guide"," plus ",[24,217,162],{"href":218,"rel":219},"https:\u002F\u002Fdevelopers.openai.com\u002Fcookbook\u002Fexamples\u002Fprompt_caching_201",[28],".",[58,222,223,224,227,228,231,232,237],{},"Model-specific discounts move quickly (cookbook tables show examples from ~50% off on cached ",[100,225,226],{},"gpt-4o"," inputs up to ~90% on newer ",[100,229,230],{},"gpt-5.x"," families)—revalidate against ",[24,233,236],{"href":234,"rel":235},"https:\u002F\u002Fopenai.com\u002Fapi-pricing\u002F",[28],"Pricing"," before quoting dollars.",[11,239,240,243,244,249],{},[15,241,242],{},"Engineering tip"," straight from the cookbook: move timestamps\u002Fdebug strings into request ",[15,245,246],{},[100,247,248],{},"metadata"," so they do not poison the prompt prefix.",[32,251,253,254],{"id":252},"anthropic-cache_control","Anthropic: ",[100,255,256],{},"cache_control",[11,258,259,260,267,268,271],{},"Claude documents ",[24,261,264],{"href":262,"rel":263},"https:\u002F\u002Fdocs.anthropic.com\u002Fen\u002Fdocs\u002Fbuild-with-claude\u002Fprompt-caching",[28],[15,265,266],{},"prompt caching"," with explicit breakpoints (",[100,269,270],{},"cache_control: { type: \"ephemeral\" }",") or automatic policies for long chats—TTL and $\u002Ftoken differ from OpenAI, so don’t reuse pricing math. Review ZDR statements if your contract requires it.",[32,273,275],{"id":274},"order-of-magnitude-expectations","Order-of-magnitude expectations",[11,277,278,279,282,283,286],{},"Vendor-published savings targets ",[15,280,281],{},"high-repeat"," workloads; if most tokens are unique user questions, ",[15,284,285],{},"do not expect large wins",". Measure anyway.",[32,288,290],{"id":289},"privacy-and-invalidation","Privacy and invalidation",[11,292,293],{},"Sensitive data inside a shared cached prefix can violate compliance in multi-tenant setups—confirm per-tenant isolation and retention statements.",[32,295,297],{"id":296},"summary","Summary",[11,299,300,303,304,307,308,311],{},[15,301,302],{},"Freeze the prefix",", push variability to the tail, ",[15,305,306],{},"measure before\u002Fafter"," using real token usage fields from API responses. Marketing percentages never replace ",[100,309,310],{},"usage.prompt_tokens_details"," (or equivalent) in your logs.",{"title":313,"searchDepth":314,"depth":314,"links":315},"",2,[316,317,318,319,320,322,323,324],{"id":34,"depth":314,"text":35},{"id":73,"depth":314,"text":74},{"id":106,"depth":314,"text":107},{"id":154,"depth":314,"text":155},{"id":252,"depth":314,"text":321},"Anthropic: cache_control",{"id":274,"depth":314,"text":275},{"id":289,"depth":314,"text":290},{"id":296,"depth":314,"text":297},"\u002Fblog\u002F24_llm_prompt_caching_measure_costs_1x.webp","How to package stable system and user prompts where providers support caching, and a protocol to estimate token savings without invented figures.",false,"md","en",{},true,"\u002Fen\u002Fblog\u002Fllm-prompt-caching-measure-costs","2026-06-07",{"title":5,"description":326},"Prompt caching: structure and measure it","llm-prompt-caching-measure-costs","en\u002Fblog\u002Fllm-prompt-caching-measure-costs",[339],"mvp","LPTTGV_aS6yUNbxvX_dan76UWRJAMB8KcD7W8gVqQHU"]