Cache Read Discount Rates by Provider and Workload Type

OpenAI, Anthropic, and Google now bill cached input reads at a small fraction of the standard input rate. The 90% discount has converged. Yet two teams paying the same list price can see very different effective bills.

The gap comes from three variables: cache hit rate, static-prefix length, and write-fee structure. They interact differently depending on how an application is built. List-price savings become realized savings only when you know which workloads actually harvest the discount and where each provider's mechanics quietly erode it.

How each provider's caching mechanism actually works, and where the fine print sits

The four major providers use three structurally different caching models, and the differences on the write side matter more than the now-converged read discount. Two implementation families exist. Automatic prefix matching, used by OpenAI, DeepSeek, and Gemini's implicit cache, detects a repeated prefix with no code change. Explicit cache control, used by Anthropic and Gemini's explicit context cache, requires the developer to mark which blocks are cacheable.

Anthropic uses cache_control breakpoints. Developers can place up to four cache checkpoints per request at natural boundaries in the prompt. OpenAI's automatic caching requires no API change. The system detects matching prefixes independently. Gemini 2.5 and newer models use implicit caching by default with no storage fee. Explicit context caching is a separate mode that bills hourly storage, not a write multiplier. DeepSeek uses automatic prefix matching with no separate write charge and no exposed TTL.

Write-fee structures fall into three categories. The first is free write with discounted read. The second category is write premium plus discounted read. The third is an hourly storage meter. Gemini's explicit context cache charges no write multiplier but bills storage per million tokens per hour. It is a running meter against wall-clock time rather than a one-time fee.

Minimum-token thresholds differ significantly and quietly exclude whole categories of workload from caching eligibility. Anthropic sets a higher minimum for Haiku than for Sonnet and Opus models. OpenAI's token minimum is low enough that most system prompts clear it easily. Gemini's explicit cache minimum is far higher than the other providers, which disqualifies short system prompts entirely.

TTL mechanics shape reuse windows. Anthropic's five-minute and one-hour tiers are explicit choices with different write costs. Gemini's explicit cache TTL is configurable. DeepSeek's TTL is not exposed to the developer at all.

The write-fee math that determines whether caching saves or costs money on each workload

Write premiums look alarming in isolation, but break-even arrives so quickly on any realistic production workload that the write fee becomes a rounding error. The danger is not overpaying to write. It is choosing the wrong TTL tier for your traffic pattern.

The break-even math runs plain. With free writes and a 0.1× read rate, which covers DeepSeek, Gemini implicit, and pre-GPT-5.6 OpenAI models, savings begin on the second call. With a 1.25× write fee and 0.1× reads, which covers Anthropic's five-minute tier and GPT-5.6 and later, break-even arrives on roughly the second reuse. With a 2× write fee and 0.1× reads, which covers Anthropic's one-hour tier, break-even lands on roughly the third total request.

For any workload where the same prefix is touched dozens or hundreds of times, the write fee is negligible against accumulated read savings. The TTL choice is where teams actually lose money. A bursty workload with several requests in a short window followed by a long quiet period benefits from the five-minute tier, because the one-hour write premium is paid for TTL that goes unused. A steady-cadence workload, one request every several minutes for hours, benefits from the one-hour tier because the prefix stays live between requests.

Gemini's explicit cache hourly storage meter inverts the calculus. A large document cached and left idle accumulates storage cost with no reuse. The meter runs whether or not requests arrive.

Which workload shapes actually harvest the discount, and which ones quietly miss it

The workloads that realize close to the full headline discount share one structural property: a large, static prefix reused at high frequency. Workloads that lack either property realize a fraction of the discount regardless of which provider they use. Think through the high-harvesting shapes. Multi-turn agents re-send conversation history every turn, so caching captures the accumulated turns at the discounted rate while only the new turn is billed at full input price. Batch evaluation runs send a fixed rubric or scoring prompt against many inputs, so you get a stable prefix and high volume, and they qualify for batch discounts that stack with cache discounts.

The low-harvesting shapes are just as concrete. Short system prompts that clear Anthropic's and OpenAI's minimums but fall far below Gemini's floor simply cannot use Gemini caching. Low-volume or sporadic traffic never clears break-even. Open-ended, creative, or highly personalized queries leave the prefix-stable portion small relative to the per-user dynamic content, so the cacheable fraction of total input tokens stays low even when caching is correctly configured.

A real customer-support agent case shows what the high-harvesting shape delivers in practice. Moving from uncached to cached required only three cache_control markers and reduced the monthly bill by 90%. The same provider pricing produces very different effective costs depending on what the application actually sends.

How provider selection shifts depending on which workload shape you are running

Once workload shape is known, provider fit is largely determined by minimum-token thresholds, write-fee structure, and whether implicit or explicit caching matches how the application is built. The read discount rate alone tells you little. For a short-to-medium stable prefix below Gemini's minimum threshold with high request volume, Anthropic and OpenAI are the natural fits. Both clear the minimum easily. Anthropic's explicit cache_control gives precise control over which blocks are cached. OpenAI's automatic caching requires no implementation work. For a very long shared document above Gemini's minimum with many users and a long session, Gemini is competitive. The minimum is cleared easily, implicit caching applies automatically, and there is no write multiplier on the implicit path. Gemini's explicit context cache adds configurable TTL but introduces the hourly storage meter. For cost-extreme, cache-dominated workloads where the prefix is reused very heavily, DeepSeek V4-Pro's sub-one-percent cache-hit rate versus miss makes it structurally cheaper per cached token than any Western lab model when cache hit rate is high. But its peak and off-peak pricing means the cache-miss rate swings significantly at peak hours, making a well-configured cache doubly important as protection against that swing.

TTL choice also follows traffic shape. You pay the lower write premium, capture the burst, and let the entry expire rather than paying the two-times write fee for idle TTL on the one-hour tier. Steady-cadence traffic fits Anthropic's one-hour tier, which breaks even on the third request and then runs cheaply for the duration.

Teams that want zero implementation overhead get it from Gemini's implicit caching and OpenAI's automatic caching, both of which apply without code changes or explicit cache management. Teams routing through OpenRouter as a gateway aggregator should also account for its per-provider cache-read multipliers: Anthropic at 0.1×, DeepSeek at 0.1×, Qwen at 0.1×, OpenAI at 0.25× or 0.50×, and Gemini at 0.25×. These multipliers shift effective prices when comparing through a gateway layer.

Why prompt-caching decisions belong in the infrastructure layer, not in application code

When caching logic is embedded in application code, every service makes independent decisions about prefix structure, TTL tier, and provider selection. None of them has visibility into the aggregate hit rate or spend that would tell them whether those decisions are working. Consider the prefix-stable prompt structure problem. The provider simply bills the full rate silently. This mistake gets replicated independently across microservices unless a shared layer enforces prefix discipline.

Cache-aware routing belongs in the gateway too. Selecting a provider based on which one is likely to produce a high cache hit rate for the current request shape requires knowing the workload's historical token mix and prefix stability. That information lives in a gateway, not in the application making the call.

Semantic caching at the gateway layer is a distinct mechanism from provider-level prompt caching. Gateway-level semantic caching and provider-level prompt caching stack. On a miss, the request goes to the provider where prompt caching applies to the prefix. A team running AI in production across multiple providers, models, and services needs one layer where prefix discipline is enforced, provider selection reflects actual cache-hit economics, and spend is attributed per team, project, and model in real time. Reconstructing that picture from end-of-month invoices across four provider dashboards is not operationally viable.

What a gateway layer should handle for teams running cache-aware AI in production

Teams evaluating gateway options for cache-aware production AI should assess four capabilities: cache-aware routing, semantic caching, real-time spend attribution granular enough to show cache efficiency per workload, and security controls that ensure sensitive data does not reach provider APIs unguarded. Semantic caching compounds provider-level savings. On a hit, no API call is made and no tokens are billed at any rate. For high-volume repetitive query workloads, the combination of semantic caching at the gateway and prompt caching at the provider represents the deepest effective cost reduction available.

Fallback and reliability matter for cache-heavy workloads. If the primary provider goes down, the gateway should automatically route to a backup provider serving the same model without a code change. Cache state does not transfer between providers, so the first requests after a failover will be cache misses. That is a cost spike, not a catastrophe, and it resolves as the new provider builds its cache.

Security controls for sensitive data are especially important for cached workloads. PII and PHI redaction should run at the gateway before any prompt reaches a provider API, because a cached prompt containing unredacted PHI is a prompt that has already been sent to a provider and potentially stored. Redaction at the gateway, enforced by policy rather than developer discipline, is the correct chokepoint.

Gateway options vary meaningfully on these dimensions. Bifrost covers more than 1,000 models across a large number of providers with dual-layer semantic caching and hierarchical virtual-key budget controls. Cloudflare AI Gateway covers a broad range of providers with exact-match-only caching. LiteLLM covers a large number of providers with backend-specific semantic caching requiring separate setup, a self-hosted option for teams that want to operate the infrastructure themselves. Kong AI Gateway uses a plugin model. OpenRouter has no semantic caching and applies its own per-provider cache-read multipliers that teams should account for in cost comparisons.

The implementation decisions that make or break cache hit rate in practice

Achieving a high cache hit rate is primarily a prompt-structure problem, not a configuration problem. The discount is available automatically on most providers, but it is silently forfeited by prompt architectures that shift content near the start of each request. Prefix stability is the single highest-leverage implementation decision. Static content such as system prompts, tool definitions, and reference documents must come first in the prompt. Dynamic content such as user messages, session context, and timestamps must come last. One changed byte early in a prefix invalidates everything after it with no error message.

On Anthropic, place explicit cache_control breakpoints at natural structural boundaries. Put them at the end of the system prompt, the end of the tool schema block, and the end of the shared document. Each block can then be cached independently, and a change in one block does not invalidate the others.

Cache hit rate measurement is not optional. Without instrumenting cached_tokens in the usage response, there is no way to know whether caching is working. The gain was only visible because hit rate was being measured.

TTL tier selection should follow traffic pattern, not default. Bursty traffic favors the shorter, cheaper write tier. Steady-cadence traffic justifies the longer TTL's higher write cost. Choosing the wrong tier is a silent cost that does not show up as an error.

For Gemini, the implicit cache is the default and correct choice for most workloads. It applies automatically, charges no write fee, and avoids the hourly storage meter of the explicit cache path. The explicit cache is only necessary when a guaranteed cache hit on a specific object is required.

Finally, stacking cache with batch discounts matters. All three major providers support 50% batch discounts on asynchronous traffic, and batch discounts stack with cache-read discounts. For workloads that tolerate asynchronous processing, such as evaluation runs, bulk document analysis, and overnight processing, stacking the two discounts produces the deepest effective cost reduction available at the API layer. Realized savings depend on how an application is built, not just on which discount rate a provider advertises.