LLM Prompt Caching: Calculate Savings Without Double-Counting Tokens
By Nicho
Prompt caching can reduce the cost of repeatedly sending the same instructions or context to an LLM. The saving depends on actual cache hits, the provider's billing categories, and how often the cached material is reused before it expires.
A low cache-read price is only part of the calculation. Writes may have a separate price, output tokens still cost money, and a reported input total may already include cached tokens. This guide explains how to build a useful estimate and check it against usage records. The worked examples use explicitly hypothetical prices, not measured application results or offers from a provider.
Identify what the provider actually caches
Prompt caching generally concerns reusable input context. It should not be confused with an application returning a previously saved answer without calling a model.
DeepSeek's context-caching documentation describes matching persisted prompt-prefix units. It is enabled by default, but operates on a best-effort basis. Similar-looking requests do not guarantee a hit, and cached input does not mean that the previous answer will simply be returned.
For your application, distinguish three mechanisms:
- Provider prompt caching: the inference service reuses eligible input processing and reports its own usage categories.
- Application response caching: your application serves a saved result, potentially avoiding a model call entirely.
- Retrieval or document caching: your application avoids repeating extraction or retrieval work, but may still send the resulting text to the model.
Measure each separately. Otherwise a change in document retrieval can be mistaken for an inference discount, or a response-cache hit can be counted as a provider cache hit.
Divide input into non-overlapping categories
Start with the selected endpoint's usage schema, not a guessed formula. DeepSeek reports prompt_cache_hit_tokens and prompt_cache_miss_tokens. Other APIs may expose different fields or nested breakdowns.
If a response reports 10,000 total input tokens, including 8,000 cached tokens, the remaining uncached input is 2,000. Charging all 10,000 at the uncached rate and then adding 8,000 at the cached rate double-counts part of the request.
For a service with separate hit and miss rates, a basic model-token estimate is:
cost = miss_tokens / 1M × miss_rate + hit_tokens / 1M × hit_rate + output_tokens / 1M × output_rate
Only use that formula when it matches the provider's definitions. If the service reports cache creation or storage separately, reconcile those categories before adding their charges. A cache-write price might replace another category or be an additional charge; the rate card must settle that distinction.
Keep the currency, units, model ID, provider, and date beside your calculation. A number quoted per thousand tokens cannot be inserted unchanged into a per-million-token formula.
Include the first write and the lifetime
Kimi's inference-billing guide distinguishes K3 cache writes by lifetime, with five-minute and one-hour categories. Its cache-hit input is charged at the hit price without another cache-write charge. Those are direct-platform rules, not proof that a gateway uses the same accounting.
A useful question is how many times a prefix will be reused while its cache entry remains eligible. Consider this hypothetical billing scheme for a one-million-token prefix:
| Charge | Assumed price |
|---|---|
| Process the prefix without caching | $1.00 per use |
| First cache write, replacing ordinary input billing | $1.25 |
| Each subsequent cache read | $0.10 |
Five uses without caching cost $5.00. One write followed by four reads costs $1.65, a 67% saving on that prefix. With only one use, the cached route costs more: $1.25 instead of $1.00.
The assumption that the write replaces ordinary input billing is essential. If a real provider charges writes additionally, the formula changes. These prices illustrate the method; they are not a current Kimi or cheapaimodel quote.
For this replacement-style scheme, compare write_cost + (uses − 1) × read_cost with uses × ordinary_cost. Then repeat the calculation for the expected number of uses within the actual lifetime. A cheap read cannot help if every request arrives after the entry has expired.
Calculate savings on the whole request
An input discount is not the same percentage discount on the final bill. Suppose an uncached workload has $10 of input charges and $20 of output charges. Assume 80% of that input becomes cache hits, reads cost 10% of the ordinary input rate, and there are no extra write charges.
The new input charge is $2 for misses plus $0.80 for hits, or $2.80. The total becomes $22.80 instead of $30: a 24% overall saving, despite a 90% discount on the cached portion's rate.
Long answers, reasoning output, tool calls, retries, and storage can change the total further. Use billable usage reported by the endpoint rather than counting only the visible final answer. For current model prices, start with the price comparison, then apply your own usage mix.
Design a useful measurement
Collect enough information to compare like with like:
- Model and provider route, request time, and relevant configuration.
- Total input, cache hits, cache misses, cache creation, and output where reported.
- Separately billed services and the account's recorded charge.
- Whether the result met the application's acceptance criteria.
Calculate a token-weighted hit share: total hit tokens divided by total eligible input tokens. Averaging per-request percentages can mislead when a tiny request and a large request receive equal weight.
Keep stable instructions and reusable context before changing task-specific material when that arrangement suits the provider's documented prefix rules. Do not rearrange security instructions or remove necessary context solely to chase a higher hit rate. Validate output quality after changing prompt structure.
Reconcile before scaling
Compare a small sample of usage records with the provider's billing view. Investigate missing categories, rounding, different pricing windows, and write or tool fees before assuming the invoice is wrong.
cheapaimodel publishes input, output, and cache-read rates on its pricing page. Confirm the applicable model's write and other charging rules before treating a three-column subtotal as a complete invoice. For K3-specific distinctions, see the Kimi API guide.
The useful outcome is a lower cost per accepted result. A larger cached-token count alone does not establish that the application became cheaper or more effective.