What an LLM bill is actually made of
Every request carries four token buckets: the system prompt S, the fixed context C that is re-sent unchanged (tool schemas, retrieved passages, style guides), the user message U and the model’s reply A. The first three are billed at the input price, the reply at the much higher output price.
The monthly bill is then just traffic: DAU × requests per user per day × days in the month × cost per request. Because it is linear in DAU, doubling users doubles the bill exactly — the interesting variation all lives in the per-request number.
That is why this page asks for a token profile rather than a monthly token total: the same 10,000 tokens cost very different amounts depending on whether they sit in a cacheable prefix, in a user message, or in a long generated answer.
Three service shapes, three very different bills
Document summarisation (200-token system prompt, 10,000-token document, 800-token summary, one request per user per day) costs $0.0376 per request on GPT-5.4 and $0.0042 on GPT-5 mini. At 1,000 DAU over 30 days that is $1,129 versus $125 a month at 2026-09-02 list prices.
A RAG assistant (400-token system prompt, 3,000 tokens of retrieved context, 2 turns, 4 sessions per user per day) bills 8,160 input tokens per session and lands at $3,163 a month on GPT-5.4 or $376 on GPT-5 mini — the retrieved context, not the question, is what you are paying for.
A coding agent (2,000-token system prompt, 20,000 tokens of file context, 8 turns, 5 sessions a day) bills 274,300 input tokens in a single session — $69,428 a month on GPT-5.4 at the same 1,000 DAU. Agent workloads are two orders of magnitude above chat, which is why they are usually metered rather than flat-priced.
What prompt caching can and cannot save
A prompt cache matches a prefix, not a set of tokens. Only S + C can hit; the user’s message is new every time and always pays the full input price. This page applies the hit rate to the prefix alone, which is why its savings look smaller — and truer — than calculators that multiply the whole input by the hit rate.
Measured on the built-in profiles: caching saves 24.5% on the chatbot, 22.8% on RAG and 50.9% on the coding agent, where the prefix is 22,000 of the tokens. The rule of thumb is simple — caching is worth configuring when the prefix is large relative to the user’s message.
Caching saves nothing when the prefix is not reused inside the cache TTL — a batch job that processes one document per user and never repeats, for instance. Anthropic also charges 1.25× (5 min) or 2× (1 h) to write the cache, so a prefix used once costs more with caching on than off.
Conversation length is the strongest lever
On the chatbot profile a session costs $0.0076 at one turn, $0.0472 at six, $0.0992 at twelve and $0.1763 at twenty — the billed input grows from 650 tokens to 125,100. Doubling six turns to twelve costs 2.1× as much; going to twenty costs 23× a single turn.
The cause is the quadratic term in N·S + N·U + (U+A)·N(N−1)/2: every turn re-sends the whole transcript. Product decisions such as “start a new chat for a new topic” or “summarise after eight turns” move the bill more than any model swap.
The other levers: output, batch, model tier
- Output length. Output is priced 4–8× input, so capping max_tokens or asking for a shorter format is often the cheapest change available.
- Batch API. On the classification profile a request drops from $0.00026 to $0.00013 — exactly half — in exchange for a delayed response. Anything that does not need to answer a waiting user should run this way.
- Model tier. The same chatbot session is $0.0472 on GPT-5.4 and $0.0059 on GPT-5 mini, an 8× gap. Routing simple turns to the small model and escalating only when needed usually beats prompt micro-optimisation.
- Fixed context. Trimming 1,000 tokens of boilerplate from the prefix of a service doing a million requests a month saves $2,500 on a $2.50/1M model — before any caching.
Cost per user, and what to charge for it
Cost per user per month does not depend on how many users you have, so it is the number a business case should quote. On the chatbot profile it is $0.36 on GPT-5 mini, $0.52 on Gemini 2.5 Flash and $2.83 on GPT-5.4 — the difference between a $5 subscription with a healthy margin and one that loses money.
Read it the other way for a fixed budget: $1,000 a month on this profile covers about 353 daily users on GPT-5.4, 1,930 on Gemini 2.5 Flash and 2,815 on GPT-5 mini. The budget line on the chart marks that crossing point for every model you selected.
For a free tier, work from the same figure: a 100-message monthly cap on the mini model costs a fraction of a dollar per signup, while the same cap on a frontier model is a marketing budget. Decide the model per tier, not per product.
Common mistakes in LLM cost estimates
- Multiplying the entire input by the cache hit rate. Only the prefix can hit, so that overstates the saving — often by a factor of two on chat workloads.
- Estimating a session as one turn. On any multi-turn product this is the single largest source of error, and it is always an under-estimate.
- Ignoring reasoning tokens. They are billed at the output price and are invisible in the response, so a “short answer” can carry a long hidden completion.
- Budgeting the average and forgetting the tail. Retries, long documents and abusive sessions sit far above the mean; a per-user token cap costs one afternoon to build and prevents the bill you cannot explain.