Building a Pre-Launch Token Cost Model for a New AI Feature

A token bill is not a surprise waiting to happen. It's a number you can calculate before a single request goes to production, the same way you'd size a database or estimate cloud spend for a new service. Most teams don't, which is why the invoice feels like a shock instead of a forecast that landed close to target.

One B2B services firm found this out the hard way. Its customer-support AI ran about $300 in the trial month. By month twelve, the bill hit roughly $14,000. The jump traced back to a single feature launch that pushed conversation volume up by dozens of times over in two weeks, with no cost cap, no usage alerts, and no fallback routing to catch the overflow. The kind of re-architecture required could have been avoided if anyone had modeled the cost curve before shipping.

That story isn't rare. Industry reporting puts the miss rate at four out of five enterprises, missing their AI cost forecasts by more than 25%. The gap exists because nobody built the model before launch. It's because nobody built the model before launch. And the stakes for getting this right keep rising: Gartner projects worldwide AI model and platform spending to grow 63% in 2026, reaching $64 billion. That kind of growth means finance departments are watching AI line items the way they watch cloud spend, and a feature team that can't explain its token economics in advance is going to have an uncomfortable budget conversation later. The rest of this piece is the method for avoiding that conversation.

What you are estimating: prompt tokens, completion tokens, and why they bill differently

A token is roughly four characters of English text. Every major provider prices per million tokens, and critically, input and output get billed at different rates. That distinction is the whole ballgame.

Output tokens run meaningfully more expensive than input tokens across the market, often several times the input rate. That asymmetry is the structural fact that should shape every cost estimate: completion length, not prompt length, is usually the most sensitive variable in the whole model.

Look at the actual spread as of September 2026, reported by TechBullion. Claude Opus 5 runs $5 per million input tokens and $25 per million output tokens. Claude Haiku 4.5 is $1/$5, a fifth of the flagship rate. Gemini 3.1 Flash-Lite is cheaper still, at $0.25/$1.50. And the gap isn't limited to model tier: Ramp's data shows GPT-5-nano priced at $0.07 per million tokens against GPT-5.5 at $1.42, a wide spread inside a single provider's own lineup. Picking a model is now also a cost decision with its own line item. It's a cost decision with its own line item.

That means any honest cost estimate has at least four moving parts before usage volume even enters the picture: input size, output size, input price, output price. Get any one of those wrong and the error compounds across every scenario built on top of it.

Retries are a trap hiding underneath the sticker price, too. A cheap model that fails often and needs a second or third pass can end up costing more than a pricier model that gets the answer right the first time. A model at $0.15 per million tokens that needs significantly more retries can be, in practical terms, more expensive than a $0.30 model with a clean first-pass rate. That has to enter the model as a quality-adjusted cost, not a raw per-token number.

None of this requires memorizing exact token counts right now. The point of this section is the structure of the estimate, so the numbers that follow have somewhere to slot in.

Measuring prompt and completion sizes before you have real traffic

Write representative prompt templates and run them through a tokenizer, or the provider's own token counter. Character counts are a rough guess at best; actual token counts are the real number, and they're one API call away.

The prompt token count needs to include everything that actually goes into the request:

• The system prompt is often a fixed cost on every single call, and teams often forget to count it

Completion size is harder to pin down, because unlike the prompt, the caller doesn't fully control how long the model's answer runs. Setting a floor, a typical case, and a ceiling depends on what the task actually asks the model to do. A classification task returns a short, bounded answer almost every time. A summarization task tracks loosely with input length. Open-ended generation, on the other hand, has real variance; a cost model needs to capture that variance rather than average it away.

The practical move: generate 20 to 50 prompts that span the realistic range of user input, short ones, typical ones, and the long edge cases, then measure the actual token counts rather than guessing from an average. Averages hide the tail, and the tail is where budgets break.

Multi-turn conversations complicate the math further. Each turn appends the prior context, so a multi-turn conversation can multiply effective prompt token consumption well beyond what a single-turn estimate would suggest. Model that explicitly, turn by turn, rather than treating a conversation as one flat request.

Agentic workflows add another layer. A multi-step agent might make many model calls to complete one user action, and each of those calls carries its own token cost. Budget tool calls and reasoning steps as a multiplier on the single-call cost, not as a rounding error tacked on at the end.

Whatever numbers come out of this exercise, write down the assumptions behind them: which prompt template version was tested, what context length was assumed, what output distribution was measured. That record becomes the audit trail when the model needs an update after launch, and it will need one.

Mapping usage patterns to a request volume forecast

Usage volume isn't a single number. It has a shape: daily rhythms, weekly cycles, and spikes tied to specific events like an onboarding push or a marketing campaign that suddenly goes wide.

Start with what's already measurable. If the AI feature replaces an existing workflow, instrument that workflow first. How many times per day does a user trigger it? How many active users touch the feature at launch, and how many are expected at six months? Those two numbers alone put a floor and a rough trajectory under the forecast.

The growth assumption matters more than the launch-week number. Budget planning should account for substantial annual growth in AI token spend, and a model built only on launch estimates goes stale within a few months of going live.

Usage doesn't distribute evenly across users, either. Power-law patterns are common: a small slice of users generates a disproportionate share of the requests. If pricing passes cost through to users, on a per-query SaaS model for instance, that skew hits margins directly and needs its own line in the forecast.

Agentic workflows shift the unit of measurement too. Multi-step agents generate many model and tool calls per single user action, so cost per token stops being a useful denominator. Cost per task, or cost per completed workflow, is the number that actually tracks what the feature is doing.

Shadow usage is a blind spot that matters here. The broader pattern of enterprises missing cost forecasts ties partly to spend that never appears in a clean reconciliation, because it comes from systems calling the AI feature indirectly. If the feature integrates with other internal tools, account for the downstream calls those tools trigger on their own.

The output of all this should be three scenarios, not one: a conservative baseline, a central estimate, and a high-growth case, each with its assumptions written down clearly enough that someone else on the team could check the math in six months.

Choosing a model (or model tier) that fits the cost target, not just the quality bar

Not every request needs the flagship model. Splitting tasks by complexity and routing each one to the cheapest model that can actually handle it is the structural lever that compresses cost the most, more than any prompt-trimming exercise ever will.

Iternal.ai's example lays out what that looks like in practice: route simple classification to a Haiku-class model at $1/$5 per million tokens, medium-complexity tasks to a Sonnet-class model at an intermediate price point, and reserve Opus-class ($5/$25) for the genuinely hard cases. Blended across a realistic task mix, that tiering produces an output cost roughly 58% lower than sending everything to the flagship model.

The ceiling on what routing can achieve is higher than that example suggests. RouteLLM, from LMSYS and UC Berkeley (ICLR 2025), showed a trained classifier routing requests across models can cut overall spend by more than 85% on standard benchmarks while holding onto 95% of a frontier model's response quality. That's the target well-tuned routing is chasing.

Routing isn't free, though. Rule-based routing, keyword or regex matching, adds under a millisecond of latency. A trained classifier doing the routing adds 50 to 200 milliseconds per request. For a feature where response time matters to the user, that overhead belongs in the cost model as a user-experience constraint, not just a backend detail.

Open-weight models have also closed a lot of the quality gap that used to justify defaulting to proprietary flagships. Iternal.ai's 2026 guide has MiniMax M2.5 scoring 80.2% on SWE-bench Verified against Claude Opus 4.6's 80.8%, a difference close enough that, for coding tasks specifically, the open-weight option is cost-competitive at a fraction of the price.

Budget-tier models can hold their own, too, but the quality varies more than the price does. A 2026 benchmark of 15 models across 38 real tasks found Google Gemini 2.5 Flash ($0.30/$2.50) hitting 97.1% quality at $0.003 per 38-task run, with a 1.1-second median response time. DeepSeek V3 ($0.27/$1.10) runs cheaper per run but trails at 88.7% quality; know that gap before defaulting to the lowest sticker price.

Build at least two model configurations into the cost model before launch, a quality-first version and a cost-optimized version, so engineering and product are choosing the trade-off on purpose instead of inheriting whatever the default happened to be. Running that kind of tiered policy across multiple providers is where a managed gateway earns its keep: it turns routing into a configurable rule instead of a separate integration for every model swap.

Stress-testing the model against realistic load scenarios

Averages lie by omission. A cost model built entirely on average token counts and average request volume will look fine right up until the month it doesn't, because the scenarios that actually blow budgets, burst traffic, unusually long outputs, edge-case prompts arriving in clusters, never appear in an average.

Run three scenarios explicitly, not as an afterthought but as the core output of the model:

• Baseline: central usage estimate times central token count per request, times the chosen model's pricing

Gateway overhead compounds in agentic workflows the same way cost does. If a gateway adds latency on every call, that latency multiplies across every step in a multi-step agent chain, and each of those steps is also a separate billable call. Model both effects together, not as two unrelated line items.

Use RouterBench as a pre-launch check on routing assumptions. It offers more than 405,000 precomputed inference outputs across eleven LLMs and seven tasks, including MMLU, MT-Bench, MBPP, HellaSwag, WinoGrande, GSM8K, and ARC, all tagged with cost and performance metadata. Teams designing a routing policy can test it against that dataset before spending real inference budget finding out it doesn't hold up.

Price trends deserve a place in the stress test too, but a careful one. LLM prices fell roughly 80% industry-wide from 2025 to 2026, yet consumption grew faster than prices dropped, so total spend still climbed. Model volume growth and price decline as separate variables, and treat falling prices as a possible offset, not a guarantee baked into the forecast.

Set budget trigger thresholds ahead of time: the spend levels that kick off an internal review, a model downgrade, a rate limit, or an escalation to stakeholders. Ramp's enterprise data puts the top 25% of AI spenders at roughly $14,843 a month and the top 10% at roughly $73,030 a month, useful reference points for where finance attention starts to sharpen. Write down what would have to be true for each scenario to actually happen. Those assumptions matter as much as the dollar figures, because they're what the team is supposed to keep an eye on after launch.

Turning the cost model into a living operational input

A cost model that never gets checked against actuals is just a guess with better formatting. Set up cost attribution before the first production request goes out, not after the first invoice lands.

There are three practical ways to do it. Direct attribution tags API calls with a team or feature identifier at request time, the most accurate method available. Proportional allocation splits cost based on each team's share of total tokens consumed. Virtual tagging uses metadata, like user IDs or request patterns, to allocate spend without touching the application code.

Once the feature is live, the metric that matters isn't cost per token in isolation. It's cost per task, per workflow, or per resolved ticket, measured against what the process cost before AI took it over. A workflow whose token bill doubled while it handled three times the volume actually got cheaper per unit of work, even though the raw invoice went up. Per-token cost alone misses that.

Real-time visibility is the mechanism that keeps the model honest. The discipline sometimes called Token FinOps tracks token consumption by vendor, team, and model, attributes it to specific projects, sets budgets with alerts attached, and steers model choice on an ongoing basis. That's the operational, always-on version of the model built before launch.

The scenario outputs from the stress test become the alert thresholds in production. The high-output scenario's ceiling turns into an alert trigger. The spike scenario's total becomes the number that trips a hard cap. And enforcement lives at the gateway level: a managed LLM gateway is where budget limits, per-team spend caps, and fallback routing actually get enforced, without rewriting application code every time a policy changes.

As spend climbs toward the range Ramp's April 2026 data flags for its top 25% of enterprise spenders, that's the point where a monthly review earns its place on the calendar. The pre-launch model itself needs a refresh on a quarterly rhythm too: new token measurements, updated usage forecasts, actuals checked against the original assumptions. The model built before launch, the attribution set up at launch, and the review that follows aren't three separate projects. They're one continuous process, and treating launch as the finish line just means reconstructing the cost story from an invoice at the end of the month, which is exactly the position a pre-launch model was supposed to prevent.