Someone will eventually ask which customer accounted for last month's model spend. If you cannot answer, you cannot price your product, identify an unprofitable account, or find the one tenant whose retry loop is costing you four figures a week.
The provider dashboard cannot answer it. It shows spend by API key, and unless you provision a key per tenant, which is its own operational problem, that is one number for everybody.
This post covers what to record, when to record it, and the three mistakes that make the resulting numbers quietly wrong.
Record it at request time, or not at all
The core insight: model prices change, and you cannot reconstruct historical cost from historical token counts.
If you store input_tokens: 1200, model: "gpt-4o-mini" and compute cost later using today's price table, every number you produce for last quarter is wrong. Providers adjust prices, deprecate models and change tier boundaries. A price table is a point-in-time fact.
So record the money at the moment of the request:
{
"request_id": "req_01J...",
"timestamp": "2026-08-24T09:14:22Z",
"tenant_id": "acme-corp",
"user_id": "u_8823",
"feature": "support-reply",
"provider": "openai",
"model_requested": "gpt-4o-mini",
"model_served": "gpt-4o-mini-2026-07-18",
"input_tokens": 1204,
"output_tokens": 318,
"cached_input_tokens": 900,
"input_price_per_1k": 0.00015,
"output_price_per_1k": 0.0006,
"cost_usd": 0.000371,
"latency_ms": 1840,
"cache_hit": true
}
Both the token counts and the prices used, plus the computed cost. Then a re-price is a decision rather than an archaeology project.
The three mistakes
1. Attributing to the API key instead of the tenant
The most common one, and it looks fine until you have a second customer.
Tenant identity has to travel with the request from your application to wherever cost is recorded. If your gateway learns the tenant from which upstream key was used, you are limited to one tenant per provider key, and you will end up managing hundreds of keys to get reporting you could have had from a header.
Pass tenant identity explicitly. Then verify it: a field that is blank on 30% of records is worse than no field, because you will build dashboards on it before you notice.
Run this query before you trust anything:
SELECT
count(*) FILTER (WHERE tenant_id IS NULL OR tenant_id = '') AS unattributed,
count(*) AS total
FROM llm_requests
WHERE timestamp > now() - interval '7 days';
If unattributed is not approximately zero, fix that before building anything else.
A related trap worth naming: on some telemetry pipelines the tenant identifier ends up in resource attributes on some spans and span attributes on others. Query the wrong map and every customer looks inactive. Pick one location, and validate with a real query rather than by reading the instrumentation.
2. Ignoring cached and reasoning tokens
Provider billing is no longer two numbers.
Cached input tokens are usually billed at a large discount. If you compute cost as input_tokens x input_price, you overstate spend for any workload with a stable system prompt, which is most of them. Record cached input separately and price it separately.
Reasoning tokens on reasoning models are billed as output but are not in the visible completion. A cost model that derives output tokens from response length will understate spend, sometimes dramatically. Take the token counts from the provider's usage object rather than counting characters.
Requested versus served model. If you route with fallback, or use an alias that resolves to a dated snapshot, the model you asked for is not always the one that billed you. Record both. This is also the first thing to check when cost moves and nobody deployed anything.
3. Recording cost without recording why
tenant_id and cost_usd tell you who spent. They do not tell you what to do about it.
Add the dimensions that make spend actionable:
- Feature or endpoint. Which product surface? This is what tells you whether a feature is priced correctly.
- User or session. Lets you find the single account inside a tenant driving the spend.
- Cache hit. Without this you cannot measure whether caching is paying for itself.
- Retry attempt number. Retries are invisible spend. A silent retry loop is one of the most common causes of an unexplained bill.
That last one deserves emphasis. If your client retries on timeout and your gateway also retries, a single logical request can bill four times. Unless attempt number is recorded, that shows up as "usage grew" rather than "we have a bug".
Where to put the recording
Three options, with a real trade.
In application code. Full control, and it works regardless of how you call providers. But it is a code change in every service, it drifts as services are added, and coverage is only ever as good as your least-maintained service.
At the gateway. If model calls route through a gateway, it sees every request and can record all of the above with no per-service work. It also sees calls from services you cannot easily instrument, which is often where the surprising spend lives. What it cannot see is context that only exists in your application, so tenant and feature have to be passed in as headers or metadata.
Both. The gateway records the money, the application enriches with business context. This is what most teams converge on.
Making it useful
Raw records are not a cost model. The four things people actually ask:
Spend per tenant per month, ranked. Sounds trivial, is the number that starts every commercial conversation.
Spend per tenant divided by revenue per tenant. The one that matters. A customer costing 40% of what they pay you is a different problem from one costing 4%.
Spend per feature. Tells you which surface to optimise or reprice.
Week-over-week change per tenant, with an alert. The retry loop is only expensive because nobody noticed for three weeks. An alert on a tenant's spend doubling week over week catches it on day two.
Cost control, not just cost visibility
Visibility is where most teams stop. It is half the job.
Once you can attribute spend, you can enforce against it: per-tenant budgets that reject rather than warn, per-tenant rate limits so one customer cannot exhaust shared capacity, and routing rules that send bulk workloads to cheaper models.
Enforcement has to happen in the request path, which is the practical argument for putting the meter at the gateway rather than in an observability pipeline. An observability tool can tell you a tenant overspent last Tuesday. Only something in the request path can decline the request.
Everstack records these fields per request at the gateway, with per-tenant limits and budgets enforced in the same place, so attribution and enforcement are the same system rather than two that have to agree. The pricing page shows how we meter our own usage on the same mechanism.