Record it at request time, or not at all
The core insight: model prices change, and you cannot reconstruct historical cost from historical token counts.
Open-source AI infrastructure
Per-Tenant LLM Cost Tracking That Survives an Audit
Why token counts from provider dashboards cannot be attributed to customers, what to record at request time so cost is reconstructable later, and the three mistakes…
The three mistakes
The most common one, and it looks fine until you have a second customer.
Where to put the recording
In application code. Full control, and it works regardless of how you call providers. But it is a code change in every service, it drifts as services are added, and coverage is only ever as good as your least-maintained service.
Making it useful
Raw records are not a cost model. The four things people actually ask:
Cost control, not just cost visibility
Once you can attribute spend, you can enforce against it: per-tenant budgets that reject rather than warn, per-tenant rate limits so one customer cannot exhaust shared capacity, and routing rules that send bulk workloads to cheaper models.