LLM cost management for CFOs.
A finance guide · July 26, 2026
AI spend is no longer an experiment line item. For many companies it is becoming one of the fastest-growing operating costs. The CFO's job is to give product teams the freedom to ship while making sure that freedom is visible, budgeted, and controlled.
1. Make spend visible first
Before you set rules, you need data. That means every LLM request must be tagged with: team, project, feature, model, provider, and environment. Without this, the monthly cloud invoice is just a number. With it, you can ask which product drove the spike and why.
Finance should own the reconciliation to the provider invoice. Gateway logs are useful, but they miss retries, fallback calls, and reasoning tokens. The invoice is the ground truth.
2. Choose the right metrics
Total spend tells you little. The metrics that matter are:
- Cost per successful task — resolved ticket, accepted suggestion, completed workflow.
- Spend concentration — the top five endpoints usually drive most of the cost.
- Budget variance — actual vs. forecast by team or feature.
- Cost per user — especially important for product-led growth businesses.
3. Set budgets and guardrails
Give teams a monthly AI budget and the tools to stay inside it. Useful guardrails include per-request cost caps, daily user limits, and alerts when a feature's spend exceeds a threshold. The goal is not zero cost; it is predictable cost.
4. Optimize with product, not against it
Cost optimization should never damage the user experience. The safest levers — model routing, prompt caching, batch routing — usually save 30–50% with no quality regression if tested against a stable baseline. For the engineering playbook, see FinOps LLM's guide to LLM cost management.
5. Run a weekly operating rhythm
Strong AI cost programs review the largest drivers weekly, not monthly. Finance and product together pick one or two optimization candidates, test them behind flags, and reconcile savings against the invoice. This turns cost control from a one-off project into a habit.