LLM usage metering for customer billing.

Operations guide · 1 July 2026

By the LLM CFO team

Usage metering is how SaaS platforms that resell LLM functionality track consumption per customer and translate it into a charge. Unlike internal chargeback, metering is customer-facing and contractual—it defines who owes what, when, and for how much. Getting metering right means choosing the right measurement points, handling provider discounts, and reconciling against the actual provider invoice.

Where metering lives in your architecture

Every LLM API call flowing through your platform passes through multiple layers. You can capture metering at any of them:

• API gateway. Your front door to the provider. Captures all requests with low latency overhead, works regardless of language or SDK, and is hardest to spoof. Best for real-time dashboards and per-customer quotas. Downside: unless you control the provider integration, you may see calls but not the full token count until the response arrives.
• SDK or client library. Metering at the point the customer calls you. Easiest to tag by customer, feature, or account, but requires you to ship and maintain SDKs and opens the surface for customer manipulation (if they control the client code).
• Provider invoice or usage API. The provider's authoritative record. Arrives days or weeks later, complete with all retries, timeouts, and edge cases the gateway missed. Use this for reconciliation only—never bill customers off the invoice without a lag.

The right approach uses all three: gateway for real-time rate limiting and customer visibility, SDKs for enriched tagging, and provider invoices for monthly reconciliation.

Billing units: tokens, requests, or outcomes

You must choose what unit you charge customers by:

• Token-based billing. Charge per token consumed (input + output). Mirrors how providers price, gives you maximum cost recovery, and is fairest when request sizes vary wildly. Downside: complex to explain to non-technical customers, requires accurate token counting, and exposes you to customer confusion over input vs. output vs. cached tokens.
• Request-based billing. Charge a flat fee per API call. Simpler to understand and calculate, works well if request sizes are similar, reduces customer surprise. Downside: does not recover cost when requests are large, incentivizes long prompts, and does not scale if output length varies.
• Outcome-based billing. Charge per completed task, translation, classification, or search result—the business value delivered, not the LLM cost. Hardest to implement, but aligns pricing with customer value instead of your infrastructure cost. Most SaaS companies move here eventually.

Most platforms start with token-based (cost recovery is clear) and gradually move to outcome-based (better positioning).

The discount math: cache-read, batch, and reasoning tokens

Providers discount certain token types dramatically:

• Cache-read tokens. OpenAI charges ~50% of input rate; Anthropic charges ~10%. If you bill customers at the full input rate for cached tokens, you keep the margin. If you pass the discount through, customers benefit when cache hit rates improve but you lose revenue. Your decision here is a pricing choice, not a technical detail—document it in the contract.
• Batch API tokens. Most providers discount batch ~50% off standard rates. Again, you can retain the margin or pass the savings to customers. If you pass it through, your revenue shrinks on batch work even though your infrastructure cost is the same.
• Reasoning tokens. Output-like tokens that the model consumes for extended thinking (OpenAI o1, Claude Opus with thinking). They are invisible to the application but visible on your bill. If customers use reasoning and you do not bill for it, you absorb the cost. If you bill for it, you must explain it clearly.

The key: if the provider gives you a discount, you must decide whether to keep it or pass it through. Do not leave this ambiguous—it will become a contract dispute.

Handling retries, tool calls, and fallback paths

One user action often triggers multiple LLM requests:

• Retries. A transient error triggers a retry. Charge once? Charge both? If you only bill on success, you undercount costs during outages. If you bill on all attempts, you charge customers for failures. Common practice: charge on all attempts, but document the policy.
• Tool calls and function invocation. An agent makes a request, gets a tool decision, calls the tool, and then makes a follow-up request. Metering must count both. If your metering only counts the first request, you are missing the full cost.
• Fallback chains. Customer request → try expensive model → hit budget or error → fall back to cheaper model. Did the customer get two charges or one? Document the rule: metering usually counts every request.

Every alternate path in your application is a metering edge case. Build a decision tree early.

Idempotency and deduplication

Your metering system must be idempotent: the same request replayed twice should not produce two charges. This is hard at scale:

• Request IDs. Tag every LLM request with a unique, customer-controlled ID (idempotency key). If you see the same ID twice, it is a retry; count it once.
• Event deduplication. Your metering pipeline must deduplicate events within a time window (usually one invoice period). Use the request ID + timestamp to detect and skip duplicates.
• Retry safety. If a metering event fails to reach your database, the system must be able to safely retry without double-charging. Use database insert-or-update semantics keyed on (request_id, timestamp).

Without idempotency, every network hiccup becomes revenue loss (double charges that customers contest) or hidden cost (charges you ignore to avoid disputes).

Monthly reconciliation and variance investigation

At month close, reconcile your metered usage against the provider invoice:

1. Sum your metered usage. All events for the month, converted to cost using your pricing table.
2. Compare to the provider invoice. The numbers rarely match exactly. Variance under 2% is acceptable; variance over 3% means something is wrong.
3. Investigate the delta. Common causes: (a) missing metering events (retries, tool calls, fallback paths you forgot to tag), (b) timestamp misalignment (requests in one month, invoiced in the next), (c) wrong pricing (you used last month's rates but the provider updated them), (d) provider bugs or adjustments.
4. Fix the root cause. Do not manually adjust. If you are consistently short, fix your metering. If you are consistently over, fix your pricing table.

Reconciliation is also your audit trail. If a customer disputes a charge, you rebuild the bill from metered events and compare to your cost.

What to meter: the minimal set

Do not meter everything. Focus on the contractual signal:

• Customer ID or account ID
• Model and provider (pricing varies by both)
• Token counts: input, output, cache-read, reasoning (separately)
• Timestamp (to the second; you will need this for reconciliation)
• Request ID (for idempotency)
• Estimated cost (compute it immediately using your pricing table, not the provider response)

Everything else (feature, endpoint, user, duration) is operational telemetry, not metering.

Related

• How to track AI token usage — the foundations for accurate token counting.
• Cache-read tokens and the baseline trap — why cache discounts corrupt your cost model if not handled separately.
• AI chargeback and showback — internal allocation (different from customer billing).
• Batch API routing: 50% off for the work that can wait — understanding batch discounts you must price for.
• LLM cost monitoring: what to track and how to control it — the observability foundation for metering.