AI gross-margin erosion: model it before you ship
September 2, 2026
An AI feature moves spend out of R&D and into cost of revenue. Model training and prototyping can sit in development budgets, but inference runs every time a customer uses the product, which makes it a variable cost of serving. If it is not modelled as one before launch, the first visible signal is a gross-margin decline that finance has to explain after the fact.
Put inference in cost of revenue
Classify recurring inference, plus its retries, tool calls, and any vendor per-seat AI add-on that scales with usage, as cost of revenue rather than R&D. The test is whether the cost recurs when a customer uses the feature again. One-off fine-tuning runs and evaluation suites usually fail that test; the serving path passes it.
The calculation
Work in cost per active account per month, not cost per token. Take measured cost per successful task, multiply by expected tasks per account per month, and add the share of failed and retried work you will still pay for. Compare that against the revenue the account already produces. The output is the new gross margin for that cohort, not a percentage saved.
Run it at three volumes: median usage, the 90th-percentile account, and the heaviest account you would still accept. The median usually looks fine. Margin failures live in the tail, because AI cost scales with usage while a flat subscription price does not.
Where the erosion actually comes from
Four causes account for most of it. Heavy users on flat pricing, where consumption has no ceiling. Retries and failed runs billed at full rate. Long conversation context resent on every turn. And agentic flows that expand one user action into many model calls. Each is measurable before launch if the feature is instrumented in staging with realistic traffic.
Decide the pricing response early
The options are a usage ceiling in the plan, a consumption component in the price, a cheaper model on the default path with the expensive one reserved for cases that need it, or accepting the margin as a deliberate acquisition cost for a fixed period. All four are defensible. What is not defensible is discovering the answer from a variance report a quarter later.
Report it as a margin line
Give the board AI cost of revenue as its own line inside gross margin, with cost per account and the margin at each usage percentile. That framing makes a rising bill readable: spend that grows with revenue at a stable cost per account is scale, and spend that grows faster is erosion.